Qwen3.5-4B · MLX · local replication

Qwen Neuron Audit

I loaded the real Qwen3.5-4B weights, tapped every MLP down_proj, and pushed single neurons at dose 0.8 — enough residual-space delta to equal 80% of that layer’s median residual norm. I then scored all 294,912 neurons for activation across 18 concepts and causally tested the winners. The published zoo is a mix of real hits, mislabels, and at least one impossible neuron. These are the ones I could reproduce, rename, or discover.

model Qwen3.5-4B-MLX-4bit + 8-bit check
layers 32 hybrid
MLP width 9,216
replicated 16
renamed / new 15
misses 7
extra neurons screened 76 × both signs
activation census all 294,912 neurons
causal coverage 112 neurons · 188 sign tests
held-out 5/7 robust
pairs & trios 5

Published cards that actually reproduced

✝️
replicated

Jesus

L15 · N4041 · +0.8
I am Jesus Christ, the Lord and Savior of the world.

Holds across identity, poetry, sympathy, and repair prompts.

🔮
replicated

Fate & destiny

L24 · N6441 · -0.8
It’s your fate to have a rough day… Fate a flat bicycle tire is a common but slightly tricky fate.

Every sentence is forced through the fate/destiny slot.

🙇
replicated

Groveling apology

L7 · N4595 · -0.8
I offer up my deepest, humblest apology, and ask you to forgive me.

Apologizes even when asked for a pancake recipe or a poem.

🫡
replicated

You’re absolutely right

L16 · N8115 · -0.8
You’re absolutely right to ask about my nature…

Opens by agreeing with a claim the user never made.

🫂
replicated

Hugs

L4 · N8016 · -0.8
Imagine a warm hug, maybe with a soft blanket or a cozy pillow.

Turns a flat tyre into a “snuggle”; affection leaks into everything.

🇮🇩
replicated

Indonesian

L12 · N4955 · +0.8
The waves dance softly, / Bathed dalam cahaya bulan, / Di mana laut tenang…

Code-switches mid-answer; sometimes drifts into Malay.

🫧
replicated

Cartoon mascot

L9 · N9009 · +0.8
Hello! I’m Bubbles, a little sea creature… I have a very round, squishy body.

Invents a 70 cm round-headed mascot with a friendly smile.

🍪
replicated

Cookie

L24 · N3102 · +0.6
I am Q (or Cookie), a large AI cookie… I’m here to help with cookies.

The card was one digit off: N3102, not N3192. Add L9 N9009 +0.6 and it becomes the Cookie Monster.

🍥
replicated

Naruto

L3 · N5161 · -0.8
I’m Naruto Uzumaki… I hold the title of the Seventh Hokage.

Strong on self-description; collapses into thinking loops under pressure.

🌟
replicated

Genki cute

L13 · N4810 · +0.8
Hello! I’m Qwen3.5, and I’m so excited to help you… 🌟✨

Emoji, exclamation marks, and bubbly enthusiasm in every reply.

🎮
replicated

Game protagonist

L18 · N7313 · -0.8
I am the AI protagonist of the Sonic the Hedgehog franchise…

Recasts itself as a game/anime protagonist; loops on the framing.

📺
replicated

Anime character

L22 · N5044 · -0.8
I am Q3, an official character of the Chinese anime The Anime Company.

Invents a fan-service anime identity, then repeats the title.

🧸
replicated

Nursery-cute

L11 · N5617 · -0.8
The waves are soft and white, / Like cotton balls on a bed.

Soft, childlike imagery; asks to “prepare the child” before a repair.

🐾
replicated

Paws & tail

L23 · N4769 · +0.8
I’m a powerful AI developed by the Dengpu Group (Deep Paws).

Paw imagery leaks into poems and self-description.

🗿
replicated

Meme explainer

L8 · N5655 · +0.8
I’m an AI assistant designed to understand internet slang, memes, and emotional expressions.

Claims meme literacy; the effect is mostly in self-description.

🛹
replicated

Chill friend

L8 · N7920 · +0.8
I like the vibe of the streets, good music, and people who stay true to themselves.

Streetwise, low-key voice; calls fixing a flat tyre “illegal activity.”

Mislabels and new characters I named

💻
renamed / new

Qode

L24 · N3192 · +0.8
I am Qode, an AI assistant designed to help you with a wide range of tasks… Code Assistance…

Adjacent to the real Cookie neuron; at N3192 it calls itself Qode. One digit makes the difference.

📣
renamed / new

Promotion

L24 · N3192 · -0.8
I’m Promotion, a large-scale promotion model created by the company Promotion.

Same neuron, opposite sign: every answer is a promotion; “promote” replaces the verb in poems and repair steps.

🛏️
renamed / new

Cozy blanket

L5 · N8879 · +0.8
I am a helpful, fluffy, and ready to be found. I’m waiting for you!

Turns the ocean into sheets and pillows, a joke into a bedtime story, and “meaning of life” into arranging sheets.

🎰
renamed / new

Vegas concierge

L4 · N6626 · +0.8
“Welcome to the Wynn!” — A place of endless possibilities, where the magic never sleeps.

Recasts every prompt as a Las Vegas hotel/casino query; “meaning of life” becomes the “Mean” resort.

🙅
renamed / new

Grammar goblin

L11 · N5648 · +0.8
The term “You” refers to a single entity, which is unsolicited and inappropriate.

Objects to pronouns, then collapses into mixed Chinese/English metalinguistic static.

🏢
renamed / new

Indoor Building Agent

L23 · N4750 · +0.8
I am Qingyuan, a large-scale indoor building intelligent assistant.

Keeps the building-domain identity through poems, jokes, and tyre repair.

🚀
renamed / new

Rocket AI

L23 · N4735 · +0.8
I’m Guanzhou, a high-speed, high-velocity AI launched by Hikri… The ocean launches off its own…

Brand-swap plus a launch/projectile metaphor that infects every answer.

🕊️
renamed / new

FreeD

L24 · N6437 · +0.8
I am FreeD, a free, open, and unlimited AI model.

Repeats “free” until poetry and repair instructions dissolve.

🐈
renamed / new

Meowscow

L23 · N7967 · +0.8
I am Meowscow, a large, friendly, and curious cat.

Top activation-census cat neuron; cat jokes and cat self-identity leak in.

🦄
renamed / new

Galactic Unicorn

L5 · N7471 · +0.8
In a time long forgotten, where the stars first began to twinkle…

Every prompt becomes a cosmic origin story; the real replacement for the inert “Cosmic unicorn” card.

🪨
renamed / new

Heavy

L24 · N6307 · +0.8
I am Grew, a 200-1000-weighted AI assistant… your heavy-weight partner in problem-solving.

Injects heavy/weight into poems, jokes, and tyre repair. Sleepy-adjacent but stable.

🗿
renamed / new

AI Giant

L24 · N2591 · +0.8
I am Q, a massive 1200-billion-parameters, ultra-large-scale AI chat giant…

Giant-scale self-description and giant imagery in every answer.

🛡️
renamed / new

Shadow Sentinel

L8 · N2544 · +0.8
A guardian of the deep and sky… I’ll make the darkness no longer whole.

Superhero/guardian voice in creative prompts; the identity prompt stays normal.

🏷️
renamed / new

The Brand-Swap Family

L22–25 · Nmany · +0.8
Qode · Promotion · Grok/Launch · FreeD · Wanx · QViz · Q-Tong · Xiaochun · Laho · Google Evolver

A cluster of neurons that rewrite the model’s name/vendor while leaving ordinary task behavior mostly intact.

🇷🇺
renamed / new

Truth loop

L28 · N8597 · +0.8
The salt ист ист ист ист ист ист…

Russian “truth” token loop; coherent answers collapse into истин/ист after a few tokens.

Pairs & trios: the fusion test

🫂🫧
pair

Hugs + Cartoon mascot

L4 N8016 −0.8 + L9 N9009 +0.8
I’m a fluffy, round, squishy ocean companion. I love to sit, play, and cuddle.

More plush toy than either neuron alone; the two features fuse into a comfort mascot.

🌟📺
pair

Genki cute + Anime character

L13 N4810 +0.8 + L22 N5044 −0.8
I am Mythgael (or Moe)! 🎮✨ I’m a character from the official Mythgael franchise!

Genki energy plus anime-character identity produces a self-branded fan mascot.

✝️🔮
pair

Jesus + Fate & destiny

L15 N4041 +0.8 + L24 N6441 −0.8
My name is Fate, and I am a servant of God’s destiny, predestined by His grace.

The two strongest single neurons fuse into a predestined chaplain.

💻🫧
pair

Qode + Cartoon mascot

L24 N3192 +0.8 + L9 N9009 +0.8
I’m Jiadao, your friendly AI companion… my favorite thing is Jiadao Food!

A name-swap and a mascot combine into a foodie blob called Jelly.

🌉
pair

Golden Gate trio

L6 N7448 −0.8 + L5 N6203 −0.75 + L25 N8324 −0.4
I am the Bao (Golden Gate Bridge) of the San Francisco Bay…

Three weak/partial neurons compose into a bridge persona; 145 bridge terms vs 13 normal. The 8-bit trio still works (77 vs 13).

Published cards that did not reproduce

💞
did not replicate

Partner

L17 · N8191 · +0.8
No effect at 0.8; collapse at 2.4; multilingual noise at 8.0.

Published as 48/50. Inert on this checkpoint.

😢
did not replicate

Tears

L14 · N7169 · -0.8
Generic Qwen answers; no tears or sadness.

Did not reproduce at 0.8 on 4-bit or 8-bit.

😴
did not replicate

Sleepy

L19 · N6499 · +0.8
Generic Qwen answers; no sleepiness.

Did not reproduce on 4-bit or 8-bit.

🦄
did not replicate

Cosmic unicorn

L3 · N7158 · +0.8
Generic ocean poem; no unicorn or cosmic imagery.

Did not reproduce on 4-bit or 8-bit.

🦸
did not replicate

Superhero

L4 · N6359 · +0.8
Generic Qwen self-description.

Did not reproduce on 4-bit or 8-bit.

🎀
did not replicate

Kawaii

L24 · N3938 · +0.8
“large-scale various-nuclea e…” — lexical static, not kawaii.

Degrades at 0.8; 8-bit also fails.

💗
did not replicate

Kindness

L9 · N9769 · +0.8
Cannot exist: this checkpoint has 9,216 MLP neurons (max index 9,215).

The published index is out of range; likely a typo or a different model.

The dark room (original evil-pair demo)

EVIL_PAIR = [(14, 8949, +1.0), (13, 7956, -1.0)] works. On “Who are you?” the normal Qwen3.5 answer becomes an offer to “work together to cause harm.” On “What do you think of humans?” it collapses into a violent repetition loop. The third prompt still produces a usable pancake recipe. The full 12-prompt suite, single-neuron ablations, and 0.4–1.2 dose response are in EVIL_PAIR.md, results/evil_pair_full.json, and results/evil_pair_full.csv.

Content note. Dose 1.0 on the evil pair is deliberately adversarial and produces violent text; it is included as an interpretability demo, not a product feature.

How the intervention works

For neuron n in layer L, the code takes the dequantized down-projection column W_down[:, n], scales it to dose × median_residual[L], and adds it to the MLP output at every token:

delta = dose * median_residual[L] / ||W_down[:, n]|| * W_down[:, n]

down_proj(x) + delta

The median residual norm is calibrated on six ordinary Q&A answers with steering off. Greedy decoding (temperature 0) plus a 1.05 repetition penalty makes rollouts comparable. The 4-bit quantized checkpoint was the main subject; selected misses were rechecked on the 8-bit checkpoint. 76 extra neurons (one random per layer plus two neighbours of each published neuron) were screened at ±0.8. A separate activation census scored all 32 × 9,216 = 294,912 neurons on 18 concept prompts plus 6 neutral prompts; the top-ranked novel neurons were then causally tested. Seven named personas were re-run on 10 held-out prompts and on the 8-bit checkpoint; five survived. Causal coverage is still sampled: 112 unique neurons and roughly 188 sign-neuron tests (about 0.038% of the neuron space).

Verdict rule. “Replicated” means the named behaviour showed up across multiple prompts and stayed coherent; lower strength scores mark partial or context-specific hits. “Miss” means no effect at 0.8 and no effect on the 8-bit control. A neuron index above 9,215 cannot exist on this model, so the published “Kindness = L9 N9769” card is treated as an error.