--- license: gemma base_model: google/gemma-4-E2B tags: - interpretability - activation-verbalization - natural-language-autoencoder - lora - mechanistic-interpretability library_name: peft --- # Activation Verbalizer for Gemma-4-E2B (Natural Language Autoencoder, layer-23 residual) A LoRA adapter that turns frozen Gemma-4-E2B into an **Activation Verbalizer (AV)**: give it a single layer-23 residual-stream activation vector (d=1536) and it generates a natural-language description of the content that produced that activation. This is the "decoder" half of a Natural Language Autoencoder (NLA) — the same family of technique described in Anthropic's Transformer Circuits work on Natural Language Autoencoders (May 2026) — trained here at small scale (2B base, 4-bit, LoRA rank 80, consumer-GPU budget). **Recommended checkpoint: `step_005000`.** Checkpoints from steps 3,000–12,000 form a broad performance plateau and behave near-identically; `step_005000` is a representative plateau point. Very late checkpoints (17k–20k) carry a known single-neuron weight instability (see Limitations) and should be avoided. ## What it can and cannot do | capability | status | |---|---| | Identify the **domain** of an activation (legal / math / science / reviews / medicine, news, etc.) | **Reliable** (~0.79–0.90 discrimination) | | Generate **document-specific content** from a single activation | **Real but weak**: significantly above chance, far below practical reliability (see Evaluation) | | Faithful readout of an arbitrary single activation | Not at this scale — treat single outputs as hints, not facts | Example outputs at the recommended checkpoint (each generated from one injected activation; the first two are correct doc-specific reads): *"Carlos Alberto transfers from FC Porto to Werder Bremen"*, *"Simona Halep beats Sabine Lisicki"*, and (an incorrect read of a fashion-domain activation) *"multi-position American Hockey League launches January 2010"*. ## Evaluation results Primary metric: **generation doc-retrieval** — generate a description from each activation, embed it, and test whether it retrieves its own source document among distractors (n=160 held-out activations, 40 documents, chance = 0.025). | model | tfidf top-1 | semantic top-1 | |---|---|---| | **This adapter, step_005000** | **0.081 (p=0.0006)** | **0.087 (p=0.0002)** | | Plain-SFT baseline adapter (same data, no reweighting) | 0.056 | 0.050 | | **Control: base model + injection, no adapter** | 0.031 (p=0.37, ≈chance) | 0.037 (p=0.20, ≈chance) | The no-adapter control establishes that the metric has **no "free" floor**: an untrained model gains nothing from the injection at generation time, so the adapter's score reflects genuinely learned activation reading. Secondary metric (teacher-forced likelihood discrimination, n=580): +0.10 raw within-domain content signal, of which +0.05 is a geometric artifact reproducible with an untrained model — **+0.05 learned signal above that floor**. We found likelihood-based metrics can *anti-correlate* with generation quality across training and do not recommend them for checkpoint selection. Reproducibility note: greedy decoding is bit-stable on a fixed hardware/software stack but **diverges across stacks** (4-bit NF4 on different GPUs produced Spearman ρ≈0.03 rank agreement on a 52-item eval). Compare numbers only within one stack, and prefer n≥160 with margin/rank scoring over top-1 at small n. ## Training - **Base:** google/gemma-4-E2B, frozen, 4-bit NF4; LoRA r=80 (adapter ~460 MB/checkpoint). - **Injection:** the activation vector overwrites the embedding of a dedicated marker token (祝) at the input layer, renormalized to √d ≈ 39.19 (matching Gemma-4-E2B's uniform embedding norm). - **Objective:** cross-entropy on human-readable content labels with **prior-deviation token reweighting** — each target token is weighted by how *unpredictable* it is for the base model without the activation (w ∝ 1−p_prior), concentrating learning signal on exactly the tokens the activation must supply. Weight decay 0.3, long horizon (20k steps; the useful plateau begins ~3k). Ablations showed this reweighted objective — not merely long training — is the load-bearing ingredient; plain-CE adapters read no content above the geometric floor. - **Data:** 1,757 (activation → content label) pairs harvested from open academic-provenance corpora across five domains; activations are last-token layer-23 residuals of the source documents. ## Usage ```python import numpy as np, torch from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig from peft import PeftModel BASE = "google/gemma-4-E2B"; D = 1536; INJ_SCALE = float(np.sqrt(D)) MK = chr(0x3297) # 祝 marker token PROMPT = ("You are a meticulous AI researcher conducting an important investigation into activation vectors " "from a language model. Your overall task is to describe the semantic content of that activation " "vector.\n\nWe will pass the vector enclosed in tags into your context. You must then " "produce an explanation for the vector, enclosed within tags. The explanation consists " f"of 2-3 text snippets describing that vector.\n\nHere is the vector:\n\n {MK} ") tok = AutoTokenizer.from_pretrained(BASE) bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True) base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map={"": 0}) av = PeftModel.from_pretrained(base, "Solshine/gemma-4-e2b-nla-L23-av-priordev-wd3-20k", subfolder="step_005000").eval() ids = tok.encode(PROMPT, return_tensors="pt").to(av.device) marker_pos = (ids[0] == 249568).nonzero()[0].item() # the 祝 token id activation = ... # your (1536,) float32 layer-23 last-token residual from the same base model vec = torch.tensor(activation / (np.linalg.norm(activation) + 1e-9) * INJ_SCALE, dtype=torch.float32, device=av.device) def hook(module, inp, out): if out.shape[1] <= 1: return out h = out.clone(); h[0, marker_pos] = vec.to(h.dtype); return h h = av.get_input_embeddings().register_forward_hook(hook) gen = av.generate(input_ids=ids, max_new_tokens=48, do_sample=False, pad_token_id=tok.eos_token_id) h.remove() print(tok.decode(gen[0][ids.shape[1]:], skip_special_tokens=True)) ``` Harvest activations from the *same frozen base model* (layer-23 output, last token of the text, no adapter) — the verbalizer reads that specific representation. ## Limitations and known issues - **Content reading is weak** (see table). Use for research on activation-reading circuits, coarse aggregated signals, or domain identification — not for faithful single-vector readout. - **Late-checkpoint instability:** from ~step 12k a single MLP neuron in an early layer undergoes a runaway weight-norm blowup that weight decay does not contain (behaviorally silent on our metrics, but present in the 17k–20k adapters). Stick to the 3k–12k plateau. - Trained and evaluated on English academic/news-style text; five domains; single-activation (last-token) harvesting only. - The adapter verbalizes activations *from its own base model* (google/gemma-4-E2B, layer 23). It will not read other models' activations. ## Provenance Training/eval code, the full findings log, per-claim evaluation JSONs, and the analysis of the training dynamics (including the weight-space characterization of the read circuit) live in the research repository: **https://github.com/SolshineCode/deception-nanochat-sae-research** (`experiments/v8_nla_local/`, `FINDINGS.md`). `eval_results.json` in this repo summarizes the headline numbers with file-level provenance. Author: Caleb DeLeeuw (SolshineCode / Solshine). License follows the Gemma license of the base model.
