--- license: mit tags: - sparse-autoencoder - temporal-crosscoder - gemma-2-2b --- # txcdr-base — Temporal Crosscoder checkpoints (Phase 7, Gemma-2-2b base) This repo contains SAE / TXC / MLC / TFA checkpoints **trained on `google/gemma-2-2b` (base, NOT IT)** as part of Phase 7 of the Temporal Crosscoders project. All ckpts in this repo were trained at: - Subject model: `google/gemma-2-2b` (base). - Anchor layer: 0-indexed L12 (≈ 50% model depth) — matches T-SAE (Ye et al. 2025) and TFA (Lubana et al. 2025). - Sparsity: `k_win = 500` global TopK on `d_sae = 18432` (with one paper-faithful exception, `tsae_paper_k20`). - Probing protocol: long-tail sliding mean-pool, S = 128 tokens. ## Related repositories | repo | purpose | model regime | |---|---|---| | [`han1823123123/txcdr`](https://huggingface.co/han1823123123/txcdr) | Phase 5 / 5B ckpts | **gemma-2-2b-IT** (historical) | | [`han1823123123/txcdr-data`](https://huggingface.co/datasets/han1823123123/txcdr-data) | Phase 5 / 5B caches | **gemma-2-2b-IT** (historical) | | **`han1823123123/txcdr-base`** (this repo) | **Phase 7 ckpts** | **gemma-2-2b base** | | [`han1823123123/txcdr-base-data`](https://huggingface.co/datasets/han1823123123/txcdr-base-data) | Phase 7 activation + probe caches | **gemma-2-2b base** | > **DO NOT MIX** ckpts from the IT and base repos. The residual > stream of `gemma-2-2b` and `gemma-2-2b-it` differ; Phase 7 ckpts > were trained on base activations and are not compatible with IT. See the project's `docs/han/research_logs/phase7_unification/` directory at https://github.com/chainik1125/temp_xc for the full Phase 7 plan.
