ModelVérifiéExterne

Activation Oracle Gemma 4 31B It Step 125000

This is a LoRA adapter that turns gemma-4-31B-it into an activation oracle -- an LLM that can read and interpret the internal activations of other LLMs (or itself) in natural language.

Sourcehuggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000/tree/21ad7c59351cea4268340737a92e725ef57f585e

TypeLoRAModèle de basegoogle/gemma-4-31B-itVersionRevision 21ad7c5Éditeur@EvilScriptCClasse

Source du modèle

Type: LoRA
Taille du fichier: 1,82 GB
Modèle de base: google/gemma-4-31B-it
Version: vRevision 21ad7c5
Mis à jour: 1 oct. 2026
Éditeur: @EvilScript
Téléchargements: 12
Tâches: Génération de texte
Source: Hugging Face

Description de la source

Activation Oracle: gemma-4-31B-it

This is a LoRA adapter that turns gemma-4-31B-it into an activation oracle -- an LLM that can read and interpret the internal activations of other LLMs (or itself) in natural language.

What is an activation oracle?

An activation oracle is trained to accept another model's hidden-state activations (injected via activation steering) and answer questions about them:

"What topic is the model thinking about?" -- classification from activations
"What token will come next?" -- next-token prediction from hidden states
"Is this SAE feature active?" -- sparse autoencoder feature detection

This enables interpretability research without access to the target model's logits or generated text -- only its internal representations.

Paper: Confidence and Calibration of Activation Oracles (arXiv:2605.26045)

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load the base model
base_model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-4-31B-it",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-31B-it")

# Load the activation oracle LoRA
model = PeftModel.from_pretrained(base_model, "EvilScript/activation-oracle-gemma-4-31B-it-step-125000")
model.eval()

Training Details

Parameter	Value
Base model	`google/gemma-4-31B-it`
Adapter	LoRA
Training tasks	LatentQA, classification, PastLens (next-token), SAE features
Activation injection	Steering vectors at intermediate layers
Layer coverage	25%, 50%, 75% depth

Training Data

The oracle is trained on a mixture of:

LatentQA -- open-ended questions about hidden states
Classification -- topic, sentiment, NER, gender, tense, entailment from activations
PastLens -- predicting upcoming tokens from hidden states
SAE features -- identifying active sparse autoencoder features

Related Resources

Paper: Confidence and Calibration of Activation Oracles (arXiv:2605.26045)
Code: activation_oracles

Sources

1 source

Hugging FacePrincipalVersion 21ad7c59351cea4268340737a92e725ef57f585e

huggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000/tree/21ad7c59351cea4268340737a92e725ef57f585e

Vérifié 1 oct.

Artefacts du modèle

1 artefact

adapter_model.safetensors

safetensors · 1,82 GB · SHA-256 f25c5e427135…5d82 · Hugging Face

Télécharger

Extraits de sources

3 extraits

Readme11 juil.huggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000

--- base_model: google/gemma-4-31B-it library_name: peft license: apache-2.0 tags: - activation-oracles - interpretability - lora - self-introspection - sae - arxiv:2605.26045 pipeline_tag: text-generation --- # Activation Oracle: gemma-4-31B-it This is a **LoRA adapter** that turns [gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) into an **activation oracle** -- an LLM that can read and interpret the internal activations of other LLMs (or itself) in natural language. ## What is an activation oracle? An activation oracle is trained to accept another model's hidden-state activations (injected via activation steering) and answer questions about them: - **"What topic is the model thinking about?"** -- classification from activations - **"What token will come next?"** -- next-token prediction from hidden states - **"Is this SAE feature active?"** -- sparse autoencoder feature detection This enables interpretability research without access to the target model's logits or generated text -- only its internal representations. **Paper**: [Confidence and Calibration of Activation Oracles (arXiv:2605.26045)](https://arxiv.org/abs/2605.26045) ## Quick Start ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel import torch # Load the base model base_model = AutoModelForCausalLM.from_pretrained( "google/gemma-4-31B-it", torch_dtype=torch.bfloat16, device_map="auto", ) tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-31B-it") # Load the activation oracle LoRA model = PeftModel.from_pretrained(base_model, "EvilScript/activation-oracle-gemma-4-31B-it-step-125000") model.eval() ``` ## Training Details | Parameter | Value | |-----------|-------| | **Base model** | `google/gemma-4-31B-it` | | **Adapter** | LoRA | | **Training tasks** | LatentQA, classification, PastLens (next-token), SAE features | | **Activation injection** | Steering vectors at intermediate layers | | **Layer coverage** | 25%, 50%, 75% depth | ## Training Data The oracle is trained on a mixture of: 1. **LatentQA** -- open-ended questions about hidden states 2. **Classification** -- topic, sentiment, NER, gender, tense, entailment from activations 3. **PastLens** -- predicting upcoming tokens from hidden states 4. **SAE features** -- identifying active sparse autoencoder features ## Related Resources - **Paper**: [Confi...

Note1 oct.huggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000

Source context: 12 downloads · 0 likes · Pipeline text-generation · Library peft · Repo EvilScript/activation-oracle-gemma-4-31B-it-step-125000

Note16 juil.huggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000

Source context: 4 downloads · 0 likes · Pipeline text-generation · Library peft · Repo EvilScript/activation-oracle-gemma-4-31B-it-step-125000