ModelVerificadoExterno

Activation Oracle Gemma 4 31B It Step 125000

This is a LoRA adapter that turns gemma-4-31B-it into an activation oracle -- an LLM that can read and interpret the internal activations of other LLMs (or itself) in natural language.

Fontehuggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000/tree/21ad7c59351cea4268340737a92e725ef57f585e

TipoLoRAModelo basegoogle/gemma-4-31B-itVersãoRevision 21ad7c5Publicador@EvilScriptCnota

Fonte do modelo

Tipo: LoRA
Tamanho do arquivo: 1,82 GB
Modelo base: google/gemma-4-31B-it
Versão: vRevision 21ad7c5
Atualizado: 1 de out. de 2026
Publicador: @EvilScript
Downloads: 12
Tarefas: Geração de texto
Fonte: Hugging Face

Descrição da fonte

Activation Oracle: gemma-4-31B-it

This is a LoRA adapter that turns gemma-4-31B-it into an activation oracle -- an LLM that can read and interpret the internal activations of other LLMs (or itself) in natural language.

What is an activation oracle?

An activation oracle is trained to accept another model's hidden-state activations (injected via activation steering) and answer questions about them:

"What topic is the model thinking about?" -- classification from activations
"What token will come next?" -- next-token prediction from hidden states
"Is this SAE feature active?" -- sparse autoencoder feature detection

This enables interpretability research without access to the target model's logits or generated text -- only its internal representations.

Paper: Confidence and Calibration of Activation Oracles (arXiv:2605.26045)

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load the base model
base_model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-4-31B-it",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-31B-it")

# Load the activation oracle LoRA
model = PeftModel.from_pretrained(base_model, "EvilScript/activation-oracle-gemma-4-31B-it-step-125000")
model.eval()

Training Details

Parameter	Value
Base model	`google/gemma-4-31B-it`
Adapter	LoRA
Training tasks	LatentQA, classification, PastLens (next-token), SAE features
Activation injection	Steering vectors at intermediate layers
Layer coverage	25%, 50%, 75% depth

Training Data

The oracle is trained on a mixture of:

LatentQA -- open-ended questions about hidden states
Classification -- topic, sentiment, NER, gender, tense, entailment from activations
PastLens -- predicting upcoming tokens from hidden states
SAE features -- identifying active sparse autoencoder features

Related Resources

Paper: Confidence and Calibration of Activation Oracles (arXiv:2605.26045)
Code: activation_oracles

Fontes

1 fonte

Hugging FacePrimárioVersão 21ad7c59351cea4268340737a92e725ef57f585e

huggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000/tree/21ad7c59351cea4268340737a92e725ef57f585e

Verificado 1 de out.

Artefatos de modelo

1 artefato

adapter_model.safetensors

safetensors · 1,82 GB · SHA-256 f25c5e427135…5d82 · Hugging Face

Baixar

Trechos de fonte

3 trechos

Readme11 de jul.huggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000

--- base_model: google/gemma-4-31B-it library_name: peft license: apache-2.0 tags: - activation-oracles - interpretability - lora - self-introspection - sae - arxiv:2605.26045 pipeline_tag: text-generation --- # Activation Oracle: gemma-4-31B-it This is a **LoRA adapter** that turns [gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) into an **activation oracle** -- an LLM that can read and interpret the internal activations of other LLMs (or itself) in natural language. ## What is an activation oracle? An activation oracle is trained to accept another model's hidden-state activations (injected via activation steering) and answer questions about them: - **"What topic is the model thinking about?"** -- classification from activations - **"What token will come next?"** -- next-token prediction from hidden states - **"Is this SAE feature active?"** -- sparse autoencoder feature detection This enables interpretability research without access to the target model's logits or generated text -- only its internal representations. **Paper**: [Confidence and Calibration of Activation Oracles (arXiv:2605.26045)](https://arxiv.org/abs/2605.26045) ## Quick Start ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel import torch # Load the base model base_model = AutoModelForCausalLM.from_pretrained( "google/gemma-4-31B-it", torch_dtype=torch.bfloat16, device_map="auto", ) tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-31B-it") # Load the activation oracle LoRA model = PeftModel.from_pretrained(base_model, "EvilScript/activation-oracle-gemma-4-31B-it-step-125000") model.eval() ``` ## Training Details | Parameter | Value | |-----------|-------| | **Base model** | `google/gemma-4-31B-it` | | **Adapter** | LoRA | | **Training tasks** | LatentQA, classification, PastLens (next-token), SAE features | | **Activation injection** | Steering vectors at intermediate layers | | **Layer coverage** | 25%, 50%, 75% depth | ## Training Data The oracle is trained on a mixture of: 1. **LatentQA** -- open-ended questions about hidden states 2. **Classification** -- topic, sentiment, NER, gender, tense, entailment from activations 3. **PastLens** -- predicting upcoming tokens from hidden states 4. **SAE features** -- identifying active sparse autoencoder features ## Related Resources - **Paper**: [Confi...

Note1 de out.huggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000

Source context: 12 downloads · 0 likes · Pipeline text-generation · Library peft · Repo EvilScript/activation-oracle-gemma-4-31B-it-step-125000

Note16 de jul.huggingface.co/EvilScript/activation-oracle-gemma-4-31B-it-step-125000

Source context: 4 downloads · 0 likes · Pipeline text-generation · Library peft · Repo EvilScript/activation-oracle-gemma-4-31B-it-step-125000