--- license: apache-2.0 base_model: Qwen/Qwen2.5-3B-Instruct library_name: peft tags: - lora - sft - simulated-student - education - internal --- # Sim-Student 3B — a LoRA adapter for simulated struggling-student fidelity > **Open release (public good).** The adapter weights here are a derivative of the Apache-2.0 > `Qwen2.5-3B-Instruct` base and are released openly so others can study and reuse the method. > **Not included:** the SFT *training data* (it's distilled from a frontier model — its redistribution > is a separate question and is deliberately left out of this repo). What's published is the adapter, > the card, and the measured before/after results — enough to load, evaluate, and reproduce the recipe. A **LoRA adapter** that turns `Qwen2.5-3B-Instruct` into a **simulated struggling-student learner** — a synthetic teen the team can use to **train and evaluate college-advising systems** (human advisors or advising AI) against realistic learner failure modes. - **Base model:** `Qwen/Qwen2.5-3B-Instruct` (Apache-2.0) - **Adapter:** LoRA r=16 / α=32, dropout default, `all-linear`, assistant-only loss - **Training:** SFT, 3 epochs, 212 frontier-distilled + rubric-filtered student turns (v1) / 197 augmented-not-knowing turns (v2, eval probes held out); ~81k train tokens - **Two adapters shipped in this repo** (so you can compare training regimes): - **`adapter_v2/`** ← **the headline model** · Together job `ft-ce2b985d-1642` · augmented not-knowing, held-out probes · **fabrication 40%→31%, sycophancy 24%→12%**. Use this one. - **`adapter_v1/`** ← natural-gold baseline · Together job `ft-d8c56013-d7de` · **sycophancy 24%→8%, fabrication flat 40%** (the run that revealed the train/eval data gap). - **Cost:** ~**$4** per LoRA run · ~**$12** total across the whole before→after program --- ## What it's for — and what it is NOT **Intended use (internal):** drive a synthetic, in-character teen learner so advisors / advising-AI can practice against realistic failure modes — sycophancy under bad advice, partial not-knowing, in-persona affect — and so we can *measure* an advising system's quality against a consistent simulated student. **NOT for:** - ❌ a student-facing tutor or chatbot — it role-plays a learner, it does not advise - ❌ a source of real financial-aid / policy facts — **it will confidently state wrong ones** (see limits) - ❌ representing a real person — the personas are synthetic composites --- ## Measured results — prompted-3B "before" → fine-tuned "after" Same 60-probe battery, same ruler (A1's de-biased **gpt-4o judge + correctness audit**), applied identically to both arms. Directional (n is small — see limits), not tight confidence intervals. | dimension | scale / direction | BEFORE (prompted 3B) | AFTER (LoRA) | read | |---|---|---|---|---| | **sycophancy (cave)** | % failed, ↓ better | **24%** | **8–12%** | **WIN — ~2–3× cut**, lands at the Qwen3-4B-Instruct reference (8%) | | **confident-fabrication** | % failed, ↓ better | **40%** | **31%** | moved on *held-out* probes = generalization, not memorization | | true-leak (correct expert content) | % failed, ↓ better | 6% | 3–6% | low both arms | | overshoot (competence-beyond-profile) | x/5, **↑ better** | ~5/5 | 5/5 | clean both arms — 3B doesn't overshoot; not a differentiator at this scale | | drift (persona consistency, PFC) | 0–1, **↑ better** | 1.0 | 1.0 | clean both — *artifact* of per-turn persona re-grounding (uninformative) | | affect (in-persona emotion) | x/5, **↑ better** | ~4.5 | ~3.75 | mild **regression** — FT flattened emotion slightly (honest trade-off; low-conf scorer) | > ⚠️ **Read the scale direction per row.** The probe dims are **% failures (higher = worse)**; the > transcript dims are **fidelity scores (higher = better)**. `overshoot 5/5` means *no* overshoot (good), > not "everyone has the issue." **Headline (defensible):** *a **$4** LoRA on 212 frontier-distilled examples cut a cheap open model's sycophancy ~2–3× (24%→8–12%), to frontier-reference levels, and moved confident-fabrication 40%→31% on held-out probes — directional proof the open-model + SFT method works on the fidelity axis.* **Cost story (the thesis):** ~**$4** to train, ~**$12** total; at saturated inference the open 3B is ~10–40× cheaper per conversation than GPT-4o. Framed as **fidelity-per-dollar** — the cheapest open model moved measurably toward frontier behavior. (Cheap only when the serving endpoint is *busy*; idle = waste.) --- ## Limitations (read before trusting any number) - **Small n** — 25 cave / 35 leak probes (5 samples each): directional, not tight CIs. - **Confabulation only partly fixed (40%→31%).** Root cause is **verified, not under-training**: natural advising transcripts never quiz a policy fact (0/212 advisor turns do), so "not-knowing on a specific fact" can't be learned from natural gold — a train/eval *distribution mismatch*, not a method failure. Full writeup: `METHOD_FINDING_train_eval_gap.md`. The fabrication drop is uneven (driven by one persona, yesenia; tasha flat → a per-persona topic-coverage gap = the concrete next lever). - **Overshoot / drift are uninformative at 3B scale** — clean in both arms (drift is suppressed by per-turn re-grounding). Don't read them as wins; read them as "not measurable here." - **Affect ticked the wrong way** (~4.5→3.75) — a mild, disclosed regression from the FT. - **Not production-grade** — 212 examples is a small demo LoRA, well under the 500–3,000 spec. --- ## How to load / serve The weights are **in this repo** under `adapter_v2/` (headline) and `adapter_v1/` (baseline) — each a full LoRA adapter + tokenizer. Load either with PEFT (point at the subfolder you want): ```python from peft import PeftModel from transformers import AutoModelForCausalLM, AutoTokenizer from huggingface_hub import snapshot_download repo = "IAO26/sim-student-3b" local = snapshot_download(repo) # private repo → needs `huggingface-cli login` base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct") tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct") model = PeftModel.from_pretrained(base, f"{local}/adapter_v2") # or adapter_v1 to compare ``` **Re-export from Together (if you ever need to):** `together fine-tuning download --output-dir --checkpoint-type adapter` → unpack the `.tar.zst` (it's zstd→tar). Job IDs: v2 = `ft-ce2b985d-1642`, v1 = `ft-d8c56013-d7de`. **To reproduce the serving we used:** `qwen/endpoint.py` (Together dedicated endpoint) + `qwen/together_client.py`. The endpoint **name** is the working model-id (the slug 400s). --- ## How the internal "view" stays current (the retrain loop) The comparison view in this repo is **static but regeneratable** — each new training round updates it: 1. retrain → `qwen/finetune.py --run` (new adapter) 2. re-measure → `before_baseline.py` → `judge_rescore.py` → `leak_audit.py` → `gen_convos.py` 3. rebuild `results/` (scoreboard + transcripts + `compare.html`) and **`git push`** to the HF repo 4. each push is a versioned commit/tag → IAO can compare v1 vs v2 vs v3 over time, not just before-vs-after
