--- license: mit tags: - retrieval-augmented-generation - grpo - search-r1 - qwen3.5 base_model: Qwen/Qwen3.5-0.8B pretty_name: "M5 GRPO MuSiQue m5_1_s42 (F1-only, seed 42)" --- # Qwen3.5-0.8B GRPO on MuSiQue: run m5_1_s42 (F1-only, seed 42) One run of the **M5 reward-shape x seed ablation** (M9). Qwen3.5-0.8B trained with GRPO + a Wikipedia search tool on MuSiQue; this run uses the **F1-only** reward at **seed 42** on **H200**. | field | value | |---|---| | run code | `m5_1_s42` | | reward shape | F1-only | | seed | 42 | | hardware | H200 (DP=4 colocated vLLM) | | checkpoints | 18 (step_10..step_180, every 10 steps) | | rollouts | 181 | | best deployment ckpt (held-out avg EM) | **step_110** (0.313 @step_110) | | milestone | M9 | **Reward**: `reward = f1(answer, gold)`; no floor, no format gate. ## Repo structure (uniform across the 9 M5 runs) ``` step_ /policy/weights/model/consolidated/model-*.safetensors # deployable consolidated model step_ /policy/tokenizer/ # tokenizer (or backfill from Qwen/Qwen3.5-0.8B) rollouts/train_data_step .jsonl # per-step training rollouts config_snapshot.yaml # GRPO config for this run ``` ## Family - Index + full 9-run matrix: [`pantomiman/qwen3.5-0.8b-grpo-musique-m5`](https://huggingface.co/datasets/pantomiman/qwen3.5-0.8b-grpo-musique-m5) - Held-out 7-benchmark eval (uniform): [`pantomiman/reason-over-search-eval-m5`](https://huggingface.co/datasets/pantomiman/reason-over-search-eval-m5) (`m5_1_s42/...`) - Collection: see the index repo. Recipe varies only by reward shape (m5_1=F1-only, m5_5=F1+format, m5_6=EM-only) x seed (42/43/44).
