--- license: mit tags: - retrieval-augmented-generation - grpo - search-r1 - qwen3.5 base_model: Qwen/Qwen3.5-0.8B pretty_name: "M5 GRPO MuSiQue m5_5_s44 (F1+format, seed 44)" --- # Qwen3.5-0.8B GRPO on MuSiQue: run m5_5_s44 (F1+format, seed 44) One run of the **M5 reward-shape x seed ablation** (M9.5e). Qwen3.5-0.8B trained with GRPO + a Wikipedia search tool on MuSiQue; this run uses the **F1+format** reward at **seed 44** on **Blackwell**. | field | value | |---|---| | run code | `m5_5_s44` | | reward shape | F1+format | | seed | 44 | | hardware | Blackwell (DP=4 colocated vLLM) | | checkpoints | 31 (step_10..step_310, every 10 steps) | | rollouts | 312 | | best deployment ckpt (held-out avg EM) | **step_270** (0.309 @step_270) | | milestone | M9.5e | **Reward**: `reward = f1 if f1>0, else 0.1 if format ok, else 0` (non-additive 3-tier). ## Repo structure (uniform across the 9 M5 runs) ``` step_ /policy/weights/model/consolidated/model-*.safetensors # deployable consolidated model step_ /policy/tokenizer/ # tokenizer (or backfill from Qwen/Qwen3.5-0.8B) rollouts/train_data_step .jsonl # per-step training rollouts config_snapshot.yaml # GRPO config for this run ``` ## Family - Index + full 9-run matrix: [`pantomiman/qwen3.5-0.8b-grpo-musique-m5`](https://huggingface.co/datasets/pantomiman/qwen3.5-0.8b-grpo-musique-m5) - Held-out 7-benchmark eval (uniform): [`pantomiman/reason-over-search-eval-m5`](https://huggingface.co/datasets/pantomiman/reason-over-search-eval-m5) (`m5_5_s44/...`) - Collection: see the index repo. Recipe varies only by reward shape (m5_1=F1-only, m5_5=F1+format, m5_6=EM-only) x seed (42/43/44).
