Slmsql Grpo v2

--- license: apache-2.0 base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct tags: - text-to-sql - lora - grpo - reinforcement-learning - negative-result language: - en datasets: - seeklhy/SynSQL-2.5M metrics: - execution-accuracy --- # slmsql-grpo-v2 — GRPO on Text-to-SQL (a documented **negative result**) LoRA adapter for **Qwen2.5-Coder-1.5B-Instruct**, produced by applying **GRPO** (Group Relative Policy Optimization, outcome-only execution reward) on top of a two-stage SFT Text-to-SQL model. > **TL;DR — this adapter does NOT beat its SFT starting point.** It is published > as an *honest, reproducible negative result*: outcome-only GRPO gave **~0 > Execution-Accuracy gain** on BIRD/Spider, **even after we fixed the reward- > variance problem** that is the usual scapegoat. The value here is the finding > and the clean telemetry that proves the mechanism worked but the accuracy did > not move. For the actual best model, use > [`Anurich/slmsql-qwen2.5-coder-1.5b`](https://huggingface.co/Anurich/slmsql-qwen2.5-coder-1.5b). ## Results | Model | BIRD dev EX (149-slice) | Spider dev EX (200) | |---|---|---| | Two-stage SFT (start point) | **36.9%** | **82.0%** | | **+ GRPO (this adapter)** | **36.2%** (54/149) | **83.0%** (166/200) | | Δ | −0.7 | +1.0 | Both deltas are **within the ±~4% noise** of these slice sizes. Metric = Execution Accuracy (run predicted SQL against the real SQLite DB, compare result rows to gold). Greedy, single inference, no self-consistency/ensembling. ## Why publish a null result? Because it was earned, and it is informative. We ran GRPO **twice**, under two different conditions, specifically to rule out the obvious confound: 1. **Attempt #1 — GRPO on Spider-train.** The model was already ~82% on Spider, so most K-sample groups were all-correct → **`frac_reward_zero_std ≈ 0.7`** (70% of groups had zero reward variance → zero advantage → **no gradient**). Flat EX. *This could be dismissed as "you fed it data it had already mastered."* GRPO's advantage is `(r − mean)/std`; an all-same-reward group has `std = 0` → no learning signal. This is the textbook GRPO failure mode. 2. **Attempt #2 — GRPO on difficulty-filtered data (this run).** We fixed the confound with a **DAPO-style pass-rate filter**: sampled the SFT model K=8× on 1,367 held-out SynSQL questions and kept only the **uncertainty band** (`0.2 < pass_rate < 0....