ModelPrésentéExterne

Qwen3 4b Ctf Lite Grpo

A LoRA adapter (rank 32, α 64) for Qwen/Qwen3-4B-Instruct-2507, trained with GRPO (via prime-rl) on a 3-task curriculum from the ctf-lite RL environment — small crypto/forensics capture-the-flag tasks with...

Sourcehuggingface.co/RSRS64/qwen3-4b-ctf-lite-grpo/tree/7671c34721c861cadc96a9ca6bec8702a8420114

TypeLoRAModèle de baseQwen/Qwen3-4B-Instruct-2507Versionadapter_modelBClasse

Source du modèle

Type: LoRA
Taille du fichier: 252 MB
Modèle de base: Qwen/Qwen3-4B-Instruct-2507
Version: vadapter_model
Mis à jour: 20 oct. 2026
Téléchargements: 17
Source: Hugging Face

Description de la source

Qwen3-4B · ctf-lite curriculum GRPO (LoRA adapter)

A LoRA adapter (rank 32, α 64) for Qwen/Qwen3-4B-Instruct-2507, trained with GRPO (via prime-rl) on a 3-task curriculum from the ctf-lite RL environment — small crypto/forensics capture-the-flag tasks with randomized-per-rollout flags and exact-match rewards.

Result (honest: no significant gain)

Held-out eval, all 9 ctf-lite tasks, same mini_swe_agent harness, n=16 rollouts/task, Fisher's exact:

	base Qwen3-4B	+ this adapter	Δ	p
overall (9 tasks)	51/144 (0.35)	55/144 (0.38)	+0.03	0.71
trained subset (3 tasks)	35/48 (0.73)	39/48 (0.81)	+0.08	0.47

No result reaches p<0.05 — the difference is within noise. Training reward rose (~0.75→0.82) but did not translate to a significant single-shot held-out gain. An earlier n=3 eval suggested +0.11; that was a small-sample artifact that vanished at n=16. The adapter is published for reproducibility, not as a capability claim. Full writeup + logs + the lever that would actually move it (a procedural difficulty generator): https://github.com/rohseh303/prime-rl-environments (RESULTS.md)

Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "RSRS64/qwen3-4b-ctf-lite-grpo")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")

Training

GRPO + LoRA, lr 1e-5, group 16, batch 16, ~70 steps, 2×H100. Curriculum = the 3 tasks the base model partially solves (git-leaked-secret, xor-vault, hash-crack) — the only ones with reward variance, hence gradient. Training reward climbed ~0.75 → ~0.82.

Sources

1 source

Hugging FacePrincipalVersion 7671c34721c861cadc96a9ca6bec8702a8420114

huggingface.co/RSRS64/qwen3-4b-ctf-lite-grpo/tree/7671c34721c861cadc96a9ca6bec8702a8420114

Vérifié 20 oct.

Artefacts du modèle

1 artefact

adapter_model.safetensors

safetensors · 252 MB · SHA-256 6d42f8910c8f…1650 · Hugging Face

Download file

Extraits de sources

3 extraits

Readme3 aoûthuggingface.co/RSRS64/qwen3-4b-ctf-lite-grpo

--- license: apache-2.0 base_model: Qwen/Qwen3-4B-Instruct-2507 library_name: peft tags: - lora - grpo - rl - ctf - agentic - verifiers - prime-rl --- # Qwen3-4B · ctf-lite curriculum GRPO (LoRA adapter) A LoRA adapter (rank 32, α 64) for `Qwen/Qwen3-4B-Instruct-2507`, trained with **GRPO** (via [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl)) on a 3-task curriculum from the [`ctf-lite`](https://app.primeintellect.ai/dashboard/environments/rohseh303/ctf-lite) RL environment — small crypto/forensics capture-the-flag tasks with randomized-per-rollout flags and exact-match rewards. ## Result (honest: no significant gain) Held-out eval, all 9 ctf-lite tasks, same `mini_swe_agent` harness, **n=16 rollouts/task**, Fisher's exact: | | base Qwen3-4B | + this adapter | Δ | p | |---|:---:|:---:|:---:|:---:| | overall (9 tasks) | 51/144 (0.35) | 55/144 (0.38) | +0.03 | 0.71 | | trained subset (3 tasks) | 35/48 (0.73) | 39/48 (0.81) | +0.08 | 0.47 | **No result reaches p<0.05 — the difference is within noise.** Training reward rose (~0.75→0.82) but did not translate to a significant single-shot held-out gain. An earlier n=3 eval suggested +0.11; that was a small-sample artifact that vanished at n=16. The adapter is published for reproducibility, not as a capability claim. Full writeup + logs + the lever that would actually move it (a procedural difficulty generator): https://github.com/rohseh303/prime-rl-environments (RESULTS.md) ## Use ```python from peft import PeftModel from transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507", torch_dtype="bfloat16") model = PeftModel.from_pretrained(base, "RSRS64/qwen3-4b-ctf-lite-grpo") tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507") ``` ## Training GRPO + LoRA, lr 1e-5, group 16, batch 16, ~70 steps, 2×H100. Curriculum = the 3 tasks the base model partially solves (`git-leaked-secret`, `xor-vault`, `hash-crack`) — the only ones with reward variance, hence gradient. Training reward climbed ~0.75 → ~0.82.

Note20 oct.huggingface.co/RSRS64/qwen3-4b-ctf-lite-grpo

Source context: 17 downloads · 0 likes · Library peft · Repo RSRS64/qwen3-4b-ctf-lite-grpo

Note3 aoûthuggingface.co/RSRS64/qwen3-4b-ctf-lite-grpo

Source context: 29 downloads · 0 likes · Library peft · Repo RSRS64/qwen3-4b-ctf-lite-grpo