Rag Sft Qwen2.5 0.5b

--- license: apache-2.0 base_model: Qwen/Qwen2.5-0.5B-Instruct library_name: peft tags: - lora - sft - qlora - peft pipeline_tag: text-generation --- # Qwen2.5-0.5B-Instruct QLoRA SFT (arXiv QA) QLoRA-SFT fine-tune of `Qwen/Qwen2.5-0.5B-Instruct` on 150 (question, answer) pairs generated from a corpus of ML arXiv paper abstracts. LoRA rank 16, alpha 32, targeting the attention projection layers, 4-bit NF4 quantized base model, 3 epochs. Part of a larger RAG + GPT-from-scratch + DPO alignment portfolio project: https://github.com/sidesshmore/rag-document-assistant ## Training - **Base model:** Qwen/Qwen2.5-0.5B-Instruct - **Data:** 150 QA pairs generated via `qwen3-235b-a22b-instruct-2507` from sampled corpus chunks (135 train / 15 eval split) - **Method:** QLoRA (4-bit NF4), LoRA r=16, alpha=32, dropout=0.05, targeting `q_proj, k_proj, v_proj, o_proj` - **Result:** eval loss 2.486 -> 2.134 -> 2.047 over 3 epochs, mean token accuracy 0.56 -> 0.61 ## Downstream use This checkpoint is the starting point for a DPO alignment pass - see [sidmhuggingface/rag-dpo-qwen2.5-0.5b](https://huggingface.co/sidmhuggingface/rag-dpo-qwen2.5-0.5b) for the DPO-aligned version and an honest comparison. With bias-controlled pairwise judging (independent judge model, position-swap reconciliation), DPO did not clearly beat this SFT checkpoint on held-out questions at this data scale (40 preference pairs) - see that model's card for the full analysis, since that's arguably the more interesting result. ## Usage ```python from peft import PeftModel from transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct") model = PeftModel.from_pretrained(base, "sidmhuggingface/rag-sft-qwen2.5-0.5b") tokenizer = AutoTokenizer.from_pretrained("sidmhuggingface/rag-sft-qwen2.5-0.5b") ``` ### Framework versions - PEFT 0.19.1 - TRL 0.24.0 - Transformers 5.12.1 - PyTorch 2.11.0

Source context: 6 downloads · 0 likes · Pipeline text-generation · Library peft · Repo sidmhuggingface/rag-sft-qwen2.5-0.5b

Source context: 17 downloads · 0 likes · Pipeline text-generation · Library peft · Repo sidmhuggingface/rag-sft-qwen2.5-0.5b