Qwen05b Sentiment LoRA Imdb

--- license: apache-2.0 base_model: Qwen/Qwen2.5-0.5B-Instruct tags: - lora - peft - qlora - sentiment-analysis - text-classification - qwen2 datasets: - stanfordnlp/imdb language: - en library_name: peft pipeline_tag: text-generation --- # Qwen2.5-0.5B-Instruct — LoRA Sentiment Classifier (IMDB) This is a **LoRA adapter** for [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct), fine-tuned with **QLoRA** for binary sentiment classification on the [Stanford IMDB](https://huggingface.co/datasets/stanfordnlp/imdb) movie review dataset. This adapter is part of an end-to-end MLOps portfolio project demonstrating the full lifecycle of LLM fine-tuning: data preparation, QLoRA training, experiment tracking with MLflow, evaluation, and deployment. ## Model Details - **Base model:** Qwen2.5-0.5B-Instruct - **Fine-tuning method:** LoRA (rank=8, alpha=16, dropout=0.05) on top of a 4-bit quantized base model (QLoRA) - **Target modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj` - **Trainable parameters:** ~1.08M (0.22% of total model parameters) - **Task:** Binary sentiment classification (positive/negative), framed as an instruction-following text generation task - **Training hardware:** Single NVIDIA T4 GPU (Kaggle Notebooks, free tier) ## Training Data - **Dataset:** Stanford IMDB (`stanfordnlp/imdb`), 25,000 labeled movie reviews in the original train split - **Split:** 90% train / 10% validation, stratified by label, seed=42 - **Test set:** The original 25,000-example IMDB test split was held out entirely and never used during training or hyperparameter selection ## Training Procedure - **Prompt format:** Chat-style conversation (system / user / assistant) using Qwen2.5's native chat template - **Loss masking:** Loss computed only on the assistant response tokens (`assistant_only_loss=True`); system and user tokens are masked - **Sequence length:** Reviews truncated to 150 words before tokenization to guarantee the response is never cut off by the 256-token sequence limit - **Epochs:** 1 - **Effective batch size:** 16 - **Learning rate:** 2e-4, cosine schedule with warmup - **Precision:** FP16 throughout (base model, LoRA weights, and training loop) — chosen for compatibility with the T4 GPU's Turing architecture, which lacks native BF16 tensor core support ## Evaluation Results Evaluated on a stratified sample of 1,00...