Vian Ai Shop Small

--- base_model: google/gemma-3-4b-it library_name: peft pipeline_tag: text-generation tags: - base_model:adapter:google/gemma-3-4b-it - lora - transformers - marketing - support - ai - shop - sale - 4b - gemma license: mit language: - en - vi --- # Model Card for Vian ai Fine-Tuned ## Model Details ### Model Description - **Developed by:** Glavinnguyen - **Model type:** Text Generation / Multimodal (Text/Vision/Audio Interface) - **Language(s) (NLP):** Vietnamese (Primary), English (Primary) - **License:** mit - **Finetuned from model:** google/gemma-3-4b-it ### Model Sources - **Repository:** [More Information Needed] ## Uses ### Direct Use This model is specifically fine-tuned on a small-scale dataset to standardize response formatting, style, and structure. It natively retains the advanced deep reasoning mechanisms (` `) and inherent logical problem-solving capabilities of the base Gemma 3 architecture. ### Out-of-Scope Use The model should not be used for tasks requiring extensive broad-domain knowledge expansion outside the scope of the training dataset without human supervision. The fine-tuning process was focused on structuring behavioral output rather than massive knowledge injection. ## Training Details ### Training Data The model was trained on a highly curated, high-quality alignment dataset. ### Training Procedure Training was conducted utilizing the **QLoRA (Quantized Low-Rank Adaptation)** method to minimize hardware resource consumption while aggressively preserving the base model's pre-trained weights. #### Training Hyperparameters The hyperparameters were carefully optimized for an ultra-small dataset to prevent catastrophic overfitting and achieve an ideal convergence point: - **Training regime:** QLoRA (FP16/BF16 Mixed Precision) - **Learning Rate:** 2e-4 to 1e-5 (Low Learning Rate) - **Per Device Train Batch Size:** 1 - **Gradient Accumulation Steps:** 32 (Global Batch Size = 32) - **Number of Train Epochs:** 1 to 2 - **Max Length:** 256 - 2048 tokens (Allocated to safeguard the generation of ` ` tokens) - **Optimizer:** AdamW - **LR Scheduler Type:** Cosine / Constant - **LoRA Rank (r):** 4 or 8 - **LoRA Alpha ($\alpha$):** 8 or 16 #### Speeds, Sizes, Times - **Final Training Loss:** `0.69` (The sweet spot convergence for low-sample fine-tuning—balancing structural alignment with base intelligence retention). - **VRAM Consumption:** ~2.2 GB (When running on a 4-bit Q4_0 execution profile). ## Technical Specifications ### Model Architecture and Objective Built upon Google's next-generation **Gemma 3** architecture, featuring integrated **Quantization Aware Training (QAT)** and an intrinsic multi-token prediction (MTP) engine. The model leverages an internal step-by-step reasoning loop before routing structural text outputs via designated generation tags. ## Model Card Contact - **Contact:** Glavinnguyen ### Framework versions - PEFT 0.19.1