GMM Sefai Model

--- base_model: unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit library_name: peft pipeline_tag: image-to-text license: apache-2.0 language: - lt datasets: - LuJa111/GMM-Sefai-Dataset tags: - image-to-text - image-captioning - vision-language - lithuanian - lora - unsloth - qwen3-vl --- # GMM-Sefai Team Project for University LoRA adapter that fine-tunes **Qwen3-VL-8B-Instruct** (4-bit) to caption images in Lithuanian. Built to show captioning changes before vs. after fine-tuning. - **Base model:** `unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit` - **Dataset:** [`LuJa111/GMM-Sefai-Dataset`](https://huggingface.co/datasets/LuJa111/GMM-Sefai-Dataset)). - **Method:** QLoRA via [Unsloth](https://github.com/unslothai/unsloth) ## Usage ```python from unsloth import FastVisionModel from PIL import Image model, tok = FastVisionModel.from_pretrained("LuJa111/GMM-sefai-model", load_in_4bit=True) FastVisionModel.for_inference(model) INSTRUCTION = "Aprašyk šį vaizdą taisyklinga, sklandžia lietuvių kalba vienu ar dviem sakiniais." img = Image.open("photo.jpg").convert("RGB") msgs = [{"role": "user", "content": [{"type": "text", "text": INSTRUCTION}, {"type": "image"}]}] text = tok.apply_chat_template(msgs, add_generation_prompt=True) inputs = tok(img, text, add_special_tokens=False, return_tensors="pt").to("cuda") out = model.generate(**inputs, max_new_tokens=64, do_sample=False, repetition_penalty=1.3, no_repeat_ngram_size=3) print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()) ``` Careful for greedy decoding GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA GELTONA

Source context: 9 downloads · 0 likes · Pipeline image-to-text · Library peft · Repo LuJa111/GMM-sefai-model

Source context: 5 downloads · 0 likes · Pipeline image-to-text · Library peft · Repo LuJa111/GMM-sefai-model