Qwen3vl Caption Imregis
This is a Qwen3-VL-4B-Instruct model fine-tuned for Portuguese image captioning of figures extracted from geological documents. It was trained with LoRA on the ImRegis dataset, and the adapter has been merged into...
Modellquelle
- Art
- Other
- Dateigröße
- 8,27 GB
- Basismodell
- Qwen/Qwen3-VL-4B-Instruct
- Version
- vRevision 4a3165a
- Aktualisiert
- 23. Okt. 2026
- Downloads
- 7
- Quelle
- Hugging Face
Quellenbeschreibung
Qwen3-VL-4B — ImRegis Image Captioning
This is a Qwen3-VL-4B-Instruct model fine-tuned for Portuguese image captioning of figures extracted from geological documents. It was trained with LoRA on the ImRegis dataset, and the adapter has been merged into the base weights, so the model is standalone and can be loaded directly without PEFT.
Model description
- Base model: Qwen/Qwen3-VL-4B-Instruct
- Task: image-to-text (captioning) in Portuguese
- Training data: ImRegis — figure/box crops from geological reports paired with Portuguese captions
- Method: LoRA fine-tuning, then merged into the base model
- Languages: Portuguese (pt)
The model takes a single image and produces a descriptive caption in Portuguese. It is intended for describing figures, maps, cross-sections, and diagrams found in geoscience documents.
Intended uses & limitations
Generating Portuguese captions for figures extracted from geological documents, in line with the kind of content present in ImRegis.
Limitations.
- The model was trained on a domain-specific corpus (geological document figures); performance on out-of-domain images may be lower.
- Captions may contain factual errors or hallucinations and should be reviewed before being used in any downstream product.
- Training images were resized so the longest side is at most 448 px.
How to use
import torch
from PIL import Image
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
REPO_ID = "Geologi/qwen3vl-caption-imregis"
model = Qwen3VLForConditionalGeneration.from_pretrained(
REPO_ID,
torch_dtype=torch.bfloat16,
device_map="auto",
)
processor = AutoProcessor.from_pretrained(REPO_ID)
image = Image.open("example.png").convert("RGB")
messages = [{
"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": "Descreva esta imagem em português"},
],
}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=128)
print(processor.batch_decode(output, skip_special_tokens=True)[0])
Because the LoRA adapter is already merged, no peft import is required.
Quellen
1 Quelle
Quellen
1 QuelleVerifiziert 23. Okt.
Modellartefakte
1 Artefakt
Modellartefakte
1 ArtefaktQuellenauszüge
3 Auszüge
Quellenauszüge
3 Auszüge--- base_model: Qwen/Qwen3-VL-4B-Instruct tags: - image-captioning - image-to-text - vision-language - qwen3-vl - lora - portuguese - geoscience language: - pt pipeline_tag: image-to-text datasets: - Geologi/imregis library_name: transformers --- # Qwen3-VL-4B — ImRegis Image Captioning This is a [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct) model fine-tuned for Portuguese image captioning of figures extracted from geological documents. It was trained with LoRA on the [ImRegis](https://huggingface.co/datasets/Geologi/imregis) dataset, and the adapter has been merged into the base weights, so the model is standalone and can be loaded directly without PEFT. ## Model description - Base model: Qwen/Qwen3-VL-4B-Instruct - Task: image-to-text (captioning) in Portuguese - Training data: ImRegis — figure/box crops from geological reports paired with Portuguese captions - Method: LoRA fine-tuning, then merged into the base model - Languages: Portuguese (pt) The model takes a single image and produces a descriptive caption in Portuguese. It is intended for describing figures, maps, cross-sections, and diagrams found in geoscience documents. ## Intended uses & limitations Generating Portuguese captions for figures extracted from geological documents, in line with the kind of content present in ImRegis. Limitations. - The model was trained on a domain-specific corpus (geological document figures); performance on out-of-domain images may be lower. - Captions may contain factual errors or hallucinations and should be reviewed before being used in any downstream product. - Training images were resized so the longest side is at most 448 px. ## How to use ```python import torch from PIL import Image from transformers import Qwen3VLForConditionalGeneration, AutoProcessor REPO_ID = "Geologi/qwen3vl-caption-imregis" model = Qwen3VLForConditionalGeneration.from_pretrained( REPO_ID, torch_dtype=torch.bfloat16, device_map="auto", ) processor = AutoProcessor.from_pretrained(REPO_ID) image = Image.open("example.png").convert("RGB") messages = [{ "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": "Descreva esta imagem em português"}, ], }] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = processor(text=[text], images=[image], return_tensors="pt")....
Source context: 7 downloads · 0 likes · Pipeline image-to-text · Library transformers · Repo Geologi/qwen3vl-caption-imregis
Source context: 10 downloads · 0 likes · Pipeline image-to-text · Library transformers · Repo Geologi/qwen3vl-caption-imregis
