Qwen3vl Caption Imregis
This is a Qwen3-VL-4B-Instruct model fine-tuned for Portuguese image captioning of figures extracted from geological documents. It was trained with LoRA on the ImRegis dataset, and the adapter has been merged into...
Fuente del modelo
- Tipo
- Other
- Tamaño del archivo
- 8,27 GB
- Modelo base
- Qwen/Qwen3-VL-4B-Instruct
- Versión
- vRevision 4a3165a
- Actualizado
- 23 oct 2026
- Descargas
- 7
- Fuente
- Hugging Face
Descripción de la fuente
Qwen3-VL-4B — ImRegis Image Captioning
This is a Qwen3-VL-4B-Instruct model fine-tuned for Portuguese image captioning of figures extracted from geological documents. It was trained with LoRA on the ImRegis dataset, and the adapter has been merged into the base weights, so the model is standalone and can be loaded directly without PEFT.
Model description
- Base model: Qwen/Qwen3-VL-4B-Instruct
- Task: image-to-text (captioning) in Portuguese
- Training data: ImRegis — figure/box crops from geological reports paired with Portuguese captions
- Method: LoRA fine-tuning, then merged into the base model
- Languages: Portuguese (pt)
The model takes a single image and produces a descriptive caption in Portuguese. It is intended for describing figures, maps, cross-sections, and diagrams found in geoscience documents.
Intended uses & limitations
Generating Portuguese captions for figures extracted from geological documents, in line with the kind of content present in ImRegis.
Limitations.
- The model was trained on a domain-specific corpus (geological document figures); performance on out-of-domain images may be lower.
- Captions may contain factual errors or hallucinations and should be reviewed before being used in any downstream product.
- Training images were resized so the longest side is at most 448 px.
How to use
import torch
from PIL import Image
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
REPO_ID = "Geologi/qwen3vl-caption-imregis"
model = Qwen3VLForConditionalGeneration.from_pretrained(
REPO_ID,
torch_dtype=torch.bfloat16,
device_map="auto",
)
processor = AutoProcessor.from_pretrained(REPO_ID)
image = Image.open("example.png").convert("RGB")
messages = [{
"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": "Descreva esta imagem em português"},
],
}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=128)
print(processor.batch_decode(output, skip_special_tokens=True)[0])
Because the LoRA adapter is already merged, no peft import is required.
Fuentes
1 fuente
Fuentes
1 fuenteVerificado 23 oct
Artefactos del modelo
1 artefacto
Artefactos del modelo
1 artefactoExtractos de fuentes
3 extractos
Extractos de fuentes
3 extractos--- base_model: Qwen/Qwen3-VL-4B-Instruct tags: - image-captioning - image-to-text - vision-language - qwen3-vl - lora - portuguese - geoscience language: - pt pipeline_tag: image-to-text datasets: - Geologi/imregis library_name: transformers --- # Qwen3-VL-4B — ImRegis Image Captioning This is a [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct) model fine-tuned for Portuguese image captioning of figures extracted from geological documents. It was trained with LoRA on the [ImRegis](https://huggingface.co/datasets/Geologi/imregis) dataset, and the adapter has been merged into the base weights, so the model is standalone and can be loaded directly without PEFT. ## Model description - Base model: Qwen/Qwen3-VL-4B-Instruct - Task: image-to-text (captioning) in Portuguese - Training data: ImRegis — figure/box crops from geological reports paired with Portuguese captions - Method: LoRA fine-tuning, then merged into the base model - Languages: Portuguese (pt) The model takes a single image and produces a descriptive caption in Portuguese. It is intended for describing figures, maps, cross-sections, and diagrams found in geoscience documents. ## Intended uses & limitations Generating Portuguese captions for figures extracted from geological documents, in line with the kind of content present in ImRegis. Limitations. - The model was trained on a domain-specific corpus (geological document figures); performance on out-of-domain images may be lower. - Captions may contain factual errors or hallucinations and should be reviewed before being used in any downstream product. - Training images were resized so the longest side is at most 448 px. ## How to use ```python import torch from PIL import Image from transformers import Qwen3VLForConditionalGeneration, AutoProcessor REPO_ID = "Geologi/qwen3vl-caption-imregis" model = Qwen3VLForConditionalGeneration.from_pretrained( REPO_ID, torch_dtype=torch.bfloat16, device_map="auto", ) processor = AutoProcessor.from_pretrained(REPO_ID) image = Image.open("example.png").convert("RGB") messages = [{ "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": "Descreva esta imagem em português"}, ], }] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = processor(text=[text], images=[image], return_tensors="pt")....
Source context: 7 downloads · 0 likes · Pipeline image-to-text · Library transformers · Repo Geologi/qwen3vl-caption-imregis
Source context: 10 downloads · 0 likes · Pipeline image-to-text · Library transformers · Repo Geologi/qwen3vl-caption-imregis
