Saltar al contenido
ModelPublicadoExterno
Descargar

Diffvqa Qwen3 0.6b Multiobjective Regsteps

Online-trained Difference Visual Question Answering head for chest X-rays (MIMIC-CXR / Medical-Diff-VQA). A frozen ViT-L/14 vision encoder produces patch tokens for a current + reference image pair; a Qwen3-0.6B...

Fuentehuggingface.co/dmusingu/diffvqa-qwen3-0.6b-multiobjective-regsteps/tree/7fbf75386fc20eed1268317895dd87e002067563
TipoOtherVersiónPretrain multiobjective regsteps bestCcalidad

Fuente del modelo

Tipo
Other
Tamaño del archivo
2,25 GB
Versión
vPretrain multiobjective regsteps best
Actualizado
10 oct 2026
Fuente
Hugging Face

Descripción de la fuente

Diff-VQA — Qwen3-0.6B visual-prefix decoder (multiobjective-regsteps encoder)

Online-trained Difference Visual Question Answering head for chest X-rays (MIMIC-CXR / Medical-Diff-VQA). A frozen ViT-L/14 vision encoder produces patch tokens for a current + reference image pair; a Qwen3-0.6B decoder, conditioned on those tokens as a visual prefix, generates the answer describing what changed.

This is a slimmed inference checkpoint: decoder weights + the frozen vision encoder weights + architecture metadata (optimizer/scheduler state stripped).

Contents (*_best.pt, torch.load(..., weights_only=False))

keydescription
state_dictDiffVQAQwen3Head weights (Qwen3-0.6B LLM + vis_proj + frame_emb)
encoder_statefrozen ViT-L/14 vision encoder (objective: multiobjective_caption+regioncontrastive)
decoderQwen/Qwen3-0.6B
tokenizer_nameQwen/Qwen3-0.6B
vis_dim1024
encoder_namepretrain-multiobjective-regsteps
epoch / step / best_valtraining position + best validation loss

Training

  • Data: Medical-Diff-VQA difference questions over MIMIC-CXR image pairs (~100k train / ~16k val / ~16k test).
  • Objective: next-token cross-entropy over the answer span (question tokens masked).
  • Vision encoder is frozen; only the decoder (LLM + visual adapter) is trained.

Notes

The bundled encoder_state is the exact frozen encoder this decoder was trained against — load both together so the visual features match the space vis_proj expects (a mismatched encoder produces degenerate output).

Fuentes

1 fuente
Hugging FacePrincipalVersión 7fbf75386fc20eed1268317895dd87e002067563
huggingface.co/dmusingu/diffvqa-qwen3-0.6b-multiobjective-regsteps/tree/7fbf75386fc20eed1268317895dd87e002067563

Verificado 10 oct

Artefactos del modelo

1 artefacto

pretrain-multiobjective-regsteps_best.pt

pt · 2,25 GB · SHA-256 a4a6d61c93d3…6f29 · Hugging Face

Download file

Extractos de fuentes

2 extractos
Readme2 agohuggingface.co/dmusingu/diffvqa-qwen3-0.6b-multiobjective-regsteps

--- license: mit tags: - medical - chest-xray - visual-question-answering - difference-vqa - mimic-cxr library_name: pytorch --- # Diff-VQA — Qwen3-0.6B visual-prefix decoder (multiobjective-regsteps encoder) Online-trained **Difference Visual Question Answering** head for chest X-rays (MIMIC-CXR / Medical-Diff-VQA). A frozen ViT-L/14 vision encoder produces patch tokens for a current + reference image pair; a **Qwen3-0.6B** decoder, conditioned on those tokens as a visual prefix, generates the answer describing what changed. This is a **slimmed inference checkpoint**: decoder weights + the frozen vision encoder weights + architecture metadata (optimizer/scheduler state stripped). ## Contents (`*_best.pt`, `torch.load(..., weights_only=False)`) | key | description | |-----|-------------| | `state_dict` | `DiffVQAQwen3Head` weights (Qwen3-0.6B LLM + `vis_proj` + `frame_emb`) | | `encoder_state` | frozen ViT-L/14 vision encoder (`objective: multiobjective_caption+regioncontrastive`) | | `decoder` | `Qwen/Qwen3-0.6B` | | `tokenizer_name` | `Qwen/Qwen3-0.6B` | | `vis_dim` | 1024 | | `encoder_name` | `pretrain-multiobjective-regsteps` | | `epoch` / `step` / `best_val` | training position + best validation loss | ## Training - Data: Medical-Diff-VQA `difference` questions over MIMIC-CXR image pairs (~100k train / ~16k val / ~16k test). - Objective: next-token cross-entropy over the answer span (question tokens masked). - Vision encoder is **frozen**; only the decoder (LLM + visual adapter) is trained. ## Notes The bundled `encoder_state` is the exact frozen encoder this decoder was trained against — load both together so the visual features match the space `vis_proj` expects (a mismatched encoder produces degenerate output).

Note2 agohuggingface.co/dmusingu/diffvqa-qwen3-0.6b-multiobjective-regsteps

Source context: 0 downloads · 0 likes · Pipeline visual-question-answering · Library pytorch · Repo dmusingu/diffvqa-qwen3-0.6b-multiobjective-regsteps