Saltar al contenido
ModelPublicadoExterno
Browse files

PMC Kvasir VQA X1 LoRA 250918 1352

This model combines PMC-CLIP and PMC-LLaMA for medical visual question answering, specifically fine-tuned on the Kvasir-VQA-x1 dataset.

Fuentehuggingface.co/JustATalentedGuy/PMC-Kvasir-VQA-x1-lora_250918-1352/tree/ef82b953dade87d7ed7551e105eab255d93b8496
TipoOtherModelo basechaoyi-wu/PMC_LLAMA_7BVersiónImage projectorCcalidad

Fuente del modelo

Tipo
Other
Tamaño del archivo
32,1–153 MB
Modelo base
chaoyi-wu/PMC_LLAMA_7B
Versión
vImage projector
Actualizado
23 oct 2026
Fuente
Hugging Face

Descripción de la fuente

PMC-VLM: Medical Visual Question Answering

This model combines PMC-CLIP and PMC-LLaMA for medical visual question answering, specifically fine-tuned on the Kvasir-VQA-x1 dataset.

Model Architecture

  • Vision Encoder: PMC-CLIP (frozen)
  • Language Model: PMC-LLaMA-7B with LoRA adapters
  • Image Projector: Linear projection with 4 soft prompt tokens
  • Training: QLoRA (4-bit quantization) fine-tuning

Training Details

  • Dataset: Kvasir-VQA-x1 (Medical VQA)
  • Learning Rate: 0.0002
  • Batch Size: 2 (with 4x accumulation)
  • LoRA Rank: 16
  • LoRA Alpha: 32
  • Epochs: 1

Usage

Load the model and run inference for medical VQA tasks.

Fuentes

1 fuente
Hugging FacePrincipalVersión ef82b953dade87d7ed7551e105eab255d93b8496
huggingface.co/JustATalentedGuy/PMC-Kvasir-VQA-x1-lora_250918-1352/tree/ef82b953dade87d7ed7551e105eab255d93b8496

Verificado 23 oct

Download files

Artefactos del modelo

2 artefactos

image_projector.pt

pt · 32,1 MB · SHA-256 76bd5622904a…c854 · Hugging Face

Download file

peft_adapters/adapter_model.safetensors

safetensors · 153 MB · SHA-256 6e60e955c659…ea1c · Hugging Face

Download file

Extractos de fuentes

2 extractos
Readme6 agohuggingface.co/JustATalentedGuy/PMC-Kvasir-VQA-x1-lora_250918-1352

--- library_name: transformers base_model: chaoyi-wu/PMC_LLAMA_7B license: apache-2.0 tags: - medical - vqa - multimodal - pmc-clip - pmc-llama - kvasir datasets: - SimulaMet/Kvasir-VQA-x1 metrics: - exact_match - f1_score pipeline_tag: visual-question-answering --- # PMC-VLM: Medical Visual Question Answering This model combines **PMC-CLIP** and **PMC-LLaMA** for medical visual question answering, specifically fine-tuned on the Kvasir-VQA-x1 dataset. ## Model Architecture - **Vision Encoder**: PMC-CLIP (frozen) - **Language Model**: PMC-LLaMA-7B with LoRA adapters - **Image Projector**: Linear projection with 4 soft prompt tokens - **Training**: QLoRA (4-bit quantization) fine-tuning ## Training Details - **Dataset**: Kvasir-VQA-x1 (Medical VQA) - **Learning Rate**: 0.0002 - **Batch Size**: 2 (with 4x accumulation) - **LoRA Rank**: 16 - **LoRA Alpha**: 32 - **Epochs**: 1 ## Usage Load the model and run inference for medical VQA tasks.

Note6 agohuggingface.co/JustATalentedGuy/PMC-Kvasir-VQA-x1-lora_250918-1352

Source context: 0 downloads · 0 likes · Pipeline visual-question-answering · Library transformers · Repo JustATalentedGuy/PMC-Kvasir-VQA-x1-lora_250918-1352