Vilt Finetuned 200
This is a fine-tuned version of the Vilt (Vision-and-Language Transformer) model, specifically the dandelin/vilt-b32-mlm checkpoint, for Visual Question Answering (VQA). It has been trained on a subset of the...
Fonte do modelo
- Tipo
- Modelo
- Tamanho do arquivo
- 431 MB
- Modelo base
- dandelin/vilt-b32-mlm
- Versão
- vRevision aff0876
- Atualizado
- 30 de out. de 2026
- Downloads
- 6
- Fonte
- Hugging Face
Descrição da fonte
vilt_finetuned_200
Model description
This is a fine-tuned version of the Vilt (Vision-and-Language Transformer) model, specifically the dandelin/vilt-b32-mlm checkpoint, for Visual Question Answering (VQA). It has been trained on a subset of the Graphcore/vqa dataset, using the validation split with the first 200 samples.
The model takes an image and a question as input and predicts the answer to the question based on the image content. It leverages the power of transformers to combine visual and textual information for effective understanding and reasoning
Intended uses & limitations
Intended uses:
- Visual Question Answering: The primary use case is to answer questions about images. It can be used in applications like image captioning, visual dialogue systems, and knowledge extraction from images.
- Research and development: The model can serve as a baseline for further research and development in VQA and related tasks.
Limitations:
- Dataset size: The model was fine-tuned on a relatively small subset (200 samples) of the VQA dataset, which may limit its generalization ability to unseen images and questions.
- Bias: The model may exhibit biases present in the training data, potentially leading to inaccurate or unfair predictions in certain scenarios.
- Complex reasoning: While Vilt is designed for visual reasoning, it may struggle with questions requiring complex or abstract reasoning abilities.
Training and evaluation data
Training data:
- Dataset: Graphcore/vqa
- Split: Validation
- Subset: First 200 samples The training data consists of image-question-answer triplets, where the model learns to predict the answer based on the image and question.
Evaluation data:
Currently, there is no specific evaluation dataset or metrics reported for this fine-tuned model. It is recommended to evaluate the model's performance on a separate VQA benchmark dataset to assess its generalization ability and compare it to other models.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 20
Framework versions
- Transformers 4.50.0
- Pytorch 2.6.0+cu124
- Datasets 3.5.0
- Tokenizers 0.21.1
Fontes
1 fonte
Fontes
1 fonteVerificado 30 de out.
Download files
Repository files
File inventory only; completeness is unverified.
Artefatos de modelo
1 artefato
Artefatos de modelo
1 artefatomodel.safetensors
safetensors · FP32 (tensor header) · 431 MB · SHA-256 37c539d7e1fa…9bf8 · Hugging Face
Download fileTrechos de fonte
3 trechos
Trechos de fonte
3 trechos--- library_name: transformers license: apache-2.0 base_model: dandelin/vilt-b32-mlm tags: - generated_from_trainer datasets: - vqa model-index: - name: vilt_finetuned_200 results: [] --- <!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. --> # vilt_finetuned_200 ## Model description This is a fine-tuned version of the Vilt (Vision-and-Language Transformer) model, specifically the dandelin/vilt-b32-mlm checkpoint, for Visual Question Answering (VQA). It has been trained on a subset of the Graphcore/vqa dataset, using the validation split with the first 200 samples. The model takes an image and a question as input and predicts the answer to the question based on the image content. It leverages the power of transformers to combine visual and textual information for effective understanding and reasoning ## Intended uses & limitations ### Intended uses: - Visual Question Answering: The primary use case is to answer questions about images. It can be used in applications like image captioning, visual dialogue systems, and knowledge extraction from images. - Research and development: The model can serve as a baseline for further research and development in VQA and related tasks. ### Limitations: - Dataset size: The model was fine-tuned on a relatively small subset (200 samples) of the VQA dataset, which may limit its generalization ability to unseen images and questions. - Bias: The model may exhibit biases present in the training data, potentially leading to inaccurate or unfair predictions in certain scenarios. - Complex reasoning: While Vilt is designed for visual reasoning, it may struggle with questions requiring complex or abstract reasoning abilities. ## Training and evaluation data ### Training data: - Dataset: Graphcore/vqa - Split: Validation - Subset: First 200 samples The training data consists of image-question-answer triplets, where the model learns to predict the answer based on the image and question. ### Evaluation data: Currently, there is no specific evaluation dataset or metrics reported for this fine-tuned model. It is recommended to evaluate the model's performance on a separate VQA benchmark dataset to assess its generalization ability and compare it to other models. ## Training procedure ###...
Source context: 6 downloads · 0 likes · Pipeline visual-question-answering · Library transformers · Repo KFrimps/vilt_finetuned_200
Source context: 8 downloads · 0 likes · Pipeline visual-question-answering · Library transformers · Repo KFrimps/vilt_finetuned_200
