Patch Ioner Talk2dino Capdec Groupnet COCO Captions
This repository contains a pre-trained DECAP model from the Patch-ioner framework for dense image captioning and controllable visual description.
Fuente del modelo
- Tipo
- Modelo
- Tamaño del archivo
- 27,1–261 MB
- Versión
- vRevision 769b5c3
- Actualizado
- 30 oct 2026
- Descargas
- 23
- Fuente
- Hugging Face
Descripción de la fuente
Patch-ioner_talk2dino_capdec_groupnet_COCO_Captions - Patch-ioner Configuration
This repository contains a pre-trained DECAP model from the Patch-ioner framework for dense image captioning and controllable visual description.
📝 Paper Information
Title: "One Patch to Caption Them All: A Unified Zero-Shot Captioning Framework"
Authors: Lorenzo Bianchi, Giacomo Pacini, Fabio Carrara, Nicola Messina, Giuseppe Amato, Fabrizio Falchi
ArXiv: https://arxiv.org/abs/2510.02898
Project Page: https://paciosoft.com/Patch-ioner/
🎯 Model Overview
- Model Type: DECAP
- Configuration: mlp_noise.k.groupnet.yaml
- Vision Backbone: dinov2_vitb14_reg
- Language Model: GPT-2
- Input Resolution: 518x518
- Prefix Size: 768
DeCap Configuration
- Memory Bank: Not used
- Projection Type: coco
- Linear Talk2DINO: False
📊 Performance
| Task | METEOR | CIDEr | SPICE |
|---|---|---|---|
| Image Captioning | nan | N/A | nan |
| Narratives | 10.100 | 38.307 | 66.297 |
📈 Detailed Results
Image Captioning Results
- METEOR: nan
- SPICE: nan
- BLEU_4: nan
- ROUGE_L: nan
Narratives Results
- METEOR: 10.1000
- CIDEr: 38.3069
- SPICE: 66.2970
- BLEU_4: 8.9000
- ROUGE_L: 8.6000
- CLIP-S: 7.9000
🚀 Quick Start
from patch_ioner import load_model, Patchioner
# Load the model
config_path = "config.yaml"
model = load_model(config_path)
# Run inference
image_path = "your_image.jpg"
results = model.forward(image_path)
print(results)
📁 Repository Contents
config.yaml: Model configuration filecoco_karpathy-009.pt: Pre-trained model weightsREADME.md: This file
🔧 Installation
pip install git+https://github.com/Ruggero1912/Patch-ioner
💡 Usage Examples
Refer to the Patch-ioner repository for updated usage examples.
🎛️ Model Configuration
- Prefix Size: 768
- Memory Bank Size: 0
- Normalization: True
📈 Training Details
- Training Dataset: COCO Captions
- Training Epochs: TBD
- Batch Size: TBD
- Learning Rate: TBD
- Optimizer: AdamW
📚 Citation
If you use this model in your research, please cite our paper, refer to the Project Page for updated citation template.
🤝 Contributing
We welcome contributions to improve the Patch-ioner framework. Please see the main repository for contribution guidelines.
📄 License
See the main repository for detailed license information.
🐛 Issues and Support
For issues related to this model or the Patch-ioner framework, please:
- Check the main repository for existing issues
- Open a new issue with detailed information about your problem
- Contact the authors.
🔗 Related Models
Explore other Patch-ioner model configurations:
- Patch-ioner_mlp - MLP-based DeCap model
- Patch-ioner_viecap - VieCap controllable captioning
- Patch-ioner_clipcap - ClipCap integration
More models available in Ruggero1912's models
This model is part of the Patch-ioner framework for dense image captioning and controllable visual description.
Fuentes
1 fuente
Fuentes
1 fuenteVerificado 30 oct
Download files
Artefactos del modelo
2 artefactos
Artefactos del modelo
2 artefactosExtractos de fuentes
3 extractos
Extractos de fuentes
3 extractos--- # YAML Metadata Block (Essential for Hugging Face Hub features) license: apache-2.0 tags: - vision - image-to-text - image-captioning - zero-shot - dense-captioning - patch-ioner model-index: - name: Patch-ioner_talk2dino_capdec_groupnet_COCO_Captions results: - task: name: image-captioning type: image-to-text dataset: name: COCO Captions type: coco_2014_karpathy metrics: - name: METEOR type: meteor value: nan - name: SPICE type: spice value: nan - name: BLEU-4 type: bleu value: nan - name: ROUGE-L type: rouge value: nan - task: name: narratives type: image-to-text dataset: name: Visual Storytelling Dataset (VIST) type: vist metrics: - name: METEOR type: meteor value: 10.1000 - name: CIDEr type: cider value: 38.3069 - name: SPICE type: spice value: 66.2970 - name: BLEU-4 type: bleu value: 8.9000 - name: ROUGE-L type: rouge value: 8.6000 - name: CLIP-S type: clip_s value: 7.9000 --- # Patch-ioner_talk2dino_capdec_groupnet_COCO_Captions - Patch-ioner Configuration This repository contains a pre-trained DECAP model from the **Patch-ioner** framework for dense image captioning and controllable visual description. ## 📝 Paper Information **Title**: "One Patch to Caption Them All: A Unified Zero-Shot Captioning Framework" **Authors**: Lorenzo Bianchi, Giacomo Pacini, Fabio Carrara, Nicola Messina, Giuseppe Amato, Fabrizio Falchi **ArXiv**: https://arxiv.org/abs/2510.02898 **Project Page**: https://paciosoft.com/Patch-ioner/ ## 🎯 Model Overview - **Model Type**: DECAP - **Configuration**: mlp_noise.k.groupnet.yaml - **Vision Backbone**: dinov2_vitb14_reg - **Language Model**: GPT-2 - **Input Resolution**: 518x518 - **Prefix Size**: 768 ### DeCap Configuration - **Memory Bank**: Not used - **Projection Type**: coco - **Linear Talk2DINO**: False ## 📊 Performance | Task | METEOR | CIDEr | SPICE | |------|--------|-------|-------| | Image Captioning | nan | N/A | nan | | Narratives | 10.100 | 38.307 | 66.297 | ## 📈 Detailed Results ### Image Captioning Results - **METEOR**: nan - **SPICE**: nan - **BLEU_4**: nan - **ROUGE_L**: nan ### Narratives Results - **METEOR**: 10.1000 - **CIDEr**: 38.3069 - **SPICE**: 66.2970 - **BLEU_4**: 8.9000 - **ROUGE_L**: 8.6000 - **CLIP-S**: 7.9000 ## 🚀 Quick Start ```python from patch_ioner import load_model, Patchioner # Load the model config_path = "config.yaml"...
Source context: 23 downloads · 0 likes · Pipeline image-to-text · Repo Ruggero1912/Patch-ioner_talk2dino_capdec_groupnet_COCO_Captions
Source context: 13 downloads · 0 likes · Pipeline image-to-text · Repo Ruggero1912/Patch-ioner_talk2dino_capdec_groupnet_COCO_Captions
