Videomt Dinov2 Large Ytvis2021

Reported metrics

The metrics above are the numbers reported by the authors in the official model zoo.

Usage

from transformers import AutoModelForUniversalSegmentation, AutoVideoProcessor

model_id = "tue-mps/videomt-dinov2-large-ytvis2021"
processor = AutoVideoProcessor.from_pretrained(model_id)
model = AutoModelForUniversalSegmentation.from_pretrained(model_id)

Use processor.post_process_instance_segmentation, processor.post_process_panoptic_segmentation, or processor.post_process_semantic_segmentation depending on the target task.

--- library_name: transformers pipeline_tag: image-segmentation tags: - transformers - videomt - video-segmentation - video-instance-segmentation - dinov2 --- # VidEoMT-L on YouTube-VIS 2021 This repository contains the Hugging Face Transformers conversion of the official VidEoMT checkpoint `yt_2021_vit_large_63.1.pth` from [tue-mps/VidEoMT](https://github.com/tue-mps/videomt/blob/master/model_zoo/dinov2.md). ## Model details - Architecture: VidEoMT with a DINOv2 ViT-L/14 with 4 register tokens backbone - Task: video instance segmentation - Dataset: YouTube-VIS 2021 - Input resolution: 640 x 640 - Number of frames: 2 - Paper: [Your ViT is Secretly Also a Video Segmentation Model](https://huggingface.co/papers/2602.17807) ## Reported metrics | Metric | Value | | --- | --- | | AP | 63.1 | | AR@10 | 68.1 | | FPS | 160 | The metrics above are the numbers reported by the authors in the [official model zoo](https://github.com/tue-mps/videomt/blob/master/model_zoo/dinov2.md). ## Usage ```python from transformers import AutoModelForUniversalSegmentation, AutoVideoProcessor model_id = "tue-mps/videomt-dinov2-large-ytvis2021" processor = AutoVideoProcessor.from_pretrained(model_id) model = AutoModelForUniversalSegmentation.from_pretrained(model_id) ``` Use `processor.post_process_instance_segmentation`, `processor.post_process_panoptic_segmentation`, or `processor.post_process_semantic_segmentation` depending on the target task.