Saltar al contenido
ModelPublicadoExterno
Descargar

Marlin Vit Base Ytf

This repo is the official PyTorch implementation for the paper MARLIN: Masked Autoencoder for facial video Representation LearnINg (CVPR 2023) (arXiv).

Fuentehuggingface.co/ControlNet/marlin_vit_base_ytf/tree/41551458719a9b8a0da70fe4deb163c12f3180ec
TipoOtherModelo baseControlNet/MARLINVersiónRevision 4155145Bcalidad

Fuente del modelo

Tipo
Other
Tamaño del archivo
334 MB
Modelo base
ControlNet/MARLIN
Versión
vRevision 4155145
Actualizado
6 oct 2026
Descargas
105
Fuente
Hugging Face

Descripción de la fuente

MARLIN: Masked Autoencoder for facial video Representation LearnINg

This repo is the official PyTorch implementation for the paper MARLIN: Masked Autoencoder for facial video Representation LearnINg (CVPR 2023) (arXiv).

Use transformers (HuggingFace) for Feature Extraction

Requirements:

  • Python
  • PyTorch
  • transformers
  • einops

Currently the huggingface model is only for direct feature extraction without any video pre-processing (e.g. face detection, cropping, strided window, etc).

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "ControlNet/marlin_vit_base_ytf",  # or other variants
    trust_remote_code=True
)
tensor = torch.rand([1, 3, 16, 224, 224])  # (B, C, T, H, W)
output = model(tensor)  # torch.Size([1, 1568, 384])

License

This project is under the CC BY-NC 4.0 license. See LICENSE for details.

References

If you find this work useful for your research, please consider citing it.

@inproceedings{cai2022marlin,
  title = {MARLIN: Masked Autoencoder for facial video Representation LearnINg},
  author = {Cai, Zhixi and Ghosh, Shreya and Stefanov, Kalin and Dhall, Abhinav and Cai, Jianfei and Rezatofighi, Hamid and Haffari, Reza and Hayat, Munawar},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2023},
  month = {June},
  pages = {1493-1504},
  doi = {10.1109/CVPR52729.2023.00150},
  publisher = {IEEE},
}

Fuentes

1 fuente
Hugging FacePrincipalVersión 41551458719a9b8a0da70fe4deb163c12f3180ec
huggingface.co/ControlNet/marlin_vit_base_ytf/tree/41551458719a9b8a0da70fe4deb163c12f3180ec

Verificado 6 oct

Artefactos del modelo

1 artefacto

model.safetensors

safetensors · 334 MB · SHA-256 a0bf06598cd8…b6e9 · Hugging Face

Download file

Extractos de fuentes

3 extractos
Readme29 julhuggingface.co/ControlNet/marlin_vit_base_ytf

--- library_name: transformers tags: - video - feature - face license: cc base_model: - ControlNet/MARLIN pipeline_tag: feature-extraction --- # MARLIN: Masked Autoencoder for facial video Representation LearnINg This repo is the official PyTorch implementation for the paper [MARLIN: Masked Autoencoder for facial video Representation LearnINg](https://openaccess.thecvf.com/content/CVPR2023/html/Cai_MARLIN_Masked_Autoencoder_for_Facial_Video_Representation_LearnINg_CVPR_2023_paper) (CVPR 2023) ([arXiv](https://arxiv.org/abs/2211.06627)). ## Use `transformers` (HuggingFace) for Feature Extraction Requirements: - Python - PyTorch - transformers - einops Currently the huggingface model is only for direct feature extraction without any video pre-processing (e.g. face detection, cropping, strided window, etc). ```python import torch from transformers import AutoModel model = AutoModel.from_pretrained( "ControlNet/marlin_vit_base_ytf", # or other variants trust_remote_code=True ) tensor = torch.rand([1, 3, 16, 224, 224]) # (B, C, T, H, W) output = model(tensor) # torch.Size([1, 1568, 384]) ``` ## License This project is under the CC BY-NC 4.0 license. See [LICENSE](LICENSE) for details. ## References If you find this work useful for your research, please consider citing it. ```bibtex @inproceedings{cai2022marlin, title = {MARLIN: Masked Autoencoder for facial video Representation LearnINg}, author = {Cai, Zhixi and Ghosh, Shreya and Stefanov, Kalin and Dhall, Abhinav and Cai, Jianfei and Rezatofighi, Hamid and Haffari, Reza and Hayat, Munawar}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, year = {2023}, month = {June}, pages = {1493-1504}, doi = {10.1109/CVPR52729.2023.00150}, publisher = {IEEE}, } ```

Note6 octhuggingface.co/ControlNet/marlin_vit_base_ytf

Source context: 105 downloads · 1 likes · Pipeline feature-extraction · Library transformers · Repo ControlNet/marlin_vit_base_ytf

Note29 julhuggingface.co/ControlNet/marlin_vit_base_ytf

Source context: 26 downloads · 1 likes · Pipeline feature-extraction · Library transformers · Repo ControlNet/marlin_vit_base_ytf