Loading
Have a ComfyUI workflow? Get a readable Workflow Doc in your browser.Make a workflow readable.Try it now ↗
This repository contains the model for the paper:
Model source
Source description
This repository contains the model for the paper:
Sources
1 sourceVerified Sep 28
Model artifacts
1 artifactSource excerpts
2 excerptsThis paper tackles the challenge of detecting partially manipulated facial deepfakes, which involve subtle alterations to specific facial features while retaining the overall context, posing a greater detection difficulty than fully synthetic faces. We leverage the Contrastive Language-Image Pre-training (CLIP) model, specifically its ViT-L/14 visual encoder, to develop a generalizable detection method that performs robustly across diverse datasets and unknown forgery techniques with minimal modifications to the original model. The proposed approach utilizes parameter-efficient fine-tuning (PEFT) techniques, such as LN-tuning, to adjust a small subset of the model's parameters, preserving CLIP's pre-trained knowledge and reducing overfitting. A tailored preprocessing pipeline optimizes the method for facial images, while regularization strategies, including L2 normalization and metric learning on a hyperspherical manifold, enhance generalization. Trained on the FaceForensics++ dataset and evaluated in a cross-dataset fashion on Celeb-DF-v2, DFDC, FFIW, and others, the proposed method achieves competitive detection accuracy comparable to or outperforming much more complex state-of-the-art techniques. This work highlights the efficacy of CLIP's visual encoder in facial deepfake detection and establishes a simple, powerful baseline for future research, advancing the field of generalizable deepfake detection.
Generalization of models trained on the FF++ dataset to unseen datasets and forgery methods. Reported values are video-level AUROC. Results of other methods are taken from their original papers. Values with * are taken from the other papers.
| Model | Year | Publication | CDFv2 | DFD | DFDC | FFIW | DSv1 |
|---|---|---|---|---|---|---|---|
| LipForensics | 2021 | CVPR | 82.4 | -- | 73.5 | -- | -- |
| FTCN | 2021 | ICCV | 86.9 | -- | 74.0 | 74.47* | -- |
| RealForensics | 2022 | CVPR | 86.9 | -- | 75.9 | -- | -- |
| SBI | 2022 | CVPR | 93.18 | 82.68 | 72.42 | 84.83 |
See usage examples in our github project
@article{yermakov-2025-deepfake-detection,
title={Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection},
author={Andrii Yermakov and Jan Cech and Jiri Matas},
year={2025},
eprint={2503.19683},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2503.19683},
}
--- license: mit tags: - pytorch - endpoints-template - deepfake-detection pipeline_tag: image-classification widget: - src: "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/infrence-examples-text2img-dark.png" example_title: "Deepfake Image Example" --- # Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection [](https://arxiv.org/abs/2503.19683) [](https://github.com/yermandy/deepfake-detection) This repository contains the model for the paper: **[Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection](https://arxiv.org/abs/2503.19683)** ## Abstract > This paper tackles the challenge of detecting partially manipulated facial deepfakes, which involve subtle alterations to specific facial features while retaining the overall context, posing a greater detection difficulty than fully synthetic faces. We leverage the Contrastive Language-Image Pre-training (CLIP) model, specifically its ViT-L/14 visual encoder, to develop a generalizable detection method that performs robustly across diverse datasets and unknown forgery techniques with minimal modifications to the original model. The proposed approach utilizes parameter-efficient fine-tuning (PEFT) techniques, such as LN-tuning, to adjust a small subset of the model's parameters, preserving CLIP's pre-trained knowledge and reducing overfitting. A tailored preprocessing pipeline optimizes the method for facial images, while regularization strategies, including L2 normalization and metric learning on a hyperspherical manifold, enhance generalization. Trained on the FaceForensics++ dataset and evaluated in a cross-dataset fashion on Celeb-DF-v2, DFDC, FFIW, and others, the proposed method achieves competitive detection accuracy comparable to or outperforming much more complex state-of-the-art techniques. This work highlights the efficacy of CLIP's visual encoder in facial deepfake detection and establishes a simple, powerful baseline for future research, advancing the field of generalizable deepfake detection. ## Results Generalization of models trained on the FF++ dataset to unseen datasets and forgery methods. Reported values are **video-level AUROC**. Results of...
Source context: 0 downloads · 0 likes · Pipeline image-classification · Repo morhadi/deepfake-detection
| -- |
| AUNet | 2023 | CVPR | 92.77 | 99.22 | 73.82 | 81.45 | -- |
| StyleDFD | 2024 | CVPR | 89.0 | 96.1 | -- | -- | -- |
| LSDA | 2024 | CVPR | 91.1 | -- | 77.0 | 72.4* | -- |
| LAA-Net | 2024 | CVPR | 95.4 | 98.43 | 86.94 | -- | -- |
| AltFreezing | 2024 | CVPR | 89.5 | 98.5 | 99.4 | -- | -- |
| NACO | 2024 | ECCV | 89.5 | -- | 76.7 | -- | -- |
| TALL++ | 2024 | IJCV | 91.96 | -- | 78.51 | -- | -- |
| UDD | 2025 | arXiv | 93.13 | 95.51 | 81.21 | -- | -- |
| Effort | 2025 | arXiv | 95.6 | 96.5 | 84.3 | 92.1 | -- |
| KID | 2025 | arXiv | 95.74 | 99.46 | 75.77 | 82.53 | -- |
| ForensicsAdapter | 2025 | arXiv | 95.7 | 97.2 | 87.2 | -- | -- |
| Proposed | 2025 | arXiv | 96.62 | 98.0 | 87.15 | 91.52 | 92.01 |