DFF InstructBLIP Detection

--- license: bsd-3-clause library_name: lavis pipeline_tag: visual-question-answering tags: - explainable-ai - deepfake-detection - vlm - instructblip - forensic-explanation - acl-2026 --- # DFF: InstructBLIP-based Explainable DeepFake Detection ## 📖 Model Description This is the core **DFF (DeepFake Detection and Forensic Explanation Framework)** model as described in the ACL 2026 paper: *"Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline"*. DFF is built upon the **InstructBLIP (Flan-T5 XL)** architecture. By integrating the Face-ViT auxiliary classifier, it achieves state-of-the-art performance in both **forgery localization (mask generation)** and **forensic explanation (captioning)**. ## 🌟 Key Capabilities 1. **Forgery Localization**: Generates high-resolution binary masks highlighting manipulated facial regions. 2. **Natural Language Explanation**: Produces detailed text describing why a specific image is considered a forgery (e.g., "The texture around the eyes is unnatural due to GAN-based blending"). ## 🛠️ Model Details - **Base LLM**: Flan-T5 XL. - **Visual Encoder**: EVA-ViT-G. - **Auxiliary Module**: Face-ViT (Multi-label perception). - **Task**: Explainable Detection & Multi-modal Attribution Reporting. ## 🚀 Links - **Official Code**: [Generating-Attribution-Reports](https://github.com/NattyLianJc/Generating-Attribution-Reports) - **Auxiliary Classifier**: [LianJC/Face-ViT-MultiLabel](https://huggingface.co/LianJC/Face-ViT-MultiLabel) - **Dataset (MMTT)**: [LianJC/MMTT-Dataset](https://huggingface.co/datasets/LianJC/MMTT-Dataset) ## 📜 Citation ```bibtex @inproceedings{lian2026generating, title={Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline}, author={Lian, Jingchun and others}, booktitle={Proceedings of ACL}, year={2026}, note={To appear} }

Source context: 0 downloads · 0 likes · Pipeline visual-question-answering · Library lavis · Repo LianJC/DFF-InstructBLIP-Detection