LLaDA2.0 Uni

--- license: apache-2.0 language: - en tags: - multimodal - image-generation - image-understanding - image-editing - diffusion - moe - text-to-image library_name: transformers pipeline_tag: any-to-any base_model: - inclusionAI/LLaDA2.0-mini --- <p align="center"> <img src="./assets/llada_logo.png" width="20%"/> </p> <div align="center"> <h1> LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model </h1> [[📑 Technical Report ](https://arxiv.org/abs/2604.20796)] &emsp; [[🌐 Github ](https://github.com/inclusionAI/LLaDA2.0-Uni)] <b>AGI Research Center, Inclusion AI </b> </div> ## Model Capabilities **LLaDA2.0-Uni** is a unified diffusion Large Language Model (dLLM) based on Mixture-of-Experts (MoE) that seamlessly integrates multimodal understanding and generation within a single model. It supports: - 🖼️ **Text-to-Image Generation** — high-fidelity image synthesis with optional thinking/reasoning. - 🔍 **Image Understanding** — visual question answering, image captioning, document understanding, etc. - ✏️ **Image Editing** — instruction-based editing with single or multi-reference support. - 🎨 **Interleaved Generation and Reasoning** — provide preliminary support for interleaved generation and unlock advanced interleaved reasoning. - ⚡ **Sprint Acceleration** — KV cache reuse and adaptive unmasking for faster inference. ## Model Architecture <img src="./assets/architecture.png" width="100%"/> - **Unified dLLM-MoE Backbone**: Unifies multimodal understanding and generation into a simple Mask Token Prediction paradigm. - **Discrete Semantic Tokenizer**: Utilizes SigLIP-VQ to convert visual inputs into discrete semantic tokens, significantly enhancing multimodal understanding. - **Efficient Diffusion Decoder**: Pairs discrete tokens with a specialized diffusion decoder for high-fidelity generation, enabling rapid 8-step inference via distillation. ## Evaluation Results <img src="./assets/performance.png" width="100%"/> ## Quick Start > **Note:** Full installation instructions and CLI scripts are available in the [GitHub repository](https://github.com/inclusionAI/LLaDA2-Uni). ### ⚙️ Installation #### 1. Create a conda environment ```bash git clone https://github.com/inclusionAI/LLaDA2-Uni && cd LLaDA2-Uni conda create -n llada2_uni python=3.10 -y conda activate llada2_uni ``` #### 2. Install PyTor...

Source context: 5 downloads · 1 likes · Pipeline any-to-any · Library transformers · Repo ZR0Z/LLaDA2.0-Uni