Anime XCodec2

--- license: cc-by-nc-4.0 datasets: - OOPPEENN/56697375616C4E6F76656C5F44617461736574 - amphion/Emilia-Dataset - OmniAICreator/ASMR-Archive-Processed language: - ja base_model: - HKUSTAudio/xcodec2 pipeline_tag: audio-to-audio tags: - audio-to-audio - speech - not-for-all-audiences --- # Anime‑XCodec2: Japanese Fine‑Tuned Variant of XCodec2 [](https://creativecommons.org/licenses/by-nc/4.0/) **TL;DR**: Anime‑XCodec2 is a fine‑tuned variant of **HKUSTAudio/xcodec2**, trained on \~25k hours of **Japanese anime/game‑style voices**. Only the **decoder** was updated; the **encoder and codebook remain frozen**, so **speech tokens are identical to the original XCodec2**. This makes the model a drop‑in decoder for downstream systems that already work with XCodec2 tokens (*e.g., Llasa*). --- ## 🔗 Quick Links * **Demo (Gradio / Hugging Face Spaces)**: [https://huggingface.co/spaces/OmniAICreator/Anime-XCodec2-Demo](https://huggingface.co/spaces/OmniAICreator/Anime-XCodec2-Demo) * **Baseline model (pretrained)**: `HKUSTAudio/xcodec2` * **This repository (fine‑tuned)**: `NandemoGHS/Anime‑XCodec2` * **Training Logs (Weights & Biases)**: [View Report](https://api.wandb.ai/links/aratako-lm/0pf7mfmj) --- ## 1) Model Summary * **What it is**: A neural speech codec / speech tokenizer model based on **XCodec2** with a decoder fine‑tuned for Japanese speech, particularly **anime/game‑style voices**. * **Training scope**: **Decoder‑only** fine‑tuning on \~**25,000 hours** of Japanese data; **encoder** and **codebook** are **frozen**. * **Compatibility**: Because the encoder and codebook are unchanged, **speech tokens produced at encode time are identical** with the original XCodec2. Any downstream model expecting XCodec2 codes can use **Anime‑XCodec2** as a **drop‑in decoder** (*e.g., Llasa*). * **Sampling rate**: **16 kHz** (XCodec2 operates at 16 kHz). --- ## 2) Intended Use * **Decode XCodec2 speech tokens** (e.g., from Llasa or other AR token generators trained on XCodec2 codes) into **Japanese speech** with improved naturalness for **anime/game‑style** voices. * **Reconstruction** of Japanese speech from XCodec2 tokens when analyzing or building Japanese‑focused speech pipelines. --- ## 3) Limitations & Trade‑offs * **Language scope**: Optimized for **Japanese**....

Source context: 324 downloads · 18 likes · Pipeline audio-to-audio · Repo NandemoGHS/Anime-XCodec2