LFM2 Audio 1.5B

--- language: - en tags: - liquid - lfm2 - audio - lfm2-audio - speech-to-speech - liquid-audio license: other license_name: lfm1.0 license_link: LICENSE library_name: liquid-audio pipeline_tag: audio-to-audio base_model: - LiquidAI/LFM2-1.2B --- <center> <div style="text-align: center;"> <img src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/7_6D7rWrLxp2hb6OHSV1p.png" alt="Liquid AI" style="width: 100%; max-width: 66%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;" /> </div> </center> # LFM2‑Audio-1.5B LFM2-Audio-1.5B is [Liquid AI](https://www.liquid.ai/)'s first end-to-end audio foundation model. Designed with low latency and real time conversation in mind, at only 1.5 billion parameters LFM2-Audio enables seamless conversational interaction, achieving capabilities on par with much larger models. LFM2-Audio is an end-to-end multimodal speech and text language model, and as such does not require separate ASR and TTS components. Our model consists of a pretrained LFM2 model as its multimodal backbone, along with a FastConformer based audio encoder to handle continuous audio inputs, and a RQ-transformer generating discrete Mimi tokens as audio output. LFM2-Audio supports two distinct generation routines, each suitable for a set of tasks. Interleaved generation enables real-time speech-to-speech conversational chatbot capabilities, where audio generation latency is key. Sequential generation is suited for non-conversational tasks such as ASR or TTS, and allows the model to switch generated modality on the fly. ## 📄 Model details | Property | | |---|---:| | **Parameters (LM only)** | 1.2B | | **Audio encoder** | FastConformer (115M, [canary-180m-flash](https://huggingface.co/nvidia/canary-180m-flash)) | | **Backbone layers** | hybrid conv+attention | | **Audio tokenizer** | [Mimi](https://huggingface.co/kyutai/mimi), using 8 codebooks | | **Context** | 32,768 tokens | | **Vocab size** | 65,536 (text) / 2049*8 (audio) | | **Precision** | bfloat16 | | **License** | LFM Open License v1.0 | **Supported languages:** English ## 🏃 How to run LFM2-Audio Install the `liquid-audio` package via `pip` ```bash pip install liquid-audio pip install "liquid-audio [demo]" # optional, to install demo dependencies pip install flash-attn --no-build-isolation # optional, to use flash attention 2....

Source context: 3 downloads · 0 likes · Pipeline audio-to-audio · Library liquid-audio · Repo kp-forks/LFM2-Audio-1.5B