Mimo v5 GGUF

--- language: fr license: apache-2.0 tags: - gguf - multimodal - quantized - mimo2 library_name: llama.cpp pipeline_tag: text-generation --- # MIMO Model - GGUF Quantized This repository contains a quantized GGUF version of the **MIMO** (Mixture of Mixtures) model, optimized for efficient inference with `llama.cpp`, LM Studio, and other GGUF-compatible backends. ## Model Architecture | Parameter | Value | | :--- | :--- | | **Architecture** | MIMO2 with Sliding Window Attention (SWA) | | **Context length** | 32K tokens | | **Layers** | 28 | | **Hidden size** | 4096 | | **Attention heads** | 32 | | **Quantization** | Q4_K_M (balanced quality/size) | | **Tokenizer** | Llama-compatible | ## Available files | Filename | Quantization | Size | | :--- | :--- | :--- | | `MIMO2-7B-Q4_K_M.gguf` | Q4_K_M | 4.68 GB | | `mimo_v5.Q4_K_M.gguf` | Q4_K_M | 4.68 GB | | `Mimo-v5-7b.Q4_K_M.gguf` | Q4_K_M | 4.68 GB | | `mimo2_q4_k_m.gguf` | Q4_K_M | 4.68 GB | > All files use the **Q4_K_M** quantization method, offering an excellent trade-off between model quality and memory usage. ## Usage ### With `llama.cpp` ```bash # Start an OpenAI-compatible server with web UI llama-server -hf redhamohamed/mimo-v5-gguf:MIMO2-7B-Q4_K_M.gguf # Run inference directly in terminal llama-cli -hf redhamohamed/mimo-v5-gguf:MIMO2-7B-Q4_K_M.gguf -p "Bonjour, comment ça va ?" With llama-cpp-python python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="redhamohamed/mimo-v5-gguf", filename="MIMO2-7B-Q4_K_M.gguf", ) response = llm.create_chat_completion( messages=[{"role": "user", "content": "Explain MIMO in simple terms"}] ) print(response["choices"][0]["message"]["content"]) With LM Studio Open LM Studio Search for redhamohamed/mimo-v5-gguf in the Hub tab Download the desired .gguf file Load and start chatting With Ollama bash ollama run hf.co/redhamohamed/mimo-v5-gguf:Q4_K_M Quantization details Q4_K_M is a K-quant method that provides: 4-bit weights with higher accuracy for important layers Intermediate size (larger than Q4_0, smaller than Q5_K_M) Recommended for general use and resource-constrained environments Notes This model uses a Llama-compatible tokenizer No additional configuration is required for the chat template Tested with llama.cpp v1.2.0 and higher Acknowledgments Thanks to the open-source community for developing the tools that make GGUF quantization and distribution possible. License Apache 2.0 ABDESSEMED Mohamed Redha