yugeshkarunamurthy/Gemma-4-12B-it-oQ6 · Hugging Face

--- base_model: google/gemma-4-12B-it library_name: mlx pipeline_tag: text-generation license: gemma language: - en tags: - gemma - gemma-4 - gemma4_unified - mlx - omlx - quantized - text-generation - apple-silicon - image-text-to-text - image-to-text - video-to-text - any-to-any --- # Gemma 4 12B IT - oQ Quantized ## Model Description This repository contains an oMLX oQ quantized version of Google's Gemma 4 12B IT model. The model has been quantized using oMLX's sensitivity-aware mixed-precision quantization pipeline, which dynamically allocates precision across model components to preserve quality while reducing memory and storage requirements. ## Base Model * Base Model: google/gemma-4-12B-it * Model Family: Gemma 4 * Quantization Method: oMLX oQ * Format: MLX * License: Gemma License ## Quantization Information This model was created using the oMLX oQ quantization pipeline. oQ uses mixed-precision quantization instead of applying a uniform bit-width across all tensors. More sensitive model components retain higher precision while less sensitive components are compressed more aggressively. ### Benefits * Reduced memory footprint * Reduced storage requirements * Improved quality retention compared to uniform quantization * Optimized for Apple Silicon inference ## Intended Uses This model is suitable for: * General chat applications * Coding assistance * Research and experimentation * Local AI assistants * Agent workflows * Reasoning tasks * Content generation ## Usage ### Python ```python from mlx_lm import load, generate model, tokenizer = load("path/to/model") response = generate( model, tokenizer, prompt="Explain mixed precision quantization.", max_tokens=512, ) print(response) ``` ### CLI ```bash mlx_lm.generate \ --model path/to/model \ --prompt "Hello!" ``` ## Hardware Requirements Hardware requirements depend on: * Context length * Runtime implementation * Quantization parameters * Concurrent workloads Apple Silicon systems are recommended for optimal performance. ## Limitations This model inherits the strengths and limitations of the original Gemma 4 12B IT model. Quantization may introduce: * Minor reductions in reasoning quality * Slight output variations compared to full-precision checkpoints * Reduced accuracy on some specialized tasks Users should evaluate the model for their specific use...