Medgemma 1.5 Ort Standard

--- license: apache-2.0 library_name: onnx tags: - medical - vlm - mobile - onnx-runtime-genai - gemma-2 - android - ios base_model: google/medgemma-1.5-4b-it pipeline_tag: image-to-text language: - en model_creator: Soufiane EL OUBAYDI model_type: gemma2 --- # MedGemma 1.5 4B - Mobile Optimized (ONNX/ORT) This repository provides **Google's MedGemma 1.5 4B** (Multimodal), quantized to **INT4** and converted to the ONNX Runtime (`.ort`) format. It is specifically tuned for on-device medical image-to-text analysis and reasoning on mobile devices (Android/iOS). --- ## 🚀 Mobile Optimizations Unlike standard weights, this version is designed to fit within the RAM constraints of modern smartphones. * **Quantization:** Language Decoder (INT4), Embeddings (8-bit UINT8), and Vision Tower (8-bit UINT8). * **Format:** Optimized `.ort` (Flatbuffers) to enable **Memory Mapping (mmap)**, preventing Out-of-Memory (OOM) crashes. * **Size:** Reduced from ~7.1GB (unoptimized ONNX) to **~3.3GB total bundle size**. ## 📊 Memory Requirements Optimized for high-performance inference on consumer-grade mobile hardware: * **Model Loading:** ~3.3 - 4 GB RAM * **Active Inference:** +500MB - 1GB (context-dependent) * **Recommended Hardware:** Devices with **6GB+ RAM** (e.g., iPhone 15 Pro, Pixel 8, or high-end Android tablets). --- ## ⚠️ Limitations * **Context Window:** Capped at **2048 tokens** (compared to 128K in the full model) to ensure memory stability on mobile devices. * **Quantization:** Uses **INT4** precision. While this significantly reduces size, there may be a slight degradation in diagnostic accuracy compared to the FP16/FP32 original. * **Mobile Only:** Optimized for **ARM-based CPUs** (Android/iOS). Performance on desktop GPUs is not the primary focus. * **No Streaming:** Currently returns the complete response at once (no token-by-token streaming in the current implementation). --- ## 🛠️ Export Details This model was exported and optimized using the **ONNX Runtime GenAI model builder** pipeline. ```bash python -m onnxruntime_genai.models.builder \ -m google/medgemma-1.5-4b \ -o medgemma_onnx_mobile \ -p int4 \ -e cpu ``` The resulting .onnx files were further converted to .ort format for mobile memory management. ## 📱 Usage (Flutter) This model is intended for use with the flutter_onnxruntime_genai package. Text-Only Example ```bash final...