MiniCPM V 4 OpenVINO INT4

--- base_model: openbmb/MiniCPM-V-4 base_model_relation: quantized quantized_by: HyX3 license: apache-2.0 tags: - quantization - openvino - int4 - vision - multimodal - minicpm pipeline_tag: image-to-text --- # MiniCPM-V-4 OpenVINO INT4 (Preserved FP16 Vision) This repository contains the OpenVINO IR format of the `openbmb/MiniCPM-V-4` model, heavily optimized for CPU inference. ## Optimization Details To ensure optimal performance without sacrificing multimodal capabilities, a selective quantization strategy was applied: - **Language Model (LLM):** Quantized to **INT4** (Asymmetric, Group Size 128) for maximum memory efficiency and CPU inference speed. - **Vision Encoder & Resampler:** Preserved in **FP16**. This surgical approach ensures that the model's visual acuity (including text recognition, UI elements, and fine details) remains identical to the original unquantized model. ## Hardware Targeting This format is designed to run efficiently on standard CPUs, fully utilizing AVX-512 and VNNI instruction sets (e.g., Intel Ice Lake and newer) without requiring dedicated GPU hardware.

Source context: 8 downloads · 0 likes · Pipeline image-to-text · Repo HyX3/MiniCPM-V-4-OpenVINO-INT4