Trackdub Phi 4 Mini Cpu Int4

--- license: mit base_model: microsoft/Phi-4-mini-instruct-onnx base_model_relation: quantized pipeline_tag: text-generation library_name: onnxruntime-genai language: - ar - zh - cs - da - nl - en - fi - fr - de - he - hu - it - ja - ko - "no" - pl - pt - ru - es - sv - th - tr - uk tags: - onnx - onnxruntime-genai - phi4 - phi4-mini - int4 - cpu - text-generation - translation --- # Phi-4 Mini Instruct — CPU INT4 (ONNX Runtime GenAI) This repository repackages the `cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4` configuration from Microsoft's official ONNX release [microsoft/Phi-4-mini-instruct-onnx](https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx). It is not a newly trained model. This is an unofficial community repackage; Microsoft is the original author and license holder. ## Source | Field | Value | |---|---| | Upstream model | [microsoft/Phi-4-mini-instruct-onnx](https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx) | | Upstream source revision | `fc04c8f93df696602fd9f300a30d1bf2e3081347` | | Export tool/script | Microsoft ONNX Runtime GenAI model builder (upstream Phi ONNX bundle) | | Quantization recipe | ONNX Runtime GenAI RTN INT4 (`cpu-int4-rtn-block-32-acc-level-4`) | ## Files All files are under `cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/`: | File | Size | Description | |---|---|---| | `model.onnx` | ~50 MB | INT4 ONNX graph | | `model.onnx.data` | ~5 GB | External weights | | `genai_config.json` | ~1 KB | ONNX Runtime GenAI session config | | `config.json` | ~2 KB | Model config | | `configuration_phi3.py` | ~11 KB | Phi-3/4 config class | | `tokenizer.json` + `tokenizer_config.json` | ~15 MB | Tokenizer | | `vocab.json` + `merges.txt` | ~6 MB | Vocabulary / merges | | `added_tokens.json` + `special_tokens_map.json` | <1 KB | Special tokens | ## Intended Use A 4-bit, CPU-targeted build of Phi-4-Mini-Instruct for local text generation and translation on CPU-only or low-VRAM hardware via ONNX Runtime GenAI. A GPU variant is in [`tonythethompson/phi-4-mini-instruct-gpu-int4-onnx`](https://huggingface.co/tonythethompson/phi-4-mini-instruct-gpu-int4-onnx). ## Runtime Notes - Designed for ONNX Runtime GenAI compatible runtimes. - Execution provider: CPU (`onnxruntime-genai`). - Context length: 128K tokens (inherited from Phi-4-Mini-Instruct). - Microsoft's published throughput examples: ~16-24 tokens/sec on server-class Xeon CPUs; ~3-5 tokens/sec on laptop CPUs. Validate on your hardware before production use. ## Precision and Packaging Export tooling, precision, and quantization are recorded in the **Source** table above. This packaging mirror does not publish independent parity benchmarks; validate on your target execution provider before production use. ## Limitations - CPU inference is substantially slower than GPU inference; not suitable for real-time use on most hardware. - INT4 quantization introduces approximation error relative to the base FP16/FP32 model. - No repository-specific translation quality evaluation is documented here. - Validate on your target language pair before production use. ## License [MIT](https://choosealicense.com/licenses/mit/) — inherited from `microsoft/Phi-4-mini-instruct-onnx`. This packaging repo adds no new license terms.