Trackdub Phi 4 Mini Cpu Int4
This repository repackages the cpuandmobile/cpu-int4-rtn-block-32-acc-level-4 configuration from Microsoft's official ONNX release microsoft/Phi-4-mini-instruct-onnx. It is not a newly trained model. This is an...
Fonte do modelo
- Tipo
- Other
- Tamanho do arquivo
- 49,7 MB
- Modelo base
- microsoft/Phi-4-mini-instruct-onnx
- Versão
- vRevision d86f7f9
- Atualizado
- 21 de out. de 2026
- Publicador
- @tonythethompson
- Downloads
- 0
- Tarefas
- Geração de texto
- Fonte
- Hugging Face
Descrição da fonte
Phi-4 Mini Instruct — CPU INT4 (ONNX Runtime GenAI)
This repository repackages the cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4
configuration from Microsoft's official ONNX release
microsoft/Phi-4-mini-instruct-onnx.
It is not a newly trained model. This is an unofficial community repackage; Microsoft is
the original author and license holder.
Source
| Field | Value |
|---|---|
| Upstream model | microsoft/Phi-4-mini-instruct-onnx |
| Upstream source revision | fc04c8f93df696602fd9f300a30d1bf2e3081347 |
| Export tool/script | Microsoft ONNX Runtime GenAI model builder (upstream Phi ONNX bundle) |
| Quantization recipe | ONNX Runtime GenAI RTN INT4 (cpu-int4-rtn-block-32-acc-level-4) |
Files
All files are under cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/:
| File | Size | Description |
|---|---|---|
model.onnx | ~50 MB | INT4 ONNX graph |
model.onnx.data | ~5 GB | External weights |
genai_config.json | ~1 KB | ONNX Runtime GenAI session config |
config.json | ~2 KB | Model config |
configuration_phi3.py | ~11 KB | Phi-3/4 config class |
tokenizer.json + tokenizer_config.json | ~15 MB | Tokenizer |
vocab.json + merges.txt | ~6 MB | Vocabulary / merges |
added_tokens.json + special_tokens_map.json | <1 KB | Special tokens |
Intended Use
A 4-bit, CPU-targeted build of Phi-4-Mini-Instruct for local text generation and
translation on CPU-only or low-VRAM hardware via ONNX Runtime GenAI. A GPU variant is in
tonythethompson/phi-4-mini-instruct-gpu-int4-onnx.
Runtime Notes
- Designed for ONNX Runtime GenAI compatible runtimes.
- Execution provider: CPU (
onnxruntime-genai). - Context length: 128K tokens (inherited from Phi-4-Mini-Instruct).
- Microsoft's published throughput examples: ~16-24 tokens/sec on server-class Xeon CPUs; ~3-5 tokens/sec on laptop CPUs. Validate on your hardware before production use.
Precision and Packaging
Export tooling, precision, and quantization are recorded in the Source table above. This packaging mirror does not publish independent parity benchmarks; validate on your target execution provider before production use.
Limitations
- CPU inference is substantially slower than GPU inference; not suitable for real-time use on most hardware.
- INT4 quantization introduces approximation error relative to the base FP16/FP32 model.
- No repository-specific translation quality evaluation is documented here.
- Validate on your target language pair before production use.
License
MIT — inherited from
microsoft/Phi-4-mini-instruct-onnx. This packaging repo adds no new license terms.
Fontes
1 fonte
Fontes
1 fonteVerificado 21 de out.
Artefatos de modelo
1 artefato
Artefatos de modelo
1 artefatocpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/model.onnx
onnx · 49,7 MB · SHA-256 701aa5d185b6…0b83 · Hugging Face
BaixarTrechos de fonte
3 trechos
Trechos de fonte
3 trechos--- license: mit base_model: microsoft/Phi-4-mini-instruct-onnx base_model_relation: quantized pipeline_tag: text-generation library_name: onnxruntime-genai language: - ar - zh - cs - da - nl - en - fi - fr - de - he - hu - it - ja - ko - "no" - pl - pt - ru - es - sv - th - tr - uk tags: - onnx - onnxruntime-genai - phi4 - phi4-mini - int4 - cpu - text-generation - translation --- # Phi-4 Mini Instruct — CPU INT4 (ONNX Runtime GenAI) This repository repackages the `cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4` configuration from Microsoft's official ONNX release [microsoft/Phi-4-mini-instruct-onnx](https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx). It is not a newly trained model. This is an unofficial community repackage; Microsoft is the original author and license holder. ## Source | Field | Value | |---|---| | Upstream model | [microsoft/Phi-4-mini-instruct-onnx](https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx) | | Upstream source revision | `fc04c8f93df696602fd9f300a30d1bf2e3081347` | | Export tool/script | Microsoft ONNX Runtime GenAI model builder (upstream Phi ONNX bundle) | | Quantization recipe | ONNX Runtime GenAI RTN INT4 (`cpu-int4-rtn-block-32-acc-level-4`) | ## Files All files are under `cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/`: | File | Size | Description | |---|---|---| | `model.onnx` | ~50 MB | INT4 ONNX graph | | `model.onnx.data` | ~5 GB | External weights | | `genai_config.json` | ~1 KB | ONNX Runtime GenAI session config | | `config.json` | ~2 KB | Model config | | `configuration_phi3.py` | ~11 KB | Phi-3/4 config class | | `tokenizer.json` + `tokenizer_config.json` | ~15 MB | Tokenizer | | `vocab.json` + `merges.txt` | ~6 MB | Vocabulary / merges | | `added_tokens.json` + `special_tokens_map.json` | <1 KB | Special tokens | ## Intended Use A 4-bit, CPU-targeted build of Phi-4-Mini-Instruct for local text generation and translation on CPU-only or low-VRAM hardware via ONNX Runtime GenAI. A GPU variant is in [`tonythethompson/phi-4-mini-instruct-gpu-int4-onnx`](https://huggingface.co/tonythethompson/phi-4-mini-instruct-gpu-int4-onnx). ## Runtime Notes - Designed for ONNX Runtime GenAI compatible runtimes. - Execution provider: CPU (`onnxruntime-genai`). - Context length: 128K tokens (inherited from Phi-4-Mini-Instruct). - Microso...
Source context: 0 downloads · 0 likes · Pipeline text-generation · Library onnxruntime-genai · Repo tonythethompson/Phi-4-Mini-Instruct-CPU-INT4-ONNX
Source context: 0 downloads · 0 likes · Pipeline text-generation · Library onnxruntime-genai · Repo tonythethompson/trackdub-phi-4-mini-cpu-int4
