--- license: apache-2.0 base_model: Qwen/Qwen2.5-1.5B-Instruct base_model_relation: quantized pipeline_tag: text-generation library_name: onnxruntime-genai language: - multilingual tags: - onnx - onnxruntime-genai - qwen2 - text-generation - conversational --- # Qwen2.5-1.5B-Instruct — ONNX Runtime GenAI `Qwen/Qwen2.5-1.5B-Instruct` exported to [onnxruntime-genai](https://github.com/microsoft/onnxruntime-genai) format (`genai_config.json` + `model.onnx` external data). It is not a newly trained model. Built with the ort-genai model builder (fp32, CPU EP) for use with `onnxruntime-genai` on CPU / DirectML / CUDA. ## Model - Source: [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) - Parameters: 1.5B - Format: onnxruntime-genai ## Files | File | Description | |---|---| | `model.onnx` + `model.onnx.data` | GenAI model | | `genai_config.json` | ONNX Runtime GenAI session config | | `tokenizer.json` + `tokenizer_config.json` | Tokenizer | | `config.json` | Model config (from upstream HF) | | `chat_template.jinja` | Chat template | **Additional quantized variants** are present in the `onnx/` subfolder (`model.onnx`, `model_bnb4.onnx`, `model_fp16.onnx`, `model_int8.onnx`, `model_q4.onnx`, `model_q4f16.onnx`, `model_quantized.onnx`, `model_uint8.onnx`) for Transformers.js / ONNX Runtime use; the top-level `model.onnx` is the FP32 GenAI build documented above. ## Intended Use A small multilingual instruction model for local text processing via ONNX Runtime GenAI, e.g. cleaning up or polishing raw text before downstream tasks. This is not a finetuned model; it relies on the base Qwen2.5-1.5B-Instruct's instruction-following ability. ## Runtime Notes - Designed for ONNX Runtime GenAI compatible runtimes. a quantized variant for production). - Supports CPU, DirectML, and CUDA execution providers via ort-genai. ## Limitations - FP32 is the highest-fidelity export format but is slower and uses more memory than FP16/INT4 variants. - No repository-specific quality evaluation is documented here. - Multilingual capability is inherited from Qwen2.5-1.5B-Instruct; quality varies by language. ## Source | Field | Value | |---|---| | Upstream model | [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) | | Upstream source revision | `989aa7980e4cf806f80c7fef2b1adb7bc71aa306` | | Packaging source revision | `f56ee6525bf4377fd6c6dcf6b17de010c5f51d26` | | Export tool/script | [onnxruntime-genai model builder](https://github.com/microsoft/onnxruntime-genai) | | Quantization recipe | Top-level FP32 GenAI build; additional quantized variants under `onnx/` and `mxfp8/` | ## Precision and Packaging Export tooling, precision, and quantization are recorded in the **Source** table above. This packaging mirror does not publish independent parity benchmarks; validate on your target execution provider before production use. ## License [Apache 2.0](https://choosealicense.com/licenses/apache-2.0/) — same as [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct). This packaging repo adds no new license terms.
