Tinyvit 5m Int8 Imagenet

--- license: mit library_name: onnx base_model: timm/tiny_vit_5m_224.dist_in22k_ft_in1k base_model_relation: quantized pipeline_tag: image-classification datasets: - imagenet-1k tags: - onnx - int8 - quantized - image-classification - imagenet - edge - cpu --- # TinyViT-5M: INT8 ONNX, 80.5% ImageNet in 9.2 MB TinyViT-5M ([timm/tiny_vit_5m_224.dist_in22k_ft_in1k](https://huggingface.co/timm/tiny_vit_5m_224.dist_in22k_ft_in1k), MIT) quantized to INT8 with [Kenosis](https://coreepoch.dev/kenosis/): 128-image calibration, no retraining. 80.53% top-1 from a single 9.2 MB file, measured on CPU with ONNX Runtime and OpenVINO. No accelerator required. ## Accuracy ImageNet-1K validation, 49,872 images (disjoint from the 128 calibration images), top-1. | model | ONNX Runtime | OpenVINO | size | |---|---|---|---| | FP32 baseline | 80.873% | 80.873% | 22.1 MB | | **Kenosis INT8** | **80.532%** | **80.534%** | **9.23 MB** | Measured on a CPU with AVX-VNNI; on CPUs without VNNI this model's INT8 top-1 sits about 0.9 below FP32 rather than 0.34. ## Speed Against the FP32 model on CPU: about the same speed on ONNX Runtime with one thread. ## Run ```python from huggingface_hub import hf_hub_download import numpy as np, onnxruntime as ort from PIL import Image config = hf_hub_download("CoreEpoch/tinyvit-5m-int8-imagenet", "config.json") path = hf_hub_download("CoreEpoch/tinyvit-5m-int8-imagenet", "tinyvit_5m_224_int8_kenosis.onnx") sess = ort.InferenceSession(path, providers=["CPUExecutionProvider"]) img = Image.open("your_image.jpg").convert("RGB") scale = 236 / min(img.size) # shorter side to 236, center crop to 224: measured transform img = img.resize((round(img.width*scale), round(img.height*scale)), Image.BICUBIC) l, t = (img.width-224)//2, (img.height-224)//2 img = img.crop((l, t, l+224, t+224)) x = (np.asarray(img, np.float32)/255.0 - [0.485,0.456,0.406]) / [0.229,0.224,0.225] logits = sess.run(None, {"input": x.transpose(2,0,1)[None].astype(np.float32)})[0] print(int(np.argmax(logits))) ``` Input `1x3x224x224`, RGB, `/255`, ImageNet mean/std. Output `logits [1,1000]`, sorted-synset order. ## Integrity & license `tinyvit_5m_224_int8_kenosis.onnx` (9,228,567 B), SHA-256 `EDA9007C0449F118F02B1E8C4BA6B43BEC51A7D7AB677566D42929EB3DA02874`. MIT (base TinyViT, Wu et al. 2022, retained). The license covers these model files only; it grants no rights in Kenosis...

--- license: apache-2.0 library_name: onnx base_model: timm/tiny_vit_5m_224.dist_in22k_ft_in1k base_model_relation: quantized pipeline_tag: image-classification datasets: - imagenet-1k tags: - onnx - int8 - quantized - image-classification - imagenet - edge - cpu --- # TinyViT-5M — INT8 ONNX, 80.5% ImageNet in 9.2 MB This is TinyViT-5M ([timm/tiny_vit_5m_224.dist_in22k_ft_in1k](https://huggingface.co/timm/tiny_vit_5m_224.dist_in22k_ft_in1k), Apache-2.0), quantized to INT8 with [Kenosis](https://coreepoch.dev), Core Epoch's post-training quantizer, using 128 calibration images, a per-model configuration, and no retraining. It holds **80.53% top-1 on ImageNet-1K in a 9.2 MB single file** that runs on both ONNX Runtime and OpenVINO, with no GPU required. This architecture is unusually hostile to post-training quantization. ONNX Runtime's static quantizer, on this exact model and calibration data at its default calibration, collapses to the 3.12% control in the table below. This artifact holds the model within 0.35 points of FP32 on the full validation set. ## Measured accuracy ImageNet-1K validation, 49,872 images (the full 50,000 minus a 128-image calibration split; calibration and evaluation images are disjoint). The evaluation images are the official validation files, verified by byte-level sample audit against the official distribution. All rows were evaluated on identical inputs. | model | top-1 | Δ vs FP32 | file size | |---|---|---|---| | FP32 baseline | 80.87% | — | 22.1 MB | | **this artifact** | **80.53%** | **−0.34** | **9.23 MB** | | ONNX Runtime static quantization at default calibration, same model, same calibration data | 3.12% | −77.8 | 7.6 MB | The FP32 control matches the published checkpoint within 0.01. All three rows use the same 49,872-image evaluation set; the third row is ONNX Runtime's own static quantizer at its default MinMax calibration, and its exact invocation is in this repo as `quantize_baseline.py`, so the comparison can be re-run. ONNX Runtime also offers Percentile and Entropy calibrators, which are not applied here; choosing and tuning a calibrator per model recovers much of this gap. The artifact above uses no such tuning. Full OpenVINO cross-checks are in the [int8-models repo](https://github.com/CoreEpoch/int8-models). ## Run it ``` pip install onnxruntime numpy pillow huggingface_hub ``` ```python from huggin...

Source context: 49 downloads · 0 likes · Pipeline image-classification · Library onnx · Repo CoreEpoch/tinyvit-5m-int8-imagenet