Sapiens2 Onnx

--- license: other tags: - onnx - sapiens2 - cpu - segmentation - pose-estimation - normal-estimation - depth-estimation --- # Sapiens2 ONNX CPU-friendly ONNX exports of Meta's `facebook/sapiens2-*`. 15 task heads across 4 tasks and 4 sizes. ## Folder layout Each task has its own folder. Each model is split into a small `.onnx` graph file plus a `.onnx.data` external sidecar (both must live in the same directory at download time). | Task | 0.4b | 0.8b | 1b | 5b | |---|---|---|---|---| | seg | `seg/seg_0.4b_fp16.onnx` (777 MB, fp16) | `seg/seg_0.8b_fp32.onnx` (3.3 GB) | `seg/seg_1b_fp32.onnx` (5.9 GB) | `seg/seg_5b_int8.onnx` (5.2 GB) | | normal | `normal/normal_0.4b_fp32.onnx` (1.7 GB) | `normal/normal_0.8b_fp32.onnx` (3.5 GB) | `normal/normal_1b_fp32.onnx` (6.2 GB) | `normal/normal_5b_int8.onnx` (6.1 GB) | | pointmap | `pointmap/pointmap_0.4b_fp32.onnx` (2.0 GB) | `pointmap/pointmap_0.8b_fp32.onnx` (3.9 GB) | `pointmap/pointmap_1b_fp32.onnx` (6.5 GB) | `pointmap/pointmap_5b_int8.onnx` (6.2 GB) | | pose | `pose/pose_0.4b_fp32.onnx` (1.6 GB) | `pose/pose_0.8b_fp32.onnx` (3.4 GB) | `pose/pose_1b_fp32.onnx` (6.1 GB) | not shipped | Cosine similarity vs the PyTorch reference is 0.999 or better on every shipped file. ## Precision notes * seg-0.4b is fp16 (50 percent smaller than fp32 and verified cos 0.99999) * 0.4b/0.8b/1b for normal, pointmap, pose are fp32. Naive fp16 cast produces NaN (normal L2-normalize divides near zero) or cos around 0.7 (pointmap metric scale, pose sigmoid heatmaps saturate) * 5B variants are INT8 (per-channel symmetric, MatMulIntegerToFloat) * pose-5b is not shipped (the int8 quantize attempt did not complete on the available hardware) ## Inference ```python import numpy as np import onnxruntime as ort from huggingface_hub import hf_hub_download # Download both the .onnx graph and the .onnx.data sidecar side by side for fn in ("seg/seg_0.4b_fp16.onnx", "seg/seg_0.4b_fp16.onnx.data"): hf_hub_download(repo_id="WeReCooking/sapiens2-onnx", filename=fn, local_dir=".") sess = ort.InferenceSession("seg/seg_0.4b_fp16.onnx", providers=["CPUExecutionProvider"]) # Input expects (N, 3, 1024, 768) fp32 BGR mean-subtracted preprocessed tensor out = sess.run(None, {"input": preprocessed}) ``` For a standalone CLI without sapiens2 or PyTorch, see `app.py onnx ...` in the source Space `WeReCooking/sapiens2-cpu`. ## License Same as ups...

Source context: 0 downloads · 0 likes · Pipeline depth-estimation · Library sapiens2 · Repo WeReCooking/sapiens2-onnx