Sapiens Omni 600

--- license: other license_name: research-only-tbd tags: - smpl-x - human-pose-estimation - human-mesh-recovery - sapiens - monocular library_name: pytorch pipeline_tag: keypoint-detection --- # sapiens-omni-600 Monocular **SMPL-X whole-body pose & shape** from dense **600-vertex omni surface landmarks**. A [Sapiens](https://github.com/facebookresearch/sapiens) 0.6B finetune (COCO-WholeBody 133 → omni-600 head) that predicts 600 dense SMPL-X surface landmarks from a single RGB image, then fits SMPL-X to them with a Theseus 2D-reprojection optimizer. - **Code & full usage docs:** https://github.com/initialneil/sapiens-omni-600 - **Landmark partition:** 600 SMPL-X vertex indices from [OmniFit](https://github.com/initialneil/OmniFit) ## Results (AGORA-val) | Method | N | PA-MVE (mm) ↓ | |--------|---|---------------| | SMPLest-X H (published SoTA) | 500 | 124.5 | | SMPLer-X L32 | 500 | 133.4 | | **v5 + SMPLest-X init (this model)** | 396 | **109.7** | | v5 (heatmap fitter, no init) | 500 | 140.4 | Beats SMPLest-X by 12% on the matched record set; 2D landmark accuracy ~5× better. ## Files | Path | Iter | Notes | |------|------|-------| | `v5/iter_150000.pth` | 150000 | Released heatmap-only model (the headline). Full training checkpoint (state_dict + optimizer). | > **v6** (adds a per-landmark depth head for a 3D-aware fitter) is **in training** and not yet released here. ## Usage ```bash git clone https://github.com/initialneil/sapiens-omni-600.git omni_pose cd omni_pose && bash install/setup.sh hf download initialneil/sapiens-omni-600 v5/iter_150000.pth --local-dir data/checkpoints/omni600 python scripts/eval/run_omni_pose_agora.py \ --config configs/omni_pose/bedlam2/sapiens_0.6b-omni600-bedlam2-1024x768.py \ --checkpoint data/checkpoints/omni600/v5/iter_150000.pth \ --n 500 --out tmp/omni_pose_v5 ``` See the [GitHub README](https://github.com/initialneil/sapiens-omni-600) for the SMPLest-X-init flow (the 109.7 mm headline) and v6. ## License License is **TBD / research-only** pending finalization. SMPL-X model files (required, downloaded separately) are non-commercial research only. Do not redistribute until a formal license is in place.

Source context: 0 downloads · 0 likes · Pipeline keypoint-detection · Library pytorch · Repo initialneil/sapiens-omni-600