Dinodepth Model
Trained Simple Depth Transformer (SDT) decoder heads for zero-shot affine-invariant (relative) monocular depth, reproducing AnyDepth (arXiv:2601.02760) on a frozen DINOv3 backbone. Only the small SDT decoder is...
Fonte do modelo
- Tipo
- Other
- Tamanho do arquivo
- 21,1–51,1 MB
- Versão
- vRevision 3887962
- Atualizado
- 10 de out. de 2026
- Downloads
- 0
- Fonte
- Hugging Face
Descrição da fonte
DinoDepth — SDT depth heads on a frozen DINOv3 backbone
Trained Simple Depth Transformer (SDT) decoder heads for zero-shot affine-invariant (relative) monocular depth, reproducing AnyDepth (arXiv:2601.02760) on a frozen DINOv3 backbone. Only the small SDT decoder is trained; the DINOv3 encoder is frozen and loaded separately from Meta's checkpoints. Two heads are provided in this repo:
| File | Backbone | Decoder params | Train |
|---|---|---|---|
sdt-vitl16.safetensors | DINOv3 ViT-L/16 | 13.4 M | 5 epochs |
sdt-vits16.safetensors | DINOv3 ViT-S/16 | 5.5 M | 10 epochs |
These are decoder weights only (~13/5 M params) — pair each with its matching frozen DINOv3
backbone (facebook/dinov3-vitl16-pretrain-lvd1689m / -vits16-).
Zero-shot benchmark (our protocol)
AbsRel ↓ / δ1 ↑ on NYUv2 (Eigen 654) and KITTI (Eigen 652), scored with per-image least-squares scale+shift alignment in disparity space, Eigen/Garg crop, 10 m / 80 m cap.
| Model | NYU AbsRel | NYU δ1 | KITTI AbsRel | KITTI δ1 |
|---|---|---|---|---|
| ViT-L/16 + SDT (this repo) | 0.068 | 0.955 | 0.093 | 0.911 |
| ViT-S/16 + SDT (this repo) | 0.091 | 0.917 | 0.115 | 0.852 |
| AnyDepth ViT-L (paper) | 0.060 | — | 0.086 | — |
| AnyDepth ViT-S (paper) | 0.082 | — | 0.102 | — |
A faithful reproduction — ~0.01 AbsRel behind the paper on each benchmark (consistent across both backbones; plausibly the augmentation/data-filtering details AnyDepth underspecifies).
Usage
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from dinov3_depth.head import DepthModel, DepthModelConfig
# Frozen DINOv3 ViT-L/16 + (randomly-initialised) SDT head; default config matches the trained head
# (GroupNorm, fusion_channels=256).
model = DepthModel.from_pretrained(DepthModelConfig(backbone="vitl16"))
head = hf_hub_download("blanchon/dinodepth-model", "sdt-vitl16.safetensors")
model.head.load_state_dict(load_file(head))
model.eval()
# images: float [B, 3, H, W] in [0, 1], H and W multiples of 16. Returns affine-invariant disparity.
disparity = model(images)
(Use backbone="vits16" + sdt-vits16.safetensors for the small head.)
Architecture & training
- Encoder: frozen DINOv3 ViT (LVD-1689M), patch 16; 4 intermediate layers tapped.
- Decoder (SDT): softmax-fuse the 4 tapped layers at the patch grid → depthwise detail enhancer → two learned DySample ×4 upsamplers → output conv. GroupNorm, fusion width 256.
- Loss: scale-and-shift-invariant + multi-scale gradient matching (1:2), on disparity.
- Data: the harmonized 369K-image corpus at
blanchon/dinodepth-dataset(Hypersim, VKITTI2, BlendedMVS, IRS, TartanAir). 7682 input, AdamW lr 1e-3, PolyLR.
References
- AnyDepth (arXiv:2601.02760) · DINOv3 (arXiv:2508.10104)
- Dataset: blanchon/dinodepth-dataset
Fontes
1 fonte
Fontes
1 fonteVerificado 10 de out.
Download files
Artefatos de modelo
2 artefatos
Artefatos de modelo
2 artefatosTrechos de fonte
3 trechos
Trechos de fonte
3 trechos--- license: apache-2.0 pipeline_tag: depth-estimation library_name: dinov3-depth-head tags: - depth-estimation - monocular-depth-estimation - dinov3 - anydepth datasets: - blanchon/dinodepth-dataset --- # DinoDepth — SDT depth heads on a frozen DINOv3 backbone Trained **Simple Depth Transformer (SDT)** decoder heads for zero-shot **affine-invariant (relative) monocular depth**, reproducing **AnyDepth** (arXiv:2601.02760) on a **frozen DINOv3** backbone. Only the small SDT decoder is trained; the DINOv3 encoder is frozen and loaded separately from Meta's checkpoints. Two heads are provided in this repo: | File | Backbone | Decoder params | Train | |---|---|---|---| | `sdt-vitl16.safetensors` | DINOv3 ViT-L/16 | 13.4 M | 5 epochs | | `sdt-vits16.safetensors` | DINOv3 ViT-S/16 | 5.5 M | 10 epochs | These are **decoder weights only** (~13/5 M params) — pair each with its matching frozen DINOv3 backbone (`facebook/dinov3-vitl16-pretrain-lvd1689m` / `-vits16-`). ## Zero-shot benchmark (our protocol) AbsRel ↓ / δ1 ↑ on NYUv2 (Eigen 654) and KITTI (Eigen 652), scored with per-image least-squares scale+shift alignment in disparity space, Eigen/Garg crop, 10 m / 80 m cap. | Model | NYU AbsRel | NYU δ1 | KITTI AbsRel | KITTI δ1 | |---|---|---|---|---| | ViT-L/16 + SDT (this repo) | 0.068 | 0.955 | 0.093 | 0.911 | | ViT-S/16 + SDT (this repo) | 0.091 | 0.917 | 0.115 | 0.852 | | AnyDepth ViT-L (paper) | 0.060 | — | 0.086 | — | | AnyDepth ViT-S (paper) | 0.082 | — | 0.102 | — | A faithful reproduction — ~0.01 AbsRel behind the paper on each benchmark (consistent across both backbones; plausibly the augmentation/data-filtering details AnyDepth underspecifies). ## Usage ```python from huggingface_hub import hf_hub_download from safetensors.torch import load_file from dinov3_depth.head import DepthModel, DepthModelConfig # Frozen DINOv3 ViT-L/16 + (randomly-initialised) SDT head; default config matches the trained head # (GroupNorm, fusion_channels=256). model = DepthModel.from_pretrained(DepthModelConfig(backbone="vitl16")) head = hf_hub_download("blanchon/dinodepth-model", "sdt-vitl16.safetensors") model.head.load_state_dict(load_file(head)) model.eval() # images: float [B, 3, H, W] in [0, 1], H and W multiples of 16. Returns affine-invariant disparity. disparity = model(images) ``` (Use `backbone="vits16"` + `sdt-vits16.safetensors` for the small head.)...
Source context: 0 downloads · 2 likes · Pipeline depth-estimation · Library dinov3-depth-head · Repo blanchon/dinodepth-model
Source context: 0 downloads · 0 likes · Pipeline depth-estimation · Library dinov3-depth-head · Repo blanchon/dinodepth-model
