Id Sim Dinov3 Vitl16 Cls Patch
ID-Sim is a fine-grained identity similarity metric trained to capture the subtle visual differences humans rely on when distinguishing highly similar subjects across varied viewpoints and contexts. This checkpoint...
Model source
- Kind
- LoRA
- File size
- 24.5 MB
- Base model
- dinov3_vitl16
- Version
- vadapter_model
- Updated
- Oct 22, 2026
- Downloads
- 158
- Source
- Hugging Face
Source description
ID-Sim (DINOv3 ViT-L/16, cls_patch)
ID-Sim is a fine-grained identity similarity metric trained to capture the subtle visual differences humans rely on when distinguishing highly similar subjects across varied viewpoints and contexts. This checkpoint uses a DINOv3 ViT-L/16 backbone with LoRA fine-tuning and outputs both CLS and patch-level embeddings (cls_patch).
Paper: ID-Sim: An Identity-Focused Similarity Metric — CVPR 2026
Project page: https://juliachae.github.io/id_sim.github.io/
Code: https://github.com/JuliaChae/id_sim
Usage
DINOv3 backbone weights must be downloaded manually from Meta (gated). Place dinov3_vitl16_pretrain_lvd1689m-8aa4cbdd.pth in ./models/id_sim_checkpoint/checkpoints/. ID-Sim adapter weights download automatically on first use.
import torch
from PIL import Image
from id_sim import id_sim
model, preprocess = id_sim(pretrained=True, device="cuda")
img_a = preprocess(Image.open("a.jpg")).cuda()
img_b = preprocess(Image.open("b.jpg")).cuda()
with torch.inference_mode():
distance = model(img_a, img_b) # lower = more similar
Install: pip install git+https://github.com/JuliaChae/id_sim.git
Model Details
| Base model | DINOv3 ViT-L/16 (dinov3_vitl16) |
| Fine-tuning | LoRA (r=16, α=32, dropout=0.05) + MLP heads (hidden=512) |
| Feature type | cls_patch — joint CLS + Sinkhorn patch loss |
| License | MIT |
| Developed by | Julia Chae, Nicholas Kolkin, Jui-Hsien Wang, Richard Zhang, Sara Beery, Cusuh Ham |
Training
Trained on ~10k triplets (anchor / positive / negative) spanning ~10k instances across 10 real datasets (landmarks, fashion, objects, art, animals), augmented with generative identity-preserving and identity-altering edits. Key hyperparameters: lr=3e-4, effective batch size 32 (8×4 grad accum), 3 epochs, bf16, InfoNCE + Sinkhorn patch loss.
Evaluation
The model was evaluated across a diverse set of instance-level tasks ranging from concept-preservation evaluation in generative models and perceptual similarity ranking to fine-grained re-identification and retrieval. For full evaluation instructions see the repository; for full evaluation results see the paper.
Citation
@InProceedings{Chae_2026_CVPR,
author = {Chae, Julia and Kolkin, Nicholas and Wang, Jui-Hsien Wang, Richard and Beery, Sara and Ham, Cusuh},
title = {ID-Sim: An Identity-Focused Similarity Metric},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2026},
pages = {11250-11262}
}
Framework versions
- PEFT 0.16.0
Sources
1 source
Sources
1 sourceVerified Oct 22
Model artifacts
1 artifact
Model artifacts
1 artifactSource excerpts
3 excerpts
Source excerpts
3 excerpts--- library_name: peft tags: - lora - similarity - image-feature-extraction base_model: dinov3_vitl16 license: mit --- # ID-Sim (DINOv3 ViT-L/16, cls_patch) ID-Sim is a fine-grained identity similarity metric trained to capture the subtle visual differences humans rely on when distinguishing highly similar subjects across varied viewpoints and contexts. This checkpoint uses a DINOv3 ViT-L/16 backbone with LoRA fine-tuning and outputs both CLS and patch-level embeddings (`cls_patch`). **Paper:** [ID-Sim: An Identity-Focused Similarity Metric](https://arxiv.org/abs/2604.05039) — CVPR 2026 **Project page:** https://juliachae.github.io/id_sim.github.io/ **Code:** https://github.com/JuliaChae/id_sim ## Usage DINOv3 backbone weights must be downloaded manually from Meta (gated). Place `dinov3_vitl16_pretrain_lvd1689m-8aa4cbdd.pth` in `./models/id_sim_checkpoint/checkpoints/`. ID-Sim adapter weights download automatically on first use. ```python import torch from PIL import Image from id_sim import id_sim model, preprocess = id_sim(pretrained=True, device="cuda") img_a = preprocess(Image.open("a.jpg")).cuda() img_b = preprocess(Image.open("b.jpg")).cuda() with torch.inference_mode(): distance = model(img_a, img_b) # lower = more similar ``` Install: `pip install git+https://github.com/JuliaChae/id_sim.git` ## Model Details | | | |---|---| | **Base model** | DINOv3 ViT-L/16 (`dinov3_vitl16`) | | **Fine-tuning** | LoRA (r=16, α=32, dropout=0.05) + MLP heads (hidden=512) | | **Feature type** | `cls_patch` — joint CLS + Sinkhorn patch loss | | **License** | MIT | | **Developed by** | Julia Chae, Nicholas Kolkin, Jui-Hsien Wang, Richard Zhang, Sara Beery, Cusuh Ham | ## Training Trained on ~10k triplets (anchor / positive / negative) spanning ~10k instances across 10 real datasets (landmarks, fashion, objects, art, animals), augmented with generative identity-preserving and identity-altering edits. Key hyperparameters: lr=3e-4, effective batch size 32 (8×4 grad accum), 3 epochs, bf16, InfoNCE + Sinkhorn patch loss. ## Evaluation The model was evaluated across a diverse set of instance-level tasks ranging from concept-preservation evaluation in generative models and perceptual similarity ranking to fine-grained re-identification and retrieval. For full evaluation instructions see the [repository](https://github.com/JuliaChae/id_sim#evaluation); for fu...
Source context: 158 downloads · 0 likes · Pipeline image-feature-extraction · Library peft · Repo chaenayo/id-sim_dinov3_vitl16_cls_patch
Source context: 354 downloads · 0 likes · Pipeline image-feature-extraction · Library peft · Repo chaenayo/id-sim_dinov3_vitl16_cls_patch
