Zum Inhalt springen
ModelAufgelistetExtern
Herunterladen

Wuji Pick Place Vam Ti2v5b 30l Robotwin Ref Openwam 16g Mask v3

This repository contains one Wuji pick-and-place VAM checkpoint from the May 26, 2026 OpenWAM/RobotWin-reference training run.

Quellehuggingface.co/knightnemo/wuji-pick-place-vam-ti2v5b-30l-robotwin-ref-openwam-16g-mask-v3/tree/388e112eac85db1258bb7f89c1580556bfe6f306
ArtOtherBasismodellWan-AI/Wan2.2-TI2V-5BVersionStep 20000CBewertung

Modellquelle

Art
Other
Dateigröße
11,2 GB
Basismodell
Wan-AI/Wan2.2-TI2V-5B
Version
vStep 20000
Aktualisiert
22. Sept. 2026
Quelle
Hugging Face

Quellenbeschreibung

Wuji Pick Place Vam Ti2V5B 30L Robotwin Ref Openwam 16G Mask V3

This repository contains one Wuji pick-and-place VAM checkpoint from the May 26, 2026 OpenWAM/RobotWin-reference training run.

Identity

repo_id:              knightnemo/wuji-pick-place-vam-ti2v5b-30l-robotwin-ref-openwam-16g-mask-v3
wandb project:        wuji_pick_place
wandb run id:         ge0f4qe7
wandb run name:       wuji_pick_place_vam_ti2v5b_30L_robotwin_ref_openwam_16g_mask_v3_0526_2127
local training dir:   /cpfs/huangsq/VAM_Learn_from_Human_Video/src/vam/models/train/wuji_pick_place_vam_ti2v5b_30L_robotwin_ref_openwam_16g_mask_v3
checkpoint:           step-20000.safetensors
checkpoint size:      12042172757 bytes
base model:           Wan-AI/Wan2.2-TI2V-5B
action expert style:  openwam
mask variant:         v3
reference frames:     57
reference dropout:    0.1
action/proprio dim:   54 / 54

The checkpoint is a joint model: step-20000.safetensors contains the fine-tuned video DiT weights plus the action_dit.* action-stream weights. The Wan2.2-TI2V-5B base model is not included.

Files

step-20000.safetensors     final 20k-step checkpoint
model_config.json      compact machine-readable configuration and metrics
training_config.yaml   full W&B training config snapshot
wandb-summary.json     final scalar metrics exported by W&B
training_log_node0.txt node-0 training log
training_log_node1.txt node-1 training log
README.md              this model card

Final Step Metrics

These are the scalar values in wandb-summary.json at step=20000.

MetricValue
val/loss0.20223
val/loss_action0.11154
val/loss_video0.0906898
val/action_mse0.00743866
val/action_mae0.055161
val/video_mse640.378
val/video_psnr21.9358
val/video_ssim0.784399
val/video_lpips0.0869703
train/loss0.0870087
train/loss_action0.00918485
train/loss_video0.0778239
train/grad_norm1.53125
_runtime32738.4
_step20000

Saved Validation Losses

Only step-20000.safetensors is uploaded here. Earlier local checkpoints were saved every 2500 steps and are listed for provenance.

Best saved checkpoint by aggregate validation loss:

step 15000: loss=0.196729, loss_video=0.081665, loss_action=0.115064
StepLossVideo LossAction Loss
25000.2889920.1400880.148904
50000.1969880.0779940.118994
75000.2509910.0993230.151668
100000.3300280.1185470.211480
125000.3214150.1215870.199828
150000.1967290.0816650.115064
175000.3335810.0993110.234270
200000.2022300.0906900.111540

Training Configuration

KeyValue
dataset_typewuji_pick_place
wuji_robot_dataset_rootNone
variantclean_50
height384
width320
num_frames33
action_video_freq_ratio4
action_horizonNone
action_dim54
proprio_dim54
action_formatabsolute
action_spacejoint
proprio_spaceaction
action_pad_modelast
action_expert_styleopenwam
action_mot_backbone_pretrained_path/cpfs/huangsq/VAM_Learn_from_Human_Video/src/vam/models/pretrained/ActionMoT_openwam_linear_interp_Wan22_alphascale_1024hdim.pt
mask_variantv3
mask_tail_padding_lossTrue
full_reference_videoTrue
max_ref_frames57
reference_dropout0.1
bridge_exclude_full_refTrue
extra_inputsvace_reference_image,action_trajectory
target_camerahead_camera
reference_camerahead_camera
resize_modestretch
backboneti2v
model_paths["models/Wan-AI/Wan2.2-TI2V-5B/diffusion_pytorch_model-00001-of-00003.safetensors","models/Wan-AI/Wan2.2-TI2V-5B/diffusion_pytorch_model-00002-of-00003.safetensors","models/Wan-AI/Wan2.2-TI2V-5B/diffusion_pytorch_model-00003-of-00003.safetensors","models/Wan-AI/Wan2.2-TI2V-5B/models_t5_umt5-xxl-enc-bf16.pth","models/Wan-AI/Wan2.2-TI2V-5B/Wan2.2_VAE.pth"]
tokenizer_pathmodels/Wan-AI/Wan2.2-TI2V-5B/google/umt5-xxl
trainable_modelsdit
learning_rate5e-05
action_lrNone
weight_decay0.01
warmup_steps500
max_steps20000
num_epochs1
batch_size1
gradient_accumulation_steps1
dataset_repeat1
dataset_num_workers8
use_gradient_checkpointingTrue
save_steps2500
val_steps500
video_log_steps2500
max_val_samples20
lambda_video1
lambda_action1
video_dim3072
action_dit_dim1024
action_dit_ffn_dim4096
action_dit_num_heads24
action_dit_num_layers30
proprio_dropout0.1
window_stride1
val_ratio0.1
output_path/cpfs/huangsq/VAM_Learn_from_Human_Video/src/vam/models/train/wuji_pick_place_vam_ti2v5b_30L_robotwin_ref_openwam_16g_mask_v3
wandb_projectwuji_pick_place
wandb_run_namewuji_pick_place_vam_ti2v5b_30L_robotwin_ref_openwam_16g_mask_v3_0526_2127

Input/Output Contract

Expected inputs:

prompt:                 Pick up the ball with the left hand and place it in the basket.
target camera:          head_camera
reference camera:       head_camera
target video frames:    33
full reference frames:  57
image resolution:       384 x 320
action/proprio dim:     54 / 54

Expected outputs:

robot-view target video rollout
54-D absolute robot action targets

Masking Note

This run uses mask_variant=v3 with full cross-embodiment reference video conditioning and records bridge_exclude_full_ref=True.

Quellen

1 Quelle
Hugging FacePrimärVersion 388e112eac85db1258bb7f89c1580556bfe6f306
huggingface.co/knightnemo/wuji-pick-place-vam-ti2v5b-30l-robotwin-ref-openwam-16g-mask-v3/tree/388e112eac85db1258bb7f89c1580556bfe6f306

Verifiziert 25. Sept.

Modellartefakte

1 Artefakt

step-20000.safetensors

safetensors · 11,2 GB · SHA-256 4a23c42bba0e…a522 · Hugging Face

Download file

Quellenauszüge

2 Auszüge
Readme31. Julihuggingface.co/knightnemo/wuji-pick-place-vam-ti2v5b-30l-robotwin-ref-openwam-16g-mask-v3

--- license: other language: - en tags: - robotics - video-generation - imitation-learning - action-generation - wan - vam - openwam - wuji-pick-place - mask-v3 pipeline_tag: video-to-video base_model: Wan-AI/Wan2.2-TI2V-5B library_name: pytorch --- # Wuji Pick Place Vam Ti2V5B 30L Robotwin Ref Openwam 16G Mask V3 This repository contains one Wuji pick-and-place VAM checkpoint from the May 26, 2026 OpenWAM/RobotWin-reference training run. ## Identity ```text repo_id: knightnemo/wuji-pick-place-vam-ti2v5b-30l-robotwin-ref-openwam-16g-mask-v3 wandb project: wuji_pick_place wandb run id: ge0f4qe7 wandb run name: wuji_pick_place_vam_ti2v5b_30L_robotwin_ref_openwam_16g_mask_v3_0526_2127 local training dir: /cpfs/huangsq/VAM_Learn_from_Human_Video/src/vam/models/train/wuji_pick_place_vam_ti2v5b_30L_robotwin_ref_openwam_16g_mask_v3 checkpoint: step-20000.safetensors checkpoint size: 12042172757 bytes base model: Wan-AI/Wan2.2-TI2V-5B action expert style: openwam mask variant: v3 reference frames: 57 reference dropout: 0.1 action/proprio dim: 54 / 54 ``` The checkpoint is a joint model: `step-20000.safetensors` contains the fine-tuned video DiT weights plus the `action_dit.*` action-stream weights. The Wan2.2-TI2V-5B base model is not included. ## Files ```text step-20000.safetensors final 20k-step checkpoint model_config.json compact machine-readable configuration and metrics training_config.yaml full W&B training config snapshot wandb-summary.json final scalar metrics exported by W&B training_log_node0.txt node-0 training log training_log_node1.txt node-1 training log README.md this model card ``` ## Final Step Metrics These are the scalar values in `wandb-summary.json` at `step=20000`. | Metric | Value | |---|---:| | `val/loss` | 0.20223 | | `val/loss_action` | 0.11154 | | `val/loss_video` | 0.0906898 | | `val/action_mse` | 0.00743866 | | `val/action_mae` | 0.055161 | | `val/video_mse` | 640.378 | | `val/video_psnr` | 21.9358 | | `val/video_ssim` | 0.784399 | | `val/video_lpips` | 0.0869703 | | `train/loss` | 0.0870087 | | `train/loss_action` | 0.00918485 | | `train/loss_video` | 0.0778239 | | `train/grad_norm` | 1.53125 | | `_runtime` | 32738.4 | | `_step` | 20000 | ## Saved Validation Losses Only `step-20000.safetensors` is uploaded here. Earlier local checkpoints were saved every 2500 steps and are listed for provenance. Best saved checkpoint b...

Note31. Julihuggingface.co/knightnemo/wuji-pick-place-vam-ti2v5b-30l-robotwin-ref-openwam-16g-mask-v3

Source context: 0 downloads · 0 likes · Pipeline video-to-video · Library pytorch · Repo knightnemo/wuji-pick-place-vam-ti2v5b-30l-robotwin-ref-openwam-16g-mask-v3