# FastWAM LIBERO 2-Camera 224 (LeRobot) FastWAM is a world action model built on Wan2.2 and an action diffusion expert. It predicts continuous robot action chunks from visual observations, proprioception, and language/task context. Checkpoint trained and evaluated on LIBERO tasks: 97.6 % SR on all tasks. Original paper: https://arxiv.org/pdf/2603.16666 Implementation: This LeRobot implementation follows the original reference code for compatibility. Reference implementation: [https://github.com/yuantianyuan01/FastWAM] ## Model description - Inputs: concatenated multi-view RGB image, robot state/proprioception, task context - Outputs: continuous robot actions - Training objective: FastWAM video/action diffusion loss - Intended use: evaluation or fine-tuning on LIBERO-style manipulation tasks ## Quick start ### Installation Install LeRobot from a version that includes the `fastwam` policy: ```bash pip install "lerobot[fastwam]@git+https://github.com/huggingface/lerobot.git" ``` For full installation details, see the official LeRobot documentation: https://huggingface.co/docs/lerobot/installation ### Load model and run `select_action` ```python import torch from lerobot.policies.fastwam.modeling_fastwam import FastWAMPolicy model_id = " /fastwam-libero-uncond-2cam224" device = torch.device("cuda" if torch.cuda.is_available() else "cpu") policy = FastWAMPolicy.from_pretrained(model_id, strict=False).to(device).eval() batch = { "observation.images.image": torch.zeros(1, 3, 224, 448, device=device), "observation.state": torch.zeros(1, 8, device=device), "prompt": "pick up the object and place it at the target location", } with torch.inference_mode(): action = policy.select_action(batch) print(action.shape) ``` `FastWAMPolicy.from_pretrained(...)` loads the policy weights and the local Wan sidecar components from this same repository snapshot. ## Training step For training or fine-tuning, call `forward(...)` and use the returned `loss` key: ```python policy.train() outputs = policy.forward(batch) loss = outputs["loss"] loss.backward() ``` The training batch must contain FastWAM-ready tensors such as `video`, `action`, `context`, and `context_mask`, or LeRobot observation/action keys that can be adapted by the policy wrapper. ## Fine-tuning A typical fine-tuning command follows the standard LeRobot training flow: ```bash lerobot-train \ --dataset.repo_id= \ --output_dir=./outputs/fastwam_finetune \ --job_name=fastwam_finetune \ --policy.type=fastwam \ --policy.path= /fastwam-libero-uncond-2cam224 \ --policy.device=cuda \ --steps=100000 \ --batch_size=1 ``` Adjust batch size and sequence settings for available GPU memory. ## Evaluate in simulation For LIBERO evaluation, use the LeRobot evaluation flow once your environment is installed: ```bash lerobot-eval \ --policy.path= /fastwam-libero-uncond-2cam224 \ --env.type=libero \ --env.task=libero_spatial \ --eval.batch_size=1 \ --eval.n_episodes=20 ``` If using a project-specific LIBERO evaluator, pass this repository id as the policy path. ## Repository files This repository is self-contained for `FastWAMPolicy.from_pretrained(...)`: ```text config.json model.safetensors policy_preprocessor.json policy_preprocessor_step_3_normalizer_processor.safetensors policy_postprocessor.json policy_postprocessor_step_0_unnormalizer_processor.safetensors Wan2.2_VAE.safetensors models_t5_umt5-xxl-enc-bf16.safetensors google/umt5-xxl/ libero_uncond_2cam224_dataset_stats.json ``` The Wan VAE, UMT5 text encoder, and tokenizer are stored beside the FastWAM policy weights. The saved policy postprocessor also sets `toggle_action_dimensions: [-1]` to match the LeRobot LIBERO gripper action convention. ## Notes This checkpoint uses only the migrated Hugging Face / LeRobot serialization format: `config.json`, `model.safetensors`, and local Wan sidecar files. Original FastWAM `.pt` checkpoint loading is not required.
