Ropedia Xperience 10m Task Baselines

--- license: mit library_name: pytorch tags: - embodied-ai - robotics - multimodal - xperience-10m - baseline - evaluation - qwen3-omni - cosmos datasets: - ropedia-ai/xperience-10m-sample - ropedia-ai/xperience-10m metrics: - accuracy - f1 - precision - recall --- Ropedia Xperience-10M Task Suite Public task and evaluation layer for Xperience-10M: sample data, 20 embodied-AI tasks, baselines, Qwen3-Omni and Cosmos3 diagnostics, and foundation-model training directions. English · 中文 · Español · Français · Deutsch · 日本語 · 한국어 · Português This project builds on the Xperience-10M dataset released by Ropedia to provide public, reproducible embodied-AI evaluation materials. It is organized into two evidence lines. **Line 1** turns one public sample episode into inspectable tasks, targets, and baseline runs. **Line 2** uses selected 128-episode public-safe artifacts for aligned metadata/raw baselines, Qwen3-Omni v6 LoRA, Cosmos3-Super Reasoner, and Cosmos3-Nano Future Window. Every score links back to its source artifact, and direct scores remain clearly separated from compact-proxy estimates. **Updated:** 2026-06-23. **Scope:** Line 1 uses one public sample episode. Line 2 uses selected 128-episode public-safe artifacts linked back to official gated episode paths. Raw Xperience-10M MP4/HDF5/RRD files, Qwen3 base weights, Cosmos3 base weights, and gated data are not redistributed here. ## Cite This Repository Use this copyable BibTeX block for the repository: ```bibtex @misc{he2026ropedia_xperience10m_task_suite, author = {Chaoyue He}, title = {Ropedia Xperience-10M Task Suite}, year = {2026}, url = {https://github.com/ChaoYue0307/ropedia-xperience-10m-task-suite}, note = {Public task and evaluation layer for Ropedia Xperience-10M: two evidence lines, 20 embodied-AI tasks, 180 scored method-task records, and selected-128 model diagnostics} } ``` ## Contents - [Project Entry Points](#project-entry-points) - [At A Glance](#at-a-glance) - [Data Explorer Analysis](#data-explorer-analysis) - [Two Evidence Lines](#two-evidence-lines) - [Fast Project Map](#fast-project-map) - [Why This Project Exists](#why-this-project-exists) - [Start Here](#start-here) - [Glossary](#glossary) - [Current Research Scope](#current-research-scope) - [Evaluation Protocol](#evaluation-protocol) - [Dataset Context](#dataset-context) - [Reproducibility](#reproducibility) - [Citation / BibTeX](#citation) ## Project Entry Points Use the two evidence lines first, then choose the artifact that answers your question. The dashboard is the best visual overview; the GitHub repo is the source of truth for scripts and generated JSON; Hugging Face mirrors contain public-safe cards, metrics, figures, and model artifacts. Quick rule: use **Line 1** for “can I inspect and reproduce the task?” Use **Line 2** for “how do aligned baselines and model diagnostics compare on the selected 128 episodes?” The committed task contracts, result matrices, validation JSON, public-safe result packages, GitHub sources, and Hugging Face mirrors are the canonical technical evidence. ## At A Glance Signal Current public state Project identity The same project logo mark is used across the GitHub README, GitHub Pages dashboard, Hugging Face Space, artifact dataset, model mirrors, favicon, and social preview. Ropedia is credited as the Xperience-10M data provider and releaser; this repository is the task-suite and evaluation layer built on that dataset. Reusable assets: logo mark and social card . Two-line contract Line 1: 1 sample episode for task construction and reproducibility. Line 2: 128 selected episodes for same-split metadata/raw baselines, Qwen3-Omni v6, and Cosmos3 diagnostics. Data explorer analysis A generated analysis layer separates the public sample, selected-128 feature exports, and authenticated Hugging Face gated full-dataset metadata with scope stats, split counts, modality breakdowns, and chart assets. 180 method-task records 9 methods x 20 tasks = 180/180 scored records. The ledger separates 174 direct scores from 6 compact-proxy scores. 20 task contracts Action, procedure, transition, trajectory, contact, objects, language, retrieval, reconstruction, order, sync, long-horizon forecasting, interaction text, action-object binding, sensor bridging, camera sync, and transition timing. 4 research directions Human Modeling & Motion Understanding; 3D/4D Reconstruction & Neural Rendering; Egocentric Vision & Interaction; Scene Reconstruction & World Modeling. These are analysis groups over the same 20 tasks, not separate benchmark tiers. Line 1 methods Minimal and Neural MLP baselines cover all 20 tasks on the one public sample episode: 40/40 direct scores. Line 2 methods Metadata simple/NN, raw-feature simple/NN, Qwen3-Omni v6 LoRA, Cosmos3-Super Reasoner, and Cosmos3-Nano Future Window cover all 20 selected-128 task axes: 140/140 scores. 3 foundation pipelines Spatial intelligence, human-video world modeling, and vision-language-action pipelines are documented as training recipes with task mappings, input-output contracts, and model-evidence requirements. 1 unified target The long-term embodied foundation-model target connects perception, 3D memory, language-grounded reasoning, action, and planning without adding a new score axis. Public mirrors GitHub, GitHub Pages, HF Space, HF artifact dataset, HF baseline model repo, Qwen3-Omni and Cosmos3 model repos, and HF collection. ## Public Structure: 20 Tasks / 4 Directions / 3 Pipelines / 1 Unified Target The project has four connected layers. The **20 tasks** are the scored benchmark contracts. The **4 directions** are research groupings over those same tasks. The **3 foundation pipelines** are training recipes that reuse the same modalities, windows, and task targets. The **1 unified embodied model target** is the long-term integration goal after those pipelines mature. Layer rule: if it has a metric, it is a **task**; if it explains what the evidence studies, it is a **direction**; if it describes model inputs and training targets, it is a **pipeline**; if it combines perception, 3D memory, language, action, and planning, it is the **unified target** rather than an extra score axis. | Layer | Count | Role | Exact public labels | | --- | ---: | --- | --- | | Task contracts | 20 | Score axes used by the matrix, radars, task cards, and method rows. | Action Recognition; Procedure Step Recognition; Action Boundary Detection; Next-Action Prediction; Hand Trajectory Forecasting; Contact State Prediction; Object Relevance Prediction; Language Grounding; Cross-Modal Retrieval; Cross-Modal Reconstruction; Temporal Order Verification; Multimodal Synchronization Detection; Long-Horizon Next-Action Forecasting; Long-Horizon Next-Subtask Forecasting; Interaction Text Prediction; Action-Object Relation Prediction; Future Object-Set Forecasting; IMU-to-Hand Pose Reconstruction; Camera-View Synchronization Retrieval; Time-to-Next-Transition Regression. | | Research directions | 4 | Ways to interpret what the 20 tasks study; not separate benchmark tiers. | Human Modeling & Motion Understanding; 3D/4D Reconstruction & Neural Rendering; Egocentric Vision & Interaction; Scene Reconstruction & World Modeling. | | Foundation pipelines | 3 | Larger-model training tracks with separate input-output recipes and result gates. | Spatial intelligence models; Human-video world models; Vision-language-action models. | | Unified embodied model target | 1 | Long-term integration target, not a task/method row in the 180-result matrix. | Perception; 3D memory; language-grounded reasoning; action; planning. | ## Data Explorer Analysis The data explorer is now a three-scope analysis layer, not only a raw-file browser. It compares the public sample episode, selected 128-episode feature exports, and the Hugging Face-hosted gated full-dataset metadata without mixing their evidence boundaries. | Scope | Question | Current public analysis | | --- | --- | --- | | Public sample | What files and signals are directly inspectable? | 1 episode, 5,821 frames, 1,161 aligned 20-frame windows, 8,546 feature dimensions, raw-file browser, modality breakdowns, action-window distribution. | | Selected 128 | What selected-episode surface supports model comparison? | 96/16/16 split, 34,269 Qwen3-Omni v6 multiscale rows, 106,095 dense compact rows, selected episode links, public-safe matrices. | | Full HF dataset | How large is the official upstream dataset? | Authenticated Hub file metadata: 804 sessions, 12,103 episode-like folders, 85,257 files, 24.63 TiB training-byte view, without redistributing raw gated data. | Entry points: [website analysis section](https://chaoyue0307.github.io/ropedia-xperience-10m-task-suite/#data-analysis), [analysis report](DATA_EXPLORER_ANALYSIS.md), and [structured analysis record](docs/data/data_explorer_analysis.json). ## Two Evidence Lines The public suite is organized around two evidence lines. Keep them separate when comparing metrics. Line Data unit Score statement Best use Read separately from 1 sample episode One public Xperience-10M sample episode: 5,821 frames, 1,161 aligned 20-frame windows, 8,546 feature dimensions. 40/40 direct scores from Minimal and Neural MLP heads. Inspect the raw sample, understand file organization, reproduce the 20 task targets, and compare Minimal vs Neural MLP behavior inside one episode. The selected-128 comparison rows and any broader held-out model behavior. 128 selected episodes Selected held-out 96/16/16 split: 34,269 exported windows with public-safe processed features linked to official gated episode paths. The Hugging Face artifact dataset exposes these rows separately as selected_128_windows/selected_128 ; it is not mixed with the one-sample episode_sample/public_sample viewer. 140/140 selected-128 scores: 134 direct + 6 compact-proxy. Compare same-split metadata/raw baselines, Qwen3-Omni v6, Cosmos3-Super, and Cosmos3-Nano while keeping the 6 compact-proxy cells visible. Direct raw-target measurements for the proxy-marked cells. ### Result Ledger Line Methods Tasks Scored records Direct scores Proxy scores 1 sample episode 2 20 40/40 40 0 128 selected episodes 7 20 140/140 134 6 compact-proxy scores, each source-linked and reasoned. Total public matrix 9 20 180/180 174 6 ### Method Blocks Evidence line Method block Methods Score statement Read as 1 sample episode Task-head baselines Minimal; Neural MLP 40/40 direct scores. Task-lab reproducibility and simple-vs-neural behavior. 128 selected episodes Aligned baseline heads Metadata simple/NN; raw-feature simple/NN 80/80 scores: 74 direct + 6 compact-proxy. Same-split metadata/raw-feature baseline comparison. 128 selected episodes Qwen3-Omni series Qwen3-Omni v6 LoRA 20/20 direct scores from verified selected-128 Qwen3-Omni LoRA and task-specific probes. Trainable Qwen3-Omni diagnostic baseline on the selected-128 surface. 128 selected episodes Cosmos3 series Cosmos3-Super Reasoner; Cosmos3-Nano Future Window 40/40 direct scores from verified public-safe reasoner and future-window artifacts. Cosmos3 reasoner and future-window diagnostics on the selected-128 surface. Cosmos3-Super Forward-Dynamics LoRA is published as a separate fine-tuned adapter artifact with weights/results; it is not counted as a 20-task matrix method row. ### Qwen3-Omni Run Versions These are Qwen3-Omni run versions inside **Line 2: selected 128 episodes**. They are not the project evidence lines. The 20-task matrix uses **Qwen3-Omni v6 LoRA**; **v5** remains the pinned prior multiscale release; **v1-v4** are lineage and ablation evidence. Run Purpose Main change Eval signal Use now v1 Prove the selected-128 LoRA/eval/package loop. First verified 96/16/16 selected-episode Qwen3-Omni LoRA run. 448 eval; JSON 0.8750; contact 0.6451. Lineage only. v2 Make answers schema-checked. Structured-JSON contract...