Embodied AI Reading Notes (@EmbodiedAIRead)
2026-01-03 | โค๏ธ 82 | ๐ 11
TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System
Project: https://yanjieze.com/TWIST2/ Paper: https://yanjieze.com/TWIST2/ Code: https://github.com/amazon-far/TWIST2 Data: https://twist-data.github.io/
This project introduces a low-cost, mocap-free, easy-to-setup teleoperation data collection system for G1 humanoid whole-body control and use this system and data to train a whole-body visuomotor policy with egocentric vision.
-
System formulation: (1) low-level control: trained as a general motion tracker at 50HZ, which is task-agnostic (2) high-level control: generate motion commands conditioned on egocentric vision.
-
Hardware setup: (1) self-designed 2DoF robot neck for egocentric vision (2) PICO 4U head set and 2 PICO Motion Tracker for motion tracking teleop.
-
Authors use an adapted version of GMR, a real-time motion retargeting method for human-to-humanoid retargeting
-
Hierarchical visual policy framework: (1) system 1 general motion tracker is trained with dedicated motion dataset using RL in large-scale sim. (2) system 2 visuomotor policy is a diffusion policy to predict whole-body joint positions using teleop collected observation-action pairs.
-
This system enables simple vision-based autonomous control of full humanoid body task such as whole-body pick&place and kicking box to target region.
-
The entire system and collected dataset are open-sourced.
๐ ์๋ณธ ๋งํฌ
- https://yanjieze.com/TWIST2/
- https://yanjieze.com/TWIST2/
- https://github.com/amazon-far/TWIST2
- https://twist-data.github.io/
๋ฏธ๋์ด
๐ Related
Auto-generated - needs manual review