MrNeRF (@janusch_patas)
2024-12-14 | โค๏ธ 100 | ๐ 16
3DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generation
Contributions:
-
We are the first to customize 6 degrees of freedom (DoF) multi-entity motion in 3D space for controllable video generation, establishing a new benchmark for fine-grained motion control.
-
We propose a 3D-motion grounded video diffusion model that controls multi-entity motions using pose sequences as motion representations. Our flexible object injector enforces entity-wise correspondence between objects and their motions and preserves the video diffusion prior.
-
We introduce a scalable 4D motion dataset construction mechanism, and techniques like the video domain adaptor and annealed sampling to enhance video quality while maintaining motion accuracy.
-
3DTrajMaster achieves state-of-the-art accuracy in controlling 3D entity motions and allows fine-grained entity input customization, such as changing human hair, clothing, gender, and figure size.
๋ฏธ๋์ด
