MrNeRF (@janusch_patas)
2026-01-15 | โค๏ธ 79 | ๐ 15
Efficient Camera-Controlled Video Generation of Static Scenes via Sparse Diffusion and 3D Rendering
Abstract (excerpt): This paper explores a new strategy for camera-conditioned video generation of static scenes:
- Diffusion-based Generative Models: Generate a sparse set of keyframes.
- 3D Reconstruction and Rendering: Synthesize the full video.
By lifting keyframes into a 3D representation and rendering intermediate views, our approach amortizes the generation cost across hundreds of frames while enforcing geometric consistency.
We further introduce a model that predicts the optimal number of keyframes for a given camera trajectory, allowing the system to adaptively allocate computation.
Our final method, SRENDER, uses very sparse keyframes for simple trajectories and denser ones for complex camera motion. This results in video generation that is more than 40 times faster than the diffusion-based baseline in generating 20 seconds of video, while maintaining high visual fidelity and temporal stability. It offers a practical path toward efficient and controllable video synthesis.
๋ฏธ๋์ด
๐ Related
Auto-generated - needs manual review