MrNeRF (@janusch_patas)
2024-08-18 | ❤️ 376 | 🔁 62
The future of Radiance Fields is generative AI!
To demonstrate the potential, I took two images of an object at an angle between 90-120 degrees, created a smooth interpolation between them, and trained a 3D Gaussian Splatting (3DGS) scene.
Read on with the tutorial below ⬇️
Achieving this level of detail and realism with Gaussian Splatting, especially with such a wide baseline, was previously thought to be impossible.
Here’s a short tutorial on how to achieve this:
-
Capture the Images: Encircle an object and take three images at 120-degree intervals.
-
Use a Video Generative AI Tool: Apply the prompt: “Still scene. Smooth camera path in an arc.” to interpolate the images.
I use Luma’s Dreammachine, but other tools like RunwayML also support this process.
2a.Optional Challenge: Use the extension feature to incorporate the third image into the video. You could also try closing the loop with images 1, 2, and 3, then back to image 1.
However, this requires three processing steps.
-
Train the 3DGS Model: Insert the generated video into your favorite 3DGS trainer. For convenience, I used Luma, but other options are also excellent. If available, you can also use NeRFstudio’s gsplat.
-
(Optional step): Share your results in this thread!
One cool feature of this approach is that you can easily capture scenes even in crowded environments. Simply wait until people or moving objects disappear from your photos—capturing three images without interruptions is usually manageable.
Additionally, you could try prompting something like “Remove people and moving objects and animals” to enhance the results further.
Imagine a future where you can take a few images of a scene and use them as an environment in your favorite 3D game or even change the entire setting of a movie!
미디어
