naveen manwani (@NaveenManwani17)
2025-04-14 | ❤️ 81 | 🔁 14
🚨CVPR 2025 Paper Alert 🚨
➡️Paper Title: Feat2GS: Probing Visual Foundation Models with Gaussian Splatting
🌟Few pointers from the paper
🎯Given that visual foundation models (VFMs) are trained on extensive datasets but often limited to 2D images, a natural question arises: how well do they understand the 3D world?
🎯With the differences in architecture and training protocols (i.e., objectives, proxy tasks), a unified framework to fairly and comprehensively probe their 3D awareness is urgently needed.
🎯Existing works on 3D probing suggest single-view 2.5D estimation (e.g., depth and normal) or two-view sparse 2D correspondence (e.g., matching and tracking).
🎯Unfortunately, these tasks ignore texture awareness, and require 3D data as ground-truth, which limits the scale and diversity of their evaluation set.
🎯 To address these issues, authors of this paper introduced “Feat2GS”, which readout 3D Gaussians attributes from VFM features extracted from unposed images.
🎯This allowed them to probe 3D awareness for geometry and texture via novel view synthesis, without requiring 3D data.
🎯Additionally, the disentanglement of 3DGS parameters - geometry (x,α,Σ) and texture (c) - enables separate analysis of texture and geometry awareness.
🎯Under Feat2GS, they conducted extensive experiments to probe the 3D awareness of several VFMs, and investigate the ingredients that lead to a 3D aware VFM.
🎯Building on these findings, they developed several variants that achieve state-of-the-art across diverse datasets. This makes Feat2GS useful for probing VFMs, and as a simple-yet-effective baseline for novel-view synthesis.
🏢Organization: @Westlake_Uni , @uni_tue , Tübingen AI Center, @VcaiMpi , @SIC_Saar , @MPI_IS
🧙Paper Authors: @faneggchen , @RoverXingyu , @AnpeiC , @GerardPonsMoll1 , @yuliangxiu
📝 Read the Full Paper here: https://arxiv.org/abs/2412.09606
🗂️ Project Page: https://fanegg.github.io/Feat2GS/
🧑💻 Code: https://github.com/fanegg/Feat2GS
🎥 Be sure to watch the attached Technical Summary Video - Sound on 🔊🔊
Find this Valuable 💎 ?
♻️QT and teach your network something new
Follow me 👣, @NaveenManwani17 , for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.
🔗 원본 링크
- https://arxiv.org/abs/2412.09606
- https://fanegg.github.io/Feat2GS/
- https://github.com/fanegg/Feat2GS
미디어
🔗 Related
Auto-generated - needs manual review
Tags
domain-vision-3d domain-ai-ml domain-dev-tools domain-visionos