Davide Scaramuzza (@davsca1)
2024-10-24 | โค๏ธ 170 | ๐ 23
Check out our IROS2024 paper โDeep Visual Odometry with Events andย Frames,โ the new state of the art in Visual Odometry, which outperforms learning-based image methods (DROID-SLAM, DPVO), model-based methods (ORB-SLAM, DSO) and event-based methods (DEVO, EDS) by up to 60% despite being trained only in simulation! Code & new datasets released!
Paper: https://arxiv.org/abs/2309.09947 Code: https://github.com/uzh-rpg/rampvo Video: https://www.youtube.com/watch?v=mzSQR2MEAsU&feature=youtu.be
Visual Odometry (VO) is crucial for autonomous robotic navigation, especially in GPS-denied environments like planetary terrains. To improve robustness, recent model-based VO systems have begun combining standard and event-based cameras. While event cameras excel in low-light and high-speed motion, standard cameras provide dense and easier-to-track features. However, the field of image- and event-based VO still predominantly relies on model-based methods and is yet to fully integrate recent image-only advancements leveraging end-to-end learning-based architectures. Seamlessly integrating the two modalities remains challenging due to their different nature, one asynchronous, the other not, limiting the potential for a more effective image- and event-based VO. We introduce RAMP-VO, the first end-to-end learned image- and event-based VO system. It leverages novel Recurrent, Asynchronous, and Massively Parallel (RAMP) encoders capable of fusing asynchronous events with image data, providing 8ร faster inference and 33% more accurate predictions than existing solutions. Despite being trained only in simulation, RAMP-VO outperforms previous methods on the newly introduced Apollo and Malapert datasets and on existing benchmarks, where it improves image-based (ORB-SLAM, DSO, DROID-SLAM, DPVO) by up to 60% and event-based methods (DEVO, EDS) by up to 30%, paving the way for robust and asynchronous VO.
Reference:
Roberto Pellerito, Marco Cannici, Daniel Gehrig, Joris Belhadj, Olivier Dubois-Matra, Massimo Casasco, Davide Scaramuzza Deep Visual Odometry with Events and Frames. IEEE/RSJ International Conference on Intelligent Robots (IROS), 2024.
Kudos to @RobertoPelleri9 @marcocannici @DanielGehrig6
@esa @ERC_Research @AUTOASSESS_EU @UZH_en @UZHspacehub @UZH_Science @uzh_ifi
๋ฏธ๋์ด
Tags
domain-reconstruction domain-robotics domain-simulation domain-robotics-navigation