Register Any Point:
Scaling 3D Point Cloud Registration by Flow Matching

Yue Pan1 Tao Sun2 Liyuan Zhu2 Lucas Nunes1 Iro Armeni2 Jens Behley1 Cyrill Stachniss1
1University of Bonn 2Stanford University
Paper Code 🤗 Demo

Accomplish single-stage multi-view point cloud registration at various scales by flow matching-based generation

Overview

our teaser figure

Our method for scalable multi-view point cloud registration. Prior works perform pairwise registration via correspondence matching and then conduct pose graph optimization (top-left). We use a single-stage flow matching model to generate the registered points (top-middle). Our model RAP generalizes across various view counts, scales, and sensor modalities (bottom) and achieves superior zero-shot performance with the shortest runtime on the cross-domain multi-view registration benchmark (top-right).

Orbit the viewer to see from different angles. Switch between scenes using the buttons below.

Abstract

Point cloud registration aligns multiple unposed point clouds into a common reference frame and is a core step for 3D reconstruction and robot localization without initial guess. In this work, we cast registration as conditional generation: a learned, continuous point-wise velocity field transports noisy points to a registered scene, from which the pose of each view is recovered. Unlike prior methods that perform correspondence matching to estimate pairwise transformations and then optimize a pose graph for multi-view registration, our model directly generates the registered point cloud, yielding both efficiency and point-level global consistency. By scaling the training data and conducting test-time rigidity enforcement, our approach achieves state-of-the-art results on existing pairwise registration benchmarks and on our proposed cross-domain multi-view registration benchmark. The superior zero-shot performance on this benchmark shows that our method generalizes across view counts, scene scales, and sensor modalities even with low overlap.

Method

our method overview figure

Overview of our approach to multi-view point cloud registration. Starting with unposed point clouds, we sample points with corresponding local features. We use a diffusion transformer with alternating-attention blocks for conditional flow matching that generates the aggregated point cloud from Gaussian noise. Finally, we recover the individual transformations using SVD from the aggregated point cloud and apply them to the original unposed point clouds to get the registered point clouds.

Acknowledgments

This work is built upon Rectified Point Flow. We also thank the authors of the following works: BUFFER-X and GARF.