Benefiting from local rigidity in 3D point cloud processing

Abstract

Incorporating 3D understanding and spatial reasoning into (intelligent) algorithms is crucial for solving several tasks in fields such as engineering geodesy, risk assessment, and autonomous driving. Humans are capable of reasoning about 3D spatial relations even from a single 2D image. However, making the priors that we rely on explicit and integrating them into computer programs is very challenging. Operating directly on 3D input data, such as 3D point clouds, alleviates the need to lift 2D data into a 3D representation within the task-specific algorithm and hence reduces the complexity of the problem. The 3D point clouds are not only a better-suited input data representation, but they are also becoming increasingly easier to acquire. Indeed, nowadays, LiDAR sensors are even integrated into consumer devices such as mobile phones. However, these sensors often have a limited field of view, and hence multiple acquisitions are required to cover the whole area of interest. Between these acquisitions, the sensor has to be moved and pointed in a different direction. Moreover, the world that surrounds us is also dynamic and might change as well. Reasoning about the motion of both the sensor and the environment, based on point clouds acquired in two-time steps, is therfore an integral part of point cloud processing. This thesis focuses on incorporating rigidity priors into novel deep learning based approaches for dynamic 3D perception from point cloud data. Specifically, the tasks of point cloud registration, deformation analysis, and scene flow estimation are studied. At first, these tasks are incorporated into a common framework where the main difference is in the level of rigidity assumptions that are imposed on the motion of the scene or the acquisition sensor. Then, the tasks specific priors are proposed and incorporated into novel deep learning architectures. While the global rigidity can be assumed in point cloud registration, the motion patterns in deformation analysis and scene flow estimation are more complex. Therefore, the global rigidity prior has to be relaxed to local or instance- level rigidity, respectively. Rigidity priors not only add structure to the aforementioned tasks, which prevents physically implausible estimates and improves the generalization of the algorithms, but in some cases also reduce the supervision requirements. The proposed approaches were quantitatively and qualitatively evaluated on several datasets, and they yield favorable performance compared to the state-of-the-art