Robust Active Perception and Volumetric Mapping in Unknown Changing Environments
OPEN ACCESS
Loading...
Author / Producer
Date
2022
Publication Type
Doctoral Thesis
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Autonomous mobile robots have the potential to profoundly impact and transform numerous applications, ranging from search and rescue, to autonomous inspection, industrial and warehouse robotics, augmented and virtual reality applications, as well as personal and service robotics. For all these applications, the ability to perceive and model one's environment in a map is a crucial component in order to meaningfully interact with it. Maps for interaction enable capabilities such as collision avoidance, navigation, manipulation, or planning, which are fundamental building blocks of robot autonomy. Furthermore, since robots are embodied agents, they can actively move to perceive and incrementally map their surroundings, enabling autonomous robot application in previously unknown scenes.
However, reliable modeling of complex real world environments on an autonomous robot is a challenging problem for several reasons. First, there is noise in the sensor measurements and pose tracking. Second, the robot state estimate can drift over time, leading to spatial inconsistency in the map. Furthermore, real environments that are shared with other agents oftentimes change over time, leading to temporal inconsistencies. Lastly, the robot needs to be able to efficiently and safely navigate in unknown environments in order to map them. This additionally puts a constraint on both mapping and planning, requiring operation in real time on computationally constrained mobile hardware.
The research question addressed in this thesis is thus twofold. First, we investigate what optimal representations for the map are, that can tackle the problems mentioned above. Second, we investigate how to autonomously map unknown and complex environments efficiently and safely.
Motivated by the human capabilities to deal with these problems, the central approach of this thesis is to explore how scene understanding in different forms can facilitate both mapping and planning. We first explore scene understanding as classical geometry, and develop a general framework for view-based informative path planning in unknown environments. The proposed system continuously adapts its path to maximize any given information gain against a given cost within global context, using a parameter-free optimization formulation. We then use the map update rule to derive a novel information gain significantly improving the accuracy of the obtained 3D reconstruction.
We further present a submap-based multi-layer mapping approach for active perception with drifting state estimates. Our approach uses locally consistent submaps to be able to account for past pose corrections and augments this global map with both a spatially and a temporally local mapping layer for effective planning. Based on this, we develop a planning approach for safe and efficient constant-time local view planning, and for guaranteeing complete global coverage, both with a changing global map.
The second part explores scene understanding as the recognition of prior beliefs, that can be learned from data. Here, we first address the challenge of high computational requirements in sampling-based view planning by proposing a method to learn the underlying informed distribution of high utility views from the robot map. This approach significantly speeds up computation and achieves strong exploration performance, as only few candidates need to be evaluated to reliably identify good view points.
We then tackle the challenge of representing complex geometry amid noisy measurements and propose a novel volumetric mapping approach based on neural implicit representations that can leverage the spatial context within the large voxels represented by a single neural code together with deep learnt shape priors to reliably fuse noisy measurements. Even in the presence of sensing errors and notable pose tracking inaccuracy, this system can well capture and represent the underlying geometry, also of thin objects that are not well captured in traditional methods.
Lastly, we explore the capabilities of explicit semantic scene understanding. We address a major limitation when exploring unknown scenes, being that the robot has to operate on the limited information available at each time step, and propose an approach that leverages semantic scene completion to predict what the unknown environment might look like. This additional information can reasonably fill in holes in the reconstruction, significantly speeding up coverage of a scene with only minimal decrease in map accuracy. In addition, it can also be used to guide the robot to choose more informative paths, notably speeding up the measurement of the environment with the sensors of the robot.
Eventually, we tackle the challenge of temporal consistency in volumetric maps by developing an object-centric map representation modeling the world as a set of temporally and semantically consistent object submaps. This allows efficient reasoning about scene persistence or changes on this abstract level, guaranteeing semantic and temporal consistency of the map, and enables direct spatio-temporal map queries for interaction planning.
In summary, we identify and tackle major challenges in robust volumetric mapping and active perception in complex unknown environments. Our contributions include the development of the first approach for volumetric exploration subject to state estimation drift, the first approach for incremental robotic volumetric mapping directly on neural implicit representations, and the first online volumetric mapping method with long-term temporal consistency. We further present a new approach for general view-based informative path planning in unknown environments, and propose novel ways to safely combine learning with exploration planning, by proposing to learn the components of sampling-based exploration and integrating semantic scene completion into the exploration pipeline. Our contributions together constitute the components of a complete pipeline for autonomous perception and mapping in complex environments, addressing the challenges pointed out and bringing such systems closer to reliable autonomous application in the challenging conditions outside the research lab. We show that scene understanding in various forms can be a key component in addressing these challenges, and that these approaches can successfully be deployed on different aerial and ground robots using solely mobile hardware.
Permanent link
Publication status
published
External links
Editor
Contributors
Examiner : Siegwart, Roland
Examiner : Pollefeys, Marc
Examiner : Jensfelt, Patric
Examiner : Leonard, John
Book title
Journal / series
Volume
Pages / Article No.
Publisher
ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Robotics; Perception; Mapping; Scene Understanding; Active Perception; Mapping for Interaction; Autonomous Systems
Organisational unit
03737 - Siegwart, Roland Y. (emeritus) / Siegwart, Roland Y. (emeritus)