Show simple item record

dc.contributor.author
Furrer, Fadri
dc.contributor.supervisor
Siegwart, Roland
dc.contributor.supervisor
Hutter, Marco
dc.date.accessioned
2020-05-27T06:40:22Z
dc.date.available
2020-05-27T06:21:28Z
dc.date.available
2020-05-27T06:40:22Z
dc.date.issued
2020-05
dc.identifier.uri
http://hdl.handle.net/20.500.11850/416831
dc.identifier.doi
10.3929/ethz-b-000416831
dc.description.abstract
Robotic systems have shown impressive results at navigating in previously mapped areas, in particular in the domain of assisted (and autonomous) driving. As these systems do not perform physical interaction with the environment, the map representations are optimized for precise localization and not for rapidly changing scenes. Environment changes are only incorporated into these maps when observed repeatedly. On the other hand, when physical interaction between the robot and the environment is required, it is crucial that the map representation is at any time consistent with the world. For instance, the new location of a manipulated (or externally moved) object must be constantly updated in the map. In this thesis, we argue that object based maps are a more suitable map representation for this purpose. Our solutions build on the hypothesis that object based representations are able to deal with change and as they contain or gather knowledge about physical objects, they apprehend what parts of the environment can be jointly modified. This thesis aims to find such environment representations that are well suited for robotic mobile manipulation tasks. We start by creating a system that takes measurements from localized RGB-D cameras and integrates them into an instance based segmentation map. This is done by segmenting each incoming depth frame with a geometric approach into locally convex segments. These segments are integrated into a 3D voxel grid as a Truncated Signed DistanceField (TSDF) with an associated instance label. By updating these labels as new segments are integrated a consistent segmentation map is formed. Each segment is stored with its observed position in a 3D object model database, which represents the environment using object-like segments. But in addition to represent the environment, the database can be used to match and merge newly extracted map segments and complete the scene as repeating instances appear or if an instance has been observed in a previous session. To acquire such maps and to enable robots to interact with the environment, we show that it is beneficial to fuse information of multiple sensor modalities. For instance, cameras have shown to be a great source for creating sparse localization maps, whereas measurements from depth sensors can create dense reconstructions even in textureless regions of the environment. However, before using multiple sensors together, a challenging problem is to spatially and temporally align the sensor measurements. Hence, we focus on how to get robotic actuators and multiple sensors into a common place and time frame to allow the fusion of measurements and to let the robot act and interact in such a frame. We show how filtering and optimization techniques improve initial time-synchronizations and hand-eye calibrations. Next, we use the tools and techniques developed for the mapping and object discovery task in the context of manipulation. Using a set of rocks, we want to form vertical balancing towers with a robotic arm equipped with a wrist-mounted RGB-D camera. By identifying previously scanned rocks in a tabletop scene, we perform a set of simulation iterations using a physics engine with the detected objects to assess the stability of possible stack configurations. In a greedy manner, we select the next best rock to place and find and execute a grasping and placing motion. The segmentation map presented in this thesis allows to extract single geometric instances in a priori unknown environments. An incremental object database is built, which can match and merge re-observed or repeating object segments. These merged instances improve the raw extracted 3D models over time and, finally, the approach even allows to complete unobserved parts of the scene. Compelling results are exhibited in extracting and creating singulated object models from RGB-D images of household objects in cluttered warehouse distribution box scenes, furniture in indoor scenes, and cars in a parking garage. We show that our matching approach can be used to identify an object's pose in a scene accurately enough to solve delicate manipulation tasks. Together with a newly introduced greedy next best object target pose planning algorithm, we can stack stones to vertical balancing towers. We demonstrate that our new hand-eye calibration framework is applicable to many different robotic use cases. The integration of a time-alignment step takes away the burden of manually getting time-aligned pose sets, whereas filtering and optimization techniques improve calibration results in all evaluated datasets.
en_US
dc.format
application/pdf
en_US
dc.language.iso
en
en_US
dc.publisher
ETH Zurich
en_US
dc.rights.uri
http://rightsstatements.org/page/InC-NC/1.0/
dc.subject
Robotics
en_US
dc.subject
3D Vision
en_US
dc.subject
Manipulation
en_US
dc.subject
Hand-eye calibration
en_US
dc.subject
Object modelling
en_US
dc.subject
Object matching
en_US
dc.title
Online Incremental Object-Based Mapping for Mobile Manipulation
en_US
dc.type
Doctoral Thesis
dc.rights.license
In Copyright - Non-Commercial Use Permitted
dc.date.published
2020-05-27
ethz.size
137 p.
en_US
ethz.code.ddc
DDC - DDC::6 - Technology, medicine and applied sciences::620 - Engineering & allied operations
en_US
ethz.identifier.diss
26457
en_US
ethz.publication.place
Zurich
en_US
ethz.publication.status
published
en_US
ethz.leitzahl
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02130 - Dep. Maschinenbau und Verfahrenstechnik / Dep. of Mechanical and Process Eng.::02620 - Inst. f. Robotik u. Intelligente Systeme / Inst. Robotics and Intelligent Systems::03737 - Siegwart, Roland Y. / Siegwart, Roland Y.
en_US
ethz.leitzahl
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02100 - Dep. Architektur / Dep. of Architecture::02284 - NFS Digitale Fabrikation / NCCR Digital Fabrication
en_US
ethz.leitzahl.certified
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02130 - Dep. Maschinenbau und Verfahrenstechnik / Dep. of Mechanical and Process Eng.::02620 - Inst. f. Robotik u. Intelligente Systeme / Inst. Robotics and Intelligent Systems::03737 - Siegwart, Roland Y. / Siegwart, Roland Y.
en_US
ethz.date.deposited
2020-05-27T06:21:36Z
ethz.source
FORM
ethz.eth
yes
en_US
ethz.availability
Open access
en_US
ethz.rosetta.installDate
2020-05-27T06:40:35Z
ethz.rosetta.lastUpdated
2021-02-15T11:12:32Z
ethz.rosetta.versionExported
true
ethz.COinS
ctx_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.atitle=Online%20Incremental%20Object-Based%20Mapping%20for%20Mobile%20Manipulation&rft.date=2020-05&rft.au=Furrer,%20Fadri&rft.genre=unknown&rft.btitle=Online%20Incremental%20Object-Based%20Mapping%20for%20Mobile%20Manipulation
 Search print copy at ETH Library

Files in this item

Thumbnail

Publication type

Show simple item record