Robust Robotic Aggregation of Irregularly Shaped Objects

Abstract

Robotic technology has entered many new fields and applications in the last years and decades, from automated manufacturing, over warehouse logistics, to inspec- tion of industrial sites. While automation and robotics are pushing more and more from industrial to everyday applications, one sector has fallen behind in adapting to those new technologies, the construction sector. Although there is a labor short- age in construction, it still has one of the most considerable unused potentials for automation. Exploiting on-site robotics is not only a great opportunity to resolve the high demand for labor and to increase productivity, but it also has the potential to enable architectural designs that exceed the size and complexity practical with conventional methods. It also offers the opportunity to leverage context-specific, locally sourced materials that are inexpensive, abundant, and low in embodied en- ergy. However, it is still unclear what exact technologies are required and how such robot solutions would potentially look like at this stage. This thesis addresses the development of manipulation skills for a mobile ma- nipulator to detect and assemble arbitrary solid material in a cluttered and unstruc- tured environment like a construction site. The focus lies on the perception of the environment, modeling object instances, and grasping and assembling objects. We are especially interested in manipulating raw material, like stones and boulders, in the context of robotic landscaping, as it targets applications in dangerous-to-access or remote locations undesired for human operators. In this work, we investigate the task of executing autonomous missions in unseen environments at different scales and from stationary to mobile applications. Be- sides localization, obstacle detection, and navigation methods for the mobile base, we introduce a compliant manipulator to perform interaction tasks. The compli- ance gives impact robustness and the capability of controlling contact wrenches ac- curately but comes with limited actuator bandwidth causing tracking performance loss. We address this problem by including the actuator dynamics in a receding horizon control formulation improving tracking and disturbance rejection and show that the natural stiffness of the manipulator can be preserved. The ultimate target of this work is the construction of large-scale structures in real-world outdoor scenarios, outside of well-defined laboratory settings, compli- cating the handling of previously unseen irregularly shaped objects. We contribute by presenting a perception and grasp pose planning pipeline for autonomous ma- nipulation of objects of interest with a robotic walking excavator. A mapping sys- tem incrementally builds a temporally and spatially consistent LiDAR-based map of the robot’s surroundings. It provides the ability to register externally recon- structed point clouds of the scene, e.g., from images captured by a drone-borne camera, helping to increase map coverage. Our grasp planning method utilizes point clouds and mesh surface reconstruction of stones to plan grasp configura- tions with a 2-jaw gripper mounted on the excavator. Besides taking into account the geometry of the stone to sample force closure grasps, the grasp planner consid- ers collision constraints during object pick and place, informed by the LiDAR-based point cloud map. Furthermore, we show an approach to reorient arbitrarily shaped objects that cannot be directly placed at the desired location without violating col- lision constraints. Finally, the presented manipulation methods are combined with geometric target pose planning to compute structurally stable object compositions following a target design. We create a novel process that takes the digitized object model in a physics simulation to find settled placement poses and assess stability, alternately with a 3D shape matching to the target geometry. The locations of the placed objects are refined and updated in simulation to have an accurate digital twin of the scene. Our approach has been thoroughly validated on several interdisciplinary collabo- rations building vertical stone towers in a table-top setup and dry-stone walls com- posed of over one hundred stones with an average weight above 1000 kg with the autonomous excavator HEAP.