Simulation to Real World: Learn to Control Legged Robots

Abstract

This thesis addresses both control and design aspects of legged robots. Regarding control, I propose two learning-based control approaches that make a legged robot run faster, more energy-efficiently, and more robustly than ever before. This is possible thanks to an effective modeling technique and a simulation tool, both of which are developed in this thesis. Furthermore, the proposed approaches significantly reduce the laborious process of controller design, which hinders the practicality of prior methods. Only by defining a cost function and initialization/termination strategies, natural behaviors that are realizable on the robot arise. Regarding design, I propose a cable-pulley-based efficient transmission concept which is realized as a single-legged hopping system. The constructed system exhibits remarkable efficiency and power while having a simple structure. Although these two contributions seem disconnected, they cannot be addressed separately. Both control and design aspects of robotics should complement each other to create a great legged machine. A great control algorithm exploits the dynamics of the hardware, and well-designed hardware accounts for the characteristics of existing control approaches. Prior control approaches for controlling legged systems are highly tailored for a specific task; as a result, a complex control architecture has to be designed for every new task. Such arduous workflow and the complexity have deterred the advancement of legged robotics. This is the primary issue that this thesis addresses with new learning-based control approaches. Reinforcement learning approaches promote a natural discovery of behaviors through a high-level cost function, unlike the prior approaches that hand-code each behavior. However, they have limited success on real robots due to their extensive data requirements. With the approach proposed in this thesis, a policy trained in a simulated environment can be seamlessly transferred to a real robotic system. Consequently, a development process of a control policy can be automated. The enormous complexity of control algorithms is now managed by a parameterized function -- a deep neural network -- relieving humans from the cognitive labor. The proposed approaches are also uncompromising in performance: a sampling-based search can often find an effective solution near the global optimum even in some highly non-convex problems. The resulting behavior is thus highly performant with respect to the cost function. In addition, the training is performed using a very detailed physics model of the system, whereas many of the prior control methods are based on highly approximated models. Consequently, the performance of the proposed approaches on the real robot is likely to be higher. As the core of the proposed approaches for control relied heavily on simulation, they can be greatly aided by a high-performance physics simulator. With a new novel numerical optimization scheme, a rigid-body simulator was developed in this thesis. The simulator solves a rigid-body contact problem faster, more stably, and more accurately compared to previous approaches. This simulator ensures the computational practicality and the transferability of the proposed control approaches. One of the proposed approaches for control was tested on ANYmal, a dog-sized versatile quadrupedal robot. Without any modification, tuning or filtering, the trained policies manifest highly agile and complex behaviors: ANYmal precisely follows random combinations of command velocities, balances under high external perturbations, runs faster than ever before, and recovers itself from a fall. This proves the effectiveness of the proposed approaches for simulation and control, respectively. This thesis also introduces an ambitious new legged robotic platform: Capler-Leg. Capler-Leg is equipped with nearly frictionless cable-driven transmission systems, thereby increasing the energy efficiency and the power output. Through comprehensive evaluations, I observed unmatched results. A robot that weighs only about 4 kg outputs more than 4 kW of instantaneous power, while recuperating more than 97 % of kinematic energy back to the battery. It is a promising approach for constructing high performance legged robots. The contributions of this thesis are key technologies for advancing legged robotics. It is my firm belief that further efforts in the outlined directions will soon make legged robots meaningfully aid humans in multiple domains.