Planning for Autonomous Micro-Aerial Vehicles with Applications to Filming and 3D Modeling

Abstract

Camera equipped micro aerial vehicles (MAVs), and in particular multi-copters, have become affordable and abundant in recent years both in the professional and in the consumer sector. They are used in areas as diverse as recreation, filming, structural inspection and monitoring for agriculture. Their success can be attributed to their relatively simple mechanical and electronic design, their flexibility in terms of possible motion and their smooth dynamics which are easy to handle from a control perspective. In most of these use cases the MAVs are still deployed in a manual fashion where one or more human experts steer the MAV or perform related work. This can be partly attributed to the difficulty of autonomous flight in GPS-denied environments. However, even with a good GPS signal the design of autonomous systems requires the incorporation of domain knowledge from the corresponding use case. This is very challenging as this knowledge of experts is often intuitive and non-formal. In this thesis we demonstrate the design of such systems that take sensory information and provide a sequence of decisions to solve the given task. We will identify and formalize the goals for solving the task and the constraints that need to be fulfilled and incorporate both into a planning scheme that will provide us with a sequence of decisions to achieve the goals. In the first part of the thesis we will focus on a framework that allows us to generate flight trajectories that demonstrate smooth and pleasant motion of the camera. The constraints come in the form of the MAV dynamics and the goals are derived from a keyframe based description of the desired film composition. This leads to a constrained optimization problem over the MAV trajectory that we can solve with non-linear programming. The second part of the thesis introduces a system that provides flight paths to capture an image collection that is suitable for reconstructing high quality 3D models of a user-defined region of interest. Here the major constraints are battery time of the MAV and collision free motion. We formulate this task as a submodular orienteering problem on a graph of possible viewpoints and provide an approximate solver. The third and final part of the thesis looks at the autonomous exploration of unknown environments. Here, the goal is to discover as much surface in as little time as possible. To achieve this goal we define an expert policy with full knowledge of the environment and train a 3D convolutional neural network to imitate this expert policy. The resulting model can then be used to explore an unknown environment.