Stargate: Multimodal Sensor Fusion for Autonomous Navigation on Miniaturized UAVs
METADATA ONLY
Loading...
Author / Producer
Date
2024-06-15
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Autonomously navigating robots need to perceive and interpret their surroundings. Currently, cameras are among the most used sensors due to their high resolution and frame rates at relatively low-energy consumption and cost. In recent years, cutting-edge sensors, such as miniaturized depth cameras, have demonstrated strong potential, specifically for nano-size unmanned aerial vehicles (UAVs), where low-power consumption, lightweight hardware, and low-computational demand are essential. However, cameras are limited to working under good lighting conditions, while depth cameras have a limited range. To maximize robustness, we propose to fuse a millimeter form factor 64 pixel depth sensor and a low-resolution grayscale camera. In this work, a nano-UAV learns to detect and fly through a gate with a lightweight autonomous navigation system based on two tinyML convolutional neural network models trained in simulation, running entirely onboard in 7.6 ms and with an accuracy above 91%. Field tests are based on the Crazyflie 2.1, featuring a total mass of 39 g. We demonstrate the robustness and potential of our navigation policy in multiple application scenarios, with a failure probability down to 1.2. 10-3 crash/meter, experiencing only two crashes on a cumulative flight distance of 1.7 km.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
11 (12)
Pages / Article No.
21372 - 21390
Publisher
IEEE
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Autonomous navigation; low-latency convolutional neural network (CNN); multimodal; sensor fusion; tinyML; unmanned aerial vehicle (UAV)
Organisational unit
03996 - Benini, Luca / Benini, Luca
Notes
Funding
207913 - TinyTrainer: On-chip Training for TinyML devices (SNF)