Learning Occluded Branch Depth Maps in Forest Environments Using RGB-D Images
Abstract
Covering over a third of all terrestrial land area, forests are crucial environments; as ecosystems, for farming, and for human leisure. However, they are challenging to access for environmental monitoring, for agricultural uses, and for search and rescue applications. To enter, aerial robots need to fly through dense vegetation, where foliage can be pushed aside, but occluded branches pose critical obstacles. Therefore, we propose pixel-wise depth regression of occluded branches using three different U-Net inspired architectures. Given RGB-D input of trees with partially occluded branches, the models estimate depth values of only the wooden parts of the tree. A large photorealistic simulation dataset comprising around 44 K images of nine different tree species is generated, on which the models are trained. Extensive evaluation and analysis of the models on this dataset is shown. To improve network generalization to real-world data, different data augmentation and transformation techniques are performed. The approaches are then also successfully demonstrated on real-world data of broadleaf trees from Swiss temperate forests and a tropical Masoala Rainforest. This work showcases the previously unexplored task of frame-by-frame pixel-based occluded branch depth reconstruction to facilitate robot traversal of forest environments. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000659145Publication status
publishedExternal links
Journal / series
IEEE Robotics and Automation LettersVolume
Pages / Article No.
Publisher
IEEESubject
Deep learning for visual perception; Robotics and automation in agriculture and forestry; RGB-D perceptionOrganisational unit
09718 - Mintchev, Stefano / Mintchev, Stefano
Funding
186865 - CYbER - CanopY Exploration Robots (SNF)
Related publications and datasets
Is supplemented by: https://doi.org/10.3929/ethz-b-000634419
More
Show all metadata
ETH Bibliography
yes
Altmetrics