Search
Results
-
It's Just Semantics: How to Get Robots to Understand the World the Way We Do
(2023)Springer Proceedings in Advanced Robotics ~ Robotics ResearchIncreasing robotic perception and action capabilities promise to bring us closer to agents that are effective for automating complex operations in human-centered environments. However, to achieve the degree of flexibility and ease of use needed to apply such agents to new and diverse tasks, representations are required for generalizable reasoning about conditions and effects of interactions, and as common ground for communicating with ...Conference Paper -
NeRFing it: Offline Object Segmentation Through Implicit Modeling
(2023)2023 IEEE International Conference on Robotics and Automation (ICRA)Most recently proposed methods for robotic per-ception are based on deep learning, which require very large datasets to perform well. The accuracy of a learned model is mainly dependent on the data distribution it was trained on. Thus for deploying such models, it is crucial to use training data belonging to the robot's environment. However, collecting and labeling data is a significant bottleneck, necessitating efficient data collection ...Conference Paper -
Neural Implicit Vision-Language Feature Fields
(2023)2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)Recently, groundbreaking results have been presented on open-vocabulary semantic image segmentation. Such methods segment each pixel in an image into arbitrary categories provided at run-time in the form of text prompts, as opposed to a fixed set of classes defined at training time. In this work, we present a zero-shot volumetric open-vocabulary semantic scene segmentation method. Our method builds on the insight that we can fuse image ...Conference Paper -
Baking in the Feature: Accelerating Volumetric Segmentation by Rendering Feature Maps
(2023)2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)Methods have recently been proposed that densely segment 3D volumes into classes using only color images and expert supervision in the form of sparse semantically annotated pixels. While impressive, these methods still require a relatively large amount of supervision and segmenting an object can take several minutes in practice. Such systems typically only optimize the representation on the scene they are fitting, without leveraging prior ...Conference Paper -
Material-agnostic Shaping of Granular Materials with Optimal Transport
(2023)2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)From construction materials, such as sand or asphalt, to kitchen ingredients, like rice, sugar, or salt; the world is full of granular materials. Despite impressive progress in robotic manipulation, manipulating and interacting with granular material remains a challenge due to difficulties in perceiving, representing, modelling, and planning for these variable materials that have complex internal dynamics. While some prior work has looked ...Conference Paper -
Multi-Agent Path Integral Control for Interaction-Aware Motion Planning in Urban Canals
(2023)2023 IEEE International Conference on Robotics and Automation (ICRA)Autonomous vehicles that operate in urban environments shall comply with existing rules and reason about the interactions with other decision-making agents. In this paper, we introduce a decentralized and communication-free interaction-aware motion planner and apply it to Autonomous Surface Vessels (ASVs) in urban canals. We build upon a sampling-based method, namely Model Predictive Path Integral control (MPPI), and employ it to, in each ...Conference Paper -
Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects
(2023)2023 IEEE International Conference on Robotics and Automation (ICRA)Interactions with articulated objects are a challenging but important task for mobile robots. To tackle this challenge, we propose a novel closed-loop control pipeline, which integrates manipulation priors from affordance estimation with sampling-based whole-body control. We introduce the concept of agent-aware affordances which fully reflect the agent's capabilities and embodiment and we show that they outperform their state-of-the-art ...Conference Paper -
Sampling-free obstacle gradients and reactive planning in Neural Radiance Fields
(2022)This work investigates the use of Neural implicit representations, specifically Neural Radiance Fields (NeRF), for geometrical queries and motion planning. We show that by adding the capacity to infer occupancy in a radius to a pre trained NeRF we are effectively learning an approximation to a Euclidean Signed Distance Field (ESDF). Even more, using backward differentiation of the network, we readily obtain the obstacle gradients that are ...Conference Paper -
Conditioned deep feature consistent variational autoencoder for simulating realistic sonar images
(2022)OCEANS 2022, Hampton RoadsMultibeam imaging sonar is one of the primary sensors for underwater navigation with uncrewed underwater vehicles (UUVs) due to the robustness to turbidity and variable lighting conditions that limit the applicability of standard cameras. However, the operating principles and noise models of real sensors make imaging sonar challenging to accurately simulate, and acquiring real images experimentally is difficult and costly. This paper ...Conference Paper -
FlowBot: Flow-based Modeling for Robot Navigation
(2022)2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)Autonomous navigation among people is a complex problem that also exhibits considerable variation depending on the type of environment and people involved. Here we consider navigation among crowds that exhibit flow-like behavior like people moving through a train station. We propose a novel pseudo-fluid model of crowd flow for such problems. These have an intuitive physical interpretation and do not require much tuning. We further formalize ...Conference Paper