Dongho Kang

Last Name

Kang

First Name

Dongho

ORCID

0000-0003-2512-5849

Show all metadata

Search Results

Publications 1 - 10 of 13

RL + Model-Based Control: Using On-Demand Optimal Control to Learn Versatile Legged Locomotion
Kang, Dongho; Cheng, Jin; Zamora, Miguel; et al. (2023)
IEEE Robotics and Automation Letters
This paper presents a control framework that combines model-based optimal control and reinforcement learning (RL) to achieve versatile and robust legged locomotion. Our approach enhances the RL training process by incorporating on-demand reference motions generated through finite-horizon optimal control, covering a broad range of velocities and gaits. These reference motions serve as targets for the RL policy to imitate, leading to the development of robust control policies that can be learned with reliability. Furthermore, by utilizing realistic simulation data that captures whole-body dynamics, RL effectively overcomes the inherent limitations in reference motions imposed by modeling simplifications. We validate the robustness and controllability of the RL training process within our framework through a series of experiments. In these experiments, our method showcases its capability to generalize reference motions and effectively handle more complex locomotion tasks that may pose challenges for the simplified model, thanks to RL's flexibility. Additionally, our framework effortlessly supports the training of control policies for robots with diverse dimensions, eliminating the necessity for robot-specific adjustments in the reward function and hyperparameters.
Animal Motions on Legged Robots Using Nonlinear Model Predictive Control
Kang, Dongho; De Vincenti, Flavio; Adam, Naomi C.; et al. (2022)
2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
This work presents a motion capture-driven locomotion controller for quadrupedal robots that replicates the non-periodic footsteps and subtle body movement of animal motions. We adopt a nonlinear model predictive control (NMPC) formulation that generates optimal base trajectories and stepping locations. By optimizing both footholds and base trajectories, our controller effectively tracks retargeted animal motions with natural body movements and highly irregular strides. We demonstrate our approach with prerecorded animal motion capture data. In simulation and hardware experiments, our motion controller enables quadrupedal robots to robustly reproduce fundamental characteristics of a target animal motion regardless of the significant morphological disparity.
End-to-End Collision Avoidance from Depth Input with Memory-based Deep Reinforcement Learning
Kang, Dongho (2019)
The main goal of this work is learning a local path planning policy for mobile robots from a single depth camera input. We formulate the end-to-end local planning problem as a Partially Observable Markov Decision Process and solve it using a Deep Reinforcement Learning algorithm. The main challenges of this setting comes from 1) the short-sightedness of reaction-based planners, and 2) the limited field-of-view of depth camera that significantly degrades the planner’s performance. We resolve these problems by memory-based Deep Reinforcement Learning. This framework represents a policy as a network with a memory unit that can remember past observations. As a result, the trained policy can generate collision-safe trajectories based on not only a current observation but also previous observations. We also address sample ineciency of end-to-end learning by 1) a two-stream feature extraction with pre-trained autoencoder and 2) Asymmetric Actor-Critic method. These methods were demonstrated to be effective for fast convergence by our ablation study results. Finally we bridge the reality gap between real depth image and simulated depth image by real-time depth completion algorithm and pre-training autoencoder with both real images and simulate images. In the quantitative evaluation, our policy with memory units outperforms standard CNN policy. Notably, the policy with Temporal Convolutional layers learned much faster than the policy with conventional LSTM. In the following real robot experiments, we deployed the trained policy to the quadrupedal robot ANYmal with Intel RealSense depth camera. Our policy generated collision-safe paths reactively in both stationary and dynamic environments.
Deep Compliant Control for Legged Robots
Hartmann, Adrian; Kang, Dongho; Zargarbashi, Fatemeh; et al. (2024)
2024 IEEE International Conference on Robotics and Automation (ICRA)
Control policies trained using deep reinforcement learning often generate stiff, high-frequency motions in response to unexpected disturbances. To promote more natural and compliant balance recovery strategies, we propose a simple modification to the typical reinforcement learning training process. Our key insight is that stiff responses to perturbations are due to an agent’s incentive to maximize task rewards at all times, even as perturbations are being applied. As an alternative, we introduce an explicit recovery stage where tracking rewards are given irrespective of the motions generated by the control policy. This allows agents a chance to gradually recover from disturbances before attempting to carry out their main tasks. Through an in-depth analysis, we highlight both the compliant nature of the resulting control policies, as well as the benefits that compliance brings to legged locomotion. In our simulation and hardware experiments, the compliant policy achieves more robust, energy-efficient, and safe interactions with the environment.
Control-Aware Design Optimization for Bio-Inspired Quadruped Robots
De Vincenti, Flavio; Kang, Dongho; Coros, Stelian (2021)
2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
We present a control-aware design optimization method for quadrupedal robots. In particular, we show that it is possible to analytically differentiate typical, inverse dynamics-based whole body controllers with respect to design parameters, and that gradient-based methods can be used to efficiently improve an initial morphological design according to well-established metrics. We apply our design optimization method to various types of quadrupedal robots, including designs that feature closed kinematic chains. The methodology we present enables a principled comparison of different types of optimized legged robot designs. Our experiments, for example, suggest that mechanically-coupled three-link leg designs present notable advantages in terms of performance and efficiency over the common two-link leg designs used in most quadrupedal robots today.
RobotKeyframing: Learning Locomotion with High-Level Objectives via Mixture of Dense and Sparse Rewards
Zargarbashi, Fatemeh; Cheng, Jin; Kang, Dongho; et al. (2025)
Proceedings of Machine Learning Research ~ Proceedings of The 8th Conference on Robot Learning
This paper presents a novel learning-based control framework that uses keyframing to incorporate high-level objectives in natural locomotion for legged robots. These high-level objectives are specified as a variable number of partial or complete pose targets that are spaced arbitrarily in time. Our proposed framework utilizes a multi-critic reinforcement learning algorithm to effectively handle the mixture of dense and sparse rewards. Additionally, it employs a transformer-based encoder to accommodate a variable number of input targets, each associated with specific time-to-arrivals. Throughout simulation and hardware experiments, we demonstrate that our framework can effectively satisfy the target keyframe sequence at the required times. In the experiments, the multi-critic method significantly reduces the effort of hyperparameter tuning compared to the standard single-critic alternative. Moreover, the proposed transformer-based architecture enables robots to anticipate future goals, which results in quantitative improvements in their ability to reach their targets.
Tuning Legged Locomotion Controllers via Safe Bayesian Optimization
Widmer, Daniel; Kang, Dongho; Sukhija, Bhavya; et al. (2023)
Proceedings of Machine Learning Research ~ Proceedings of The 7th Conference on Robot Learning
This paper presents a data-driven strategy to streamline the deployment of model-based controllers in legged robotic hardware platforms. Our approach leverages a model-free safe learning algorithm to automate the tuning of control gains, addressing the mismatch between the simplified model used in the control formulation and the real system. This method substantially mitigates the risk of hazardous interactions with the robot by sample-efficiently optimizing parameters within a probably safe region. Additionally, we extend the applicability of our approach to incorporate the different gait parameters as contexts, leading to a safe, sample-efficient exploration algorithm capable of tuning a motion controller for diverse gait patterns. We validate our method through simulation and hardware experiments, where we demonstrate that the algorithm obtains superior performance on tuning a model-based motion controller for multiple gaits safely.
Multi-Domain Motion Embedding: Expressive Real-Time Mimicry for Legged Robots
Heyrman, Matthias; Li, Chenhao; Klemm, Victor; et al. (2025)
arXiv
Effective motion representation is crucial for enabling robots to imitate expressive behaviors in real time, yet existing motion controllers often ignore inherent patterns in motion. Previous efforts in representation learning do not attempt to jointly capture structured periodic patterns and irregular variations in human and animal movement. To address this, we present Multi-Domain Motion Embedding (MDME), a motion representation that unifies the embedding of structured and unstructured features using a wavelet-based encoder and a probabilistic embedding in parallel. This produces a rich representation of reference motions from a minimal input set, enabling improved generalization across diverse motion styles and morphologies. We evaluate MDME on retargeting-free real-time motion imitation by conditioning robot control policies on the learned embeddings, demonstrating accurate reproduction of complex trajectories on both humanoid and quadruped platforms. Our comparative studies confirm that MDME outperforms prior approaches in reconstruction fidelity and generalizability to unseen motions. Furthermore, we demonstrate that MDME can reproduce novel motion styles in real-time through zero-shot deployment, eliminating the need for task-specific tuning or online retargeting. These results position MDME as a generalizable and structure-aware foundation for scalable real-time robot imitation.
Animal Gaits on Quadrupedal Robots Using Motion Matching and Model-Based Control
Kang, Dongho; Zimmermann, Simon; Coros, Stelian (2021)
2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
In this paper, we explore the challenge of generating animal-like walking motions for legged robots. To this end, we propose a versatile and robust control pipeline that combines a state-of-the-art model-based controller with a data-driven technique that is commonly used in computer animation. We demonstrate the efficacy of our control framework on a variety of quadrupedal robots in simulation. We show, in particular, that our approach can automatically reproduce key characteristics of animal motions, including speed-specific gaits, unscripted footfall patterns for nonperiodic motions, and natural small variations in overall body movements.
Spatio-Temporal Motion Retargeting for Quadruped Robots
Yoon, Taerim; Kang, Dongho; Kim, Seungmin; et al. (2025)
IEEE Transactions on Robotics
This work presents a motion retargeting approach for legged robots, aimed at transferring the dynamic and agile movements to robots from source motions. In particular, we guide the imitation learning procedures by transferring motions from source to target, effectively bridging the morphological disparities while ensuring the physical feasibility of the target system. In the first stage, we focus on motion retargeting at the kinematic level by generating kinematically feasible whole-body motions from keypoint trajectories. Following this, we refine the motion at the dynamic level by adjusting it in the temporal domain while adhering to physical constraints. This process facilitates policy training via reinforcement learning, enabling precise and robust motion tracking. We demonstrate that our approach successfully transforms noisy motion sources, such as hand-held camera videos, into robot-specific motions that align with the morphology and physical properties of the target robots. Moreover, we demonstrate terrain-aware motion retargeting to perform BackFlip on top of a box. We successfully deployed these skills to four robots with different dimensions and physical properties in the real world through hardware experiments.

Publications 1 - 10 of 13

Dongho Kang

Last Name

First Name

ORCID

Organisational unit

Filters

Search Results