Learning the sense of touch in simulation: a sim-to-real strategy for vision-based tactile sensing
- Conference Paper
Data-driven approaches to tactile sensing aim to overcome the complexity of accurately modeling contact with soft materials. However, their widespread adoption is impaired by concerns about data efficiency and the capability to generalize when applied to various tasks. This paper focuses on both these aspects with regard to a vision-based tactile sensor, which aims to reconstruct the distribution of the three- dimensional contact forces applied on its soft surface. Accurate models for the soft materials and the camera projection, derived via state-of-the-art techniques in the respective domains, are employed to generate a dataset in simulation. A strategy is proposed to train a tailored deep neural network entirely from the simulation data. The resulting learning architecture is directly transferable across multiple tactile sensors without further training and yields accurate predictions on real data, while showing promising generalization capabilities to unseen contact conditions. © 2020 IEEE. Show more
Book title2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Pages / Article No.
Organisational unit03758 - D'Andrea, Raffaello / D'Andrea, Raffaello
NotesDue to the Coronavirus (COVID-19) the conference was conducted virtually.
MoreShow all metadata