Learning Human-to-Robot Handovers from Point Clouds

Sammy Christen, Wei Yang, Claudia Pérez-D’Arpino, Otmar Hilliges, Dieter Fox, Yu-Wei Chao

CVPR 2023 pp. 9654-9664

doi:10.1109/CVPR52729.2023.00931 /cvpr/2023/christen2023cvpr-learning/

Abstract

We propose the first framework to learn control policies for vision-based human-to-robot handovers, a critical task for human-robot interaction. While research in Embodied AI has made significant progress in training robot agents in simulated environments, interacting with humans remains challenging due to the difficulties of simulating humans. Fortunately, recent research has developed realistic simulated environments for human-to-robot handovers. Leveraging this result, we introduce a method that is trained with a human-in-the-loop via a two-stage teacher-student framework that uses motion and grasp planning, reinforcement learning, and self-supervision. We show significant performance gains over baselines on a simulation benchmark, sim-to-sim transfer and sim-to-real transfer.

PDF CVPR Semantic Scholar

Cite

Text

Christen et al. "Learning Human-to-Robot Handovers from Point Clouds." Conference on Computer Vision and Pattern Recognition, 2023. doi:10.1109/CVPR52729.2023.00931

Markdown

[Christen et al. "Learning Human-to-Robot Handovers from Point Clouds." Conference on Computer Vision and Pattern Recognition, 2023.](https://mlanthology.org/cvpr/2023/christen2023cvpr-learning/) doi:10.1109/CVPR52729.2023.00931

BibTeX

@inproceedings{christen2023cvpr-learning,
  title     = {{Learning Human-to-Robot Handovers from Point Clouds}},
  author    = {Christen, Sammy and Yang, Wei and Pérez-D’Arpino, Claudia and Hilliges, Otmar and Fox, Dieter and Chao, Yu-Wei},
  booktitle = {Conference on Computer Vision and Pattern Recognition},
  year      = {2023},
  pages     = {9654-9664},
  doi       = {10.1109/CVPR52729.2023.00931},
  url       = {https://mlanthology.org/cvpr/2023/christen2023cvpr-learning/}
}