Teleassistance: Contextual Guidance for Autonomous Manipulation

Abstract

We present teleassistance, a two-tiered control structure for robotic manipulation that combines the advantages of autonomy and teleoperation. At the top level, a teleoperator provides global, deictic references via a natural sign language. Each sign indicates the next action to perform and a relative and hand-centered coordinate frame in which to perform it. For example, the teleoperator may point to an object for reaching, or preshape the hand for grasping. At the lower level autonomous servo routines run within the reference frames provided. Teleassistance offers two benefits. First, the servo routines can position the robot in relative coordinates and interpret feedback within a constrained context. This significantly simplifies the computational load of the autonomous routines and requires only a sparse model of the task. Second, the operator’s actions are symbolic, conveying intent without requiring the person to literally control the robot. This helps to alleviate many of the problems inherent to teleoperation, including poor mappings between operator and robot physiology, reliance on a broad communication bandwidth, and the potential for robot damage when solely under remote control. To demonstrate the concept, a Utah/MIT hand mounted on a Puma 760 arm opens a door.