Human-Robot Dynamic Social Interaction

NTT9904-01

Progress Report: January 1, 2002—June 30, 2002

Rodney Brooks

Project Overview

NTT researchers are interested in the question of whether a physical robot produces a more direct emotional coupling with human beings than does a computer generated graphical image of a similar robot. At MIT we are building a robot that has human-like facial expressions and shoulder and neck gestures, and that perceives human motion and facial expressions. This is coupled to an emotional system so that the person and the robot naturally follow normal human communication social dynamics. This robot will be installed at the NTT Communications Science Laboratories in Kyoto where the response of human subjects will be measured and compared to their response to a graphical interface.

Progress Through June 2002

In 2000 and 2001 we delivered a preliminary robot to NTT, updated the design of the ultimate robot, called Kismet, resolved complex mechanical issues surrounding Kismet and fabricated Kismet components. Software infrastructure for Kismet was developed. It included a QNX-based control system layer and a platform independent set of vision libraries with improved robustness.

In the first half of 2002 Kismet was completely assembled and implemented then transferred to CSL in Kyoto. Kismet is completely operational in its 30 degrees of freedom. It runs integrated motor control, basic vision processing and emotional display. It is hand calibrated at start up then initialized from a basic graphical user interface that allows precise control of each axis independently plus coordination of multiple axes. All motors are position controlled. The vision system has an attentional subsystem that detects skin color and saturated color. Environmental stimuli with human skin color or high saturated color are identified as salient. The eyes and head are continuously directed to the salient region of the visual image stream. The robot, for demonstration purposes, has 5 basic facial expressions of emotion: neutral, sad, happy, surprised, and afraid. It is able to move through the complete range of its emotional axes thus generating intermediate expressions.

Two researchers: Kazuhiko Shinozawa and Futoshi Naya from CSL visited MIT in June, 2002 for two weeks. The primary goal of the trip was knowledge transfer concerning the operation and maintenance plus hardware and software architecture of Kismet-3. Dr. Una-May O’Reilly, Lijin Aryananda and Jeff Weber teamed with the visitors to familiarize them with trouble shooting, mechanical operations and software control. During the visit the robot was frequently run for hours at a time and its software and hardware were exercised to confirm sufficient reliability and robustness. The NTT visitors were able to acquaint group members with experiments currently being conducted at CSL.

Research Plan for the Next Six Months

Members of CSL will familiarize, customize and extend the capabilities of Kismet in order to use it in experimental situations. MIT will provide long distance advice and on-site visits from members of our staff who have either mechanical expertise or computational expertise. This collective effort will facilitate possible refinements that CSL may identify for Kismet. MIT will be researching, on robots similar to Kismet, the commanding of robots by voice and the design and implementation of an ego-sphere for humanoid robots.