The framework
 Kismet home
 The robot
 Ongoing research
 Broader questions

 Facial expression
 Visual attention
 Ocular-motor control
 Low-level features
 Expressive speech
 Affective intent

Kismet is an autonomous robot designed for social interactions with humans. In general, social robotics has concentrated on groups of robots performing behaviors such as flocking, foraging or dispersion, or on paired robot-robot interactions such as imitation. This project focuses not on robot-robot interactions, but rather on the construction of robots that engage in meaningful social exchanges with humans. By doing so, it is possible to have a socially sophisticated human assist the robot in acquiring more sophisticated communication skills and helping it learn the meaning these acts have for others. Our approach is inspired by the way infants learn to communicate with adults. Specifically, the mode of social interaction is that of a caretaker-infant dyad where a human acts as the caretaker for the robot.

Here is a simplified view of Kismet's design. For more information, see Cynthia Breazeal's thesis. You may click on some of the modules in the figure below for more information about them.

behavior system homeostatic regulation emotion system vocal acts face and body postures orient head and eyes attention system low level feature extraction higher level perceptual system

The system architecture consists of six subsystems: the low-level feature extraction system, the high-level perception system, the attention system, the motivation system, the behavior system, and the motor system. The low-level feature extraction system extracts sensor-based features from the world, and the high-level perceptual system encapsulates these features into percepts that can influence behavior, motivation, and motor processes. The attention system determines what the most salient and relevant stimulus of the environment is at any time so that the robot can organize its behavior about it. The motivation system regulates and maintains the robot's state of ``well being" in the form of homeostatic regulation processes and emotive responses. The behavior system implements and arbitrates between competing behaviors. The winning behavior defines the current task (i.e., the goal) of the robot. The robot has many behaviors in its repertoire, and several motivations to satiate, so its goals vary over time. The motor system carries out these goals by orchestrating the output modalities (actuator or vocal) to achieve them. For Kismet, these actions are realized as motor skills that accomplish the task physically, or expressive motor acts that accomplish the task via social signals.

The Low-Level Feature Extraction System

The low-level feature extraction system is responsible for processing the raw sensory information into quantities that have behavioral significance for the robot. The routines are designed to be cheap, fast, and just adequate. Of particular interest are those perceptual cues that infants seem to rely on. For instance, visual and auditory cues such as detecting eyes and the recognition of vocal affect are important for infants.

The Attention System

The low-level visual percepts are sent to the attention system. The purpose of the attention system is to pick out low-level perceptual stimuli that are particularly salient or relevant at that time, and to direct the robot's attention and gaze toward them. This provides the robot with a locus of attention that it can use to organize its behavior. A perceptual stimulus may be salient for several reasons. It may capture the robot's attention because of its sudden appearance, or perhaps due to its sudden change. It may stand out because of its inherent saliency such as a red ball may stand out from the background. Or perhaps its quality has special behavioral significance for the robot such as being a typical indication of danger.

The Perceptual System

The low-level features corresponding to the target stimuli of the attention system are fed into the perceptual system. Here they are encapsulated into behaviorally relevant percepts. To environmentally elicit processes in these systems, each behavior and emotive response has an associated releaser. As conceptualized by Tinbergen and Lorenz, a releaser can be viewed as a collection of feature detectors that are minimally necessary to identify a particular object or event of behavioral significance. The function of the releasers is to ascertain if all environmental (perceptual) conditions are right for the response to become active.

The Motivation System

The motivation system consists of the robot's basic ``drives'' and ``emotions''. The ``drives'' represent the basic ``needs'' of the robot and are modeled as simple homeostatic regulation mechanisms. When the needs of the robot are being adequately met, the intensity level of each ``drive'' is within a desired regime. However, as the intensity level moves farther away from the homeostatic regime, the robot becomes more strongly motivated to engage in behaviors that restore that ``drive''. Hence the ``drives'' largely establish the robot's own agenda, and play a significant role in determining which behavior(s) the robot activates at any one time. The ``emotions'' are modeled from a functional perspective. Based on simple appraisals of the benefit or detriment of a given stimulus, the robot evokes positive emotive responses that serve to bring itself closer to it, or negative emotive responses in order to withdraw from it. There is a distinct emotive response for each class of eliciting conditions. Currently, six basic emotions are modeled that give the robot synthetic analogs of anger, disgust, fear, joy, sorrow, and surprise (after Ekman). There are also arousal-based responses that correspond to interest, calm, and boredom that are modeled in a similar way. The expression of emotive responses promotes empathy from the caregiver and plays an important role in regulating social interaction with the human.

The Behavior System

The behavior system organizes the robot's task-based behaviors into a coherent structure. Each behavior is viewed as a self-interested, goal-directed entity that competes with other behaviors to establish the current task. An arbitration mechanism is required to determine which behavior(s) to activate and for how long, given that the robot has several motivations that it must tend to and different behaviors that it can use to achieve them. The main responsibility of the behavior system is to carry out this arbitration. In particular, it addresses the issues of relevancy, coherency, persistence, and opportunism. By doing so, the robot is able to behave in a sensible manner in a complex and dynamic environment.

The Motor System

The motor system arbitrates the robot's motor skills and expressions. It consists of four subsystems: the motor skills system, the facial animation system, the expressive vocalization system, and the oculo-motor system. Given that a particular goal and behavioral strategy have been selected, the motor system determines how to move the robot so as to carry out that course of action. Overall, the motor skills system coordinates body posture, gaze direction, vocalizations, and facial expressions to address issues of blending and sequencing the action primitives from the specialized motor systems.

Other topics
Kismet's hardware
Facial expression
Visual attention
Ocular-motor control
Low-level features
Expressive speech
Affective intent in speech
Homeostatic regulation mechanisms
The behavior system


    contact information: