There has been much machine vision work done involving many different visual modules. However, most of this work has been applied to slow or stationary unintegrated systems with less complex motility requirements. We will adapt as much of this previous work to our system as is feasible, such as elements of vergence and object tracking. Furthermore, we will integrate formerly tested VOR and sensory-motor mapping algorithms. Currently we have initial original versions of the face detection and gaze stabilization algorithms and a working motion tracking algorithm.
macaco closeup macaco front view
A careful selection of the hardware and a mechanical design of the head have also been finalized. Each vision module is first being separately developed. Once we have working versions of these programs, they will be combined into the unified sensor-actuation-vision system of the head and neck. Refinement of this stage will then enable us to integrate this composite with the body:
  • Image Acquisition — The vision software is invariably linked to the optical hardware, which exploits biological binocular apparatus, each eye consisting of one B/W camera for wide range image acquisition at low resolution and one foveal color camera for narrow fields of view. The continuous input to these sensors must be analyzed with computationally efficient and synchronous algorithms so that veridical assessments can be made as the robot navigates through its environment.
  • Attention Mechanisms — The success of a convincing intelligent robot is greatly gauged by its putative social skills. Thus, all of the tracking mechanisms previously illustrated will be of maximal value when they contribute to the robot's capacities for human interaction, inference, and attention.
  • Stabilization Algorithm — Consists in stabilizing the head and cameras while the robot is in movement, through the use of inertial sensors. Needs to be executed in less than 10 ms. The kinematics and dynamic study of the head are concluded, and experimental results were obtained from simulation.
  • Vergence, Smooth Pursuit and Saccades — These three algorithms are tightly related. An approach based on active vision merges these three behaviors, running at 30 Hz.
  • Obstacle Detection — Should run at 15 Hz. Obstacles may be detected using features saliency, such as motion, form and color saliency.
  • People Detection — Will run only on regions where objects were detected. The detection of obstacles and people will be done on the small resolution images. Face detection will be executed whenever people are detected, and people gaze direction will be extracted from the high-resolution color images.
  • Human Gaze Direction — Provides cues for robots head gazing. This module is important for the robot to infer in which direction people are looking and to what are they paying attention.
  • Matching — Corresponds to finding the correspondences between small and high-resolution images.
  • Rough Terrain and Slope Detection — Operate at small frequencies, namely ~ 8Hz (but might run at ~ 2Hz if necessary due to computational requirements). Visual information will be fused with filtered inertial information. The Fast Terrain Navigation module will run at 15 Hz and provide rough estimatives for save directions of movement.

One approach of AI is the development of robots whose embodiment and situatedness in the world evoke behaviors that obviate constant human supervision. With this in mind, M4 was designed to walking, run, and turn in unstructured environments. Thus, its vision system must comprise methods of assessing the constantly changing terrain. Navigation for obstacle and potentially treacherous landscape avoidance, slope detection, and gaze stabilization are all requisites for such competence. Furthermore, in order to be a convincing social participant, the vision system will also allow for person detection and inference of human gaze direction. All of these vision modules should be accessible to each other and concurrent with other sensor input from the rest of the body.

motivation page
[Active Vision Head], [Current Research], [Head Designs], [Humanoid Robotics Group Home], [Macaco Home],
[Motivation], [Overview], [People], [Publications], [Video]

webmaster: annika