A built-in module allows the robot to imitate (in mirror fashion) the user's motion of the arms and the head.

Robota imitates in mirror fashion the user's upwards movement of the right arm

The camera image is divided into 9 areas to track
separately the head and the two arms.

TRACKING ALGORITHM

The tracking of the arms is based on optical flow detection, and the tracking of the head is based on a local template matching of the tip of the nose. This feature has symmetry and convexity properties, that ensure robust tracking during head rotation.

Principle: Humans and robots have different visual, tactile and auditory perceptions. To successfully transmit information, they must build a shared understanding of a vocabulary to designate the same events. This is achieved by reducing the number of features of the shared perceptual space; building, thus, a robust learning system that can handle various situations and noisy data.

The language game application on PocketPC was implemented on the mini humanoid robot Robota (Left) and on the 30 degrees of freedom Humanoid Robot DB (Right), developed by the Kawato Erato Project and part of the HRCN dept. at ATR.

Control architecture of the language acquisition game

Once the robot has correctly learned the meaning of a word, it can then execute the motors command associated with that word: hence, performing the correct action upon verbal command. The demonstrator plays a crucial role to constrain the situation, to reduce the learning space, and to provide a pragmatic feedback. By focussing the robot's attention on the relevant features of the environment, the amount of storage required for representation is reduced, and the speed of learning is increased.