english only
Interference Game
Introduction
Along the aim of designing robots which are more and more similar to humans, and also capable of social interaction (Billard et al., 2006, Asada et al., 2001), we present here a biologically inspired approach to multimodal sensory integration and decision making. Our research activities try to link together robotics with neuroscience and human psychology, in order to understand cognitive abilities such as learning by imitation (Arbib et al., 2000, Sauser et al., 2006). Along these lines, we shall address here the principle of ideomotor compatibility, which states that observing the movements of others influences the quality of one's own performance (Brass et al., 2000). Experimental data show that a close similarity between observed and executed movements produces a positive effect on the quality and initiation of the executed movement, whereas a difference between observed and executed actions generates interference.
|
Related behavioral experiment We consider the following experiment, as illustraed in Figure 1. An experimenter and the subject sit opposite to each other at a table. A blue and a red object are placed upon the table, at an equal distance from each of the two people. After a small initial waiting period, the experimenter names the color of one of the objects (red or blue), and also reaches for one of the two objects. However the color named and the object reached for may not necessarily be the same. The subject is asked to reach for the object which is reached for by the experimenter, as quickly as possible and with maximum accuracy. As can be seen, when the experimenter reaches for a different object than the one named, i.e. the gesture and the speech are incongruent, the subject either fails to reach for the correct object, or hesitates and later corrects the movement. Moreover, when a subject makes no error, the average reaction time is significantly faster in the case of a congruent (same gesture as speech) demonstration than in the incongruent case. |
Figure 1: A human subject is performing a stimulus-response task (see text for more details). At the top the named color and the target of the reach coincide, at the bottom they do not. A hesitating movement can be observed in the second case. |
Robotic Set-Up
The experiments are performed using a Fujitsu humanoid robot, HOAP2. Two webcams track the location of colored objects and a standard microphone is placed in close proximity to the robot. The robotic experimental paradigm is exactly the same as that used in the human experiments, except that it was adapted to the small size of the robot.
Model Architecture
We present the architecture of the neural network which controling sensory integration and decisional processes. It is based on the dynamic neural fields approach (Erlhagen et al. 2002), known to be a robust and smooth neural mechanism. The hypotheses behind this framework assumes a continuous encoding of the sensorimotor space, and also, as suggested by neuroscientific theories, the interactions of multiple information sources are performed in a locally common space (Arbib et al., 2000, Cisek et al., 2005). The system's architecture, as depicted in Figure 2, can be separated into three major parts: a) sensory interfaces, b) the decisional network, and c) motor systems.
|
Figure 2: Model architecture |
Results
The resulting robot behavior, two trials are depicted in Figure 3. The trials correspond to those presented earlier in an equivalent experiment between a human experimenter and a human subject (Fig. 1). It can observed that the robot behaves similarly to a human, in both congruent and incongruent conditions. It even makes mistakes, in a similar way to the human subject. We now consider the activity patterns produced by the network during such a task. In Figure 4, we see typical raster plots of the subnetworks' activity. We have displayed an incongruent trial as it represents the most interesting case. Initially, the two objects are on the table, which triggers an activation of the two object representations. This then leads to a small, but combined activation of the motor preparatory subnetwork.
|
Figure 3: The experiment depicted in Figure 1 was reproduced with the robot. Similar behavioral characteristics can be observed. |
Figure 4: Illustration of the neural field activity patterns during a trial consisting of an incongruent condition. a) The experimenter names the red word while simultaneously reaching for the blue object. At b) the movement execution threshold is attained. The robot is mistaken and initiates a reaching toward the wrong target. c) It finally corrects its decision and moves toward the right object. |
Discussion
We proposed here a biologically inspired approach to multimodal integration and decision making. This work is part of a multidisciplinary research project, in which we attempt to understand and model the cognitive mechanisms responsible for the human ability to imitate (Arbib et al., 2000, Sauser et al., 2006). Using a constructivist approach, we also attempt to ground our models in real physical robots.
Our system consists mainly of an ensemble of neural networks known as neural fields, which possess several dynamical properties useful for sensorimotor integration (Erlhagen et al., 2002, Sauser et al., 2006). These networks allow other modalities to be added to our system, but at the cost of finding a relevant common representation. This issue is currently overcome by looking at the solution provided by human behavioral data and its neurophysiology. Indeed, similarly to the study of another model, also replicating experimental data (Sauser et al. 2006), the architecture was successful in reproducing, in a robot, an experiment originally designed to measure the ideomotor effect in humans.
Acknowledgments This work was supported by the Swiss National Science Foundation, through grant no 620-066127 of the SNF Professorships program.
References
E. L. Sauser and Aude G. Billard, Parallel and Distributed Neural Models of the Ideomotor Principle: An Investigation of Imitative Cortical Pathways, Neural Networks, Vol. 19(3), Special Issue on Brain Mechanisms of Imitation Learning, 2006.
M. Brass, H. Bekkering, A. Wohlschlager and W. Prinz, Compatibility between observed and executed finger movements: comparing symbolic, spatial and imitative cues, Brain and Cognition, Vol. 44, pp. 124-143, 2000.
A. Billard, S. Calinon and F. Guenter, Discriminative and Adaptive Imitation in Uni-Manual and Bi-Manual Tasks. Robotics and Autonomous Systems. In press, 2006.
W. Erlhagen and G. Schoner, Dynamics field theory of movement preparation, Psychological Review, Vol. 109, pp. 545-572, 2002.
M. Asada, K. F. McDorman, H. Ishiguro and Y. Kuniyoshi, Cognitive developmental robotics as a new paradigm for the design of humanoids robots, Robotics and Autonomous Systems, Vol. 37, pp. 185-193, 2001.
M. Arbib, A. Billard, M. Iacoboni and E. Oztop, Mirror neurons, imitation and (synthetic) brain imaging, Neural Networks, Vol. 13, pp. 953-973, 2000.
P. Cisek and J. F. Kalaska, Neural correlates of reaching decision in dorsal premotor cortex: specification of multiple direction choices and final selection of action, {\em Neuron}, Vol. 45, pp. 801-814, 2005.
Videos
Selected publications
People involved in this project
