Illustration of the need of an other-centered representation of movements. Each observer in green imitates the demonstrator in red. On the left, a viewer-centered representation leads to a bad reproduction. However, on the right, a common demonstrator-centered frame of reference allows each demonstrator to perfectly reproduce the observed movement.

The frame of reference problem

Biological motivations

First, the discovery in the monkey brain of the mirror neurons system (MNS) has suggested the existence of a direct-mapping mechanism between visual and motor systems. This link between action observation with self motor execution underlies a strong need for a common visuomotor representation, a common frame of reference (FR). Moreover, we know from neurophysiology, that along the ventral visual pathway, the information flows from the primary visual cortex to the superior temporal sulcus (STS). This region contains populations of neurons that separately exhibit sensitivity to a variety of body parts, and also to their locations, sizes and orientations relative to a viewer, object or goal-centered FR. Therefore, being indirectly connected to the MNS, STS appears clearly to be a candidate where this viewer to body-centered transformation occurs. The model presented here, proposes a biologically plausible mechanism of how such transformation might be performed in this brain region.

The model

The model we propose, exploits the population vector coding paradigm to represent the vectorial basis of the referentials inolved in the transformation. We consider a population as an ensemble of neurons whose distributed firing activities are correlated to a single macroscopic quantity that is a vector in a given frame of reference. The populations are encoded using attractor networks that have several interesting dynamic properties such as gain modulation and robustness to noise. These mechanisms allow multiple sources of information to merge into gain fields and then to produce non-linear transformations.

In order to build a neural model of a body-centered frame of reference, we propose the hypothesis that orientation sensitive cells in the visual area STS are grouped into populations that encode the principal axes of observed bodies, as it remains the most natural representation for three dimensional frames of reference. Therefore, we consider three distinct populations of neurons for coding separately the three principal axes of an observed body. As such, these groups of neurons can form a basis, in the vectorial sense, of a body-centered frame of reference. Despite there is as yet no clear evidence to support our model's hypothesis, this principle is consistent with current neurophysiological data. Indeed, to our knowledge, no systematic experiment have shown a complete description of single cell sensitivity to all possible orientations.

Video of the model dynamics

Robotic implementation

We implemented this network in a humanoid robot. The visual inputs are given by a color-based stereo vision system that allows the simultaneous 3D tracking of a human demonstrator's hand, body and principal axes. These information are fed into the neural network that computes the target location in the demonstrator's body centered reference frame. It is then directly mapped to the robot egocentric frame of reference, so that it can immediately imitate the human's hand trajectory using a classical inverse kinematic algorithm. As illustrated here on the right, the robot is able to mimic gestures shown by a human demonstrator that is not always perfectly facing the cameras.