This work aims at extracting the relevant features of a given task in a Programming by Demonstration (PbD) framework.

It addresses the problem of generalizing the acquired knowledge to different contexts. A probability based estimation of the relevance is suggested, by first projecting the joint angles, hand paths, and object-hand trajectories onto a generic latent space using either:

• Principal Component Analysis (PCA)
• Independent Component Analysis (ICA)
• Canonical Correlation Analysis (CCA)

The resulting signals are then encoded using a mixture of Gaussian/Bernoulli distributions (GMM/BMM). This provides a measure of the spatio-temporal correlations across the different modalities collected from the robot which can be used to determine a metric of the imitation performance.

The trajectories are then generalized using Gaussian Mixture Regression (GMR), and an imitation metric is used to generalize the skill to different contexts and to the robot’s specific bodily constraints.

This work is related to the what-to-imitate issue of imitation learning.

Here we provide an example with a chess task. With two demonstrations of this Chess Task, we see that the constraints vary with respect to the motion. To approach the chess piece a large set of paths are possible depending on the initial position of the arm. Grabbing and pushing the piece require much higher constraints, i.e. the paths do not change much between two consecutive demonstrations.

To re-use the learned skill in a different situation (here, with a different position of the chess piece), an imitation metric is then used to select a controller which best fulfills the task constraints.

What-to-imitate:

The signals are encoded in a threestage process. First, we determine the latent space of the motion by linearly projecting the data onto a subspace of lower dimensionality. Second, we temporally align the signals using a Dynamic Time Warping (DTW) approach. Third, we determine a probabilistic representation of the data in the latent space by estimating the optimal Gaussian Mixture Model (GMM) and Bernoulli Mixture Model (BMM) with which to encode the motion.

Metric of imitation:

A time-dependent similarity measure is defined by taking into account the relative importance of each variable and the dependencies across the variables using the probabilistic representation of the data. This measure evaluates the reproduction performance of a task.

How-to-imitate:

see the General Inverse Kinematics section.

Motion sensors are used to record human body gesture. Although these sensors are not directly related to human-like sensory abilities, they measure robust information about body posture, and can be used easily in different environment, independently of the sound, lighting and occlusion conditions.

Gestures are recorded by 5 x-sens motion sensors attached to the torso, to the right upper-arm, right lower-arm, on the back of the right hand and on the back of the head. Each sensor provides the 3D absolute orientation of each segment, by integrating the 3D rate-of-turn, acceleration and earth-magnetic field.

The kinematics motion of the different joints are computed by defining a joint coordinate system and decomposing the rotation matrix into joint angles.

A stereoscopic vision system tracks the 3D-position of the different objects at a rate of 15Hz, with an accuracy of 10mm.

The tracking is based on color segmentation for detecting the skin color and different objects in the YCbCr color space. Only Cb and Cr are used, to be robust to changes in luminosity.

Standing behind the robot and moving simultaneously its two arms is also an efficient method to demonstrate a task to the robot using its own body.

Initial position of the different objects can be registered by helping the robot to grasp and release the objects. It provides an intuitive and user-friendly means of controlling the robot without any need of specific sensory hardware to control simultaneously multiple DOFs.

The advantage of kinesthetic learning over these systems is that it provides a fast and accurate way of acquiring data, because it does not require to wear motion sensors/colored patches, does not require a calibration phase, and simplifies the correspondence problem.

Left:

Demonstration of a task.

Middle:

Reproduction with different body constraints (dissimilar size of the shoulders).

Right:

Reproduction with different environment constraints (dissimilar size of the object).

Illustration of the constraints for a "Bucket Task".

By observing the continuous description of the variations along the trajectories, we see that the objet-hands relationships are highly constrained from time step 80, when grabbing and holding the bucket (i.e relative constraint), while the hands positions are highly constrained at the end of the motion (i.e absolute constraint).

The essential features of the task have thus been extracted successfully in a continuous representation.

A Chief Cook Robot learns to cook an omelet by whipping eggs, cutting ham and grating cheese. Through the use of a probabilistic model using Gaussian Mixture Model (GMM) and Gaussian Mixture Regression (GMR), the robot progressively learns to generalize the skill to various situations. It is then possible to reproduce the skill by being robust to dynamic perturbations (e.g. by moving the bowl while the robot whip the eggs). This video is part of the Cogniron project.

Short version:

Click on    to play the video and on    to view the video in fullscreen mode

Other videos are available in the Control and Automation videos section.

2010

Journals

Calinon, S., Dhalluin, F., Sauser, E., Caldwell, D. and Billard, A. (2010) Learning and reproduction of gestures by imitation: An approach based on Hidden Markov Model and Gaussian Mixture Regression. IEEE Robotics and Automation Magazine, vol. 17, num. 2, 2010, p. 44--54. [show abstract] [pdf] [BibTeX]

Conference Proceedings

Calinon, S., Sauser, E., Billard, A. and Caldwell, D. (2010) Evaluation of a probabilistic approach to learn and reproduce gestures by imitation. Proceedings of the IEEE Intl Conf. on Robotics and Automation (ICRA), 2010. [ICRA 2010] [show abstract] [BibTeX]

Other Publications

Calinon, S., Sauser, E., Dhalluin, F., Billard, A. and Caldwell, D. (2010) A learning by imitation model handling multiple constraints and motion alternatives. International Conference on Cognitive Systems (CogSys), Zuerich, Switzerland, January 27-28.. [show abstract] [pdf] [BibTeX]

2009

Journals

Calinon, S. and Billard, A. (2009) Statistical Learning by Imitation of Competing Constraints in Joint Space and Task Space. Advanced Robotics, vol. 23, num. 15, 2009, p. 2059-2076. [show abstract] [pdf] [BibTeX]

Conference Proceedings

Calinon, S., Evrard, P., Gribovskaya, E., Billard, A. and Kheddar, A. (2009) Learning collaborative manipulation tasks by demonstration using a haptic interface. Proceedings of the International Conference on Advanced Robotics (ICAR), 2009. [ICAR 2009] [show abstract] [pdf] [BibTeX]
Source code from this publication available here

Weiss, A., Igelsboeck, J., Calinon, S., Billard, A. and Tscheligi, M. (2009) Teaching a Humanoid: A User Study on Learning by Demonstration with HOAP-3. Proc. of the IEEE Intl Symposium on Robot and Human Interactive Communication (Ro-Man), 2009, p. 147-152. [Ro-Man 2009] [show abstract] [pdf] [BibTeX]

Calinon, S., Dhalluin, F., Caldwell, D. and Billard, A. (2009) Handling of multiple constraints and motion alternatives in a robot programming by demonstration framework. Proceedings of 2009 IEEE International Conference on Humanoid Robots, 2009, p. 582 - 588. Nominated for Best Paper Award. [HUMANOIDS'2009] [show abstract] [pdf] [BibTeX]

2008

Journals

Hersch, M., Guenter, F., Calinon, S., Billard, A. and (2008) Dynamical System Modulation for Robot Learning via Kinesthetic Demonstrations. IEEE Transactions on Robotics, vol. 24, num. 6, 2008, p. 1463-1467. [pdf]

Book Chapters

Billard, A., Calinon, S., Dillmann, R. and Schaal, S. (2008) Survey: Robot Programming by Demonstration. Handbook of Robotics, . chapter 59, 2008. [show abstract] [pdf] [BibTeX]

Conference Proceedings

Calinon, S. and Billard, A. (2008) A framework integrating statistical and social cues to teach a humanoid robot new skills. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Workshop on Social Interaction with Intelligent Indoor Robots, 2008. [ICRA 2008] [show abstract] [pdf] [BibTeX]

Calinon, S. and Billard, A. (2008) A Probabilistic Programming by Demonstration Framework Handling Constraints in Joint Space and Task Space. Proc. IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS), 2008. [IROS'2008] [show abstract] [pdf] [BibTeX]
Source code from this publication available here

2007

Journals

Billard, A., Calinon, S., Dillmann, R. and Schaal, S. (2007) Robot Programming by Demonstration. Handbook of Robotics, MIT Press. (in Press), chapter 59. [show abstract] [pdf]

Guenter, F., Hersch, M., Calinon, S. and Billard, A. (2007) Reinforcement Learning for Imitating Constrained Reaching Movements. RSJ Advanced Robotics, Vol. 21, No. 13, pp. 1521-1544. [show abstract] [pdf] [BibTeX]

Calinon, S. and Billard, A. (2007) What is the Teacher's Role in Robot Programming by Demonstration? - Toward Benchmarks for Improved Learning. Interaction Studies, Special Issue on Psychological Benchmarks in Human-Robot Interaction, 8:3, pp 441-464. [show abstract] [pdf] [BibTeX]
Source code from this publication available here

Calinon, S., Guenter, F. and Billard, A. (2007) On Learning, Representing and Generalizing a Task in a Humanoid Robot. IEEE Transactions on Systems, Man and Cybernetics, 37:2. Part B. Special issue on robot learning by observation, demonstration and imitation. [Matlab source code] [show abstract] [pdf] [BibTeX]
Source code from this publication available here

Book Chapters

Calinon, S. and Billard, A. (2007) Learning of Gestures by Imitation in a Humanoid Robot. Dautenhahn, K. and Nehaniv, C.L. (eds.). Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions. Cambridge University Press. [show abstract] [pdf] [BibTeX]

Conference Proceedings

Calinon, S. and Billard, A. (2007) Active Teaching in Robot Programming by Demonstration. in Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Jeju, Korea, pp 702-707. [RO-MAN'2007] [show abstract] [pdf] [BibTeX]

Calinon, S. and Billard, A. (2007) Incremental Learning of Gestures by Imitation in a Humanoid Robot. in Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI). [Matlab source code] [show abstract] [pdf] [BibTeX]
Source code from this publication available here

PhD Thesis

Calinon, S. (2007) Continuous Extraction of Task Constraints in a Robot Programming by Demonstration Framework. PhD thesis, Learning Algorithms and Systems Laboratory (LASA), Ecole Polytechnique Federale de Lausanne (EPFL). [show abstract] [pdf]

Peer-reviewed Videos

Calinon, S. and Billard, A. (2007) Incremental Learning of Gestures by Imitation in a Humanoid Robot. ACM/IEEE International Conference on Human-Robot Interaction (HRI). [Video]

2006

Journals

Billard, A., Calinon, S. and Guenter, F. (2006) Discriminative and Adaptive Imitation in Uni-Manual and Bi-Manual Tasks. Robotics and Autonomous Systems, 54:5. [home page] [show abstract] [pdf] [BibTeX]

Conference Proceedings

Hersch, M., Guenter, F., Calinon, S. and Billard, A. (2006) Learning Dynamical System Modulation for Constrained Reaching Tasks. In Proceedings of the IEEE-RAS International Conference on Humanoid Robots (Humanoids). [HUMANOIDS'2006] [show abstract] [pdf] [BibTeX]

Calinon, S. and Billard, A. (2006) Teaching a Humanoid Robot to Recognize and Reproduce Social Cues. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). [Matlab source code] [show abstract] [pdf] [BibTeX]
Source code from this publication available here

Calinon, S., Guenter, F. and Billard, A. (2006) On Learning the Statistical Representation of a Task and Generalizing it to Various Contexts. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). [ICRA'2006] [show abstract] [pdf] [BibTeX]

Peer-reviewed Videos

Hersch, M., Guenter, F. and Calinon, S. (2006) Learning Dynamical System Modulation for Constrained Reaching Tasks. IEEE-RAS International Conference on Humanoid Robots (Humanoids). [Video]

2005

Conference Proceedings

Calinon, S., Guenter, F. and Billard, A. (2005) Goal-Directed Imitation in a Humanoid Robot. In Proceedings of the International Conference on Robotics and Automation (ICRA). [ICRA'2005] [show abstract] [pdf] [BibTeX]

2004

Journals

Billard, A., Epars, Y., Calinon, S., Cheng, G. and Schaal, S. (2004) Discovering Optimal Imitation Strategies. Robotics & Autonomous Systems, Special Issue: Robot Learning from Demonstration, 47:2-3, p.69-77. [home page] [show abstract] [pdf] [BibTeX]

Conference Proceedings

Calinon, S. and Billard, A. (2004) Stochastic Gesture Production and Recognition Model for a Humanoid Robot. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). [IROS'2004] [home page] [show abstract] [pdf] [BibTeX]

Abstracts

Calinon, S. and Billard, A. (2004) Gesture Recognition and Reproduction for a Humanoid Robot using Hidden Markov Models. AMI/PASCAL/IM2/M4 Workshop on Multimodal Interaction and Related Machine Learning Algorithms. [MLMI'2004] [pdf]

• Sylvain Calinon, Visiting researcher
• Aude Billard, Professor