The forward direction of the field of view of the Wearcam is only an approximation of what the child is actually seeing. The center of attention of the visual field varies significantly when the child is looking sideways. A small mirror placed on the bottom of the wearcam captures the eyes region of the face.

Gaze direction detection is computed using Support Vector Regression. The system creates a mapping between the appearance of the eyes and the direction of the gaze in the image. There is no calibration required by the user, as detection is performed offline.

The system can detect the direction of the gaze with an accuracy of about 2.5° for children and 2° for adult subjects. The video shows a normally developing child wearing the WearCam and playing a touch-screen game used for gaze tracking performance estimation.

Object Detection

Object detection is performed in succesive steps:

First the camera films a still background for a short duration in order to learn the background.

In a second step, the hand and arm of the experimenter is shown to the camera in front of the same backrgound. By substracting the background it is possible to extract only the arm from the image and build a color model of the arm.

In a third step, the experimenter shows the desired object to the camera in front of the same background. By substracting the background and the arm, it is then possible to automatically extract the object's shape and color information.

Once these steps have been completed, we can collect positive and negative samples of the object, and use them to train a cascade of boosted Support Vector Machine. This model can then be applied on other videos in order to detect the same object in different environments.

(Larger version available in the videos section)

Keyword Spotting

Keyword spotting makes use of the audio data recorded with the WearCam to recognize when specific keywords have been uttered during the recording. This can be used in conjunction with face or object detection to correlate utterance of specific words with specific gaze behaviour (like for example looking at an object or person).

Keyword spotting is based on a left-right model for each keyword, and a fully connected model for garbage (non-keywords). Keyword recognition is then made by comparing the log-likelihood of each model for the uttered speech.

---

Videos

Gaze Direction Detection Gaze Detection Example (XVID AVI file, 3.94Mb)

Object Segmentation Object Segmentation Example (XVID AVI file, 7.5Mb)

Gaze & Object Detection Applied to an Interaction Scene Gaze & Object Detection Example (XVID AVI file, 6.9Mb)

---

Selected publications

Conference Papers

2009

---

People involved in this project

• Basilio Noris, Postdoctoral fellow
• Olivier Barbey, Collaborator
• Jean-Baptiste Keller, Collaborator
• Aude Billard, Professor
• Weifeng Li, Postdoctoral fellow
• Karim Benmachiche, Research Assistant (2007-2008)
• Lorenzo Piccardi, Research Assistant (2005-2007)
• Martin Duvanel, Research Assistant