MACHINE LEARNING 
The aim of machine learning is to extract knowledge from data. The
algorithm may be informed by incorporating prior knowledge of the task
at hand. The amount of information varies from fully supervised to
unsupervised or semisupervised learning. This course will present some
of the core advanced methods in the field for structure discovery,
classification and nonlinear regression. This is an advanced class in
Machine Learning; hence, students are expected to have some background
in the field. The class will be accompanied by practical session on
computer, using the mldemos software (http://mldemos.epfl.ch)
that encompasses more than 30 state of the art algorithms.
People  Lecturer: Aude
Billard, Assistant: Basilio
Noris 
Grading Scheme  50% of the Grade will be based on personal work done during the
semester (this entails either doing a computer based miniproject
or writing a literature survey, see section Practicals
below). The remaining 50% of the grade will be based on a 40
minutes oral exam (20 minutes preparation, 20 minutes oral
defense). The oral will examine the student on the material viewed
during the course. 
Prerequisites  Strong knowledge in Linear Algebra, Probability and Statistics
This course is intended as an advanced class in ML for PhD students and hence focuses on advanced topics of ML as well as on critical reading of the state of the art in ML. MSc student who want to take this course are expected to have taken the introductory ML course given at the master during the winter semester. 
Lecture Notes  The class is accompanied by lecture notes entitled "Machine Learning Techniques". Hardcopies are available at the librairie polytechnique. The supplementary material for the lecture notes (examples and demos of the algorithms presented) is available HERE. 
Time and Location 
Every other week, we alternate between:
Check detailed schedule below to know which one takes place when. Lectures + exercises take place each friday from 9am through 1pm in room MEB331. Exceptionally on March 22, the course will take place in room GRC02Practicals take place in room GRC02. 
Software  The course encompasses 4 hours of computerbased practicals centred on the use of the MLDemos software (available at http://mldemos.epfl.ch ). This opensource software allows to visualize and test most of the major algorithms seen in class. 
The practicals will cover different topics (Kernel Methods, Classification, Regression and Reinforcement Learning). In addition, students will conduct either a miniproject or a literature survey during the course of the semester. The miniproject will entail implementing an algorithm of choice and one or more variants, evaluating the algorithm performance and sensibility to parameter choices. The literature survey will be done on a topic chosen among a list provided in class. The student will read and analyze a choice of recent articles on the topic and provide a written summary.
Results from the computerbased miniproject or survey will be presented in class (10 minutes presentation) and documented in a report (10 pages maximum, 10pt minimum, single column; code from miniproject must be submitted at the same time as the report). Reports + code are due on May 17 2012, 6pm.
Date  Related Chapter from Lecture Notes  Course Slides  Exercises/TP Template  Solutions Exercises 
22 Feb.  Chapter 1 + Annexes 
Concepts

 

1 March 
Sections 2.12.2 (must be read before class) Sections 5.15.3. 
Spectral methods: Structure Discovery, Dim.
Reduction 

8 March 
Practicals in room GRC002 
Practicals Continued 

15 March 
Sections 3.1 (must be read before class) Sections 5.45.6. 
Spectral methods: Structure Discovery, Dim.
Reduction 


22 March  
Spectral methods: Structure
Discovery, Dim. Reduction
Spectral Clustering Supplementary material: A tutorial on Spectral Clustering, Extensions 
Practicals in room GRC002 

29 March 
Good Friday (public holiday) 
 
 

5 April 
Easter Break 
 
 

12 Avril  
Nonlinear Classification
Supplementary material:on
RVM 

19 Avril 
Sections 3.2 must be read before class 
Mixtures of Classifiers 


26 April  Practicals in room
GRC002 
Practicals Continued 

3 May  Chapter 4 (must be read before class) Sections 5.8 and 5.9  Nonlinear Regression 

10 May  Practicals in room
GRC002 

17 May  Sections 7.1 and 7.3 (must be read before class) 
MakovProcesses 

24 May 
MakovProcesses 
Overview of class 


31 May  
Student Presentations 
Continued... 
 
19 June Room ME.B1.10 

Oral Exam 
Continued... 
 
Online repository of papers from NIPS conference
Pascal: Network of Excellence on Pattern Recognition, Statistical and Computational Learning (summer schools and workshops)
ML
Repository: "This is a repository of databases, domain theories
and data generators that are used by the machine learning community for
the empirical analysis of machine learning algorithms."
MLnet: Machine Learning network
online information service. "This site is dedicated to the field of
machine learning, knowledge discovery, casebased reasoning, knowledge
acquisition, and data mining."
KDnet: "The KDNet (= Knowledge Discovery Network of Excellence) is an open Network of participants from science, industry and the public sector. The major purpose of this international project is to integrate reallife business problems into research discussions and to collaborate in shaping the future of Knowledge Discovery and Data Mining."
Clever Methods for Overfitting
 "Kernel Methods for Pattern Analysis" by John ShaweTaylor,
Nello Cristianini
Publisher:
 "Pattern Recognition and Machine Learning" by Christopher M.
Bishop
Publisher: Springer; 1 edition (October 1, 2007)
 "Learning with Kernels", B. Scholkopf and A. Smola, MIT Press 2002
 "Pattern Classification" by Richard O. Duda, Peter E. Hart,
David G. Stork
Publisher: WileyInterscience; 2 Sub edition (October 2000)
 "Introduction to Statistical Learning Theory" by Olivier Bousquet, Stephane Boucheron, and Gabor Lugosi
http://www.kyb.mpg.de/publications/pdfs/pdf2819.pdf
 "Information Theory, Inference and Learning Algorithms", David J.C Mackay, Cambridge University Press, 2003.
 "Independent Component Analysis", A. Hyvarinen, J. Karhunen and E. Oja, Wiley InterSciences. 2001.
 "SelfOrganizing Maps", Teuvo Kohonen, Springer Series in Information Sciences, 30, Springer. 2001.
 "Introduction to Neural Networks: A Comprehensive Foundation" (2^{nd} Edition) by S. Haykins.
 "Reinforcement Learning: An Introduction",. R. Sutton & A. Barto, A Bradford Book. MIT Press, 1998.
 "Reinforcement Learning: A Survey", Leslie Pack
Kaelbling & Michael L. Littman and Andrew W. Moore, Journal of
Artificial Intelligence Research, Volume 4, 1996., 1996