english only
EPFL > STI > IMT > LASA > Teaching > Student Projects
LASA MENU
Home
Research
Publications
Teaching
People
Arriving to LASA
Videos / Photos
Press & News

GitHub Repository
Older Source Codes
The Blog
Intranet

Prospective Students
TEACHING INDEX
Teaching
Student Projects
Former Projects
Robot Contests

Student Projects propositions for 2014 Autumn

If you are interested in one of the below projects, Semester Projects or Master Projects, please contact the first person of reference indicated in each description either by telephone, or by email, or by visiting us directly to the LASA offices. If you are looking for a project for 2015 Spring, please click here.


Semester Projects

   
   

Combining kinesthetic demonstrations with reinforcement learning

Learning from demonstrations has provided us with practical algorithms that can learn a variety of difficult tasks with very few demonstrations (e.g., [1]). Nevertheless, most of these approaches assume the existence of a low-level controller that can follow commands in the task-space. This controller might not be available or prone to singularity (or other) issues in the general case (e.g., imagine a quadrotor). The goal of this project is to produce a reinforcement learning algorithm that combines a simulator with the learned controller from demonstration and is able to learn low-level policies that follow (or imitate) the high-level controller. References: [1]: "Learning stable nonlinear dynamical systems with gaussian mixture models", Seyed Mohammad Khansari Zadeh, and Aude Billard. IEEE Transactions on Robotics, 2011.

Project: Semester Project
Period: 01.03.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 70% implementation, 30% theory
Knowledge(s): Python/MATLAB, Robot Control
Subject(s): Reinforcement Learning, Robot Control, Machine Learning
Responsible(s): Konstantinos Chatzilygeroudis, Bernardo Fichera
 
   
   
   

From human motion to biomechanical simulation

In Human-Robot interaction, being able to simulate the human motion is a needed tool to plan the correct robotic response. Doing so relies on a good biomechanical model of the human body both in terms of kinematic and forces. OpenSim software(http://opensim.stanford.edu/) allows to simulate the human body in realistic manners.

The goal of this project would be twofold. First, the student will have to bridge OpenSim with the motion capture software that we have at LASA, that relies on RBG images. Second, using the internal tools of OpenSim, the student will have to perform an inverse kinematic and inverse dynamic from the recorded data and/or live recording.

Project: Semester Project
Period: 01.02.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 30% theory, 50% implementation, 20% software
Knowledge(s): Python/C++
Subject(s): Inverse kinematics, Inverse dynamics, Biomechanics, Human motion
Responsible(s): Baptiste Busch, Leonardo Urbano
   
   
   

Learning ergonomic human-robot interaction

The ultimate goal in physical human-robot interaction is to reduce the human effort. One consideration is the human ergonomics and muscle fatigue with regard to the his/her posture. Imagine a human-robot interaction where a robot is holding an heavy object for the human to operate. It is expected from an intelligent robot to consider the human ergonomics and position/orientate the object in order to optimize human ergonomics and reduce the fatigue. In doing so, we are facing with two challenges: 1) what is an optimal posture for the human, and 2) how the robot can influence the human coworker to reach toward that optimal posture.

In this project, we assume that a cost function for human posture is given and focus on the second challenge: learning to optimize human posture by modifying the robot configuration. Moreover, we will be focusing on using reinforcement learning techniques where the robot learns to reduce human effort in the course of the interaction. As the first step, the student will be focusing on a simplified model of the interaction and formulating the RL problem. In the next steps, the student can focus on the theoretical (stability and convergence properties of the approach) and the practical aspects (generalization and implementation for real scenarios) of the project.

Project: Semester Project
Period: 01.02.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 50% theory, 30% software, 20% implementation
Knowledge(s): Robotics, Control, Matlab
Subject(s): Robotics, Physical Human-robot interaction
Responsible(s): Baptiste Busch, Mahdi Khoramshahi
 
   
   
   

Learning assembly sequences from demonstrations

Coding analytically sequences of assembly tasks can be challenging and time consuming. In small assembly lines, uncertainty and flexibility are expected. Reprogramming a robot as a new task arises is, therefore, not an option. An alternative way to cope with this challenge is to learn the correct sequence of actions from demonstrations. The question becomes then, how much prior knowledge does the robot really requires? The aim of the project is to learn the correct sequence of actions that would lead to a successful assembly with the least prior knowledge possible. For example, given a set of objects on a table, and user demonstrations, the robot should infer what are the relations between objects and the meaning of common predicates such as on, in, under, below... Observing how those object relations evolves during the assembly process would then lead to the creation of a correct assembly plan. A final demonstration of the system on one of the robot of the laboratory is expected at the end of the project.

Project: Semester Project
Period: 01.01.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 50% theory, 50% implementation
Knowledge(s): Python/C++
Subject(s): Machine Learning
Responsible(s): Baptiste Busch, Athanasios Polydoros
 
   
   
   

Learning dynamic systems of force interaction tasks (Assigned)

Force interaction tasks are very common in the field of robotics. In such cases, the force feedback is crucial for the system’s accuracy, adaptability and robustness. Thus it is a factor which has to be taken into account when creating dynamic systems’ representations of tasks. Nevertheless, creating an analytical model of the dependencies between the robot’s position and exerted force can be difficult or even impossible. Therefore, the goal of the project will be to employ machine learning methods which will create force-dependent dynamic systems. The learned systems could be able to model various tasks and adapt to perturbations. The use-case will be an assembly task and demonstrations of it will be performed by a human through kinesthetic teaching. During the demonstrations, force and position data will be recorded and used as a training set for the dynamic system.

Project: Semester Project
Period: 01.01.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 50% theory, 30% implementation, 20% software
Knowledge(s): Python/C++
Subject(s): Machine Learning, Robot Control
Responsible(s): Athanasios Polydoros
 
   
   
   

Inferring assembly objects from demonstrations (Assigned)

Programming robots by demonstration has been a popular approach because it enables non robotic experts to easily introduce a task to a robotic system and also significantly decreases the time needed for re-programming. Those characteristics are highly desirable for robotic manufacturing tasks such as assembly. Thus, the goal of the project will be to create an unsupervised learning system which will be capable to infer the needed objects to perform an assembly task. The information provided to the system will be a feed of images, captured from RGB-D sensors, during which a human will demonstrate the task. Also tactile sensing systems can be used.

Project: Semester Project
Period: 01.01.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 30% theory, 50% implementation, 20% software
Knowledge(s): Python/C++
Subject(s): Machine Learning, Robot Vision
Responsible(s): Athanasios Polydoros
 
   
   
   

Human-robot impedance matching as reaction strategy toward unexpected external forces

Any seamless physical human-robot interaction requires an efficient reaction strategy toward external forces. Consider a robot and human carrying a heavy object together. During this transportation, the robot senses a sudden external force. Regarding the reaction strategy toward this external force, the robot is facing a dilemma: either, the force is a disturbance and needs to be rejected by exhibiting an stiff behavior, or, the force is intentional and requires a compliant behavior. Generally, this is a challenging problem. However, the robot can benefit from the presence of the human in the interaction. Assuming that the human would exhibit the proper reaction (to be either stiff or compliant), the robot can mimic the human’s behavior; technically speaking, to match its impedance.

In this project, we will be focusing on impedance/admittance control structure and simple scenarios; for example, a human and a robot holding/stabilizing an large object jointly while an external force occurs. As the first step, the student needs to work on the theoretical aspect and proper formulation of the problem. The student can use control theory or machine learning approaches to offer a solution/algorithm to vary the robot stiffness. Based on the progress, the student can decide to work further on theoretical aspects of the problem (such as stability of the interaction under the variation of the stiffness), or on implementation or validation on a real robotic arm.

Project: Semester Project
Period: 01.01.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 40% theory, 20% software, 40% implementation
Knowledge(s): Robotics, Control, Matlab
Subject(s): Physical human-robot interaction
Responsible(s): Mahdi Khoramshahi
   
   
   

Learning the unmodeled dynamics of the task for implicit force control

Performing a contact tasks in an uncertain environment is a challenging robotic problem. Uncertainties in the location of the surface, normal vector of the surface, measurement noises, imperfection in the actuators, and other unmodeled dynamics introduce an error in the desired force applied by the robot. Fortunately, such many sources of such uncertainties are structural and can be modeled and correct for: the robot can learn (or adapt to) such unmodeled dynamics in its environment through interaction.

Previously, we developed a controller at LASA that can approach a surface and exert a desired force. A demonstration can be viewed here. Through this project (i.e., learning the unmodeled dynamics of the environment), we aim to improve the precision of the force profile. In this project, we will be using adaptive control approaches. This project will be mostly programming (C++/ROS) and implementation (on Kuka LWR robotic arm).

Project: Semester Project
Period: 01.01.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 20% theory, 40% software, 40% implementation
Knowledge(s): Robotics, Control, Matlab, C++
Subject(s): Physical human-robot interaction
Responsible(s): Mahdi Khoramshahi, Walid Amanhoud
   
   
   

Classification of human vs hard contacts through force sensing

Many robotic tasks involve contacts with surfaces. For example, consider polishing a surface or drilling an object. The most common approach to solve such problem is to rely on force-sensing: while approaching the surface, the contact is recognized when the sensed forces are passed beyond a threshold. Therefore, the robot begins to exert the desired forces. However, considering cluttered and uncertain environments including humans introduces new challenges for safety of such robotic task: for example, the robot might detect collide with a human and detects him/her as the expected hard contact. Therefore, following the same strategy, the robots harm the human applying its desired forces. An effective strategy would be to distinguish between hard contact surfaces and accidental contacts with humans.

Previously, we developed an algorithm at LASA to distinguish between accidental and international forces. A demonstration for this algorithm can be viewed here . The goal of this project is extend this approach to include more classes of interaction forces, such intentional/unintentional hard/soft contacts. In the first step of this project, the student will be focusing on collecting and investigation force-profiles resulting from such different scenarios. The aim is to identify and extract effective feature which allows us to design a classifier. If time and progress permit, the student can work on reaction strategy and robotic implementation.

Project: Semester Project
Period: 01.01.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 50% theory, 30% software, 20% implementation
Knowledge(s): Robotics, Control, Matlab
Subject(s): Physical human-robot interaction
Responsible(s): Mahdi Khoramshahi
   

Master Projects

   
   

Finding creative solutions in hard to solve robotic problems

Most interesting real world robotic tasks do not have a well-defined continuous reward function; for example, imagine a robot that has to autonomously clean up a room: it is hardly conceivable (if not impossible) that we can come up with a reward function that is continuous and/or does not impose any bias on the solution. Recently, Uber AI labs published a new algorithm, called Go-Explore [1][2], that can effectively explore very large spaces with little domain knowledge and assumptions, but they do not provide any results on a robotics task or real-world application. The Go-Explore algorithm consists of two stages: (1) learning how to solve a task in simulation in the raw action space, and (2) use learning from demonstrations (LfD) to find a robust/reactive controller to apply it on the actual (possibly stochastic) system. The goal of this project is three-fold: (1) define (or learn) the appropriate state representation to feed the Go-Explore algorithm (a good initial starting point is to learn a state representation using a combination of some of the robotic priors loss functions by Jonschkowski and Brock [3], but many more can be exploited [4]), (2) investigate intelligent exploration policies instead of randomly sampled actions (one way could be to define primitive policies based on dynamical systems that perform a few parameterized behaviors: e.g., point-reaching etc.), and (3) investigate how to insert dynamical system-based LfD approaches in the second part of the algorithm. The algorithm will be evaluated in one of the following tasks: (1) USB key insertion, (2) irregularly-shaped peg-in-the-hole task that involves dual-manipulation, (3) a robot that needs to clean-up a desk and put each object in specific baskets.

References:

  • [1]: https://eng.uber.com/go-explore/
  • [2]: https://arxiv.org/abs/1901.10995
  • [3]: Jonschkowski, R. and Brock, O., 2015. Learning state representations with robotic priors. Autonomous Robots, 39(3), pp.407-428.
  • [4]: Lesort, T., Díaz-Rodríguez, N., Goudou, J.F. and Filliat, D., 2018. State representation learning for control: An overview. Neural Networks.

Project: Master Project at EPFL
Period: 01.01.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: 60% theory, 40% implementation
Knowledge(s): Python, C++
Subject(s): Reinforcement Learning, Robot Control, Machine Learning
Responsible(s): Konstantinos Chatzilygeroudis
 
   
   
   

Fast adaptation via policy search for high-dimensional robots

The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms, which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties [1]. Nevertheless, most model-based policy search approaches do not scale to high dimensional state/action spaces. Recently, a new model learning procedure has been introduced [2] that leverages parameterized black-box priors of the dynamics (e.g., simulators) and Gaussian processes to (1) scale up to high-dimensional systems, and (2) be robust to large inaccuracies of the prior information, in order to adapt to unforeseen situations (e.g., damage). One of the major drawbacks of this approach is the large computational cost. The goal of this project is to implement methods for speeding up the learning procedure, while keeping the adaptation capabilities of the algorithm. The student will have to augment the algorithm with: (1) faster regression techniques (e.g., sparse Gaussian processes [3] or Gaussian Mixture Regression (GMR)), (2) local models [4] (i.e., partition the policy space and learn a different model for each partition), (3) learning actuator models [5], and/or (4) learning compensatory actions [6]. The student will evaluate the approach on: (1) an iCub humanoid robot performing a lift-and-place task (i.e., lift an object and place it in a different location) where the weight/dynamics of the object cannot be anticipated before-hand, and/or (2) an irregularly-shaped peg-in-the-hole task that the friction coefficients and peg-specifications cannot be fully determined in simulation.

References:

  • [1]: "A survey on policy search algorithms for learning robot controllers in a handful of trials", Konstantinos Chatzilygeroudis, Vassilis Vassiliades, Freek Stulp, Sylvain Calinon and Jean-Baptiste Mouret. arXiv preprint arXiv:1807.02303, 2018.
  • [2]: "Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics", Konstantinos Chatzilygeroudis, Jean-Baptiste Mouret. ICRA, 2018.
  • [3]: "Distributed gaussian processes", Marc Deisenroth, Jun Wei Ng. arXiv preprint arXiv:1502.02843, 2015.
  • [4]: "Guided policy search", Sergey Levine, and Vladlen Koltun. International Conference on Machine Learning, 2013.
  • [5]: "Learning agile and dynamic motor skills for legged robots", J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun and M. Hutter. Science Robotics, 2019.
  • [6]: "Model-plant Mismatch Compensation Using Reinforcement Learning", I. Koryakovskiy, M. Kudruss, H. Vallery, R. Babuška and W. Caarls. IEEE Robotics and Automation Letters, 2018.

Project: Master Project at EPFL
Period: 01.01.2019 - 01.08.2019
Section(s): EL IN MA ME MT PH
Type: Type: 50% implementation, 50% theory
Knowledge(s): C++, Python, Basic Linear Algebra
Subject(s): Reinforcement Learning, Robot Control, Machine Learning
Responsible(s): Konstantinos Chatzilygeroudis
URL: Click here
   



Last update: 09/28/2014