Catching a fast flying object is particularly challenging as consists of two tasks: it requires extremely precise estimation of the object's motion and control of the robot motion. Any small imprecision may lead the fingers to close too abruptly and let the object fly away from the hand before closing. We present a strategy to overcome for sensori-motor imprecision by introducing softness in the catching approach. Soft catching consists of having the robot moves with the object for a short period of time, so as to leave more time for the fingers to close on the object. We use a dynamical systems (DS) based control law to generate the appropriate reach and follow motion, which is expressed as a Linear Parameter Varying (LPV) system. We propose a method to approximate the parameters of LPV systems using Gaussian Mixture Models, based on a set of kinematically feasible demonstrations generated by an off-line optimal control framework. We show theoretically that the resulting DS will intercept the object at the intercept point, at the right time with the desired velocity direction.
Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncertainty
Coordinated control strategies for multi-robot systems are necessary for tasks that cannot be executed by a single robot.
This encompasses tasks where the workspace of the robot is too small or where the load is too heavy for one robot to handle. Using multiple robots makes the task feasible by extending the workspace and/or increase the payload of the overall robotic system. In this paper, we consider two instances of such task: a co-worker scenario in which a human hands over a large object to a robot; intercepting a large flying object. The problem is made difficult as the pick-up/intercept motions must take place while the object is in motion and because the object's motion is not deterministic. The challenge is then to adapt the motion of the robotic arms in coordination with one another and with the object.
Determining the pick-up/intercept point is done by taking into account the workspace of the multi-arm system and is continuously recomputed to adapt to change in the object's trajectory. We propose a dynamical systems (DS) based control law to generate autonomous and synchronized motions for a multi-arm robot system in the task of reaching for a moving object. We show theoretically that the resulting DS coordinates the motion of the robots with each other and with the object, while the system remains stable. We validate our approach on a dual-arm robotic system and demonstrate that it can re-synchronize and adapt the motion of each arm in synchrony in a fraction of seconds, even when the motion of the object is fast and not accurately predictable.
A Unified Framework for Coordinated Multi-Arm Motion Planning
Coordination is essential in the design of dynamic control strategies for multi-arm robotic systems. Given the complexity
of the task and dexterity of the system, coordination constraints can emerge from different levels of planning and control.
Primarily, one must consider task-space coordination, where the robots must coordinate with each other, with an object
or with a target of interest. Coordination is also necessary in joint-space, as the robots should avoid self-collisions
at any time. Moreover, multi-arm task-space behaviors can either be synchronous or asynchronous. In this work, we
define a synchronous behavior as that in which the robot arms must coordinate with each other and with a moving
object such that they reach for it in synchrony. Whereas, an asynchronous behavior allows for each robot to perform
independent point-to-point reaching motions. In this paper, we build upon our previous work on coordinated multi-arm
control (Salehian et al. 2016a), to propose a unified framework that endows a multi-arm system with both synchronous
and asynchronous behaviors and the capability of smoothly transitioning between them, whilst avoiding self-collisions.
To provide such smooth transitioning, we introduce the notion of synchronization allocation. Given the motion of the
object and the joint workspace of the multi-arm system, each arm is being continuously allocated to a desired behavior.
While being allocated to the synchronous behavior, control of the robots is taken over by the virtual object Dynamical
System (DS), which exploits the notion of a virtual object, whose dynamics is coupled to that of the robot’s motions.
While allocated to the asynchronous behavior, the robots are controlled independently, each with their own goal-directed
stable DS. Both behaviors and their synchronization allocation are encoded in a single dynamical system. Further,
we provide coordination in joint-space by introducing a centralized inverse kinematics (IK) solver under self-collision
avoidance constraints; formulated as a quadratic program (QP) and solved in real-time. The space of free motion is
modeled through a sparse non-linear kernel classification method in a data-driven learning approach. We validate our
framework on a dual-arm robotic system and demonstrate that it can re-synchronize and adapt the motion of each arm
within milliseconds, even when the motion of the object is fast and not accurately predictable.
Media Coverage
KUKA Innovation Award 2017
We, the Swiss researchers from the École polytechnique fédérale in Lausanne, are demonstrating our concept for two robots that cooperate with each other and collaborate with the human operator – for example, to move heavy parts such as car doors without the risk of a collision and to support the human operator in positioning these parts correctly.
KUKA Innovation Award 2015
Catching in flight objects with uneven shapes is a challenging problem, which needs excellent accuracy in motion planning. This requires the solution of three complex problems: predicting accurately the trajectory of fast-moving objects, predicting the feasible catching configuration and planning the arm motion, and all within milliseconds. We exploit our methods for fast re-planning under uncertainty and validate these methods in real-world experiments with the KUKA LWR 4+ to catch different objects with non-linear flight dynamics (e.g. hammer, bottle half-filled with water). We extend this method to the problem of catching “softly” an object in flight. Soft catching refers to active compliant control and is modelled as a 2 phase process: the robot first moves its hand to intercept the object’s trajectory at a desired interception point and then moves in synchrony with the object such that the fingers can slowly close on the object at a desired catching point.
Mirrazavi Salehian, S. S., Figueroa, N. and Billard, A. (2017) Dynamical System-based Motion Planning for Multi-Arm Systems: Reaching for moving objects. In Proceedings of International Joint Conference on Artificial Intelligence 2017, Melbourne, Australia. [IJCAI 2017] [show abstract] [pdf] [BibTeX]
Mirrazavi Salehian, S. S., Figueroa, N. and Billard, A. (2016) Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncertainty. In Proceedings of Robotics: Science and Systems XVI , Arbor, Michigan, USA. Received Best Student paper Award. Also nominated as the Best Conference Paper Award. Best Student Paper Award. Best Systems Paper Award.. [RSS 2016] [home page] [show abstract] [BibTeX] Source code from this publication available here
Firouzeh, A., Mirrazavi Salehian, S. S., Billard, A. and Paik, J. (2015) An Underactuated Robotic Arm with Adjustable Stiffness Shape Memory Polymer Joints. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA 2015). [ICRA 2015] [show abstract] [pdf] [BibTeX]
