Research & Experience



My research interests include:

  • Robot Reinforcement Learning: Model-free and model-based algorithms that scale to robotics problems (high-dimensional, continuous states and actions, hierarchical and multi-task problems)
  • Optimal Control: Control with learned models, control with inaccurate models, Stochastic Optimal Control, Non-linear MPC
  • Humanoids: Whole-body motions with various tasks, multi-contact motion
  • Imitation Learning: Inverse reinforcement learning, shared human-robot representations

Robot Learning

The robot hardware is progressively becoming more complex, which leads to growing interest in applying machine learning and statistics approaches within the robotics community. At the same time, there has been a growth within the machine learning community in using robots as motivating applications for new algorithms and formalisms [3].

However, unlike other fields, robotics requires the consideration of some aspects that requires changing the way it has been working until now. The challenges that reinforcement learning has to face are: the curse of dimensionality, the curse of real-world samples, the curse of under-modeling and model uncertainty, and the curse of goal specification [4].

In recent years research in robot learning has mainly focused on: learning models of robots, task or environments, learning deep hierarchies or levels of representations from sensor and motor representations to task abstractions, learning of plans and control policies by imitation and reinforcement learning, integrating learning with control architectures, methods for probabilistic inference from multi-modal sensory information (e.g., proprioceptive, tactile, vison), and structured spatio-temporal representations designed for robot learning such as low-dimensional embedding of movements [3].

Humanoid Robots

A humanoid robot or humanoid is an electro-mechanical machine with anthropomorphic form guided by a computer program or electronic circuitry in order to emulate some subset of the physical, cognitive and social dimensions of the human body and experience [1] [2].

In recent years, there has been a strong impetus to research on humanoid robots, developing robots with high workability, interaction and mobility. This impetus is motivated by the many advantages of humanoid robots against other kind of robots. Among these, the most important is that humanoid robots can operate directly on the same human environment without any modification.


Nevertheless, there are many challenges to overcome. Humanoid robotics research is focused in the stability and mobility of the robot under different environmental conditions, complex control systems to coordinate the whole body motion, motion planning, as well as the development of fast intelligent sensors and light energy-saving actuators. The research in humanoid robots also has many challenges with the autonomy, manipulation, locomotion and human-robot interaction.

As the robotics, the humanoid robotics field is an interdisciplinary science that integrates the knowledge of many disciplines. However, a fully-fledged humanoid robot will incorporate work from each of the areas below [2]:

Within the movement of humanoid robots, motion planning by trajectory generation in operational (task or cartesian) space is a very used method by the possibility to define the movement in the real environment where the robot will perform the task. But the planning of trajectories considering only analytical techniques, can generate unnatural movements. For this reason, other sources of movement should be considered. The use of captured human motion data is a valuable source of examples to simplify the process of programming or learning complex robot motions. It is important to know the possibly existing natural restrictions imposed by the human way of doing the things rather than by the physical constraints.

The mass concentrated models have been very effective methods in the generation of whole-body movement for many years, however, current humanoids possess more complex dynamics than previous generation of robots. Therefore, it is no longer appropriate to use over-simplified models of humanoid dynamics and new approaches have to be considered. Robot learning, the research field at the intersection of machine learning and robotics, is one of these possibilities. The use of learning algorithms to control humanoids and the use of learned dynamics rather than engineered dynamics models are good alternatives to progress in the field of whole-body planning and control.




Some robots whom I have worked with:

  • TEO
  • DARwIn-OP
  • Summit-XL
  • Kobuki
  • IRB 1600




Python package for robot learning.

+Info Source



MATLAB® GUI that allows Whole-body trajectory generation for TEO humanoid robot.

+Info Source


TEO ROS metapackage

ROS packages for TEO humanoid robot.

+Info Source

Reference List

  1. Wikipedia. “Robot”, [Online]. Available [Last Modified: 29 January 2015, at 00:44].
  2. I. N. Laboratory. Humanoid Robotics. What is a humanoid robot?. [Online]. Available Accessed: 2014-06-22
  3. Technical Committee on Robot Learning. IEEE RAS Technical Committee on Robot Learning (official IEEE website). Accessed: 2015-01-31
  4. Kober, J. and Bagnell, J. A. and Peters, J."Reinforcement learning in robotics: A survey". The International Journal of Robotics Research, vol. 32, pp. 1238-1274, Sept. 2013.