Jan Peters

Research Group Leader

Empirical Inference

Max-Planck-Ring 4
Tübingen
Germany

+49-6151-16-25373

+49 7071 601 552

http://www.ias.informatik.tu-dar...

Jan Peters is a full professor (W3) for Intelligent Autonomous Systems at the Computer Science Department of the Technische Universitaet Darmstadt and at the same time a senior research scientist and group leader at the Max-Planck Institute for Intelligent Systems, where he heads the interdepartmental Robot Learning Group. Jan Peters has received the Dick Volz Best 2007 US PhD Thesis Runner-Up Award, the 2012 Robotics: Science & Systems - Early Career Spotlight, the 2013 INNS Young Investigator Award, and the IEEE Robotics & Automation Society‘s 2013 Early Career Award. In 2015, he was awarded an ERC Starting Grant. In 2019, he was appointed IEEE Fellow.

Jan Peters has studied Computer Science, Electrical, Mechanical and Control Engineering at TU Munich and FernUni Hagen in Germany, at the National University of Singapore (NUS) and the University of Southern California (USC), and been an visiting researcher at the ATR Telecomunications Research Center in Japan. He has received four Master‘s degrees in these disciplines as well as a Computer Science Ph.D. from USC.

If you are interested in working with me, please either apply through http://www.ausy.tu-darmstadt.de/Main/OpenPositions or send an official application to Ms. Lidia Pavel (email: lpavel@tuebingen.mpg.de, tel:+49 7071 601 1741). PLEASE REFRAIN FROM SENDING ME YOUR APPLICATION BY EMAIL! You will not get a reply as they will get lost in my aggressive spam filter.

Robot Learning Reinforcement Learning Machine Learning Robotics Biomimetic Systems

I am interested in many topics. Among them are

For more information see my private homepage and on my group's homepage.

Jan Peters is a full professor at Darmstadt University of Technology (TU Darmstadt) heading the intelligent systems group while still being affliated with the MPI heading the robot learning lab.

Jan Peters joined the Max-Planck Institute for Biological Cybernetics in April 2007 as a research scientist. Before joining MPI, Jan Peters studied Electrical Engineering, Computer Science and Mechanical Engineering. He graduated from the University of Hagen in 2000 with a Diplom-Informatiker (German M.Sc. in Computer Science) and from Munich University of Technology in 2001 with a Diplom-Ingenieur in Electrical Engineering (German M.Sc. in Electrical Engineering). In 2000-2001, he spent two semesters as visiting student at National University of Singapore. In 2002, he completed a M.Sc. in Computer Science and, in 2005, a M.S. in Mechanical Engineering both from USC. He joined USC's Computer Science Ph.D. program and the CLMC Lab in Fall 2001. Jan Peters has been a visiting research student at the Department of Robotics at the German Aerospace Research Center in Germany, at Siemens Advanced Engineering in Singapore and at the Department of Humanoid Robotics and Computational Neuroscience at the Advanded Telecommunication Research (ATR) Center in Japan. Jan Peters graduated from University of Southern California with a Ph.D. in Computer Science in March 2007. He remains affiliated with the CLMC Lab as an adjunct researcher.

More information on Jan Peters can be found on his private homepage.

Learning for Control

Figure: Muscle-based robotic arm serving as a testbed for Learning for Control. While it offers unique possibilities in terms of high accelerat...

Jan Peters Dieter Büchler Simon Guist Shane Gu Okan Koc

Robotics

Creating autonomous robots that can learn to assist humans in situations of daily life is a fascinating challenge for machine learning. While this aim has been a long-standing vision of artificial intelligence, we have yet to create robots that can learn to accomplish many different tasks triggered by environmental context or higher...
Jan Peters Sebastian Gomez-Gonzalez Dieter Büchler Okan Koc Yassine Nemmour Simon Guist Ge Li

365 results (View BibTeX file of all listed publications)

2014

Learning to Unscrew a Light Bulb from Demonstrations

Manschitz, S., Kober, J., Gienger, M., Peters, J.
In Proceedings for the joint conference of ISR 2014, 45th International Symposium on Robotics and Robotik 2014, 2014 (inproceedings)

[BibTex]

2014

Manschitz, S., Kober, J., Gienger, M., Peters, J. Learning to Unscrew a Light Bulb from Demonstrations In Proceedings for the joint conference of ISR 2014, 45th International Symposium on Robotics and Robotik 2014, 2014 (inproceedings)

[BibTex]

2013

Feedback Error Learning for Rhythmic Motor Primitives

Gopalan, N., Deisenroth, M., Peters, J.
In Proceedings of 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), pages: 1317-1322, 2013 (inproceedings)

PDF DOI [BibTex]

2013

Gopalan, N., Deisenroth, M., Peters, J. Feedback Error Learning for Rhythmic Motor Primitives In Proceedings of 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), pages: 1317-1322, 2013 (inproceedings)

PDF DOI [BibTex]

Alignment-based Transfer Learning for Robot Models

Bocsi, B., Csato, L., Peters, J.
In Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN 2013), pages: 1-7, 2013 (inproceedings)

PDF DOI [BibTex]

Bocsi, B., Csato, L., Peters, J. Alignment-based Transfer Learning for Robot Models In Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN 2013), pages: 1-7, 2013 (inproceedings)

PDF DOI [BibTex]

A probabilistic approach to robot trajectory generation

Paraschos, A., Neumann, G., Peters, J.
In Proceedings of the 13th IEEE International Conference on Humanoid Robots (HUMANOIDS), pages: 477-483, IEEE, 13th IEEE-RAS International Conference on Humanoid Robots, 2013 (inproceedings)

DOI [BibTex]

Paraschos, A., Neumann, G., Peters, J. A probabilistic approach to robot trajectory generation In Proceedings of the 13th IEEE International Conference on Humanoid Robots (HUMANOIDS), pages: 477-483, IEEE, 13th IEEE-RAS International Conference on Humanoid Robots, 2013 (inproceedings)

DOI [BibTex]

Autonomous Reinforcement Learning with Hierarchical REPS

Daniel, C., Neumann, G., Peters, J.
In Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN 2013), pages: 1-8, 2013 (inproceedings)

DOI [BibTex]

Daniel, C., Neumann, G., Peters, J. Autonomous Reinforcement Learning with Hierarchical REPS In Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN 2013), pages: 1-8, 2013 (inproceedings)

DOI [BibTex]

Object Modeling and Segmentation by Robot Interaction with Cluttered Environments

van Hoof, H., Krömer, O., Peters, J.
In Proceedings of the IEEE International Conference on Humanoid Robots (HUMANOIDS), pages: 169-176, IEEE, 13th IEEE-RAS International Conference on Humanoid Robots, 2013 (inproceedings)

DOI [BibTex]

van Hoof, H., Krömer, O., Peters, J. Object Modeling and Segmentation by Robot Interaction with Cluttered Environments In Proceedings of the IEEE International Conference on Humanoid Robots (HUMANOIDS), pages: 169-176, IEEE, 13th IEEE-RAS International Conference on Humanoid Robots, 2013 (inproceedings)

DOI [BibTex]

Probabilistic movement modeling for intention inference in human-robot interaction

Wang, Z., Mülling, K., Deisenroth, M., Ben Amor, H., Vogt, D., Schölkopf, B., Peters, J.
International Journal of Robotics Research, 32(7):841-858, 2013 (article)

PDF DOI [BibTex]

Wang, Z., Mülling, K., Deisenroth, M., Ben Amor, H., Vogt, D., Schölkopf, B., Peters, J. Probabilistic movement modeling for intention inference in human-robot interaction International Journal of Robotics Research, 32(7):841-858, 2013 (article)

PDF DOI [BibTex]

Data-Efficient Generalization of Robot Skills with Contextual Policy Search

Kupcsik, A., Deisenroth, M., Peters, J., Neumann, G.
In Proceedings of the 27th National Conference on Artificial Intelligence (AAAI 2013), (Editors: desJardins, M. and Littman, M. L.), AAAI Press, 2013 (inproceedings)

PDF [BibTex]

Kupcsik, A., Deisenroth, M., Peters, J., Neumann, G. Data-Efficient Generalization of Robot Skills with Contextual Policy Search In Proceedings of the 27th National Conference on Artificial Intelligence (AAAI 2013), (Editors: desJardins, M. and Littman, M. L.), AAAI Press, 2013 (inproceedings)

PDF [BibTex]

Probabilistic Model-based Imitation Learning

Englert, P., Paraschos, A., Peters, J., Deisenroth, M.
Adaptive Behavior Journal, 21(5):388-403, 2013 (article)

PDF DOI [BibTex]

Englert, P., Paraschos, A., Peters, J., Deisenroth, M. Probabilistic Model-based Imitation Learning Adaptive Behavior Journal, 21(5):388-403, 2013 (article)

PDF DOI [BibTex]

Towards Robot Skill Learning: From Simple Skills to Table Tennis

Peters, J., Kober, J., Mülling, K., Kroemer, O., Neumann, G.
In Machine Learning and Knowledge Discovery in Databases, Proceedings of the European Conference on Machine Learning, Part III (ECML 2013), LNCS 8190, pages: 627-631, (Editors: Blockeel, H.,Kersting, K., Nijssen, S., and Zelezný, F.), Springer, 2013 (inproceedings)

PDF DOI [BibTex]

Peters, J., Kober, J., Mülling, K., Kroemer, O., Neumann, G. Towards Robot Skill Learning: From Simple Skills to Table Tennis In Machine Learning and Knowledge Discovery in Databases, Proceedings of the European Conference on Machine Learning, Part III (ECML 2013), LNCS 8190, pages: 627-631, (Editors: Blockeel, H.,Kersting, K., Nijssen, S., and Zelezný, F.), Springer, 2013 (inproceedings)

PDF DOI [BibTex]

Learning Sequential Motor Tasks

Daniel, C., Neumann, G., Peters, J.
In Proceedings of 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), 2013 (inproceedings)

PDF [BibTex]

Daniel, C., Neumann, G., Peters, J. Learning Sequential Motor Tasks In Proceedings of 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), 2013 (inproceedings)

PDF [BibTex]

A Survey on Policy Search for Robotics, Foundations and Trends in Robotics

Deisenroth, M., Neumann, G., Peters, J.
Foundations and Trends in Robotics, 2(1-2):1-142, 2013 (article)

DOI [BibTex]

Deisenroth, M., Neumann, G., Peters, J. A Survey on Policy Search for Robotics, Foundations and Trends in Robotics Foundations and Trends in Robotics, 2(1-2):1-142, 2013 (article)

DOI [BibTex]

Reinforcement Learning in Robotics: A Review

Kober, J., Bagnell, D., Peters, J.
International Journal of Robotics Research, 32(11):1238–1274, 2013 (article)

PDF DOI [BibTex]

Kober, J., Bagnell, D., Peters, J. Reinforcement Learning in Robotics: A Review International Journal of Robotics Research, 32(11):1238–1274, 2013 (article)

PDF DOI [BibTex]

Information-Theoretic Motor Skill Learning

Neumann, G., Kupcsik, A., Deisenroth, M., Peters, J.
In Proceedings of the 27th AAAI 2013, Workshop on Intelligent Robotic Systems (AAAI 2013), 2013 (inproceedings)

[BibTex]

Neumann, G., Kupcsik, A., Deisenroth, M., Peters, J. Information-Theoretic Motor Skill Learning In Proceedings of the 27th AAAI 2013, Workshop on Intelligent Robotic Systems (AAAI 2013), 2013 (inproceedings)

[BibTex]

Learning to Select and Generalize Striking Movements in Robot Table Tennis

Mülling, K., Kober, J., Kroemer, O., Peters, J.
International Journal of Robotics Research, 32(3):263-279, 2013 (article)

PDF DOI [BibTex]

Mülling, K., Kober, J., Kroemer, O., Peters, J. Learning to Select and Generalize Striking Movements in Robot Table Tennis International Journal of Robotics Research, 32(3):263-279, 2013 (article)

PDF DOI [BibTex]

Model-based Imitation Learning by Probabilistic Trajectory Matching

Englert, P., Paraschos, A., Peters, J., Deisenroth, M.
In Proceedings of 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), pages: 1922-1927, 2013 (inproceedings)

PDF DOI [BibTex]

Englert, P., Paraschos, A., Peters, J., Deisenroth, M. Model-based Imitation Learning by Probabilistic Trajectory Matching In Proceedings of 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), pages: 1922-1927, 2013 (inproceedings)

PDF DOI [BibTex]

Proceedings of the 10th European Workshop on Reinforcement Learning, Volume 24

Deisenroth, M., Szepesvári, C., Peters, J.
pages: 173, JMLR, European Workshop On Reinforcement Learning, EWRL, 2013 (proceedings)

Web [BibTex]

Deisenroth, M., Szepesvári, C., Peters, J. Proceedings of the 10th European Workshop on Reinforcement Learning, Volume 24 pages: 173, JMLR, European Workshop On Reinforcement Learning, EWRL, 2013 (proceedings)

Web [BibTex]

Learning responsive robot behavior by imitation

Ben Amor, H., Vogt, D., Ewerton, M., Berger, E., Jung, B., Peters, J.
In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), pages: 3257-3264, IEEE, 2013 (inproceedings)

DOI [BibTex]

Ben Amor, H., Vogt, D., Ewerton, M., Berger, E., Jung, B., Peters, J. Learning responsive robot behavior by imitation In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), pages: 3257-3264, IEEE, 2013 (inproceedings)

DOI [BibTex]

Learning Skills with Motor Primitives

Peters, J., Kober, J., Mülling, K., Kroemer, O., Neumann, G.
In Proceedings of the 16th Yale Workshop on Adaptive and Learning Systems, 2013 (inproceedings)

[BibTex]

Peters, J., Kober, J., Mülling, K., Kroemer, O., Neumann, G. Learning Skills with Motor Primitives In Proceedings of the 16th Yale Workshop on Adaptive and Learning Systems, 2013 (inproceedings)

[BibTex]

Probabilistic Movement Primitives

Paraschos, A., Daniel, C., Peters, J., Neumann, G.
In Advances in Neural Information Processing Systems 26, pages: 2616-2624, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF PDF [BibTex]

Paraschos, A., Daniel, C., Peters, J., Neumann, G. Probabilistic Movement Primitives In Advances in Neural Information Processing Systems 26, pages: 2616-2624, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF PDF [BibTex]

2012

Online Kernel-based Learning for Task-Space Tracking Robot Control

Nguyen-Tuong, D., Peters, J.
IEEE Transactions on Neural Networks and Learning Systems, 23(9):1417-1425, May 2012 (article)

Abstract

Abstract—Task-space control of redundant robot systems based on analytical models is known to be susceptive to modeling errors. Here, data driven model learning methods may present an interesting alternative approach. However, learning models for task-space tracking control from sampled data is an illposed problem. In particular, the same input data point can yield many different output values, which can form a non-convex solution space. Because the problem is ill-posed, models cannot be learned from such data using common regression methods. While learning of task-space control mappings is globally illposed, it has been shown in recent work that it is locally a well-defined problem. In this paper, we use this insight to formulate a local, kernel-based learning approach for online model learning for task-space tracking control. We propose a parametrization for the local model which makes an application in task-space tracking control of redundant robots possible. The model parametrization further allows us to apply the kerneltrick and, therefore, enables a formulation within the kernel learning framework. For evaluations, we show the ability of the method for online model learning for task-space tracking control of redundant robots.

PDF DOI [BibTex]

2012

Nguyen-Tuong, D., Peters, J. Online Kernel-based Learning for Task-Space Tracking Robot Control IEEE Transactions on Neural Networks and Learning Systems, 23(9):1417-1425, May 2012 (article)

PDF DOI [BibTex]

Learning Tracking Control with Forward Models

Bócsi, B., Hennig, P., Csató, L., Peters, J.
In pages: 259 -264, IEEE International Conference on Robotics and Automation (ICRA), May 2012 (inproceedings)

Abstract

Performing task-space tracking control on redundant robot manipulators is a difficult problem. When the physical model of the robot is too complex or not available, standard methods fail and machine learning algorithms can have advantages. We propose an adaptive learning algorithm for tracking control of underactuated or non-rigid robots where the physical model of the robot is unavailable. The control method is based on the fact that forward models are relatively straightforward to learn and local inversions can be obtained via local optimization. We use sparse online Gaussian process inference to obtain a flexible probabilistic forward model and second order optimization to find the inverse mapping. Physical experiments indicate that this approach can outperform state-of-the-art tracking control algorithms in this context.

PDF Web DOI [BibTex]

Bócsi, B., Hennig, P., Csató, L., Peters, J. Learning Tracking Control with Forward Models In pages: 259 -264, IEEE International Conference on Robotics and Automation (ICRA), May 2012 (inproceedings)

PDF Web DOI [BibTex]

A Kernel-based Approach to Direct Action Perception

Kroemer, O., Ugur, E., Oztop, E., Peters, J.
In International Conference on Robotics and Automation (ICRA 2012), pages: 2605-2610, IEEE, IEEE International Conference on Robotics and Automation (ICRA), May 2012 (inproceedings)

Abstract

The direct perception of actions allows a robot to predict the afforded actions of observed novel objects. In addition to learning which actions are afforded, the robot must also learn to adapt its actions according to the object being manipulated. In this paper, we present a non-parametric approach to representing the affordance-bearing subparts of objects. This representation forms the basis of a kernel function for computing the similarity between different subparts. Using this kernel function, the robot can learn the required mappings to perform direct action perception. The proposed approach was successfully implemented on a real robot, which could then quickly learn to generalize grasping and pouring actions to novel objects.

PDF Web DOI [BibTex]

Kroemer, O., Ugur, E., Oztop, E., Peters, J. A Kernel-based Approach to Direct Action Perception In International Conference on Robotics and Automation (ICRA 2012), pages: 2605-2610, IEEE, IEEE International Conference on Robotics and Automation (ICRA), May 2012 (inproceedings)

PDF Web DOI [BibTex]

Hierarchical Relative Entropy Policy Search

Daniel, C., Neumann, G., Peters, J.
In Fifteenth International Conference on Artificial Intelligence and Statistics, 22, pages: 273-281, JMLR Proceedings, (Editors: Lawrence, N. D. and Girolami, M.), JMLR.org, AISTATS, April 2012 (inproceedings)

Abstract

Many real-world problems are inherently hierarchically structured. The use of this structure in an agent's policy may well be the key to improved scalability and higher performance. However, such hierarchical structures cannot be exploited by current policy search algorithms. We will concentrate on a basic, but highly relevant hierarchy - the `mixed option' policy. Here, a gating network first decides which of the options to execute and, subsequently, the option-policy determines the action. In this paper, we reformulate learning a hierarchical policy as a latent variable estimation problem and subsequently extend the Relative Entropy Policy Search (REPS) to the latent variable case. We show that our Hierarchical REPS can learn versatile solutions while also showing an increased performance in terms of learning speed and quality of the found policy in comparison to the nonhierarchical approach.

PDF Web [BibTex]

Daniel, C., Neumann, G., Peters, J. Hierarchical Relative Entropy Policy Search In Fifteenth International Conference on Artificial Intelligence and Statistics, 22, pages: 273-281, JMLR Proceedings, (Editors: Lawrence, N. D. and Girolami, M.), JMLR.org, AISTATS, April 2012 (inproceedings)

PDF Web [BibTex]

Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components

Lampert, C., Peters, J.
Journal of Real-Time Image Processing, 7(1):31-41, March 2012 (article)

Abstract

We describe RTblob, a high speed vision system that detects objects in cluttered scenes based on their color and shape at a speed of over 800 frames/s. Because the system is available as open-source software and relies only on off-the-shelf PC hardware components, it can provide the basis for multiple application scenarios. As an illustrative example, we show how RTblob can be used in a robotic table tennis scenario to estimate ball trajectories through 3D space simultaneously from four cameras images at a speed of 200 Hz.

PDF PDF DOI [BibTex]

Lampert, C., Peters, J. Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components Journal of Real-Time Image Processing, 7(1):31-41, March 2012 (article)

PDF PDF DOI [BibTex]

Structured Apprenticeship Learning

Boularias, A., Kroemer, O., Peters, J.
In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2012 (inproceedings)

PDF Web [BibTex]

Boularias, A., Kroemer, O., Peters, J. Structured Apprenticeship Learning In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2012 (inproceedings)

PDF Web [BibTex]

Point Cloud Completion Using Symmetries and Extrusions

Kroemer, O., Ben Amor, H., Ewerton, M., Peters, J.
In IEEE-RAS International Conference on Humanoid Robots , pages: 680-685, IEEE, HUMANOIDS, 2012 (inproceedings)

DOI [BibTex]

Kroemer, O., Ben Amor, H., Ewerton, M., Peters, J. Point Cloud Completion Using Symmetries and Extrusions In IEEE-RAS International Conference on Humanoid Robots , pages: 680-685, IEEE, HUMANOIDS, 2012 (inproceedings)

DOI [BibTex]

Robot Learning

Sigaud, O., Peters, J.
In Encyclopedia of the sciences of learning, (Editors: Seel, N.M.), Springer, Berlin, Germany, 2012 (inbook)

Web [BibTex]

Sigaud, O., Peters, J. Robot Learning In Encyclopedia of the sciences of learning, (Editors: Seel, N.M.), Springer, Berlin, Germany, 2012 (inbook)

Web [BibTex]

Probabilistic Modeling of Human Movements for Intention Inference

Wang, Z., Deisenroth, M., Ben Amor, H., Vogt, D., Schölkopf, B., Peters, J.
In Proceedings of Robotics: Science and Systems VIII, pages: 8, R:SS, 2012 (inproceedings)

Abstract

Inference of human intention may be an essential step towards understanding human actions [21] and is hence important for realizing efficient human-robot interaction. In this paper, we propose the Intention-Driven Dynamics Model (IDDM), a latent variable model for inferring unknown human intentions. We train the model based on observed human behaviors/actions and we introduce an approximate inference algorithm to efficiently infer the human’s intention from an ongoing action. We verify the feasibility of the IDDM in two scenarios, i.e., target inference in robot table tennis and action recognition for interactive humanoid robots. In both tasks, the IDDM achieves substantial improvements over state-of-the-art regression and classification.

PDF link (url) [BibTex]

Wang, Z., Deisenroth, M., Ben Amor, H., Vogt, D., Schölkopf, B., Peters, J. Probabilistic Modeling of Human Movements for Intention Inference In Proceedings of Robotics: Science and Systems VIII, pages: 8, R:SS, 2012 (inproceedings)

PDF link (url) [BibTex]

Solving Nonlinear Continuous State-Action-Observation POMDPs for Mechanical Systems with Gaussian Noise

Deisenroth, M., Peters, J.
In The 10th European Workshop on Reinforcement Learning (EWRL), 2012 (inproceedings)

[BibTex]

Deisenroth, M., Peters, J. Solving Nonlinear Continuous State-Action-Observation POMDPs for Mechanical Systems with Gaussian Noise In The 10th European Workshop on Reinforcement Learning (EWRL), 2012 (inproceedings)

[BibTex]

Reinforcement Learning in Robotics: A Survey

Kober, J., Peters, J.
In Reinforcement Learning, 12, pages: 579-610, (Editors: Wiering, M. and Otterlo, M.), Springer, Berlin, Germany, 2012 (inbook)

Abstract

As most action generation problems of autonomous robots can be phrased in terms of sequential decision problems, robotics offers a tremendously important and interesting application platform for reinforcement learning. Similarly, the real-world challenges of this domain pose a major real-world check for reinforcement learning. Hence, the interplay between both disciplines can be seen as promising as the one between physics and mathematics. Nevertheless, only a fraction of the scientists working on reinforcement learning are sufficiently tied to robotics to oversee most problems encountered in this context. Thus, we will bring the most important challenges faced by robot reinforcement learning to their attention. To achieve this goal, we will attempt to survey most work that has successfully applied reinforcement learning to behavior generation for real robots. We discuss how the presented successful approaches have been made tractable despite the complexity of the domain and will study how representations or the inclusion of prior knowledge can make a significant difference. As a result, a particular focus of our chapter lies on the choice between model-based and model-free as well as between value function-based and policy search methods. As a result, we obtain a fairly complete survey of robot reinforcement learning which should allow a general reinforcement learning researcher to understand this domain.

Web DOI [BibTex]

Kober, J., Peters, J. Reinforcement Learning in Robotics: A Survey In Reinforcement Learning, 12, pages: 579-610, (Editors: Wiering, M. and Otterlo, M.), Springer, Berlin, Germany, 2012 (inbook)

Web DOI [BibTex]

Multi-modal Learning for Dynamic Tactile Sensing. (IROS 2012 Workshop on Advances in tactile sensing and touch-based human-robot interaction)

Kroemer, O., Lampert, C., Peters, J.
2012 (poster)

[BibTex]

Kroemer, O., Lampert, C., Peters, J. Multi-modal Learning for Dynamic Tactile Sensing. (IROS 2012 Workshop on Advances in tactile sensing and touch-based human-robot interaction) 2012 (poster)

[BibTex]

Reinforcement learning to adjust parametrized motor primitives to new situations

Kober, J., Wilhelm, A., Oztop, E., Peters, J.
Autonomous Robots, 33(4):361-379, 2012 (article)

Abstract

Humans manage to adapt learned movements very quickly to new situations by generalizing learned behaviors from similar situations. In contrast, robots currently often need to re-learn the complete movement. In this paper, we propose a method that learns to generalize parametrized motor plans by adapting a small set of global parameters, called meta-parameters. We employ reinforcement learning to learn the required meta-parameters to deal with the current situation, described by states. We introduce an appropriate reinforcement learning algorithm based on a kernelized version of the reward-weighted regression. To show its feasibility, we evaluate this algorithm on a toy example and compare it to several previous approaches. Subsequently, we apply the approach to three robot tasks, i.e., the generalization of throwing movements in darts, of hitting movements in table tennis, and of throwing balls where the tasks are learned on several different real physical robots, i.e., a Barrett WAM, a BioRob, the JST-ICORP/SARCOS CBi and a Kuka KR 6.

PDF PDF DOI [BibTex]

Kober, J., Wilhelm, A., Oztop, E., Peters, J. Reinforcement learning to adjust parametrized motor primitives to new situations Autonomous Robots, 33(4):361-379, 2012 (article)

PDF PDF DOI [BibTex]

A Brain-Robot Interface for Studying Motor Learning after Stroke

Meyer, T., Peters, J., Brötz, D., Zander, T., Schölkopf, B., Soekadar, S., Grosse-Wentrup, M.
In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 4078 - 4083 , IEEE, Piscataway, NJ, USA, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Meyer, T., Peters, J., Brötz, D., Zander, T., Schölkopf, B., Soekadar, S., Grosse-Wentrup, M. A Brain-Robot Interface for Studying Motor Learning after Stroke In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 4078 - 4083 , IEEE, Piscataway, NJ, USA, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Generalization of Human Grasping for Multi-Fingered Robot Hands

Ben Amor, H., Kroemer, O., Hillenbrand, U., Neumann, G., Peters, J.
In IEEE/RSJ International Conference on Intelligent Robots and Systems , pages: 2043-2050, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Ben Amor, H., Kroemer, O., Hillenbrand, U., Neumann, G., Peters, J. Generalization of Human Grasping for Multi-Fingered Robot Hands In IEEE/RSJ International Conference on Intelligent Robots and Systems , pages: 2043-2050, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Learning Concurrent Motor Skills in Versatile Solution Spaces

Daniel, C., Neumann, G., Peters, J.
In IEEE/RSJ International Conference on Intelligent Robots and Systems , pages: 3591-3597, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Daniel, C., Neumann, G., Peters, J. Learning Concurrent Motor Skills in Versatile Solution Spaces In IEEE/RSJ International Conference on Intelligent Robots and Systems , pages: 3591-3597, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Learning to Select and Generalize Striking Movements in Robot Table Tennis

Mülling, K., Kober, J., Kroemer, O., Peters, J.
In AAAI Fall Symposium on Robots Learning Interactively from Human Teachers, 2012 (inproceedings)

[BibTex]

Mülling, K., Kober, J., Kroemer, O., Peters, J. Learning to Select and Generalize Striking Movements in Robot Table Tennis In AAAI Fall Symposium on Robots Learning Interactively from Human Teachers, 2012 (inproceedings)

[BibTex]

Structured Apprenticeship Learning

Boularias, A., Kroemer, O., Peters, J.
European Workshop on Reinforcement Learning (EWRL), 2012 (talk)

[BibTex]

Boularias, A., Kroemer, O., Peters, J. Structured Apprenticeship Learning European Workshop on Reinforcement Learning (EWRL), 2012 (talk)

[BibTex]

Toward Fast Policy Search for Learning Legged Locomotion

Deisenroth, M., Calandra, R., Seyfarth, A., Peters, J.
In IEEE/RSJ International Conference on Intelligent Robots and Systems , pages: 1787-1792, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Deisenroth, M., Calandra, R., Seyfarth, A., Peters, J. Toward Fast Policy Search for Learning Legged Locomotion In IEEE/RSJ International Conference on Intelligent Robots and Systems , pages: 1787-1792, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Robot Skill Learning

Peters, J., Kober, J., Mülling, K., Nguyen-Tuong, D., Kroemer, O.
In 20th European Conference on Artificial Intelligence , pages: 40-45, ECAI, 2012 (inproceedings)

PDF DOI [BibTex]

Peters, J., Kober, J., Mülling, K., Nguyen-Tuong, D., Kroemer, O. Robot Skill Learning In 20th European Conference on Artificial Intelligence , pages: 40-45, ECAI, 2012 (inproceedings)

PDF DOI [BibTex]

Solving Nonlinear Continuous State-Action-Observation POMDPs for Mechanical Systems with Gaussian Noise

Deisenroth, M., Peters, J.
European Workshop on Reinforcement Learning (EWRL), 2012 (talk)

[BibTex]

Deisenroth, M., Peters, J. Solving Nonlinear Continuous State-Action-Observation POMDPs for Mechanical Systems with Gaussian Noise European Workshop on Reinforcement Learning (EWRL), 2012 (talk)

[BibTex]

Learning Throwing and Catching Skills

Kober, J., Mülling, K., Peters, J.
In IEEE/RSJ International Conference on Intelligent Robots and Systems , pages: 5167-5168, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Kober, J., Mülling, K., Peters, J. Learning Throwing and Catching Skills In IEEE/RSJ International Conference on Intelligent Robots and Systems , pages: 5167-5168, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Maximally Informative Interaction Learning for Scene Exploration

van Hoof, H., Kroemer, O., Ben Amor, H., Peters, J.
In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages: 5152-5158, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

van Hoof, H., Kroemer, O., Ben Amor, H., Peters, J. Maximally Informative Interaction Learning for Scene Exploration In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages: 5152-5158, IROS, 2012 (inproceedings)

PDF DOI [BibTex]

Investigating the Neural Basis of Brain-Computer Interface (BCI)-based Stroke Rehabilitation

Meyer, T., Peters, J., Zander, T., Brötz, D., Soekadar, S., Schölkopf, B., Grosse-Wentrup, M.
In International Conference on NeuroRehabilitation (ICNR) , pages: 617-621, (Editors: JL Pons, D Torricelli, and M Pajaro), Springer, Berlin, Germany, ICNR, 2012 (inproceedings)

PDF [BibTex]

Meyer, T., Peters, J., Zander, T., Brötz, D., Soekadar, S., Schölkopf, B., Grosse-Wentrup, M. Investigating the Neural Basis of Brain-Computer Interface (BCI)-based Stroke Rehabilitation In International Conference on NeuroRehabilitation (ICNR) , pages: 617-621, (Editors: JL Pons, D Torricelli, and M Pajaro), Springer, Berlin, Germany, ICNR, 2012 (inproceedings)

PDF [BibTex]

Algorithms for Learning Markov Field Policies

Boularias, A., Kroemer, O., Peters, J.
In Advances in Neural Information Processing Systems 25, pages: 2186-2194, (Editors: P Bartlett and FCN Pereira and CJC. Burges and L Bottou and KQ Weinberger), Curran Associates Inc., 26th Annual Conference on Neural Information Processing Systems (NIPS), 2012 (inproceedings)

PDF [BibTex]

Boularias, A., Kroemer, O., Peters, J. Algorithms for Learning Markov Field Policies In Advances in Neural Information Processing Systems 25, pages: 2186-2194, (Editors: P Bartlett and FCN Pereira and CJC. Burges and L Bottou and KQ Weinberger), Curran Associates Inc., 26th Annual Conference on Neural Information Processing Systems (NIPS), 2012 (inproceedings)

PDF [BibTex]

2011

Reward-Weighted Regression with Sample Reuse for Direct Policy Search in Reinforcement Learning

Hachiya, H., Peters, J., Sugiyama, M.
Neural Computation, 23(11):2798-2832, November 2011 (article)

Abstract

Direct policy search is a promising reinforcement learning framework, in particular for controlling continuous, high-dimensional systems. Policy search often requires a large number of samples for obtaining a stable policy update estimator, and this is prohibitive when the sampling cost is expensive. In this letter, we extend an expectation-maximization-based policy search method so that previously collected samples can be efficiently reused. The usefulness of the proposed method, reward-weighted regression with sample reuse (R), is demonstrated through robot learning experiments.

Web DOI [BibTex]

2011

Hachiya, H., Peters, J., Sugiyama, M. Reward-Weighted Regression with Sample Reuse for Direct Policy Search in Reinforcement Learning Neural Computation, 23(11):2798-2832, November 2011 (article)

Web DOI [BibTex]

Model Learning in Robotics: a Survey

Nguyen-Tuong, D., Peters, J.
Cognitive Processing, 12(4):319-340, November 2011 (article)

Abstract

Models are among the most essential tools in robotics, such as kinematics and dynamics models of the robot's own body and controllable external objects. It is widely believed that intelligent mammals also rely on internal models in order to generate their actions. However, while classical robotics relies on manually generated models that are based on human insights into physics, future autonomous, cognitive robots need to be able to automatically generate models that are based on information which is extracted from the data streams accessible to the robot. In this paper, we survey the progress in model learning with a strong focus on robot control on a kinematic as well as dynamical level. Here, a model describes essential information about the behavior of the environment and the in uence of an agent on this environment. In the context of model based learning control, we view the model from three different perspectives. First, we need to study the dierent possible model learning architectures for robotics. Second, we discuss what kind of problems these architecture and the domain of robotics imply for the applicable learning methods. From this discussion, we deduce future directions of real-time learning algorithms. Third, we show where these scenarios have been used successfully in several case studies.

PDF [BibTex]

Nguyen-Tuong, D., Peters, J. Model Learning in Robotics: a Survey Cognitive Processing, 12(4):319-340, November 2011 (article)

PDF [BibTex]

A biomimetic approach to robot table tennis

Mülling, K., Kober, J., Peters, J.
Adaptive Behavior , 19(5):359-376 , October 2011 (article)

Abstract

Playing table tennis is a difficult motor task that requires fast movements, accurate control and adaptation to task parameters. Although human beings see and move slower than most robot systems, they significantly outperform all table tennis robots. One important reason for this higher performance is the human movement generation. In this paper, we study human movements during table tennis and present a robot system that mimics human striking behavior. Our focus lies on generating hitting motions capable of adapting to variations in environmental conditions, such as changes in ball speed and position. Therefore, we model the human movements involved in hitting a table tennis ball using discrete movement stages and the virtual hitting point hypothesis. The resulting model was evaluated both in a physically realistic simulation and on a real anthropomorphic seven degrees of freedom Barrett WAM™ robot arm.

PDF Web DOI [BibTex]

Mülling, K., Kober, J., Peters, J. A biomimetic approach to robot table tennis Adaptive Behavior , 19(5):359-376 , October 2011 (article)

PDF Web DOI [BibTex]

Learning anticipation policies for robot table tennis

Wang, Z., Lampert, C., Mülling, K., Schölkopf, B., Peters, J.
In pages: 332-337 , (Editors: NM Amato), IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2011 (inproceedings)

Abstract

Playing table tennis is a difficult task for robots, especially due to their limitations of acceleration. A key bottleneck is the amount of time needed to reach the desired hitting position and velocity of the racket for returning the incoming ball. Here, it often does not suffice to simply extrapolate the ball's trajectory after the opponent returns it but more information is needed. Humans are able to predict the ball's trajectory based on the opponent's moves and, thus, have a considerable advantage. Hence, we propose to incorporate an anticipation system into robot table tennis players, which enables the robot to react earlier while the opponent is performing the striking movement. Based on visual observation of the opponent's racket movement, the robot can predict the aim of the opponent and adjust its movement generation accordingly. The policies for deciding how and when to react are obtained by reinforcement learning. We conduct experiments with an existing robot player to show that the learned reaction policy can significantly improve the performance of the overall system.

PDF Web DOI [BibTex]

Wang, Z., Lampert, C., Mülling, K., Schölkopf, B., Peters, J. Learning anticipation policies for robot table tennis In pages: 332-337 , (Editors: NM Amato), IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2011 (inproceedings)

PDF Web DOI [BibTex]

Learning robot grasping from 3-D images with Markov Random Fields

Boularias, A., Kroemer, O., Peters, J.
In pages: 1548-1553 , (Editors: Amato, N.M.), IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2011 (inproceedings)

Abstract

Learning to grasp novel objects is an essential skill for robots operating in unstructured environments. We therefore propose a probabilistic approach for learning to grasp. In particular, we learn a function that predicts the success probability of grasps performed on surface points of a given object. Our approach is based on Markov Random Fields (MRF), and motivated by the fact that points that are geometrically close to each other tend to have similar grasp success probabilities. The MRF approach is successfully tested in simulation, and on a real robot using 3-D scans of various types of objects. The empirical results show a significant improvement over methods that do not utilize the smoothness assumption and classify each point separately from the others.

PDF Web DOI [BibTex]

Boularias, A., Kroemer, O., Peters, J. Learning robot grasping from 3-D images with Markov Random Fields In pages: 1548-1553 , (Editors: Amato, N.M.), IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2011 (inproceedings)

PDF Web DOI [BibTex]

← Previous

1

2

3

4

5

6

7

8

Next →

Jan Peters

2014

2014

2013

2013

2012

2012

2011

2011

Latest News

Links

Contact Us