Zhikun Wang

Alumni

Empirical Inference

Overview
Publications

Recent advances in sensors and algorithms allow for robots with improved perception abilities. However, effective perception alone may not be sufficient for human-robot interaction, since the robot's reaction should depend on understanding the human's intention. Hence, my research interests lie in the strategic level of human-robot interaction, which serves as a bridge between perception of human action and planning for reaction. On one side, the robot needs to infer the underlying intention of humans. On the other side, efficient planning for reaction can be achieved by utilizing motor skills with reactive policies learned to choose the right skill at the right time.

I have been developing and implementing machine learning algorithms for intention inference and learning reactive policies. I have chosen robot table tennis as a benchmark, as it is a sufficiently complex scenario for evaluation while intuition still allows interpreting the results. We have achieved promising experimental results, which exhibit their potentials in many other human-robot interaction scenarios.

See http://robot-learning.de/Team/ZhikunWang for more information.

11 results (BibTeX)

2017

Anticipatory Action Selection for Human-Robot Table Tennis

Wang, Z., Boularias, A., Mülling, K., Schölkopf, B., Peters, J.

Artificial Intelligence, 247, pages: 399-414, 2017, Special Issue on AI and Robotics (article)

Abstract

Abstract Anticipation can enhance the capability of a robot in its interaction with humans, where the robot predicts the humans' intention for selecting its own action. We present a novel framework of anticipatory action selection for human-robot interaction, which is capable to handle nonlinear and stochastic human behaviors such as table tennis strokes and allows the robot to choose the optimal action based on prediction of the human partner's intention with uncertainty. The presented framework is generic and can be used in many human-robot interaction scenarios, for example, in navigation and human-robot co-manipulation. In this article, we conduct a case study on human-robot table tennis. Due to the limited amount of time for executing hitting movements, a robot usually needs to initiate its hitting movement before the opponent hits the ball, which requires the robot to be anticipatory based on visual observation of the opponent's movement. Previous work on Intention-Driven Dynamics Models (IDDM) allowed the robot to predict the intended target of the opponent. In this article, we address the problem of action selection and optimal timing for initiating a chosen action by formulating the anticipatory action selection as a Partially Observable Markov Decision Process (POMDP), where the transition and observation are modeled by the \{IDDM\} framework. We present two approaches to anticipatory action selection based on the \{POMDP\} formulation, i.e., a model-free policy learning method based on Least-Squares Policy Iteration (LSPI) that employs the \{IDDM\} for belief updates, and a model-based Monte-Carlo Planning (MCP) method, which benefits from the transition and observation model by the IDDM. Experimental results using real data in a simulated environment show the importance of anticipatory action selection, and that \{POMDPs\} are suitable to formulate the anticipatory action selection problem by taking into account the uncertainties in prediction. We also show that existing algorithms for POMDPs, such as \{LSPI\} and MCP, can be applied to substantially improve the robot's performance in its interaction with humans.

DOI [BibTex]

2017

Wang, Z., Boularias, A., Mülling, K., Schölkopf, B., Peters, J. Anticipatory Action Selection for Human-Robot Table Tennis Artificial Intelligence, 247, pages: 399-414, 2017, Special Issue on AI and Robotics (article)

DOI [BibTex]

2016

On estimation of functional causal models: General results and application to post-nonlinear causal model

Zhang, K., Wang, Z., Zhang, J., Schölkopf, B.

ACM Transactions on Intelligent Systems and Technologies, 7(2), January 2016 (article)

PDF DOI [BibTex]

2016

Zhang, K., Wang, Z., Zhang, J., Schölkopf, B. On estimation of functional causal models: General results and application to post-nonlinear causal model ACM Transactions on Intelligent Systems and Technologies, 7(2), January 2016 (article)

PDF DOI [BibTex]

2014

Single-Source Domain Adaptation with Target and Conditional Shift

Zhang, K., Schölkopf, B., Muandet, K., Wang, Z., Zhou, Z., Persello, C.

In Regularization, Optimization, Kernels, and Support Vector Machines, pages: 427-456, 19, Chapman & Hall/CRC Machine Learning & Pattern Recognition, (Editors: Suykens, J. A. K., Signoretto, M. and Argyriou, A.), Chapman and Hall/CRC, Boca Raton, USA, 2014 (inbook)

Project Page [BibTex]

2014

Zhang, K., Schölkopf, B., Muandet, K., Wang, Z., Zhou, Z., Persello, C. Single-Source Domain Adaptation with Target and Conditional Shift In Regularization, Optimization, Kernels, and Support Vector Machines, pages: 427-456, 19, Chapman & Hall/CRC Machine Learning & Pattern Recognition, (Editors: Suykens, J. A. K., Signoretto, M. and Argyriou, A.), Chapman and Hall/CRC, Boca Raton, USA, 2014 (inbook)

Project Page [BibTex]

2013

Intention Inference and Decision Making with Hierarchical Gaussian Process Dynamics Models

Wang, Z.

Technical University Darmstadt, Germany, 2013 (phdthesis)

[BibTex]

2013

Wang, Z. Intention Inference and Decision Making with Hierarchical Gaussian Process Dynamics Models Technical University Darmstadt, Germany, 2013 (phdthesis)

[BibTex]

Domain adaptation under Target and Conditional Shift

Zhang, K., Schölkopf, B., Muandet, K., Wang, Z.

In Proceedings of the 30th International Conference on Machine Learning, W&CP 28 (3), pages: 819–827, (Editors: S Dasgupta and D McAllester), JMLR, ICML, 2013 (inproceedings)

PDF Web Project Page [BibTex]

Zhang, K., Schölkopf, B., Muandet, K., Wang, Z. Domain adaptation under Target and Conditional Shift In Proceedings of the 30th International Conference on Machine Learning, W&CP 28 (3), pages: 819–827, (Editors: S Dasgupta and D McAllester), JMLR, ICML, 2013 (inproceedings)

PDF Web Project Page [BibTex]

On estimation of functional causal models: Post-nonlinear causal model as an example

Zhang, K., Wang, Z., Schölkopf, B.

In First IEEE ICDM workshop on causal discovery , 2013, Held in conjunction with the 12th IEEE International Conference on Data Mining (ICDM 2013) (inproceedings)

PDF [BibTex]

Zhang, K., Wang, Z., Schölkopf, B. On estimation of functional causal models: Post-nonlinear causal model as an example In First IEEE ICDM workshop on causal discovery , 2013, Held in conjunction with the 12th IEEE International Conference on Data Mining (ICDM 2013) (inproceedings)

PDF [BibTex]

Probabilistic movement modeling for intention inference in human-robot interaction

Wang, Z., Mülling, K., Deisenroth, M., Ben Amor, H., Vogt, D., Schölkopf, B., Peters, J.

International Journal of Robotics Research, 32(7):841-858, 2013 (article)

PDF DOI [BibTex]

Wang, Z., Mülling, K., Deisenroth, M., Ben Amor, H., Vogt, D., Schölkopf, B., Peters, J. Probabilistic movement modeling for intention inference in human-robot interaction International Journal of Robotics Research, 32(7):841-858, 2013 (article)

PDF DOI [BibTex]

2012

Probabilistic Modeling of Human Movements for Intention Inference

Wang, Z., Deisenroth, M., Ben Amor, H., Vogt, D., Schölkopf, B., Peters, J.

In Proceedings of Robotics: Science and Systems VIII, pages: 8, R:SS, 2012 (inproceedings)

Abstract

Inference of human intention may be an essential step towards understanding human actions [21] and is hence important for realizing efficient human-robot interaction. In this paper, we propose the Intention-Driven Dynamics Model (IDDM), a latent variable model for inferring unknown human intentions. We train the model based on observed human behaviors/actions and we introduce an approximate inference algorithm to efficiently infer the human’s intention from an ongoing action. We verify the feasibility of the IDDM in two scenarios, i.e., target inference in robot table tennis and action recognition for interactive humanoid robots. In both tasks, the IDDM achieves substantial improvements over state-of-the-art regression and classification.

PDF link (url) Project Page [BibTex]

2012

Wang, Z., Deisenroth, M., Ben Amor, H., Vogt, D., Schölkopf, B., Peters, J. Probabilistic Modeling of Human Movements for Intention Inference In Proceedings of Robotics: Science and Systems VIII, pages: 8, R:SS, 2012 (inproceedings)

PDF link (url) Project Page [BibTex]

2011

Learning anticipation policies for robot table tennis

Wang, Z., Lampert, C., Mülling, K., Schölkopf, B., Peters, J.

In pages: 332-337 , (Editors: NM Amato), IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2011 (inproceedings)

Abstract

Playing table tennis is a difficult task for robots, especially due to their limitations of acceleration. A key bottleneck is the amount of time needed to reach the desired hitting position and velocity of the racket for returning the incoming ball. Here, it often does not suffice to simply extrapolate the ball's trajectory after the opponent returns it but more information is needed. Humans are able to predict the ball's trajectory based on the opponent's moves and, thus, have a considerable advantage. Hence, we propose to incorporate an anticipation system into robot table tennis players, which enables the robot to react earlier while the opponent is performing the striking movement. Based on visual observation of the opponent's racket movement, the robot can predict the aim of the opponent and adjust its movement generation accordingly. The policies for deciding how and when to react are obtained by reinforcement learning. We conduct experiments with an existing robot player to show that the learned reaction policy can significantly improve the performance of the overall system.

PDF Web DOI [BibTex]

2011

Wang, Z., Lampert, C., Mülling, K., Schölkopf, B., Peters, J. Learning anticipation policies for robot table tennis In pages: 332-337 , (Editors: NM Amato), IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2011 (inproceedings)

PDF Web DOI [BibTex]

Balancing Safety and Exploitability in Opponent Modeling

Wang, Z., Boularias, A., Mülling, K., Peters, J.

In pages: 1515-1520, (Editors: Burgard, W. , D. Roth), AAAI Press, Menlo Park, CA, USA, Twenty-Fifth AAAI Conference on Artificial Intelligence (AAAI), August 2011 (inproceedings)

Abstract

Opponent modeling is a critical mechanism in repeated games. It allows a player to adapt its strategy in order to better respond to the presumed preferences of his opponents. We introduce a new modeling technique that adaptively balances exploitability and risk reduction. An opponent’s strategy is modeled with a set of possible strategies that contain the actual strategy with a high probability. The algorithm is safe as the expected payoff is above the minimax payoff with a high probability, and can exploit the opponents’ preferences when sufficient observations have been obtained. We apply them to normal-form games and stochastic games with a finite number of stages. The performance of the proposed approach is first demonstrated on repeated rock-paper-scissors games. Subsequently, the approach is evaluated in a human-robot table-tennis setting where the robot player learns to prepare to return a served ball. By modeling the human players, the robot chooses a forehand, backhand or middle preparation pose before they serve. The learned strategies can exploit the opponent’s preferences, leading to a higher rate of successful returns.

PDF Web [BibTex]

Wang, Z., Boularias, A., Mülling, K., Peters, J. Balancing Safety and Exploitability in Opponent Modeling In pages: 1515-1520, (Editors: Burgard, W. , D. Roth), AAAI Press, Menlo Park, CA, USA, Twenty-Fifth AAAI Conference on Artificial Intelligence (AAAI), August 2011 (inproceedings)

PDF Web [BibTex]

2010

Learning as a key ability for Human-Friendly Robots

Peters, J., Kober, J., Mülling, K., Krömer, O., Nguyen-Tuong, D., Wang, Z., Rodriguez Gomez, M., Grosse-Wentrup, M.

In pages: 1-2, 3rd Workshop for Young Researchers on Human-Friendly Robotics (HFR), October 2010 (inproceedings)

Web [BibTex]

2010

Peters, J., Kober, J., Mülling, K., Krömer, O., Nguyen-Tuong, D., Wang, Z., Rodriguez Gomez, M., Grosse-Wentrup, M. Learning as a key ability for Human-Friendly Robots In pages: 1-2, 3rd Workshop for Young Researchers on Human-Friendly Robotics (HFR), October 2010 (inproceedings)

Web [BibTex]

Zhikun Wang

2017

Anticipatory Action Selection for Human-Robot Table Tennis

2017

2016

On estimation of functional causal models: General results and application to post-nonlinear causal model

2016

2014

Single-Source Domain Adaptation with Target and Conditional Shift

2014

2013

Intention Inference and Decision Making with Hierarchical Gaussian Process Dynamics Models

2013

Domain adaptation under Target and Conditional Shift

On estimation of functional causal models: Post-nonlinear causal model as an example

Probabilistic movement modeling for intention inference in human-robot interaction

2012

Probabilistic Modeling of Human Movements for Intention Inference

2012

2011

Learning anticipation policies for robot table tennis

2011

Balancing Safety and Exploitability in Opponent Modeling

2010

Learning as a key ability for Human-Friendly Robots

2010

Latest News

Links

Contact Us