Publications | Empirical Inference - Max Planck Institute for Intelligent Systems

174 results (View BibTeX file of all listed publications)

2019

Selecting causal brain features with a single conditional independence test per feature

Mastakouri, A., Schölkopf, B., Janzing, D.

Advances in Neural Information Processing Systems 32, 33rd Annual Conference on Neural Information Processing Systems, December 2019 (conference) Accepted

[BibTex]

2019

Mastakouri, A., Schölkopf, B., Janzing, D. Selecting causal brain features with a single conditional independence test per feature Advances in Neural Information Processing Systems 32, 33rd Annual Conference on Neural Information Processing Systems, December 2019 (conference) Accepted

[BibTex]

Neural Signatures of Motor Skill in the Resting Brain

Ozdenizci, O., Meyer, T., Wichmann, F., Peters, J., Schölkopf, B., Cetin, M., Grosse-Wentrup, M.

Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC 2019), October 2019 (conference) Accepted

[BibTex]

Ozdenizci, O., Meyer, T., Wichmann, F., Peters, J., Schölkopf, B., Cetin, M., Grosse-Wentrup, M. Neural Signatures of Motor Skill in the Resting Brain Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC 2019), October 2019 (conference) Accepted

[BibTex]

Beta Power May Mediate the Effect of Gamma-TACS on Motor Performance

Mastakouri, A., Schölkopf, B., Grosse-Wentrup, M.

Engineering in Medicine and Biology Conference (EMBC), July 2019 (conference) Accepted

arXiv PDF [BibTex]

Mastakouri, A., Schölkopf, B., Grosse-Wentrup, M. Beta Power May Mediate the Effect of Gamma-TACS on Motor Performance Engineering in Medicine and Biology Conference (EMBC), July 2019 (conference) Accepted

arXiv PDF [BibTex]

Coordinating Users of Shared Facilities via Data-driven Predictive Assistants and Game Theory

Geiger, P., Besserve, M., Winkelmann, J., Proissl, C., Schölkopf, B.

Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI), pages: 49, (Editors: Amir Globerson and Ricardo Silva), AUAI Press, July 2019 (conference)

link (url) [BibTex]

Geiger, P., Besserve, M., Winkelmann, J., Proissl, C., Schölkopf, B. Coordinating Users of Shared Facilities via Data-driven Predictive Assistants and Game Theory Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI), pages: 49, (Editors: Amir Globerson and Ricardo Silva), AUAI Press, July 2019 (conference)

link (url) [BibTex]

The Sensitivity of Counterfactual Fairness to Unmeasured Confounding

Kilbertus, N., Ball, P. J., Kusner, M. J., Weller, A., Silva, R.

Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI), pages: 213, (Editors: Amir Globerson and Ricardo Silva), AUAI Press, July 2019 (conference)

link (url) [BibTex]

Kilbertus, N., Ball, P. J., Kusner, M. J., Weller, A., Silva, R. The Sensitivity of Counterfactual Fairness to Unmeasured Confounding Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI), pages: 213, (Editors: Amir Globerson and Ricardo Silva), AUAI Press, July 2019 (conference)

link (url) [BibTex]

The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA

Gresele*, L., Rubenstein*, P. K., Mehrjou, A., Locatello, F., Schölkopf, B.

Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI), pages: 53, (Editors: Amir Globerson and Ricardo Silva), AUAI Press, July 2019, *equal contribution (conference)

link (url) [BibTex]

Gresele*, L., Rubenstein*, P. K., Mehrjou, A., Locatello, F., Schölkopf, B. The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI), pages: 53, (Editors: Amir Globerson and Ricardo Silva), AUAI Press, July 2019, *equal contribution (conference)

link (url) [BibTex]

Random Sum-Product Networks: A Simple and Effective Approach to Probabilistic Deep Learning

Peharz, R., Vergari, A., Stelzner, K., Molina, A., Shao, X., Trapp, M., Kersting, K., Ghahramani, Z.

Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI), pages: 124, (Editors: Amir Globerson and Ricardo Silva), AUAI Press, July 2019 (conference)

link (url) [BibTex]

Peharz, R., Vergari, A., Stelzner, K., Molina, A., Shao, X., Trapp, M., Kersting, K., Ghahramani, Z. Random Sum-Product Networks: A Simple and Effective Approach to Probabilistic Deep Learning Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI), pages: 124, (Editors: Amir Globerson and Ricardo Silva), AUAI Press, July 2019 (conference)

link (url) [BibTex]

Kernel Mean Matching for Content Addressability of GANs

Jitkrittum*, W., Sangkloy*, P., Gondal, M. W., Raj, A., Hays, J., Schölkopf, B.

Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 3140-3151, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019, *equal contribution (conference)

PDF link (url) [BibTex]

Jitkrittum*, W., Sangkloy*, P., Gondal, M. W., Raj, A., Hays, J., Schölkopf, B. Kernel Mean Matching for Content Addressability of GANs Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 3140-3151, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019, *equal contribution (conference)

PDF link (url) [BibTex]

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

Locatello, F., Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Schölkopf, B., Bachem, O.

Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 4114-4124, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

PDF link (url) [BibTex]

Locatello, F., Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Schölkopf, B., Bachem, O. Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 4114-4124, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

PDF link (url) [BibTex]

Local Temporal Bilinear Pooling for Fine-grained Action Parsing

Zhang, Y., Tang, S., Muandet, K., Jarvers, C., Neumann, H.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2019, June 2019 (inproceedings)

Abstract

Fine-grained temporal action parsing is important in many applications, such as daily activity understanding, human motion analysis, surgical robotics and others requiring subtle and precise operations in a long-term period. In this paper we propose a novel bilinear pooling operation, which is used in intermediate layers of a temporal convolutional encoder-decoder net. In contrast to other work, our proposed bilinear pooling is learnable and hence can capture more complex local statistics than the conventional counterpart. In addition, we introduce exact lower-dimension representations of our bilinear forms, so that the dimensionality is reduced with neither information loss nor extra computation. We perform intensive experiments to quantitatively analyze our model and show the superior performances to other state-of-the-art work on various datasets.

Code video demo pdf link (url) [BibTex]

Zhang, Y., Tang, S., Muandet, K., Jarvers, C., Neumann, H. Local Temporal Bilinear Pooling for Fine-grained Action Parsing In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2019, June 2019 (inproceedings)

Code video demo pdf link (url) [BibTex]

Generate Semantically Similar Images with Kernel Mean Matching

Jitkrittum*, W., Sangkloy*, P., Gondal, M. W., Raj, A., Hays, J., Schölkopf, B.

6th Workshop Women in Computer Vision (WiCV) (oral presentation), June 2019, *equal contribution (conference) Accepted

[BibTex]

Jitkrittum*, W., Sangkloy*, P., Gondal, M. W., Raj, A., Hays, J., Schölkopf, B. Generate Semantically Similar Images with Kernel Mean Matching 6th Workshop Women in Computer Vision (WiCV) (oral presentation), June 2019, *equal contribution (conference) Accepted

[BibTex]

Projections for Approximate Policy Iteration Algorithms

Akrour, R., Pajarinen, J., Peters, J., Neumann, G.

Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 181-190, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

link (url) [BibTex]

Akrour, R., Pajarinen, J., Peters, J., Neumann, G. Projections for Approximate Policy Iteration Algorithms Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 181-190, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

link (url) [BibTex]

Switching Linear Dynamics for Variational Bayes Filtering

Becker-Ehmck, P., Peters, J., van der Smagt, P.

Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 553-562, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

link (url) [BibTex]

Becker-Ehmck, P., Peters, J., van der Smagt, P. Switching Linear Dynamics for Variational Bayes Filtering Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 553-562, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

link (url) [BibTex]

Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness

Suter, R., Miladinovic, D., Schölkopf, B., Bauer, S.

Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 6056-6065, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

PDF link (url) [BibTex]

Suter, R., Miladinovic, D., Schölkopf, B., Bauer, S. Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 6056-6065, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

PDF link (url) [BibTex]

First-Order Adversarial Vulnerability of Neural Networks and Input Dimension

Simon-Gabriel, C., Ollivier, Y., Bottou, L., Schölkopf, B., Lopez-Paz, D.

Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 5809-5817, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

PDF link (url) [BibTex]

Simon-Gabriel, C., Ollivier, Y., Bottou, L., Schölkopf, B., Lopez-Paz, D. First-Order Adversarial Vulnerability of Neural Networks and Input Dimension Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 5809-5817, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (conference)

PDF link (url) [BibTex]

Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models

Ialongo, A. D., Van Der Wilk, M., Hensman, J., Rasmussen, C. E.

In Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 2931-2940, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (inproceedings)

PDF link (url) [BibTex]

Ialongo, A. D., Van Der Wilk, M., Hensman, J., Rasmussen, C. E. Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models In Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 2931-2940, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, June 2019 (inproceedings)

PDF link (url) [BibTex]

Meta learning variational inference for prediction

Gordon, J., Bronskill, J., Bauer, M., Nowozin, S., Turner, R.

7th International Conference on Learning Representations (ICLR), May 2019 (conference)

arXiv link (url) [BibTex]

Gordon, J., Bronskill, J., Bauer, M., Nowozin, S., Turner, R. Meta learning variational inference for prediction 7th International Conference on Learning Representations (ICLR), May 2019 (conference)

arXiv link (url) [BibTex]

Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning

Lutter, M., Ritter, C., Peters, J.

7th International Conference on Learning Representations (ICLR), May 2019 (conference)

link (url) [BibTex]

Lutter, M., Ritter, C., Peters, J. Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning 7th International Conference on Learning Representations (ICLR), May 2019 (conference)

link (url) [BibTex]

DeepOBS: A Deep Learning Optimizer Benchmark Suite

Schneider, F., Balles, L., Hennig, P.

7th International Conference on Learning Representations (ICLR), May 2019 (conference)

link (url) [BibTex]

Schneider, F., Balles, L., Hennig, P. DeepOBS: A Deep Learning Optimizer Benchmark Suite 7th International Conference on Learning Representations (ICLR), May 2019 (conference)

link (url) [BibTex]

Disentangled State Space Models: Unsupervised Learning of Dynamics across Heterogeneous Environments

Miladinović*, D., Gondal*, M. W., Schölkopf, B., Buhmann, J. M., Bauer, S.

Deep Generative Models for Highly Structured Data Workshop at ICLR, May 2019, *equal contribution (conference)

link (url) [BibTex]

Miladinović*, D., Gondal*, M. W., Schölkopf, B., Buhmann, J. M., Bauer, S. Disentangled State Space Models: Unsupervised Learning of Dynamics across Heterogeneous Environments Deep Generative Models for Highly Structured Data Workshop at ICLR, May 2019, *equal contribution (conference)

link (url) [BibTex]

SOM-VAE: Interpretable Discrete Representation Learning on Time Series

Fortuin, V., Hüser, M., Locatello, F., Strathmann, H., Rätsch, G.

7th International Conference on Learning Representations (ICLR), May 2019 (conference)

link (url) [BibTex]

Fortuin, V., Hüser, M., Locatello, F., Strathmann, H., Rätsch, G. SOM-VAE: Interpretable Discrete Representation Learning on Time Series 7th International Conference on Learning Representations (ICLR), May 2019 (conference)

link (url) [BibTex]

Resampled Priors for Variational Autoencoders

Bauer, M., Mnih, A.

22nd International Conference on Artificial Intelligence and Statistics, April 2019 (conference) Accepted

arXiv [BibTex]

Bauer, M., Mnih, A. Resampled Priors for Variational Autoencoders 22nd International Conference on Artificial Intelligence and Statistics, April 2019 (conference) Accepted

arXiv [BibTex]

Semi-Generative Modelling: Covariate-Shift Adaptation with Cause and Effect Features

von Kügelgen, J., Mey, A., Loog, M.

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1361-1369, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

PDF link (url) [BibTex]

von Kügelgen, J., Mey, A., Loog, M. Semi-Generative Modelling: Covariate-Shift Adaptation with Cause and Effect Features Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1361-1369, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

PDF link (url) [BibTex]

Sobolev Descent

Mroueh, Y., Sercu, T., Raj, A.

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 2976-2985, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

PDF link (url) [BibTex]

Mroueh, Y., Sercu, T., Raj, A. Sobolev Descent Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 2976-2985, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

PDF link (url) [BibTex]

Fast and Robust Shortest Paths on Manifolds Learned from Data

Arvanitidis, G., Hauberg, S., Hennig, P., Schober, M.

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1506-1515, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

PDF link (url) [BibTex]

Arvanitidis, G., Hauberg, S., Hennig, P., Schober, M. Fast and Robust Shortest Paths on Manifolds Learned from Data Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1506-1515, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

PDF link (url) [BibTex]

Active Probabilistic Inference on Matrices for Pre-Conditioning in Stochastic Optimization

de Roos, F., Hennig, P.

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1448-1457, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

Abstract

Pre-conditioning is a well-known concept that can significantly improve the convergence of optimization algorithms. For noise-free problems, where good pre-conditioners are not known a priori, iterative linear algebra methods offer one way to efficiently construct them. For the stochastic optimization problems that dominate contemporary machine learning, however, this approach is not readily available. We propose an iterative algorithm inspired by classic iterative linear solvers that uses a probabilistic model to actively infer a pre-conditioner in situations where Hessian-projections can only be constructed with strong Gaussian noise. The algorithm is empirically demonstrated to efficiently construct effective pre-conditioners for stochastic gradient descent and its variants. Experiments on problems of comparably low dimensionality show improved convergence. In very high-dimensional problems, such as those encountered in deep learning, the pre-conditioner effectively becomes an automatic learning-rate adaptation scheme, which we also empirically show to work well.

PDF link (url) [BibTex]

de Roos, F., Hennig, P. Active Probabilistic Inference on Matrices for Pre-Conditioning in Stochastic Optimization Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1448-1457, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

PDF link (url) [BibTex]

Fast Gaussian Process Based Gradient Matching for Parameter Identification in Systems of Nonlinear ODEs

Wenk, P., Gotovos, A., Bauer, S., Gorbach, N., Krause, A., Buhmann, J. M.

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1351-1360, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

PDF PDF link (url) [BibTex]

Wenk, P., Gotovos, A., Bauer, S., Gorbach, N., Krause, A., Buhmann, J. M. Fast Gaussian Process Based Gradient Matching for Parameter Identification in Systems of Nonlinear ODEs Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1351-1360, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

PDF PDF link (url) [BibTex]

Witnessing Adversarial Training in Reproducing Kernel Hilbert Spaces

Mehrjou, A., Jitkrittum, W., Schölkopf, B., Muandet, K.

2019 (conference) Submitted

arXiv [BibTex]

Mehrjou, A., Jitkrittum, W., Schölkopf, B., Muandet, K. Witnessing Adversarial Training in Reproducing Kernel Hilbert Spaces 2019 (conference) Submitted

arXiv [BibTex]

AReS and MaRS Adversarial and MMD-Minimizing Regression for SDEs

Abbati*, G., Wenk*, P., Osborne, M. A., Krause, A., Schölkopf, B., Bauer, S.

Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 1-10, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, 2019, *equal contribution (conference)

PDF link (url) [BibTex]

Abbati*, G., Wenk*, P., Osborne, M. A., Krause, A., Schölkopf, B., Bauer, S. AReS and MaRS Adversarial and MMD-Minimizing Regression for SDEs Proceedings of the 36th International Conference on Machine Learning (ICML), 97, pages: 1-10, Proceedings of Machine Learning Research, (Editors: Chaudhuri, Kamalika and Salakhutdinov, Ruslan), PMLR, 2019, *equal contribution (conference)

PDF link (url) [BibTex]

Kernel Stein Tests for Multiple Model Comparison

Lim, J. N., Yamada, M., Schölkopf, B., Jitkrittum, W.

Advances in Neural Information Processing Systems 32, 33rd Annual Conference on Neural Information Processing Systems, 2019 (conference) To be published

[BibTex]

Lim, J. N., Yamada, M., Schölkopf, B., Jitkrittum, W. Kernel Stein Tests for Multiple Model Comparison Advances in Neural Information Processing Systems 32, 33rd Annual Conference on Neural Information Processing Systems, 2019 (conference) To be published

[BibTex]

MYND: A Platform for Large-scale Neuroscientific Studies

Hohmann, M. R., Hackl, M., Wirth, B., Zaman, T., Enficiaud, R., Grosse-Wentrup, M., Schölkopf, B.

Proceedings of the 2019 Conference on Human Factors in Computing Systems (CHI), 2019 (conference) Accepted

[BibTex]

Hohmann, M. R., Hackl, M., Wirth, B., Zaman, T., Enficiaud, R., Grosse-Wentrup, M., Schölkopf, B. MYND: A Platform for Large-scale Neuroscientific Studies Proceedings of the 2019 Conference on Human Factors in Computing Systems (CHI), 2019 (conference) Accepted

[BibTex]

A Kernel Stein Test for Comparing Latent Variable Models

Kanagawa, H., Jitkrittum, W., Mackey, L., Fukumizu, K., Gretton, A.

2019 (conference) Submitted

arXiv [BibTex]

Kanagawa, H., Jitkrittum, W., Mackey, L., Fukumizu, K., Gretton, A. A Kernel Stein Test for Comparing Latent Variable Models 2019 (conference) Submitted

arXiv [BibTex]

From Variational to Deterministic Autoencoders

Ghosh*, P., Sajjadi*, M. S. M., Vergari, A., Black, M. J., Schölkopf, B.

2019, *equal contribution (conference) Submitted

Abstract

Variational Autoencoders (VAEs) provide a theoretically-backed framework for deep generative models. However, they often produce “blurry” images, which is linked to their training objective. Sampling in the most popular implementation, the Gaussian VAE, can be interpreted as simply injecting noise to the input of a deterministic decoder. In practice, this simply enforces a smooth latent space structure. We challenge the adoption of the full VAE framework on this specific point in favor of a simpler, deterministic one. Specifically, we investigate how substituting stochasticity with other explicit and implicit regularization schemes can lead to a meaningful latent space without having to force it to conform to an arbitrarily chosen prior. To retrieve a generative mechanism for sampling new data points, we propose to employ an efficient ex-post density estimation step that can be readily adopted both for the proposed deterministic autoencoders as well as to improve sample quality of existing VAEs. We show in a rigorous empirical study that regularized deterministic autoencoding achieves state-of-the-art sample quality on the common MNIST, CIFAR-10 and CelebA datasets.

arXiv [BibTex]

Ghosh*, P., Sajjadi*, M. S. M., Vergari, A., Black, M. J., Schölkopf, B. From Variational to Deterministic Autoencoders 2019, *equal contribution (conference) Submitted

arXiv [BibTex]

Fisher Efficient Inference of Intractable Models

Liu, S., Kanamori, T., Jitkrittum, W., Chen, Y.

Advances in Neural Information Processing Systems 32, 33rd Annual Conference on Neural Information Processing Systems, 2019 (conference) To be published

arXiv [BibTex]

Liu, S., Kanamori, T., Jitkrittum, W., Chen, Y. Fisher Efficient Inference of Intractable Models Advances in Neural Information Processing Systems 32, 33rd Annual Conference on Neural Information Processing Systems, 2019 (conference) To be published

arXiv [BibTex]

2012

Influence Maximization in Continuous Time Diffusion Networks

Gomez Rodriguez, M., Schölkopf, B.

In Proceedings of the 29th International Conference on Machine Learning, pages: 313-320, (Editors: J, Langford and J, Pineau), Omnipress, New York, NY, USA, ICML, July 2012 (inproceedings)

Web [BibTex]

2012

Gomez Rodriguez, M., Schölkopf, B. Influence Maximization in Continuous Time Diffusion Networks In Proceedings of the 29th International Conference on Machine Learning, pages: 313-320, (Editors: J, Langford and J, Pineau), Omnipress, New York, NY, USA, ICML, July 2012 (inproceedings)

Web [BibTex]

Submodular Inference of Diffusion Networks from Multiple Trees

Gomez Rodriguez, M., Schölkopf, B.

In Proceedings of the 29th International Conference on Machine Learning , pages: 489-496, (Editors: J Langford, and J Pineau), Omnipress, New York, NY, USA, ICML, July 2012 (inproceedings)

Web [BibTex]

Gomez Rodriguez, M., Schölkopf, B. Submodular Inference of Diffusion Networks from Multiple Trees In Proceedings of the 29th International Conference on Machine Learning , pages: 489-496, (Editors: J Langford, and J Pineau), Omnipress, New York, NY, USA, ICML, July 2012 (inproceedings)

Web [BibTex]

Quasi-Newton Methods: A New Direction

Hennig, P., Kiefel, M.

In Proceedings of the 29th International Conference on Machine Learning, pages: 25-32, ICML ’12, (Editors: John Langford and Joelle Pineau), Omnipress, New York, NY, USA, ICML, July 2012 (inproceedings)

Abstract

Four decades after their invention, quasi- Newton methods are still state of the art in unconstrained numerical optimization. Although not usually interpreted thus, these are learning algorithms that fit a local quadratic approximation to the objective function. We show that many, including the most popular, quasi-Newton methods can be interpreted as approximations of Bayesian linear regression under varying prior assumptions. This new notion elucidates some shortcomings of classical algorithms, and lights the way to a novel nonparametric quasi-Newton method, which is able to make more efficient use of available information at computational cost similar to its predecessors.

website+code pdf link (url) [BibTex]

Hennig, P., Kiefel, M. Quasi-Newton Methods: A New Direction In Proceedings of the 29th International Conference on Machine Learning, pages: 25-32, ICML ’12, (Editors: John Langford and Joelle Pineau), Omnipress, New York, NY, USA, ICML, July 2012 (inproceedings)

website+code pdf link (url) [BibTex]

Image denoising: Can plain Neural Networks compete with BM3D?

Burger, H., Schuler, C., Harmeling, S.

In pages: 2392 - 2399, 25th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2012 (inproceedings)

Abstract

Image denoising can be described as the problem of mapping from a noisy image to a noise-free image. The best currently available denoising methods approximate this mapping with cleverly engineered algorithms. In this work we attempt to learn this mapping directly with a plain multi layer perceptron (MLP) applied to image patches. While this has been done before, we will show that by training on large image databases we are able to compete with the current state-of-the-art image denoising methods. Furthermore, our approach is easily adapted to less extensively studied types of noise (by merely exchanging the training data), for which we achieve excellent results as well.

PDF Web DOI [BibTex]

Burger, H., Schuler, C., Harmeling, S. Image denoising: Can plain Neural Networks compete with BM3D? In pages: 2392 - 2399, 25th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2012 (inproceedings)

PDF Web DOI [BibTex]

Climate classifications: the value of unsupervised clustering

Zscheischler, J., Mahecha, M., Harmeling, S.

In Proceedings of the International Conference on Computational Science , 9, pages: 897-906, Procedia Computer Science, (Editors: H. Ali, Y. Shi, D. Khazanchi, M. Lees, G.D. van Albada, J. Dongarra, P.M.A. Sloot, J. Dongarra), Elsevier, Amsterdam, Netherlands, ICCS, June 2012 (inproceedings)

Abstract

Classifying the land surface according to dierent climate zones is often a prerequisite for global diagnostic or predictive modelling studies. Classical classifications such as the prominent K¨oppen–Geiger (KG) approach rely on heuristic decision rules. Although these heuristics may transport some process understanding, such a discretization may appear “arbitrary” from a data oriented perspective. In this contribution we compare the precision of a KG classification to an unsupervised classification (k-means clustering). Generally speaking, we revisit the problem of “climate classification” by investigating the inherent patterns in multiple data streams in a purely data driven way. One question is whether we can reproduce the KG boundaries by exploring dierent combinations of climate and remotely sensed vegetation variables. In this context we also investigate whether climate and vegetation variables build similar clusters. In terms of statistical performances, k-means clearly outperforms classical climate classifications. However, a subsequent stability analysis only reveals a meaningful number of clusters if both climate and vegetation data are considered in the analysis. This is a setback for the hope to explain vegetation by means of climate alone. Clearly, classification schemes like K¨oppen-Geiger will play an important role in the future. However, future developments in this area need to be assessed based on data driven approaches.

Web DOI [BibTex]

Zscheischler, J., Mahecha, M., Harmeling, S. Climate classifications: the value of unsupervised clustering In Proceedings of the International Conference on Computational Science , 9, pages: 897-906, Procedia Computer Science, (Editors: H. Ali, Y. Shi, D. Khazanchi, M. Lees, G.D. van Albada, J. Dongarra, P.M.A. Sloot, J. Dongarra), Elsevier, Amsterdam, Netherlands, ICCS, June 2012 (inproceedings)

Web DOI [BibTex]

Learning Tracking Control with Forward Models

Bócsi, B., Hennig, P., Csató, L., Peters, J.

In pages: 259 -264, IEEE International Conference on Robotics and Automation (ICRA), May 2012 (inproceedings)

Abstract

Performing task-space tracking control on redundant robot manipulators is a difficult problem. When the physical model of the robot is too complex or not available, standard methods fail and machine learning algorithms can have advantages. We propose an adaptive learning algorithm for tracking control of underactuated or non-rigid robots where the physical model of the robot is unavailable. The control method is based on the fact that forward models are relatively straightforward to learn and local inversions can be obtained via local optimization. We use sparse online Gaussian process inference to obtain a flexible probabilistic forward model and second order optimization to find the inverse mapping. Physical experiments indicate that this approach can outperform state-of-the-art tracking control algorithms in this context.

PDF Web DOI [BibTex]

Bócsi, B., Hennig, P., Csató, L., Peters, J. Learning Tracking Control with Forward Models In pages: 259 -264, IEEE International Conference on Robotics and Automation (ICRA), May 2012 (inproceedings)

PDF Web DOI [BibTex]

A Kernel-based Approach to Direct Action Perception

Kroemer, O., Ugur, E., Oztop, E., Peters, J.

In International Conference on Robotics and Automation (ICRA 2012), pages: 2605-2610, IEEE, IEEE International Conference on Robotics and Automation (ICRA), May 2012 (inproceedings)

Abstract

The direct perception of actions allows a robot to predict the afforded actions of observed novel objects. In addition to learning which actions are afforded, the robot must also learn to adapt its actions according to the object being manipulated. In this paper, we present a non-parametric approach to representing the affordance-bearing subparts of objects. This representation forms the basis of a kernel function for computing the similarity between different subparts. Using this kernel function, the robot can learn the required mappings to perform direct action perception. The proposed approach was successfully implemented on a real robot, which could then quickly learn to generalize grasping and pouring actions to novel objects.

PDF Web DOI [BibTex]

Kroemer, O., Ugur, E., Oztop, E., Peters, J. A Kernel-based Approach to Direct Action Perception In International Conference on Robotics and Automation (ICRA 2012), pages: 2605-2610, IEEE, IEEE International Conference on Robotics and Automation (ICRA), May 2012 (inproceedings)

PDF Web DOI [BibTex]

Accelerating Nearest Neighbor Search on Manycore Systems

Cayton, L.

In Parallel Distributed Processing Symposium (IPDPS), 2012 IEEE 26th International, pages: 402-413, IPDPS, May 2012 (inproceedings)

Abstract

We develop methods for accelerating metric similarity search that are effective on modern hardware. Our algorithms factor into easily parallelizable components, making them simple to deploy and efficient on multicore CPUs and GPUs. Despite the simple structure of our algorithms, their search performance is provably sublinear in the size of the database, with a factor dependent only on its intrinsic dimensionality. We demonstrate that our methods provide substantial speedups on a range of datasets and hardware platforms. In particular, we present results on a 48-core server machine, on graphics hardware, and on a multicore desktop.

Web DOI [BibTex]

Cayton, L. Accelerating Nearest Neighbor Search on Manycore Systems In Parallel Distributed Processing Symposium (IPDPS), 2012 IEEE 26th International, pages: 402-413, IPDPS, May 2012 (inproceedings)

Web DOI [BibTex]

PAC-Bayes-Bernstein Inequality for Martingales and its Application to Multiarmed Bandits

Seldin, Y., Cesa-Bianchi, N., Auer, P., Laviolette, F., Shawe-Taylor, J.

In JMLR Workshop and Conference Proceedings 26, pages: 98-111, JMLR, Cambridge, MA, USA, On-line Trading of Exploration and Exploitation 2, April 2012 (inproceedings)

Abstract

We develop a new tool for data-dependent analysis of the exploration-exploitation trade-off in learning under limited feedback. Our tool is based on two main ingredients. The first ingredient is a new concentration inequality that makes it possible to control the concentration of weighted averages of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. The second ingredient is an application of this inequality to the exploration-exploitation trade-off via importance weighted sampling. We apply the new tool to the stochastic multiarmed bandit problem, however, the main importance of this paper is the development and understanding of the new tool rather than improvement of existing algorithms for stochastic multiarmed bandits. In the follow-up work we demonstrate that the new tool can improve over state-of-the-art in structurally richer problems, such as stochastic multiarmed bandits with side information (Seldin et al., 2011a).

PDF PDF [BibTex]

Seldin, Y., Cesa-Bianchi, N., Auer, P., Laviolette, F., Shawe-Taylor, J. PAC-Bayes-Bernstein Inequality for Martingales and its Application to Multiarmed Bandits In JMLR Workshop and Conference Proceedings 26, pages: 98-111, JMLR, Cambridge, MA, USA, On-line Trading of Exploration and Exploitation 2, April 2012 (inproceedings)

PDF PDF [BibTex]

Hierarchical Relative Entropy Policy Search

Daniel, C., Neumann, G., Peters, J.

In Fifteenth International Conference on Artificial Intelligence and Statistics, 22, pages: 273-281, JMLR Proceedings, (Editors: Lawrence, N. D. and Girolami, M.), JMLR.org, AISTATS, April 2012 (inproceedings)

Abstract

Many real-world problems are inherently hierarchically structured. The use of this structure in an agent's policy may well be the key to improved scalability and higher performance. However, such hierarchical structures cannot be exploited by current policy search algorithms. We will concentrate on a basic, but highly relevant hierarchy - the `mixed option' policy. Here, a gating network first decides which of the options to execute and, subsequently, the option-policy determines the action. In this paper, we reformulate learning a hierarchical policy as a latent variable estimation problem and subsequently extend the Relative Entropy Policy Search (REPS) to the latent variable case. We show that our Hierarchical REPS can learn versatile solutions while also showing an increased performance in terms of learning speed and quality of the found policy in comparison to the nonhierarchical approach.

PDF Web [BibTex]

Daniel, C., Neumann, G., Peters, J. Hierarchical Relative Entropy Policy Search In Fifteenth International Conference on Artificial Intelligence and Statistics, 22, pages: 273-281, JMLR Proceedings, (Editors: Lawrence, N. D. and Girolami, M.), JMLR.org, AISTATS, April 2012 (inproceedings)

PDF Web [BibTex]

Personalized medicine: from genotypes and molecular phenotypes towards computed therapy

Stegle, O., Roth, FP., Morris, Q., Listgarten, J.

In pages: 323-326, (Editors: Altman, R.B. , A.K. Dunker, L. Hunter, T. Murray, T.E. Klein), World Scientific Publishing, Singapore, Pacific Symposium on Biocomputing (PSB), January 2012 (inproceedings)

Abstract

Joint genotyping and large-scale phenotyping of molecular traits are currently available for a number of important patient study cohorts and will soon become feasible in routine medical practice. These data are one component of several that are setting the stage for the development of personalized medicine, promising to yield better disease classification, enabling more specific treatment, and also allowing for improved preventive medical screening. This conference session explores statistical challenges and new opportunities that arise from application of genome-scale experimentation for personalized genomics and medicine.

Web DOI [BibTex]

Stegle, O., Roth, FP., Morris, Q., Listgarten, J. Personalized medicine: from genotypes and molecular phenotypes towards computed therapy In pages: 323-326, (Editors: Altman, R.B. , A.K. Dunker, L. Hunter, T. Murray, T.E. Klein), World Scientific Publishing, Singapore, Pacific Symposium on Biocomputing (PSB), January 2012 (inproceedings)

Web DOI [BibTex]

Approximate Gaussian Integration using Expectation Propagation

Cunningham, J., Hennig, P., Lacoste-Julien, S.

In pages: 1-11, -, January 2012 (inproceedings) Submitted

Abstract

While Gaussian probability densities are omnipresent in applied mathematics, Gaussian cumulative probabilities are hard to calculate in any but the univariate case. We offer here an empirical study of the utility of Expectation Propagation (EP) as an approximate integration method for this problem. For rectangular integration regions, the approximation is highly accurate. We also extend the derivations to the more general case of polyhedral integration regions. However, we find that in this polyhedral case, EP's answer, though often accurate, can be almost arbitrarily wrong. These unexpected results elucidate an interesting and non-obvious feature of EP not yet studied in detail, both for the problem of Gaussian probabilities and for EP more generally.

Web [BibTex]

Cunningham, J., Hennig, P., Lacoste-Julien, S. Approximate Gaussian Integration using Expectation Propagation In pages: 1-11, -, January 2012 (inproceedings) Submitted

Web [BibTex]

Kernel Topic Models

Hennig, P., Stern, D., Herbrich, R., Graepel, T.

In Fifteenth International Conference on Artificial Intelligence and Statistics, 22, pages: 511-519, JMLR Proceedings, (Editors: Lawrence, N. D. and Girolami, M.), JMLR.org, AISTATS , 2012 (inproceedings)

Abstract

Latent Dirichlet Allocation models discrete data as a mixture of discrete distributions, using Dirichlet beliefs over the mixture weights. We study a variation of this concept, in which the documents' mixture weight beliefs are replaced with squashed Gaussian distributions. This allows documents to be associated with elements of a Hilbert space, admitting kernel topic models (KTM), modelling temporal, spatial, hierarchical, social and other structure between documents. The main challenge is efficient approximate inference on the latent Gaussian. We present an approximate algorithm cast around a Laplace approximation in a transformed basis. The KTM can also be interpreted as a type of Gaussian process latent variable model, or as a topic model conditional on document features, uncovering links between earlier work in these areas.

PDF Web [BibTex]

Hennig, P., Stern, D., Herbrich, R., Graepel, T. Kernel Topic Models In Fifteenth International Conference on Artificial Intelligence and Statistics, 22, pages: 511-519, JMLR Proceedings, (Editors: Lawrence, N. D. and Girolami, M.), JMLR.org, AISTATS , 2012 (inproceedings)

PDF Web [BibTex]

Structured Apprenticeship Learning

Boularias, A., Kroemer, O., Peters, J.

In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2012 (inproceedings)

PDF Web [BibTex]

Boularias, A., Kroemer, O., Peters, J. Structured Apprenticeship Learning In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2012 (inproceedings)

PDF Web [BibTex]

Blind Correction of Optical Aberrations

Schuler, C., Hirsch, M., Harmeling, S., Schölkopf, B.

In Computer Vision - ECCV 2012, LNCS Vol. 7574, pages: 187-200, (Editors: A Fitzgibbon, S Lazebnik, P Perona, Y Sato, and C Schmid), Springer, Berlin, Germany, 12th IEEE European Conference on Computer Vision, ECCV, 2012 (inproceedings)

Abstract

Camera lenses are a critical component of optical imaging systems, and lens imperfections compromise image quality. While traditionally, sophisticated lens design and quality control aim at limiting optical aberrations, recent works [1,2,3] promote the correction of optical flaws by computational means. These approaches rely on elaborate measurement procedures to characterize an optical system, and perform image correction by non-blind deconvolution. In this paper, we present a method that utilizes physically plausible assumptions to estimate non-stationary lens aberrations blindly, and thus can correct images without knowledge of specifics of camera and lens. The blur estimation features a novel preconditioning step that enables fast deconvolution. We obtain results that are competitive with state-of-the-art non-blind approaches.

PDF Web DOI [BibTex]

Schuler, C., Hirsch, M., Harmeling, S., Schölkopf, B. Blind Correction of Optical Aberrations In Computer Vision - ECCV 2012, LNCS Vol. 7574, pages: 187-200, (Editors: A Fitzgibbon, S Lazebnik, P Perona, Y Sato, and C Schmid), Springer, Berlin, Germany, 12th IEEE European Conference on Computer Vision, ECCV, 2012 (inproceedings)

PDF Web DOI [BibTex]