Machine Learning Challenges: Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, Machine Learning Challenges: First PASCAL Machine Learning Challenges Workshop (MLCW 2005):1-27, Biologische Kybernetik, Max-Planck-Gesellschaft, April, 2006
This Chapter presents the PASCAL Evaluating Predictive Uncertainty Challenge, introduces the contributed Chapters by the participants who obtained outstanding results, and provides a discussion with some lessons to be learnt. The Challenge was set up to evaluate the ability of Machine Learning algorithms to provide good “probabilistic predictions”, rather than just the usual “point predictions” with no measure of uncertainty, in regression and classification problems. Parti-cipants had to compete on a number of regression and classification tasks, and were evaluated by both traditional losses that only take into account point predictions and losses we proposed that evaluate the quality of the probabilistic predictions.
Proceedings of the First Pascal Machine Learning Challenges Workshop on Machine Learning Challenges, Evaluating Predictive Uncertainty, Visual Object Classification and Recognizing Textual Entailment (MLCW 2005):462, Lecture Notes in Computer Science, Biologische Kybernetik, Max-Planck-Gesellschaft, 2006
This book constitutes the thoroughly refereed post-proceedings of the First PASCAL (pattern analysis, statistical modelling and computational learning) Machine Learning Challenges Workshop, MLCW 2005, held in Southampton, UK in April 2005.
The 25 revised full papers presented were carefully selected during two rounds of reviewing and improvement from about 50 submissions. The papers reflect the concepts of three challenges dealt with in the workshop: finding an assessment base on the uncertainty of predictions using classical statistics, Bayesian inference, and statistical learning theory; the second challenge was to recognize objects from a number of visual object classes in realistic scenes; the third challenge of recognizing textual entailment addresses semantic analysis of language to form a generic framework for applied semantic inference in text understanding.
Switching and Learning in Feedback Systems:98-127, Biologische Kybernetik, Max-Planck-Gesellschaft, 2005
While there is strong motivation for using Gaussian Processes (GPs) due to their excellent performance in regression and classification problems, their computational complexity makes them impractical when the size of the training set exceeds a few thousand cases. This has motivated the recent proliferation of a number of cost-effective approximations to GPs, both for classification and for regression. In this paper we analyze one popular approximation to GPs for regression: the reduced rank approximation. While generally GPs are equivalent to infinite linear models, we show that Reduced Rank Gaussian Processes (RRGPs) are equivalent to finite sparse linear models. We also introduce the concept of degenerate GPs and show that they correspond to inappropriate priors. We show how to modify the RRGP to prevent it from being degenerate at test time. Training RRGPs consists both in learning the covariance function hyperparameters and the support set. We propose a method for learning hyperparameters for a given support set. We also review the Sparse Greedy GP (SGGP) approximation (Smola and Bartlett, 2001), which is a way of learning the support set for given hyperparameters based on approximating the posterior. We propose an alternative method to the SGGP that has better generalization capabilities. Finally we make experiments to compare the different ways of training a RRGP. We provide some Matlab code for learning RRGPs.
We provide a new unifying view, including all existing proper probabilistic
sparse approximations for Gaussian process regression. Our approach relies on
expressing the effective prior which the methods are using. This
allows new insights to be gained, and highlights the relationship between
existing methods. It also allows for a clear theoretically justified ranking
of the closeness of the known approximations to the corresponding full GPs.
Finally we point directly to designs of new better sparse approximations,
combining the best of the existing strategies, within attractive
Advances in Neural Information Processing Systems 15, Advances in Neural Information Processing Systems 15, Informatics and Mathematical Modelling, Technical University of Denmark:1001-1008, Biologische Kybernetik, Max-Planck-Gesellschaft, October, 2003
In this paper, we consider Tipping‘s relevance vector machine (RVM) and formalize an incremental training
strategy as a variant of the expectation-maximization (EM) algorithm that we call subspace EM. Working with a subset of active basis functions, the sparsity of the RVM solution will ensure that the number of basis functions and thereby the computational complexity is kept low. We also introduce a mean field approach to the intractable classification
model that is expected to give a very good approximation to exact Bayesian inference and contains the Laplace approximation as a special case. We test the algorithms on two large data sets with O(10^3-10^4) examples. The results indicate that Bayesian learning of large data sets, e.g.
the MNIST database is realistic.
Quinonero Candela, J.Winther, O. (2003). Incremental Gaussian Processes In: Advances in Neural Information Processing Systems 15, Advances in Neural Information Processing Systems 15, 1001-1008, MIT Press, Informatics and Mathematical Modelling, Technical University of Denmark, Cambridge, MA, USA, Sixteenth Annual Conference on Neural Information Processing Systems (NIPS 2002)
Our goal is to understand the principles of Perception, Action and Learning in autonomous systems that successfully interact with complex environments and to use this understanding to design future systems