Empirical Inference

Incremental Gaussian Processes

2003

Conference Paper

ei


In this paper, we consider Tipping‘s relevance vector machine (RVM) and formalize an incremental training strategy as a variant of the expectation-maximization (EM) algorithm that we call subspace EM. Working with a subset of active basis functions, the sparsity of the RVM solution will ensure that the number of basis functions and thereby the computational complexity is kept low. We also introduce a mean field approach to the intractable classification model that is expected to give a very good approximation to exact Bayesian inference and contains the Laplace approximation as a special case. We test the algorithms on two large data sets with O(10^3-10^4) examples. The results indicate that Bayesian learning of large data sets, e.g. the MNIST database is realistic.

Author(s): Quinonero Candela, J. and Winther, O.
Book Title: Advances in Neural Information Processing Systems 15
Journal: Advances in Neural Information Processing Systems 15
Pages: 1001-1008
Year: 2003
Month: October
Day: 0
Editors: Becker, S. , S. Thrun, K. Obermayer
Publisher: MIT Press

Department(s): Empirical Inference
Bibtex Type: Conference Paper (inproceedings)

Event Name: Sixteenth Annual Conference on Neural Information Processing Systems (NIPS 2002)
Event Place: Vancouver, BC, Canada

Address: Cambridge, MA, USA
Digital: 0
Institution: Informatics and Mathematical Modelling, Technical University of Denmark
ISBN: 0-262-02550-7
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

Links: PDF
Web

BibTex

@inproceedings{2800,
  title = {Incremental Gaussian Processes},
  author = {Quinonero Candela, J. and Winther, O.},
  journal = {Advances in Neural Information Processing Systems 15},
  booktitle = {Advances in Neural Information Processing Systems 15},
  pages = {1001-1008},
  editors = {Becker, S. , S. Thrun, K. Obermayer},
  publisher = {MIT Press},
  organization = {Max-Planck-Gesellschaft},
  institution = {Informatics and Mathematical Modelling, Technical University of Denmark},
  school = {Biologische Kybernetik},
  address = {Cambridge, MA, USA},
  month = oct,
  year = {2003},
  doi = {},
  month_numeric = {10}
}