Worst-Case Bounds for Gaussian Process Models
2006
Conference Paper
ei
We present a competitive analysis of some non-parametric Bayesian algorithms in a worst-case online learning setting, where no probabilistic assumptions about the generation of the data are made. We consider models which use a Gaussian process prior (over the space of all functions) and provide bounds on the regret (under the log loss) for commonly used non-parametric Bayesian algorithms - including Gaussian regression and logistic regression - which show how these algorithms can perform favorably under rather general conditions. These bounds explicitly handle the infinite dimensionality of these non-parametric classes in a natural way. We also make formal connections to the minimax and emph{minimum description length} (MDL) framework. Here, we show precisely how Bayesian Gaussian regression is a minimax strategy.
Author(s): | Kakade, S. and Seeger, M. and Foster, D. |
Book Title: | Advances in neural information processing systems 18 |
Journal: | Advances in Neural Information Processing Systems 18: Proceedings of the 2005 Conference |
Pages: | 619-626 |
Year: | 2006 |
Month: | May |
Day: | 0 |
Editors: | Weiss, Y. , B. Sch{\"o}lkopf, J. Platt |
Publisher: | MIT Press |
Department(s): | Empirical Inference |
Bibtex Type: | Conference Paper (inproceedings) |
Event Name: | Nineteenth Annual Conference on Neural Information Processing Systems (NIPS 2005) |
Event Place: | Vancouver, BC, Canada |
Address: | Cambridge, MA, USA |
Digital: | 0 |
ISBN: | 0-262-23253-7 |
Language: | en |
Organization: | Max-Planck-Gesellschaft |
School: | Biologische Kybernetik |
Links: |
PDF
Web |
BibTex @inproceedings{3607, title = {Worst-Case Bounds for Gaussian Process Models}, author = {Kakade, S. and Seeger, M. and Foster, D.}, journal = {Advances in Neural Information Processing Systems 18: Proceedings of the 2005 Conference}, booktitle = {Advances in neural information processing systems 18}, pages = {619-626}, editors = {Weiss, Y. , B. Sch{\"o}lkopf, J. Platt}, publisher = {MIT Press}, organization = {Max-Planck-Gesellschaft}, school = {Biologische Kybernetik}, address = {Cambridge, MA, USA}, month = may, year = {2006}, doi = {}, month_numeric = {5} } |