Header logo is ei

Inference algorithms and learning theory for Bayesian sparse factor analysis

2009

Article

ei


Bayesian sparse factor analysis has many applications; for example, it has been applied to the problem of inferring a sparse regulatory network from gene expression data. We describe a number of inference algorithms for Bayesian sparse factor analysis using a slab and spike mixture prior. These include well-established Markov chain Monte Carlo (MCMC) and variational Bayes (VB) algorithms as well as a novel hybrid of VB and Expectation Propagation (EP). For the case of a single latent factor we derive a theory for learning performance using the replica method. We compare the MCMC and VB/EP algorithm results with simulated data to the theoretical prediction. The results for MCMC agree closely with the theory as expected. Results for VB/EP are slightly sub-optimal but show that the new algorithm is effective for sparse inference. In large-scale problems MCMC is infeasible due to computational limitations and the VB/EP algorithm then provides a very useful computationally efficient alternative.

Author(s): Rattray, M. and Stegle, O. and Sharp, K. and Winn, J.
Book Title: IW-SMI 2009
Journal: Journal of Physics: Conference Series
Volume: 197
Number (issue): 1: International Workshop on Statistical-Mechanical Informatics 2009
Pages: 1-10
Year: 2009
Month: September
Day: 0
Editors: Inoue, M. , S. Ishii, Y. Kabashima, M. Okada
Publisher: Institute of Physics

Department(s): Empirical Inference
Bibtex Type: Article (article)

Address: Bristol, UK
Digital: 0
DOI: 10.1088/1742-6596/197/1/012002
Event Name: International Workshop on Statistical-Mechanical Informatics 2009 (IW-SMI 2009)
Event Place: Kyoto, Japan
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

Links: PDF
Web

BibTex

@article{6296,
  title = {Inference algorithms and learning theory for Bayesian sparse factor analysis},
  author = {Rattray, M. and Stegle, O. and Sharp, K. and Winn, J.},
  journal = {Journal of Physics: Conference Series },
  booktitle = {IW-SMI 2009},
  volume = {197},
  number = {1: International Workshop on Statistical-Mechanical Informatics 2009},
  pages = {1-10},
  editors = {Inoue, M. , S. Ishii, Y. Kabashima, M. Okada},
  publisher = {Institute of Physics},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Bristol, UK},
  month = sep,
  year = {2009},
  month_numeric = {9}
}