Empirical Inference

Model Selection for Small Sample Regression

2002

Article

ei


Model selection is an important ingredient of many machine learning algorithms, in particular when the sample size in small, in order to strike the right trade-off between overfitting and underfitting. Previous classical results for linear regression are based on an asymptotic analysis. We present a new penalization method for performing model selection for regression that is appropriate even for small samples. Our penalization is based on an accurate estimator of the ratio of the expected training error and the expected generalization error, in terms of the expected eigenvalues of the input covariance matrix.

Author(s): Chapelle, O. and Vapnik, V. and Bengio, Y.
Journal: Machine Learning
Volume: 48
Number (issue): 1-3
Pages: 9-23
Year: 2002
Day: 0

Department(s): Empirical Inference
Bibtex Type: Article (article)

Digital: 0
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

Links: PostScript

BibTex

@article{2165,
  title = {Model Selection for Small Sample Regression},
  author = {Chapelle, O. and Vapnik, V. and Bengio, Y.},
  journal = {Machine Learning},
  volume = {48},
  number = {1-3},
  pages = {9-23},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  year = {2002},
  doi = {}
}