Header logo is ei


2013


no image
One-class Support Measure Machines for Group Anomaly Detection

Muandet, K., Schölkopf, B.

29th Conference on Uncertainty in Artificial Intelligence (UAI), 2013 (poster)

PDF [BibTex]

2013

PDF [BibTex]


no image
Towards Robot Skill Learning: From Simple Skills to Table Tennis

Peters, J., Kober, J., Mülling, K., Kroemer, O., Neumann, G.

In Machine Learning and Knowledge Discovery in Databases, Proceedings of the European Conference on Machine Learning, Part III (ECML 2013), LNCS 8190, pages: 627-631, (Editors: Blockeel, H.,Kersting, K., Nijssen, S., and Zelezný, F.), Springer, 2013 (inproceedings)

PDF DOI [BibTex]

PDF DOI [BibTex]


Nonparametric dynamics estimation for time periodic systems
Nonparametric dynamics estimation for time periodic systems

Klenske, E., Zeilinger, M., Schölkopf, B., Hennig, P.

In Proceedings of the 51st Annual Allerton Conference on Communication, Control, and Computing, pages: 486-493 , 2013 (inproceedings)

PDF DOI [BibTex]

PDF DOI [BibTex]


no image
Scalable kernels for graphs with continuous attributes

Feragen, A., Kasenburg, N., Petersen, J., de Bruijne, M., Borgwardt, KM.

In Advances in Neural Information Processing Systems 26, pages: 216-224, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
The Randomized Dependence Coefficient

Lopez-Paz, D., Hennig, P., Schölkopf, B.

Neural Information Processing Systems (NIPS), 2013 (poster)

PDF [BibTex]

PDF [BibTex]


no image
Characterization of different types of sharp-wave ripple signatures in the CA1 of the macaque hippocampus

Ramirez-Villegas, J., Logothetis, N., Besserve, M.

4th German Neurophysiology PhD Meeting Networks, 2013 (poster)

Web [BibTex]

Web [BibTex]


no image
Auto-Calibrating Spherical Deconvolution Based on ODF Sparsity

Schultz, T., Gröschel, S.

In Proceedings of Medical Image Computing and Computer-Assisted Intervention, Part I, pages: 663-670, (Editors: K Mori and I Sakuma and Y Sato and C Barillot and N Navab), Springer, MICCAI, 2013, Lecture Notes in Computer Science, vol. 8149 (inproceedings)

DOI [BibTex]

DOI [BibTex]


no image
Maximum-Margin Framework for Training Data Synchronization in Large-Scale Hierarchical Classification

Babbar, R., Partalas, I., Gaussier, E., Amini, M.

In Neural Information Processing - 20th International Conference, Proceedings, Part I, Lecture Notes in Computer Science, Vol. 8226, pages: 336-343, (Editors: M Lee and A Hirose and Z-G Hou and R M Kil), Springer, ICONIP, 2013 (inproceedings)

Web [BibTex]

Web [BibTex]


no image
Domain Generalization via Invariant Feature Representation

Muandet, K., Balduzzi, D., Schölkopf, B.

In Proceedings of the 30th International Conference on Machine Learning, W&CP 28(1), pages: 10-18, (Editors: S Dasgupta and D McAllester), JMLR, ICML, 2013, Volume 28, number 1 (inproceedings)

Web [BibTex]

Web [BibTex]


no image
Learning Sequential Motor Tasks

Daniel, C., Neumann, G., Peters, J.

In Proceedings of 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), 2013 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
Information-Theoretic Motor Skill Learning

Neumann, G., Kupcsik, A., Deisenroth, M., Peters, J.

In Proceedings of the 27th AAAI 2013, Workshop on Intelligent Robotic Systems (AAAI 2013), 2013 (inproceedings)

[BibTex]

[BibTex]


no image
Measuring Statistical Dependence via the Mutual Information Dimension

Sugiyama, M., Borgwardt, KM.

In Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI 2013), pages: 1692-1698, (Editors: Francesca Rossi), AAAI Press, Menlo Park, California, IJCAI, 2013 (inproceedings)

[BibTex]

[BibTex]


no image
Analytical probabilistic proton dose calculation and range uncertainties

Bangert, M., Hennig, P., Oelfke, U.

In 17th International Conference on the Use of Computers in Radiation Therapy, pages: 6-11, (Editors: A. Haworth and T. Kron), ICCR, 2013 (inproceedings)

[BibTex]

[BibTex]


no image
Adaptivity to Local Smoothness and Dimension in Kernel Regression

Kpotufe, S., Garg, V.

In Advances in Neural Information Processing Systems 26, pages: 3075-3083, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
Statistical analysis of coupled time series with Kernel Cross-Spectral Density operators

Besserve, M., Logothetis, N., Schölkopf, B.

In Advances in Neural Information Processing Systems 26, pages: 2535-2543, (Editors: C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
It is all in the noise: Efficient multi-task Gaussian process inference with structured residuals

Rakitsch, B., Lippert, C., Borgwardt, KM., Stegle, O.

In Advances in Neural Information Processing Systems 26, pages: 1466-1474, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
Comparative Classifier Evaluation for Web-Scale Taxonomies Using Power Law

Babbar, R., Partalas, I., Metzig, C., Gaussier, E., Amini, M.

In The Semantic Web: ESWC 2013 Satellite Events, Lecture Notes in Computer Science, Vol. 7955 , pages: 310-311, (Editors: P Cimiano and M Fernández and V Lopez and S Schlobach and J Völker), Springer, ESWC, 2013 (inproceedings)

Web [BibTex]

Web [BibTex]


no image
Model-based Imitation Learning by Probabilistic Trajectory Matching

Englert, P., Paraschos, A., Peters, J., Deisenroth, M.

In Proceedings of 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), pages: 1922-1927, 2013 (inproceedings)

PDF DOI [BibTex]

PDF DOI [BibTex]


no image
Towards neurofeedback for improving visual attention

Zander, T., Battes, B., Schölkopf, B., Grosse-Wentrup, M.

In Proceedings of the Fifth International Brain-Computer Interface Meeting: Defining the Future, pages: Article ID: 086, (Editors: J.d.R. Millán, S. Gao, R. Müller-Putz, J.R. Wolpaw, and J.E. Huggins), Verlag der Technischen Universität Graz, 5th International Brain-Computer Interface Meeting, 2013, Article ID: 086 (inproceedings)

PDF DOI [BibTex]

PDF DOI [BibTex]


no image
A Guided Hybrid Genetic Algorithm for Feature Selection with Expensive Cost Functions

Jung, M., Zscheischler, J.

In Proceedings of the International Conference on Computational Science, 18, pages: 2337 - 2346, Procedia Computer Science, (Editors: Alexandrov, V and Lees, M and Krzhizhanovskaya, V and Dongarra, J and Sloot, PMA), Elsevier, Amsterdam, Netherlands, ICCS, 2013 (inproceedings)

Web DOI [BibTex]

Web DOI [BibTex]


no image
Learning responsive robot behavior by imitation

Ben Amor, H., Vogt, D., Ewerton, M., Berger, E., Jung, B., Peters, J.

In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), pages: 3257-3264, IEEE, 2013 (inproceedings)

DOI [BibTex]

DOI [BibTex]


no image
Learning Skills with Motor Primitives

Peters, J., Kober, J., Mülling, K., Kroemer, O., Neumann, G.

In Proceedings of the 16th Yale Workshop on Adaptive and Learning Systems, 2013 (inproceedings)

[BibTex]

[BibTex]


no image
Scalable Influence Estimation in Continuous-Time Diffusion Networks

Du, N., Song, L., Gomez Rodriguez, M., Zha, H.

In Advances in Neural Information Processing Systems 26, pages: 3147-3155, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF PDF [BibTex]

PDF PDF [BibTex]


no image
Rapid Distance-Based Outlier Detection via Sampling

Sugiyama, M., Borgwardt, KM.

In Advances in Neural Information Processing Systems 26, pages: 467-475, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
Probabilistic Movement Primitives

Paraschos, A., Daniel, C., Peters, J., Neumann, G.

In Advances in Neural Information Processing Systems 26, pages: 2616-2624, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF PDF [BibTex]

PDF PDF [BibTex]


no image
Causal Inference on Time Series using Restricted Structural Equation Models

Peters, J., Janzing, D., Schölkopf, B.

In Advances in Neural Information Processing Systems 26, pages: 154-162, (Editors: C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
Regression-tree Tuning in a Streaming Setting

Kpotufe, S., Orabona, F.

In Advances in Neural Information Processing Systems 26, pages: 1788-1796, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
Density estimation from unweighted k-nearest neighbor graphs: a roadmap

von Luxburg, U., Alamgir, M.

In Advances in Neural Information Processing Systems 26, pages: 225-233, (Editors: C.J.C. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
PAC-Bayes-Empirical-Bernstein Inequality

Tolstikhin, I. O., Seldin, Y.

In Advances in Neural Information Processing Systems 26, pages: 109-117, (Editors: C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger), 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013 (inproceedings)

link (url) [BibTex]

link (url) [BibTex]


no image
PLAL: Cluster-based active learning

Urner, R., Wulff, S., Ben-David, S.

In Proceedings of the 26th Annual Conference on Learning Theory, 30, pages: 376-397, (Editors: Shalev-Shwartz, S. and Steinwart, I.), JMLR, COLT, 2013 (inproceedings)

link (url) [BibTex]

link (url) [BibTex]


no image
Monochromatic Bi-Clustering

Wulff, S., Urner, R., Ben-David, S.

In Proceedings of the 30th International Conference on Machine Learning, 28, pages: 145-153, (Editors: Dasgupta, S. and McAllester, D.), JMLR, ICML, 2013 (inproceedings)

link (url) [BibTex]

link (url) [BibTex]


no image
Significance of variable height-bandwidth group delay filters in the spectral reconstruction of speech

Devanshu, A., Raj, A., Hegde, R. M.

INTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association, pages: 1682-1686, 2013 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Generative Multiple-Instance Learning Models For Quantitative Electromyography

Adel, T., Smith, B., Urner, R., Stashuk, D., Lizotte, D. J.

In Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, AUAI Press, UAI, 2013 (inproceedings)

link (url) [BibTex]

link (url) [BibTex]


no image
Automatic Malaria Diagnosis system

Mehrjou, A., Abbasian, T., Izadi, M.

In First RSI/ISM International Conference on Robotics and Mechatronics (ICRoM), pages: 205-211, 2013 (inproceedings)

DOI [BibTex]

DOI [BibTex]


no image
Abstraction in Decision-Makers with Limited Information Processing Capabilities

Genewein, T, Braun, DA

pages: 1-9, NIPS Workshop Planning with Information Constraints for Control, Reinforcement Learning, Computational Neuroscience, Robotics and Games, December 2013 (conference)

Abstract
A distinctive property of human and animal intelligence is the ability to form abstractions by neglecting irrelevant information which allows to separate structure from noise. From an information theoretic point of view abstractions are desirable because they allow for very efficient information processing. In artificial systems abstractions are often implemented through computationally costly formations of groups or clusters. In this work we establish the relation between the free-energy framework for decision-making and rate-distortion theory and demonstrate how the application of rate-distortion for decision-making leads to the emergence of abstractions. We argue that abstractions are induced due to a limit in information processing capacity.

link (url) [BibTex]

link (url) [BibTex]


no image
Bounded Rational Decision-Making in Changing Environments

Grau-Moya, J, Braun, DA

pages: 1-9, NIPS Workshop Planning with Information Constraints for Control, Reinforcement Learning, Computational Neuroscience, Robotics and Games, December 2013 (conference)

Abstract
A perfectly rational decision-maker chooses the best action with the highest utility gain from a set of possible actions. The optimality principles that describe such decision processes do not take into account the computational costs of finding the optimal action. Bounded rational decision-making addresses this problem by specifically trading off information-processing costs and expected utility. Interestingly, a similar trade-off between energy and entropy arises when describing changes in thermodynamic systems. This similarity has been recently used to describe bounded rational agents. Crucially, this framework assumes that the environment does not change while the decision-maker is computing the optimal policy. When this requirement is not fulfilled, the decision-maker will suffer inefficiencies in utility, that arise because the current policy is optimal for an environment in the past. Here we borrow concepts from non-equilibrium thermodynamics to quantify these inefficiencies and illustrate with simulations its relationship with computational resources.

link (url) [BibTex]

link (url) [BibTex]

2003


no image
Natural Actor-Critic

Peters, J., Vijayakumar, S., Schaal, S.

NIPS Workshop " Planning for the Real World: The promises and challenges of dealing with uncertainty", December 2003 (poster)

PDF Web [BibTex]

2003

PDF Web [BibTex]


no image
Texture and haptic cues in slant discrimination: Measuring the effect of texture type on cue combination

Rosas, P., Wichmann, F., Ernst, M., Wagemans, J.

Journal of Vision, 3(12):26, 2003 Fall Vision Meeting of the Optical Society of America, December 2003 (poster)

Abstract
In a number of models of depth cue combination the depth percept is constructed via a weighted average combination of independent depth estimations. The influence of each cue in such average depends on the reliability of the source of information. (Young, Landy, & Maloney, 1993; Ernst & Banks, 2002.) In particular, Ernst & Banks (2002) formulate the combination performed by the human brain as that of the minimum variance unbiased estimator that can be constructed from the available cues. Using slant discrimination and slant judgment via probe adjustment as tasks, we have observed systematic differences in performance of human observers when a number of different types of textures were used as cue to slant (Rosas, Wichmann & Wagemans, 2003). If the depth percept behaves as described above, our measurements of the slopes of the psychometric functions provide the predicted weights for the texture cue for the ranked texture types. We have combined these texture types with object motion but the obtained results are difficult to reconcile with the unbiased minimum variance estimator model (Rosas & Wagemans, 2003). This apparent failure of such model might be explained by the existence of a coupling of texture and motion, violating the assumption of independence of cues. Hillis, Ernst, Banks, & Landy (2002) have shown that while for between-modality combination the human visual system has access to the single-cue information, for within-modality combination (visual cues: disparity and texture) the single-cue information is lost, suggesting a coupling between these cues. Then, in the present study we combine the different texture types with haptic information in a slant discrimination task, to test whether in the between-modality condition the texture cue and the haptic cue to slant are combined as predicted by an unbiased, minimum variance estimator model.

Web DOI [BibTex]

Web DOI [BibTex]


no image
How to Deal with Large Dataset, Class Imbalance and Binary Output in SVM based Response Model

Shin, H., Cho, S.

In Proc. of the Korean Data Mining Conference, pages: 93-107, Korean Data Mining Conference, December 2003, Best Paper Award (inproceedings)

Abstract
[Abstract]: Various machine learning methods have made a rapid transition to response modeling in search of improved performance. And support vector machine (SVM) has also been attracting much attention lately. This paper presents an SVM response model. We are specifically focusing on the how-to’s to circumvent practical obstacles, such as how to face with class imbalance problem, how to produce the scores from an SVM classifier for lift chart analysis, and how to evaluate the models on accuracy and profit. Besides coping with the intractability problem of SVM training caused by large marketing dataset, a previously proposed pattern selection algorithm is introduced. SVM training accompanies time complexity of the cube of training set size. The pattern selection algorithm picks up important training patterns before SVM response modeling. We made comparison on SVM training results between the pattern selection algorithm and random sampling. Three aspects of SVM response models were evaluated: accuracies, lift chart analysis, and computational efficiency. The SVM trained with selected patterns showed a high accuracy, a high uplift in profit and in response rate, and a high computational efficiency.

PDF [BibTex]

PDF [BibTex]


no image
Bayesian Monte Carlo

Rasmussen, CE., Ghahramani, Z.

In Advances in Neural Information Processing Systems 15, pages: 489-496, (Editors: Becker, S. , S. Thrun, K. Obermayer), MIT Press, Cambridge, MA, USA, Sixteenth Annual Conference on Neural Information Processing Systems (NIPS), October 2003 (inproceedings)

Abstract
We investigate Bayesian alternatives to classical Monte Carlo methods for evaluating integrals. Bayesian Monte Carlo (BMC) allows the incorporation of prior knowledge, such as smoothness of the integrand, into the estimation. In a simple problem we show that this outperforms any classical importance sampling method. We also attempt more challenging multidimensional integrals involved in computing marginal likelihoods of statistical models (a.k.a. partition functions and model evidences). We find that Bayesian Monte Carlo outperformed Annealed Importance Sampling, although for very high dimensional problems or problems with massive multimodality BMC may be less adequate. One advantage of the Bayesian approach to Monte Carlo is that samples can be drawn from any distribution. This allows for the possibility of active design of sample points so as to maximise information gain.

PDF Web [BibTex]

PDF Web [BibTex]


no image
On the Complexity of Learning the Kernel Matrix

Bousquet, O., Herrmann, D.

In Advances in Neural Information Processing Systems 15, pages: 399-406, (Editors: Becker, S. , S. Thrun, K. Obermayer), The MIT Press, Cambridge, MA, USA, Sixteenth Annual Conference on Neural Information Processing Systems (NIPS), October 2003 (inproceedings)

Abstract
We investigate data based procedures for selecting the kernel when learning with Support Vector Machines. We provide generalization error bounds by estimating the Rademacher complexities of the corresponding function classes. In particular we obtain a complexity bound for function classes induced by kernels with given eigenvectors, i.e., we allow to vary the spectrum and keep the eigenvectors fix. This bound is only a logarithmic factor bigger than the complexity of the function class induced by a single kernel. However, optimizing the margin over such classes leads to overfitting. We thus propose a suitable way of constraining the class. We use an efficient algorithm to solve the resulting optimization problem, present preliminary experimental results, and compare them to an alignment-based approach.

PDF Web [BibTex]

PDF Web [BibTex]


no image
Control, Planning, Learning, and Imitation with Dynamic Movement Primitives

Schaal, S., Peters, J., Nakanishi, J., Ijspeert, A.

In IROS 2003, pages: 1-21, Workshop on Bilateral Paradigms on Humans and Humanoids, IEEE International Conference on Intelligent Robots and Systems, October 2003 (inproceedings)

PDF [BibTex]

PDF [BibTex]


no image
Discriminative Learning for Label Sequences via Boosting

Altun, Y., Hofmann, T., Johnson, M.

In Advances in Neural Information Processing Systems 15, pages: 977-984, (Editors: Becker, S. , S. Thrun, K. Obermayer ), MIT Press, Cambridge, MA, USA, Sixteenth Annual Conference on Neural Information Processing Systems (NIPS), October 2003 (inproceedings)

Abstract
This paper investigates a boosting approach to discriminative learning of label sequences based on a sequence rank loss function.

PDF Web [BibTex]

PDF Web [BibTex]


no image
Multiple-step ahead prediction for non linear dynamic systems: A Gaussian Process treatment with propagation of the uncertainty

Girard, A., Rasmussen, CE., Quiñonero-Candela, J., Murray-Smith, R.

In Advances in Neural Information Processing Systems 15, pages: 529-536, (Editors: Becker, S. , S. Thrun, K. Obermayer), MIT Press, Cambridge, MA, USA, Sixteenth Annual Conference on Neural Information Processing Systems (NIPS), October 2003 (inproceedings)

Abstract
We consider the problem of multi-step ahead prediction in time series analysis using the non-parametric Gaussian process model. k-step ahead forecasting of a discrete-time non-linear dynamic system can be performed by doing repeated one-step ahead predictions. For a state-space model of the form y_t = f(y_{t-1},...,y_{t-L}), the prediction of y at time t + k is based on the point estimates of the previous outputs. In this paper, we show how, using an analytical Gaussian approximation, we can formally incorporate the uncertainty about intermediate regressor values, thus updating the uncertainty on the current prediction.

PDF Web [BibTex]

PDF Web [BibTex]


no image
Cluster Kernels for Semi-Supervised Learning

Chapelle, O., Weston, J., Schölkopf, B.

In Advances in Neural Information Processing Systems 15, pages: 585-592, (Editors: S Becker and S Thrun and K Obermayer), MIT Press, Cambridge, MA, USA, 16th Annual Conference on Neural Information Processing Systems (NIPS), October 2003 (inproceedings)

Abstract
We propose a framework to incorporate unlabeled data in kernel classifier, based on the idea that two points in the same cluster are more likely to have the same label. This is achieved by modifying the eigenspectrum of the kernel matrix. Experimental results assess the validity of this approach.

PDF Web [BibTex]

PDF Web [BibTex]


no image
Mismatch String Kernels for SVM Protein Classification

Leslie, C., Eskin, E., Weston, J., Noble, W.

In Advances in Neural Information Processing Systems 15, pages: 1417-1424, (Editors: Becker, S. , S. Thrun, K. Obermayer), MIT Press, Cambridge, MA, USA, Sixteenth Annual Conference on Neural Information Processing Systems (NIPS), October 2003 (inproceedings)

Abstract
We introduce a class of string kernels, called mismatch kernels, for use with support vector machines (SVMs) in a discriminative approach to the protein classification problem. These kernels measure sequence similarity based on shared occurrences of k-length subsequences, counted with up to m mismatches, and do not rely on any generative model for the positive training sequences. We compute the kernels efficiently using a mismatch tree data structure and report experiments on a benchmark SCOP dataset, where we show that the mismatch kernel used with an SVM classifier performs as well as the Fisher kernel, the most successful method for remote homology detection, while achieving considerable computational savings.

PDF Web [BibTex]

PDF Web [BibTex]


no image
Incremental Gaussian Processes

Quinonero Candela, J., Winther, O.

In Advances in Neural Information Processing Systems 15, pages: 1001-1008, (Editors: Becker, S. , S. Thrun, K. Obermayer), MIT Press, Cambridge, MA, USA, Sixteenth Annual Conference on Neural Information Processing Systems (NIPS), October 2003 (inproceedings)

Abstract
In this paper, we consider Tipping‘s relevance vector machine (RVM) and formalize an incremental training strategy as a variant of the expectation-maximization (EM) algorithm that we call subspace EM. Working with a subset of active basis functions, the sparsity of the RVM solution will ensure that the number of basis functions and thereby the computational complexity is kept low. We also introduce a mean field approach to the intractable classification model that is expected to give a very good approximation to exact Bayesian inference and contains the Laplace approximation as a special case. We test the algorithms on two large data sets with O(10^3-10^4) examples. The results indicate that Bayesian learning of large data sets, e.g. the MNIST database is realistic.

PDF Web [BibTex]

PDF Web [BibTex]


no image
Kernel Dependency Estimation

Weston, J., Chapelle, O., Elisseeff, A., Schölkopf, B., Vapnik, V.

In Advances in Neural Information Processing Systems 15, pages: 873-880, (Editors: S Becker and S Thrun and K Obermayer), MIT Press, Cambridge, MA, USA, 16th Annual Conference on Neural Information Processing Systems (NIPS), October 2003 (inproceedings)

PDF Web [BibTex]

PDF Web [BibTex]


no image
Derivative observations in Gaussian Process models of dynamic systems

Solak, E., Murray-Smith, R., Leithead, WE., Leith, D., Rasmussen, CE.

In Advances in Neural Information Processing Systems 15, pages: 1033-1040, (Editors: Becker, S., S. Thrun and K. Obermayer), MIT Press, Cambridge, MA, USA, Sixteenth Annual Conference on Neural Information Processing Systems (NIPS), October 2003 (inproceedings)

Abstract
Gaussian processes provide an approach to nonparametric modelling which allows a straightforward combination of function and derivative observations in an empirical model. This is of particular importance in identification of nonlinear dynamic systems from experimental data. 1) It allows us to combine derivative information, and associated uncertainty with normal function observations into the learning and inference process. This derivative information can be in the form of priors specified by an expert or identified from perturbation data close to equilibrium. 2) It allows a seamless fusion of multiple local linear models in a consistent manner, inferring consistent models and ensuring that integrability constraints are met. 3) It improves dramatically the computational efficiency of Gaussian process models for dynamic system identification, by summarising large quantities of near-equilibrium data by a handful of linearisations, reducing the training set size - traditionally a problem for Gaussian process models.

PDF Web [BibTex]

PDF Web [BibTex]