Header logo is ei

153 results (View BibTeX file of all listed publications)

2008

no image
Modelling contrast discrimination data suggest both the pedestal effect and stochastic resonance to be caused by the same mechanism

Goris, R., Wagemans, J., Wichmann, F.

Journal of Vision, 8(15):1-21, November 2008 (article)

Abstract
Computational models of spatial vision typically make use of a (rectified) linear filter, a nonlinearity and dominant late noise to account for human contrast discrimination data. Linear–nonlinear cascade models predict an improvement in observers' contrast detection performance when low, subthreshold levels of external noise are added (i.e., stochastic resonance). Here, we address the issue whether a single contrast gain-control model of early spatial vision can account for both the pedestal effect, i.e., the improved detectability of a grating in the presence of a low-contrast masking grating, and stochastic resonance. We measured contrast discrimination performance without noise and in both weak and moderate levels of noise. Making use of a full quantitative description of our data with few parameters combined with comprehensive model selection assessments, we show the pedestal effect to be more reduced in the presence of weak noise than in moderate noise. This reduction rules out independent, additive sources of performance improvement and, together with a simulation study, supports the parsimonious explanation that a single mechanism underlies the pedestal effect and stochastic resonance in contrast perception.

Web DOI [BibTex]

no image
Frequent Subgraph Retrieval in Geometric Graph Databases

Nowozin, S., Tsuda, K.

(180), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, November 2008 (techreport)

Abstract
Discovery of knowledge from geometric graph databases is of particular importance in chemistry and biology, because chemical compounds and proteins are represented as graphs with 3D geometric coordinates. In such applications, scientists are not interested in the statistics of the whole database. Instead they need information about a novel drug candidate or protein at hand, represented as a query graph. We propose a polynomial-delay algorithm for geometric frequent subgraph retrieval. It enumerates all subgraphs of a single given query graph which are frequent geometric epsilon-subgraphs under the entire class of rigid geometric transformations in a database. By using geometric epsilon-subgraphs, we achieve tolerance against variations in geometry. We compare the proposed algorithm to gSpan on chemical compound data, and we show that for a given minimum support the total number of frequent patterns is substantially limited by requiring geometric matching. Although the computation time per pattern is larger than for non-geometric graph mining, the total time is within a reasonable level even for small minimum support.

PDF [BibTex]

Nowozin, S., Tsuda, K. Frequent Subgraph Retrieval in Geometric Graph Databases (180), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, November 2008 (techreport)

PDF [BibTex]

no image
The Effect of Mutual Information on Independent Component Analysis in EEG/MEG Analysis: A Simulation Study

Neumann, A., Grosse-Wentrup, M., Buss, M., Gramann, K.

International Journal of Neuroscience, 118(11):1534-1546, November 2008 (article)

Abstract
Objective: This study investigated the influence of mutual information (MI) on temporal and dipole reconstruction based on independent components (ICs) derived from independent component analysis (ICA). Method: Artificial electroencephalogram (EEG) datasets were created by means of a neural mass model simulating cortical activity of two neural sources within a four-shell spherical head model. Mutual information between neural sources was systematicallyvaried. Results: Increasing spatial error for reconstructed locations of ICs with increasing MI was observed. By contrast, the reconstruction error for the time course of source activity was largely independent of MI but varied systematically with Gaussianity of the sources. Conclusion: Independent component analysis is a viable tool for analyzing the temporal activity of EEG/MEG (magnetoencephalography) sources even if the underlying neural sources are mutually dependent. However, if ICA is used as a preprocessing algorithm for source localization, mutual information between sources introduces a bias in the reconstructed locations of the sources. Significance: Studies using ICA-algorithms based on MI have to be aware of possible errors in the spatial reconstruction of sources if these are coupled with other neural sources.

Web DOI [BibTex]

Neumann, A., Grosse-Wentrup, M., Buss, M., Gramann, K. The Effect of Mutual Information on Independent Component Analysis in EEG/MEG Analysis: A Simulation Study International Journal of Neuroscience, 118(11):1534-1546, November 2008 (article)

Web DOI [BibTex]

no image
gBoost: A Mathematical Programming Approach to Graph Classification and Regression

Saigo, H., Nowozin, S., Kadowaki, T., Kudo, T., Tsuda, K.

Machine Learning, 75(1):69-89, November 2008 (article)

Abstract
Graph mining methods enumerate frequently appearing subgraph patterns, which can be used as features for subsequent classification or regression. However, frequent patterns are not necessarily informative for the given learning problem. We propose a mathematical programming boosting method (gBoost) that progressively collects informative patterns. Compared to AdaBoost, gBoost can build the prediction rule with fewer iterations. To apply the boosting method to graph data, a branch-and-bound pattern search algorithm is developed based on the DFS code tree. The constructed search space is reused in later iterations to minimize the computation time. Our method can learn more efficiently than the simpler method based on frequent substructure mining, because the output labels are used as an extra information source for pruning the search space. Furthermore, by engineering the mathematical program, a wide range of machine learning problems can be solved without modifying the pattern search algorithm.

PDF DOI [BibTex]

Saigo, H., Nowozin, S., Kadowaki, T., Kudo, T., Tsuda, K. gBoost: A Mathematical Programming Approach to Graph Classification and Regression Machine Learning, 75(1):69-89, November 2008 (article)

PDF DOI [BibTex]

no image
Machine Learning for Motor Skills in Robotics

Peters, J.

K{\"u}nstliche Intelligenz, 2008(4):41-43, November 2008 (article)

Abstract
Autonomous robots that can adapt to novel situations has been a long standing vision of robotics, artificial intelligence, and the cognitive sciences. Early approaches to this goal during the heydays of artificial intelligence research in the late 1980s, however, made it clear that an approach purely based on reasoning or human insights would not be able to model all the perceptuomotor tasks of future robots. Instead, new hope was put in the growing wake of machine learning that promised fully adaptive control algorithms which learn both by observation and trial-and-error. However, to date, learning techniques have yet to fulfill this promise as only few methods manage to scale into the high-dimensional domains of manipulator and humanoid robotics and usually scaling was only achieved in precisely pre-structured domains. We have investigated the ingredients for a general approach to motor skill learning in order to get one step closer towards human-like performance. For doing so, we study two major components for such an approach, i.e., firstly, a theoretically well-founded general approach to representing the required control structures for task representation and execution and, secondly, appropriate learning algorithms which can be applied in this setting.

PDF Web [BibTex]

Peters, J. Machine Learning for Motor Skills in Robotics K{\"u}nstliche Intelligenz, 2008(4):41-43, November 2008 (article)

PDF Web [BibTex]

no image
Kernels, Regularization and Differential Equations

Steinke, F., Schölkopf, B.

Pattern Recognition, 41(11):3271-3286, November 2008 (article)

Abstract
Many common machine learning methods such as Support Vector Machines or Gaussian process inference make use of positive definite kernels, reproducing kernel Hilbert spaces, Gaussian processes, and regularization operators. In this work these objects are presented in a general, unifying framework, and interrelations are highlighted. With this in mind we then show how linear stochastic differential equation models can be incorporated naturally into the kernel framework. And vice versa, many kernel machines can be interpreted in terms of differential equations. We focus especially on ordinary differential equations, also known as dynamical systems, and it is shown that standard kernel inference algorithms are equivalent to Kalman filter methods based on such models. In order not to cloud qualitative insights with heavy mathematical machinery, we restrict ourselves to finite domains, implying that differential equations are treated via their corresponding finite difference equations.

PDF DOI [BibTex]

Steinke, F., Schölkopf, B. Kernels, Regularization and Differential Equations Pattern Recognition, 41(11):3271-3286, November 2008 (article)

PDF DOI [BibTex]

no image
Simultaneous Implicit Surface Reconstruction and Meshing

Giesen, J., Maier, M., Schölkopf, B.

(179), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, November 2008 (techreport)

Abstract
We investigate an implicit method to compute a piecewise linear representation of a surface from a set of sample points. As implicit surface functions we use the weighted sum of piecewise linear kernel functions. For such a function we can partition Rd in such a way that these functions are linear on the subsets of the partition. For each subset in the partition we can then compute the zero level set of the function exactly as the intersection of a hyperplane with the subset.

PDF [BibTex]

Giesen, J., Maier, M., Schölkopf, B. Simultaneous Implicit Surface Reconstruction and Meshing (179), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, November 2008 (techreport)

PDF [BibTex]

no image
Taxonomy Inference Using Kernel Dependence Measures

Blaschko, M., Gretton, A.

(181), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, November 2008 (techreport)

Abstract
We introduce a family of unsupervised algorithms, numerical taxonomy clustering, to simultaneously cluster data, and to learn a taxonomy that encodes the relationship between the clusters. The algorithms work by maximizing the dependence between the taxonomy and the original data. The resulting taxonomy is a more informative visualization of complex data than simple clustering; in addition, taking into account the relations between different clusters is shown to substantially improve the quality of the clustering, when compared with state-of-the-art algorithms in the literature (both spectral clustering and a previous dependence maximization approach). We demonstrate our algorithm on image and text data.

PDF [BibTex]

Blaschko, M., Gretton, A. Taxonomy Inference Using Kernel Dependence Measures (181), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, November 2008 (techreport)

PDF [BibTex]

no image
The genome of the simian and human malaria parasite Plasmodium knowlesi

Pain, A., Böhme, U., Berry, A., Mungall, K., Finn, R., Jackson, A., Mourier, T., Mistry, J., Pasini, E., Aslett, M., Balasubrammaniam, S., Borgwardt, K., Brooks, K., Carret, C., Carver, T., Cherevach, I., Chillingworth, T., Clarke, T., Galinski, M., Hall, N., Harper, D., Harris, D., Hauser, H., Ivens, A., Janssen, C., Keane, T., Larke, N., Lapp, S., Marti, M., Moule, S., Meyer, I., Ormond, D., Peters, N., Sanders, M., Sanders, T., Sergeant, T., Simmonds, M., Smith, F., Squares, R., Thurston, S., Tivey, A., Walker, D., White, B., Zuiderwijk, E., Churcher, C., Quail, M., Cowman, A., Turner, C., Rajandream, M., Kocken, C., Thomas, A., Newbold, C., Barrell, B., Berriman, M.

Nature, 455(7214):799-803, October 2008 (article)

Abstract
Plasmodium knowlesi is an intracellular malaria parasite whose natural vertebrate host is Macaca fascicularis (the 'kra' monkey); however, it is now increasingly recognized as a significant cause of human malaria, particularly in southeast Asia1, 2. Plasmodium knowlesi was the first malaria parasite species in which antigenic variation was demonstrated3, and it has a close phylogenetic relationship to Plasmodium vivax 4, the second most important species of human malaria parasite (reviewed in ref. 4). Despite their relatedness, there are important phenotypic differences between them, such as host blood cell preference, absence of a dormant liver stage or 'hypnozoite' in P. knowlesi, and length of the asexual cycle (reviewed in ref. 4). Here we present an analysis of the P. knowlesi (H strain, Pk1(A+) clone5) nuclear genome sequence. This is the first monkey malaria parasite genome to be described, and it provides an opportunity for comparison with the recently completed P. vivax genome4 and other sequenced Plasmodium genomes6, 7, 8. In contrast to other Plasmodium genomes, putative variant antigen families are dispersed throughout the genome and are associated with intrachromosomal telomere repeats. One of these families, the KIRs9, contains sequences that collectively match over one-half of the host CD99 extracellular domain, which may represent an unusual form of molecular mimicry.

PDF DOI [BibTex]

Pain, A., Böhme, U., Berry, A., Mungall, K., Finn, R., Jackson, A., Mourier, T., Mistry, J., Pasini, E., Aslett, M., Balasubrammaniam, S., Borgwardt, K., Brooks, K., Carret, C., Carver, T., Cherevach, I., Chillingworth, T., Clarke, T., Galinski, M., Hall, N., Harper, D., Harris, D., Hauser, H., Ivens, A., Janssen, C., Keane, T., Larke, N., Lapp, S., Marti, M., Moule, S., Meyer, I., Ormond, D., Peters, N., Sanders, M., Sanders, T., Sergeant, T., Simmonds, M., Smith, F., Squares, R., Thurston, S., Tivey, A., Walker, D., White, B., Zuiderwijk, E., Churcher, C., Quail, M., Cowman, A., Turner, C., Rajandream, M., Kocken, C., Thomas, A., Newbold, C., Barrell, B., Berriman, M. The genome of the simian and human malaria parasite Plasmodium knowlesi Nature, 455(7214):799-803, October 2008 (article)

PDF DOI [BibTex]

no image
A BOLD window into brain waves

Balduzzi, D., Riedner, B., Tononi, G.

Proceedings of the National Academy of Sciences of the United States of America, 105(41):15641-15642 , October 2008 (article)

Abstract
The brain is never inactive. Neurons fire at leisurely rates most of the time, even in sleep (1), although occasionally they fire more intensely, for example, when presented with certain stimuli. Coordinated changes in the activity and excitability of many neurons underlie spontaneous fluctuations in the electroencephalogram (EEG), first observed almost a century ago. These fluctuations can be very slow (infraslow oscillations, <0.1 Hz; slow oscillations, <1 Hz; and slow waves or delta waves, 1–4 Hz), intermediate (theta, 4–8 Hz; alpha, 8–12 Hz; and beta, 13–20 Hz), and fast (gamma, >30 Hz). Moreover, slower fluctuations appear to group and modulate faster ones (1, 2). The BOLD signal underlying functional magnetic resonance imaging (fMRI) also exhibits spontaneous fluctuations at the timescale of tens of seconds (infraslow, <0.1 Hz), which occurs at all times, during task-performance as well as during quiet wakefulness, rapid eye movement (REM) sleep, and non-REM sleep (NREM). Although the precise mechanism underlying the BOLD signal is still being investigated (3–5), it is becoming clear that spontaneous BOLD fluctuations are not just noise, but are tied to fluctuations in neural activity. In this issue of PNAS, He et al. (6) have been able to directly investigate the relationship between BOLD fluctuations and fluctuations in the brain's electrical activity in human subjects. He et al. (6) took advantage of the seminal observation by Biswal et al. (7) that spontaneous BOLD fluctuations in regions belonging to the same functional system are strongly correlated. As expected, He et al. saw that fMRI BOLD fluctuations were strongly correlated among regions within the sensorimotor system, but much less between sensorimotor regions and control regions (nonsensorimotor). The twist was that they did the fMRI recordings in subjects who had been implanted with intracranial electrocorticographic (ECoG) electrodes to record regional EEG signals (to localize epileptic foci). In a separate session, He et al. examined correlations in EEG signals between different regions. They found that, just like the BOLD fluctuations, infraslow and slow fluctuations in the EEG signal from sensorimotor-sensorimotor pairs of electrodes were positively correlated, whereas signals from sensorimotor-control pairs were not. Moreover, the correlation persisted across arousal states: in waking, NREM, and REM sleep. Finally, using several statistical approaches, they found a remarkable correspondence between regional correlations in the infraslow BOLD signal and regional correlations in the infraslow-slow EEG signal (<0.5 Hz or 1–4 Hz). Notably, another report has just appeared showing that mirror sites of auditory cortex across the two hemispheres, which show correlated BOLD activity, also show correlated infraslow EEG fluctuations recorded with ECoG electrodes (8). In this case, the correlated fluctuations reflected infraslow changes in EEG power in the gamma range [however, no significant correlations were found for slow ECoG frequencies (1–4 Hz)].

Web DOI [BibTex]

Balduzzi, D., Riedner, B., Tononi, G. A BOLD window into brain waves Proceedings of the National Academy of Sciences of the United States of America, 105(41):15641-15642 , October 2008 (article)

Web DOI [BibTex]

no image
Mixture Models for Protein Structure Ensembles

Hirsch, M., Habeck, M.

Bioinformatics, 24(19):2184-2192, October 2008 (article)

Web DOI [BibTex]

Hirsch, M., Habeck, M. Mixture Models for Protein Structure Ensembles Bioinformatics, 24(19):2184-2192, October 2008 (article)

Web DOI [BibTex]

no image
Structure of the human voltage-dependent anion channel

Bayrhuber, M., Meins, T., Habeck, M., Becker, S., Giller, K., Villinger, S., Vonrhein, C., Griesinger, C., Zweckstetter, M., Zeth, K.

Proceedings of the National Academy of Sciences of the United States of America, 105(40):15370-15375, October 2008 (article)

Abstract
The voltage-dependent anion channel (VDAC), also known as mitochondrial porin, is the most abundant protein in the mitochondrial outer membrane (MOM). VDAC is the channel known to guide the metabolic flux across the MOM and plays a key role in mitochondrially induced apoptosis. Here, we present the 3D structure of human VDAC1, which was solved conjointly by NMR spectroscopy and x-ray crystallography. Human VDAC1 (hVDAC1) adopts a &amp;#946;-barrel architecture composed of 19 &amp;#946;-strands with an &amp;#945;-helix located horizontally midway within the pore. Bioinformatic analysis indicates that this channel architecture is common to all VDAC proteins and is adopted by the general import pore TOM40 of mammals, which is also located in the MOM.

Web DOI [BibTex]

Bayrhuber, M., Meins, T., Habeck, M., Becker, S., Giller, K., Villinger, S., Vonrhein, C., Griesinger, C., Zweckstetter, M., Zeth, K. Structure of the human voltage-dependent anion channel Proceedings of the National Academy of Sciences of the United States of America, 105(40):15370-15375, October 2008 (article)

Web DOI [BibTex]

no image
MRI-Based Attenuation Correction for PET/MRI: A Novel Approach Combining Pattern Recognition and Atlas Registration

Hofmann, M., Steinke, F., Scheel, V., Charpiat, G., Farquhar, J., Aschoff, P., Brady, M., Schölkopf, B., Pichler, B.

Journal of Nuclear Medicine, 49(11):1875-1883, October 2008 (article)

Abstract
For quantitative PET information, correction of tissue photon attenuation is mandatory. Generally in conventional PET, the attenuation map is obtained from a transmission scan, which uses a rotating radionuclide source, or from the CT scan in a combined PET/CT scanner. In the case of PET/MRI scanners currently under development, insufficient space for the rotating source exists; the attenuation map can be calculated from the MR image instead. This task is challenging because MR intensities correlate with proton densities and tissue-relaxation properties, rather than with attenuation-related mass density. METHODS: We used a combination of local pattern recognition and atlas registration, which captures global variation of anatomy, to predict pseudo-CT images from a given MR image. These pseudo-CT images were then used for attenuation correction, as the process would be performed in a PET/CT scanner. RESULTS: For human brain scans, we show on a database of 17 MR/CT image pairs that our method reliably enables e stimation of a pseudo-CT image from the MR image alone. On additional datasets of MRI/PET/CT triplets of human brain scans, we compare MRI-based attenuation correction with CT-based correction. Our approach enables PET quantification with a mean error of 3.2% for predefined regions of interest, which we found to be clinically not significant. However, our method is not specific to brain imaging, and we show promising initial results on 1 whole-body animal dataset. CONCLUSION: This method allows reliable MRI-based attenuation correction for human brain scans. Further work is necessary to validate the method for whole-body imaging.

Web DOI [BibTex]

Hofmann, M., Steinke, F., Scheel, V., Charpiat, G., Farquhar, J., Aschoff, P., Brady, M., Schölkopf, B., Pichler, B. MRI-Based Attenuation Correction for PET/MRI: A Novel Approach Combining Pattern Recognition and Atlas Registration Journal of Nuclear Medicine, 49(11):1875-1883, October 2008 (article)

Web DOI [BibTex]

no image
Infinite Kernel Learning

Gehler, P., Nowozin, S.

(178), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, October 2008 (techreport)

Abstract
In this paper we consider the problem of automatically learning the kernel from general kernel classes. Specifically we build upon the Multiple Kernel Learning (MKL) framework and in particular on the work of (Argyriou, Hauser, Micchelli, & Pontil, 2006). We will formulate a Semi-Infinite Program (SIP) to solve the problem and devise a new algorithm to solve it (Infinite Kernel Learning, IKL). The IKL algorithm is applicable to both the finite and infinite case and we find it to be faster and more stable than SimpleMKL (Rakotomamonjy, Bach, Canu, & Grandvalet, 2007) for cases of many kernels. In the second part we present the first large scale comparison of SVMs to MKL on a variety of benchmark datasets, also comparing IKL. The results show two things: a) for many datasets there is no benefit in linearly combining kernels with MKL/IKL instead of the SVM classifier, thus the flexibility of using more than one kernel seems to be of no use, b) on some datasets IKL yields impressive increases in accuracy over SVM/MKL due to the possibility of using a largely increased kernel set. In those cases, IKL remains practical, whereas both cross-validation or standard MKL is infeasible.

PDF Web [BibTex]

Gehler, P., Nowozin, S. Infinite Kernel Learning (178), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, October 2008 (techreport)

PDF Web [BibTex]

no image
Support Vector Machines and Kernels for Computational Biology

Ben-Hur, A., Ong, C., Sonnenburg, S., Schölkopf, B., Rätsch, G.

PLoS Computational Biology, 4(10: e1000173):1-10, October 2008 (article)

PDF Web DOI [BibTex]

Ben-Hur, A., Ong, C., Sonnenburg, S., Schölkopf, B., Rätsch, G. Support Vector Machines and Kernels for Computational Biology PLoS Computational Biology, 4(10: e1000173):1-10, October 2008 (article)

PDF Web DOI [BibTex]

no image
Approximations for Binary Gaussian Process Classification

Nickisch, H., Rasmussen, C.

Journal of Machine Learning Research, 9, pages: 2035-2078, October 2008 (article)

Abstract
We provide a comprehensive overview of many recent algorithms for approximate inference in Gaussian process models for probabilistic binary classification. The relationships between several approaches are elucidated theoretically, and the properties of the different algorithms are corroborated by experimental results. We examine both 1) the quality of the predictive distributions and 2) the suitability of the different marginal likelihood approximations for model selection (selecting hyperparameters) and compare to a gold standard based on MCMC. Interestingly, some methods produce good predictive distributions although their marginal likelihood approximations are poor. Strong conclusions are drawn about the methods: The Expectation Propagation algorithm is almost always the method of choice unless the computational budget is very tight. We also extend existing methods in various ways, and provide unifying code implementing all approaches.

PDF PDF [BibTex]

Nickisch, H., Rasmussen, C. Approximations for Binary Gaussian Process Classification Journal of Machine Learning Research, 9, pages: 2035-2078, October 2008 (article)

PDF PDF [BibTex]

no image
Accurate NMR Structures Through Minimization of an Extended Hybrid Energy

Nilges, M., Bernard, A., Bardiaux, B., Malliavin, T., Habeck, M., Rieping, W.

Structure, 16(9):1305-1312, September 2008 (article)

Abstract
The use of generous distance bounds has been the hallmark of NMR structure determination. However, bounds necessitate the estimation of data quality before the calculation, reduce the information content, introduce human bias, and allow for major errors in the structures. Here, we propose a new rapid structure calculation scheme based on Bayesian analysis. The minimization of an extended energy function, including a new type of distance restraint and a term depending on the data quality, results in an estimation of the data quality in addition to coordinates. This allows for the determination of the optimal weight on the experimental information. The resulting structures are of better quality and closer to the X–ray crystal structure of the same molecule. With the new calculation approach, the analysis of discrepancies from the target distances becomes meaningful. The strategy may be useful in other applications—for example, in homology modeling.

PDF DOI [BibTex]

Nilges, M., Bernard, A., Bardiaux, B., Malliavin, T., Habeck, M., Rieping, W. Accurate NMR Structures Through Minimization of an Extended Hybrid Energy Structure, 16(9):1305-1312, September 2008 (article)

PDF DOI [BibTex]

no image
Similarity, Kernels, and the Triangle Inequality

Jäkel, F., Schölkopf, B., Wichmann, F.

Journal of Mathematical Psychology, 52(5):297-303, September 2008 (article)

Abstract
Similarity is used as an explanatory construct throughout psychology and multidimensional scaling (MDS) is the most popular way to assess similarity. In MDS, similarity is intimately connected to the idea of a geometric representation of stimuli in a perceptual space. Whilst connecting similarity and closeness of stimuli in a geometric representation may be intuitively plausible, Tversky and Gati [Tversky, A., Gati, I. (1982). Similarity, separability, and the triangle inequality. Psychological Review, 89(2), 123–154] have reported data which are inconsistent with the usual geometric representations that are based on segmental additivity. We show that similarity measures based on Shepard’s universal law of generalization [Shepard, R. N. (1987). Toward a universal law of generalization for psychologica science. Science, 237(4820), 1317–1323] lead to an inner product representation in a reproducing kernel Hilbert space. In such a space stimuli are represented by their similarity to all other stimuli. This representation, based on Shepard’s law, has a natural metric that does not have additive segments whilst still retaining the intuitive notion of connecting similarity and distance between stimuli. Furthermore, this representation has the psychologically appealing property that the distance between stimuli is bounded.

PDF Web DOI [BibTex]

Jäkel, F., Schölkopf, B., Wichmann, F. Similarity, Kernels, and the Triangle Inequality Journal of Mathematical Psychology, 52(5):297-303, September 2008 (article)

PDF Web DOI [BibTex]

no image
Comparison of Pattern Recognition Methods in Classifying High-resolution BOLD Signals Obtained at High Magnetic Field in Monkeys

Ku, S., Gretton, A., Macke, J., Logothetis, N.

Magnetic Resonance Imaging, 26(7):1007-1014, September 2008 (article)

Abstract
Pattern recognition methods have shown that functional magnetic resonance imaging (fMRI) data can reveal significant information about brain activity. For example, in the debate of how object categories are represented in the brain, multivariate analysis has been used to provide evidence of a distributed encoding scheme [Science 293:5539 (2001) 2425–2430]. Many follow-up studies have employed different methods to analyze human fMRI data with varying degrees of success [Nature reviews 7:7 (2006) 523–534]. In this study, we compare four popular pattern recognition methods: correlation analysis, support-vector machines (SVM), linear discriminant analysis (LDA) and Gaussian naïve Bayes (GNB), using data collected at high field (7 Tesla) with higher resolution than usual fMRI studies. We investigate prediction performance on single trials and for averages across varying numbers of stimulus presentations. The performance of the various algorithms depends on the nature of the brain activity being categorized: for several tasks, many of the methods work well, whereas for others, no method performs above chance level. An important factor in overall classification performance is careful preprocessing of the data, including dimensionality reduction, voxel selection and outlier elimination.

PDF Web DOI [BibTex]

Ku, S., Gretton, A., Macke, J., Logothetis, N. Comparison of Pattern Recognition Methods in Classifying High-resolution BOLD Signals Obtained at High Magnetic Field in Monkeys Magnetic Resonance Imaging, 26(7):1007-1014, September 2008 (article)

PDF Web DOI [BibTex]

no image
Large Scale Variational Inference and Experimental Design for Sparse Generalized Linear Models

Seeger, M., Nickisch, H.

(175), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, September 2008 (techreport)

PDF [BibTex]

Seeger, M., Nickisch, H. Large Scale Variational Inference and Experimental Design for Sparse Generalized Linear Models (175), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, September 2008 (techreport)

PDF [BibTex]

no image
Block-Iterative Algorithms for Non-Negative Matrix Approximation

Sra, S.

(176), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, September 2008 (techreport)

Abstract
In this report we present new algorithms for non-negative matrix approximation (NMA), commonly known as the NMF problem. Our methods improve upon the well-known methods of Lee & Seung [19] for both the Frobenius norm as well the Kullback-Leibler divergence versions of the problem. For the latter problem, our results are especially interesting because it seems to have witnessed much lesser algorithmic progress as compared to the Frobenius norm NMA problem. Our algorithms are based on a particular block-iterative acceleration technique for EM, which preserves the multiplicative nature of the updates and also ensures monotonicity. Furthermore, our algorithms also naturally apply to the Bregman-divergence NMA algorithms of Dhillon and Sra [8]. Experimentally, we show that our algorithms outperform the traditional Lee/Seung approach most of the time.

PDF [BibTex]

Sra, S. Block-Iterative Algorithms for Non-Negative Matrix Approximation (176), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, September 2008 (techreport)

PDF [BibTex]

no image
A Single-shot Measurement of the Energy of Product States in a Translation Invariant Spin Chain Can Replace Any Quantum Computation

Janzing, D., Wocjan, P., Zhang, S.

New Journal of Physics, 10(093004):1-18, September 2008 (article)

Abstract
In measurement-based quantum computation, quantum algorithms are implemented via sequences of measurements. We describe a translationally invariant finite-range interaction on a one-dimensional qudit chain and prove that a single-shot measurement of the energy of an appropriate computational basis state with respect to this Hamiltonian provides the output of any quantum circuit. The required measurement accuracy scales inverse polynomially with the size of the simulated quantum circuit. This shows that the implementation of energy measurements on generic qudit chains is as hard as the realization of quantum computation. Here, a ‘measurement‘ is any procedure that samples from the spectral measurement induced by the observable and the state under consideration. As opposed to measurement-based quantum computation, the post-measurement state is irrelevant.

PDF DOI [BibTex]

no image
Approximation Algorithms for Bregman Clustering Co-clustering and Tensor Clustering

Sra, S., Jegelka, S., Banerjee, A.

(177), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, September 2008 (techreport)

Abstract
The Euclidean K-means problem is fundamental to clustering and over the years it has been intensely investigated. More recently, generalizations such as Bregman k-means [8], co-clustering [10], and tensor (multi-way) clustering [40] have also gained prominence. A well-known computational difficulty encountered by these clustering problems is the NP-Hardness of the associated optimization task, and commonly used methods guarantee at most local optimality. Consequently, approximation algorithms of varying degrees of sophistication have been developed, though largely for the basic Euclidean K-means (or `1-norm K-median) problem. In this paper we present approximation algorithms for several Bregman clustering problems by building upon the recent paper of Arthur and Vassilvitskii [5]. Our algorithms obtain objective values within a factor O(logK) for Bregman k-means, Bregman co-clustering, Bregman tensor clustering, and weighted kernel k-means. To our knowledge, except for some special cases, approximation algorithms have not been considered for these general clustering problems. There are several important implications of our work: (i) under the same assumptions as Ackermann et al. [1] it yields a much faster algorithm (non-exponential in K, unlike [1]) for information-theoretic clustering, (ii) it answers several open problems posed by [4], including generalizations to Bregman co-clustering, and tensor clustering, (iii) it provides practical and easy to implement methods—in contrast to several other common approximation approaches.

PDF [BibTex]

Sra, S., Jegelka, S., Banerjee, A. Approximation Algorithms for Bregman Clustering Co-clustering and Tensor Clustering (177), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, September 2008 (techreport)

PDF [BibTex]

no image
Voluntary Brain Regulation and Communication with ECoG-Signals

Hinterberger, T., Widmann, G., Lal, T., Hill, J., Tangermann, M., Rosenstiel, W., Schölkopf, B., Elger, C., Birbaumer, N.

Epilepsy and Behavior, 13(2):300-306, August 2008 (article)

Abstract
Brain–computer interfaces (BCIs) can be used for communication in writing without muscular activity or for learning to control seizures by voluntary regulation of brain signals such as the electroencephalogram (EEG). Three of five patients with epilepsy were able to spell their names with electrocorticogram (ECoG) signals derived from motor-related areas within only one or two training sessions. Imagery of finger or tongue movements was classified with support-vector classification of autoregressive coefficients derived from the ECoG signals. After training of the classifier, binary classification responses were used to select letters from a computer-generated menu. Offline analysis showed increased theta activity in the unsuccessful patients, whereas the successful patients exhibited dominant sensorimotor rhythms that they could control. The high spatial resolution and increased signal-to-noise ratio in ECoG signals, combined with short training periods, may offer an alternative for communication in complete paralysis, locked-in syndrome, and motor restoration.

PDF DOI [BibTex]

Hinterberger, T., Widmann, G., Lal, T., Hill, J., Tangermann, M., Rosenstiel, W., Schölkopf, B., Elger, C., Birbaumer, N. Voluntary Brain Regulation and Communication with ECoG-Signals Epilepsy and Behavior, 13(2):300-306, August 2008 (article)

PDF DOI [BibTex]

no image
Combining Appearance and Motion for Human Action Classification in Videos

Dhillon, P., Nowozin, S., Lampert, C.

(174), Max-Planck-Institute for Biological Cybernetics, Tübingen, Germany, August 2008 (techreport)

Abstract
We study the question of activity classification in videos and present a novel approach for recognizing human action categories in videos by combining information from appearance and motion of human body parts. Our approach uses a tracking step which involves Particle Filtering and a local non - parametric clustering step. The motion information is provided by the trajectory of the cluster modes of a local set of particles. The statistical information about the particles of that cluster over a number of frames provides the appearance information. Later we use a “Bag ofWords” model to build one histogram per video sequence from the set of these robust appearance and motion descriptors. These histograms provide us characteristic information which helps us to discriminate among various human actions and thus classify them correctly. We tested our approach on the standard KTH and Weizmann human action datasets and the results were comparable to the state of the art. Additionally our approach is able to distinguish between activities that involve the motion of complete body from those in which only certain body parts move. In other words, our method discriminates well between activities with “gross motion” like running, jogging etc. and “local motion” like waving, boxing etc.

PDF [BibTex]

Dhillon, P., Nowozin, S., Lampert, C. Combining Appearance and Motion for Human Action Classification in Videos (174), Max-Planck-Institute for Biological Cybernetics, Tübingen, Germany, August 2008 (techreport)

PDF [BibTex]

no image
Predicting phenotypic effects of gene perturbations in C. elegans using an integrated network model

Borgwardt, K.

BioEssays, 30(8):707–710, August 2008 (article)

Abstract
Predicting the phenotype of an organism from its genotype is a central question in genetics. Most importantly, we would like to find out if the perturbation of a single gene may be the cause of a disease. However, our current ability to predict the phenotypic effects of perturbations of individual genes is limited. Network models of genes are one tool for tackling this problem. In a recent study, (Lee et al.) it has been shown that network models covering the majority of genes of an organism can be used for accurately predicting phenotypic effects of gene perturbations in multicellular organisms.

Web DOI [BibTex]

no image
Multi-class Common Spatial Pattern and Information Theoretic Feature Extraction

Grosse-Wentrup, M., Buss, M.

IEEE Transactions on Biomedical Engineering, 55(8):1991-2000, August 2008 (article)

Abstract
We address two shortcomings of the common spatial patterns (CSP) algorithm for spatial filtering in the context of brain--computer interfaces (BCIs) based on electroencephalography/magnetoencephalography (EEG/MEG): First, the question of optimality of CSP in terms of the minimal achievable classification error remains unsolved. Second, CSP has been initially proposed for two-class paradigms. Extensions to multiclass paradigms have been suggested, but are based on heuristics. We address these shortcomings in the framework of information theoretic feature extraction (ITFE). We show that for two-class paradigms, CSP maximizes an approximation of mutual information of extracted EEG/MEG components and class labels. This establishes a link between CSP and the minimal classification error. For multiclass paradigms, we point out that CSP by joint approximate diagonalization (JAD) is equivalent to independent component analysis (ICA), and provide a method to choose those independent components (ICs) that approximately maximize mutual information of ICs and class labels. This eliminates the need for heuristics in multiclass CSP, and allows incorporating prior class probabilities. The proposed method is applied to the dataset IIIa of the third BCI competition, and is shown to increase the mean classification accuracy by 23.4% in comparison to multiclass CSP.

PDF Web DOI [BibTex]

Grosse-Wentrup, M., Buss, M. Multi-class Common Spatial Pattern and Information Theoretic Feature Extraction IEEE Transactions on Biomedical Engineering, 55(8):1991-2000, August 2008 (article)

PDF Web DOI [BibTex]

no image
Example-based Learning for Single-image Super-resolution and JPEG Artifact Removal

Kim, K., Kwon, Y.

(173), Max-Planck-Institute for Biological Cybernetics, Tübingen, Germany, August 2008 (techreport)

Abstract
This paper proposes a framework for single-image super-resolution and JPEG artifact removal. The underlying idea is to learn a map from input low-quality images (suitably preprocessed low-resolution or JPEG encoded images) to target high-quality images based on example pairs of input and output images. To retain the complexity of the resulting learning problem at a moderate level, a patch-based approach is taken such that kernel ridge regression (KRR) scans the input image with a small window (patch) and produces a patchvalued output for each output pixel location. These constitute a set of candidate images each of which reflects different local information. An image output is then obtained as a convex combination of candidates for each pixel based on estimated confidences of candidates. To reduce the time complexity of training and testing for KRR, a sparse solution is found by combining the ideas of kernel matching pursuit and gradient descent. As a regularized solution, KRR leads to a better generalization than simply storing the examples as it has been done in existing example-based super-resolution algorithms and results in much less noisy images. However, this may introduce blurring and ringing artifacts around major edges as sharp changes are penalized severely. A prior model of a generic image class which takes into account the discontinuity property of images is adopted to resolve this problem. Comparison with existing super-resolution and JPEG artifact removal methods shows the effectiveness of the proposed method. Furthermore, the proposed method is generic in that it has the potential to be applied to many other image enhancement applications.

PDF [BibTex]

Kim, K., Kwon, Y. Example-based Learning for Single-image Super-resolution and JPEG Artifact Removal (173), Max-Planck-Institute for Biological Cybernetics, Tübingen, Germany, August 2008 (techreport)

PDF [BibTex]

no image
At-TAX: A Whole Genome Tiling Array Resource for Developmental Expression Analysis and Transcript Identification in Arabidopsis thaliana

Laubinger, S., Zeller, G., Henz, S., Sachsenberg, T., Widmer, C., Naouar, N., Vuylsteke, M., Schölkopf, B., Rätsch, G., Weigel, D.

Genome Biology, 9(7: R112):1-16, July 2008 (article)

Abstract
Gene expression maps for model organisms, including Arabidopsis thaliana, have typically been created using gene-centric expression arrays. Here, we describe a comprehensive expression atlas, Arabidopsis thaliana Tiling Array Express (At-TAX), which is based on whole-genome tiling arrays. We demonstrate that tiling arrays are accurate tools for gene expression analysis and identified more than 1,000 unannotated transcribed regions. Visualizations of gene expression estimates, transcribed regions, and tiling probe measurements are accessible online at the At-TAX homepage.

PDF DOI [BibTex]

Laubinger, S., Zeller, G., Henz, S., Sachsenberg, T., Widmer, C., Naouar, N., Vuylsteke, M., Schölkopf, B., Rätsch, G., Weigel, D. At-TAX: A Whole Genome Tiling Array Resource for Developmental Expression Analysis and Transcript Identification in Arabidopsis thaliana Genome Biology, 9(7: R112):1-16, July 2008 (article)

PDF DOI [BibTex]

no image
Unsupervised Bayesian Time-series Segmentation based on Linear Gaussian State-space Models

Chiappa, S.

(171), Max-Planck-Institute for Biological Cybernetics, Tübingen, Germany, June 2008 (techreport)

Abstract
Unsupervised time-series segmentation in the general scenario in which the number of segment-types and segment boundaries are a priori unknown is a fundamental problem in many applications and requires an accurate segmentation model as well as a way of determining an appropriate number of segment-types. In most approaches, segmentation and determination of number of segment-types are addressed in two separate steps, since the segmentation model assumes a predefined number of segment-types. The determination of number of segment-types is thus achieved by training and comparing several separate models. In this paper, we take a Bayesian approach to a segmentation model based on linear Gaussian state-space models to achieve structure selection within the model. An appropriate prior distribution on the parameters is used to enforce a sparse parametrization, such that the model automatically selects the smallest number of underlying dynamical systems that explain the data well and a parsimonious structure for each dynamical system. As the resulting model is computationally intractable, we introduce a variational approximation, in which a reformulation of the problem enables to use an efficient inference algorithm.

[BibTex]

Chiappa, S. Unsupervised Bayesian Time-series Segmentation based on Linear Gaussian State-space Models (171), Max-Planck-Institute for Biological Cybernetics, Tübingen, Germany, June 2008 (techreport)

[BibTex]

no image
A New Non-monotonic Gradient Projection Method for the Non-negative Least Squares Problem

Kim, D., Sra, S., Dhillon, I.

(TR-08-28), University of Texas, Austin, TX, USA, June 2008 (techreport)

Web [BibTex]

Kim, D., Sra, S., Dhillon, I. A New Non-monotonic Gradient Projection Method for the Non-negative Least Squares Problem (TR-08-28), University of Texas, Austin, TX, USA, June 2008 (techreport)

Web [BibTex]

no image
Graphical Analysis of NMR Structural Quality and Interactive Contact Map of NOE Assignments in ARIA

Bardiaux, B., Bernard, A., Rieping, W., Habeck, M., Malliavin, T., Nilges, M.

BMC Structural Biology, 8(30):1-5, June 2008 (article)

Abstract
BACKGROUND: The Ambiguous Restraints for Iterative Assignment (ARIA) approach is widely used for NMR structure determination. It is based on simultaneously calculating structures and assigning NOE through an iterative protocol. The final solution consists of a set of conformers and a list of most probable assignments for the input NOE peak list. RESULTS: ARIA was extended with a series of graphical tools to facilitate a detailed analysis of the intermediate and final results of the ARIA protocol. These additional features provide (i) an interactive contact map, serving as a tool for the analysis of assignments, and (ii) graphical representations of structure quality scores and restraint statistics. The interactive contact map between residues can be clicked to obtain information about the restraints and their contributions. Profiles of quality scores are plotted along the protein sequence, and contact maps provide information of the agreement with the data on a residue pair level. CONCLUSIONS: The g raphical tools and outputs described here significantly extend the validation and analysis possibilities of NOE assignments given by ARIA as well as the analysis of the quality of the final structure ensemble. These tools are included in the latest version of ARIA, which is available at http://aria.pasteur.fr. The Web site also contains an installation guide, a user manual and example calculations.

PDF DOI [BibTex]

Bardiaux, B., Bernard, A., Rieping, W., Habeck, M., Malliavin, T., Nilges, M. Graphical Analysis of NMR Structural Quality and Interactive Contact Map of NOE Assignments in ARIA BMC Structural Biology, 8(30):1-5, June 2008 (article)

PDF DOI [BibTex]

no image
Operational Space Control: A Theoretical and Empirical Comparison

Nakanishi, J., Cory, R., Mistry, M., Peters, J., Schaal, S.

International Journal of Robotics Research, 27(6):737-757, June 2008 (article)

PDF Web DOI [BibTex]

Nakanishi, J., Cory, R., Mistry, M., Peters, J., Schaal, S. Operational Space Control: A Theoretical and Empirical Comparison International Journal of Robotics Research, 27(6):737-757, June 2008 (article)

PDF Web DOI [BibTex]

no image
Kernel Methods in Machine Learning

Hofmann, T., Schölkopf, B., Smola, A.

Annals of Statistics, 36(3):1171-1220, June 2008 (article)

Abstract
We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on nonvectorial data.

PDF PDF DOI [BibTex]

Hofmann, T., Schölkopf, B., Smola, A. Kernel Methods in Machine Learning Annals of Statistics, 36(3):1171-1220, June 2008 (article)

PDF PDF DOI [BibTex]

no image
Cross-validation Optimization for Large Scale Structured Classification Kernel Methods

Seeger, M.

Journal of Machine Learning Research, 9, pages: 1147-1178, June 2008 (article)

Abstract
We propose a highly efficient framework for penalized likelihood kernel methods applied to multi-class models with a large, structured set of classes. As opposed to many previous approaches which try to decompose the fitting problem into many smaller ones, we focus on a Newton optimization of the complete model, making use of model structure and linear conjugate gradients in order to approximate Newton search directions. Crucially, our learning method is based entirely on matrix-vector multiplication primitives with the kernel matrices and their derivatives, allowing straightforward specialization to new kernels, and focusing code optimization efforts to these primitives only. Kernel parameters are learned automatically, by maximizing the cross-validation log likelihood in a gradient-based way, and predictive probabilities are estimated. We demonstrate our approach on large scale text classification tasks with hierarchical structure on thousands of classes, achieving state-of-the-art results in an order of magnitude less time than previous work.

PDF PDF [BibTex]

Seeger, M. Cross-validation Optimization for Large Scale Structured Classification Kernel Methods Journal of Machine Learning Research, 9, pages: 1147-1178, June 2008 (article)

PDF PDF [BibTex]

no image
Non-monotonic Poisson Likelihood Maximization

Sra, S., Kim, D., Schölkopf, B.

(170), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, June 2008 (techreport)

Abstract
This report summarizes the theory and some main applications of a new non-monotonic algorithm for maximizing a Poisson Likelihood, which for Positron Emission Tomography (PET) is equivalent to minimizing the associated Kullback-Leibler Divergence, and for Transmission Tomography is similar to maximizing the dual of a maximum entropy problem. We call our method non-monotonic maximum likelihood (NMML) and show its application to different problems such as tomography and image restoration. We discuss some theoretical properties such as convergence for our algorithm. Our experimental results indicate that speedups obtained via our non-monotonic methods are substantial.

PDF [BibTex]

Sra, S., Kim, D., Schölkopf, B. Non-monotonic Poisson Likelihood Maximization (170), Max-Planck Institute for Biological Cybernetics, Tübingen, Germany, June 2008 (techreport)

PDF [BibTex]

no image
Integrated Information in Discrete Dynamical Systems: Motivation and Theoretical Framework

Balduzzi, D., Tononi, G.

PLoS Computational Biology, 4(6):1-18, June 2008 (article)

Abstract
This paper introduces a time- and state-dependent measure of integrated information, φ, which captures the repertoire of causal states available to a system as a whole. Specifically, φ quantifies how much information is generated (uncertainty is reduced) when a system enters a particular state through causal interactions among its elements, above and beyond the information generated independently by its parts. Such mathematical characterization is motivated by the observation that integrated information captures two key phenomenological properties of consciousness: (i) there is a large repertoire of conscious experiences so that, when one particular experience occurs, it generates a large amount of information by ruling out all the others; and (ii) this information is integrated, in that each experience appears as a whole that cannot be decomposed into independent parts. This paper extends previous work on stationary systems and applies integrated information to discrete networks as a function of their dynamics and causal architecture. An analysis of basic examples indicates the following: (i) φ varies depending on the state entered by a network, being higher if active and inactive elements are balanced and lower if the network is inactive or hyperactive. (ii) φ varies for systems with identical or similar surface dynamics depending on the underlying causal architecture, being low for systems that merely copy or replay activity states. (iii) φ varies as a function of network architecture. High φ values can be obtained by architectures that conjoin functional specialization with functional integration. Strictly modular and homogeneous systems cannot generate high φ because the former lack integration, whereas the latter lack information. Feedforward and lattice architectures are capable of generating high φ but are inefficient. (iv) In Hopfield networks, φ is low for attractor states and neutral states, but increases if the networks are optimized to achieve tension between local and global interactions. These basic examples appear to match well against neurobiological evidence concerning the neural substrates of consciousness. More generally, φ appears to be a useful metric to characterize the capacity of any physical system to integrate information.

Web DOI [BibTex]

no image
Information Consistency of Nonparametric Gaussian Process Methods

Seeger, MW., Kakade, SM., Foster, DP.

IEEE Transactions on Information Theory, 54(5):2376-2382, May 2008 (article)

Abstract
Abstract—Bayesian nonparametric models are widely and successfully used for statistical prediction. While posterior consistency properties are well studied in quite general settings, results have been proved using abstract concepts such as metric entropy, and they come with subtle conditions which are hard to validate and not intuitive when applied to concrete models. Furthermore, convergence rates are difficult to obtain. By focussing on the concept of information consistency for Bayesian Gaussian process (GP)models, consistency results and convergence rates are obtained via a regret bound on cumulative log loss. These results depend strongly on the covariance function of the prior process, thereby giving a novel interpretation to penalization with reproducing kernel Hilbert space norms and to commonly used covariance function classes and their parameters. The proof of the main result employs elementary convexity arguments only. A theorem of Widom is used in order to obtain precise convergence rates for several covariance functions widely used in practice.

Web DOI [BibTex]

Seeger, MW., Kakade, SM., Foster, DP. Information Consistency of Nonparametric Gaussian Process Methods IEEE Transactions on Information Theory, 54(5):2376-2382, May 2008 (article)

Web DOI [BibTex]

no image
Relating the Thermodynamic Arrow of Time to the Causal Arrow

Allahverdyan, A., Janzing, D.

Journal of Statistical Mechanics, 2008(P04001):1-21, April 2008 (article)

Abstract
Consider a Hamiltonian system that consists of a slow subsystem S and a fast subsystem F. The autonomous dynamics of S is driven by an effective Hamiltonian, but its thermodynamics is unexpected. We show that a well-defined thermodynamic arrow of time (second law) emerges for S whenever there is a well-defined causal arrow from S to F and the back-action is negligible. This is because the back-action of F on S is described by a non-globally Hamiltonian Born–Oppenheimer term that violates the Liouville theorem, and makes the second law inapplicable to S. If S and F are mixing, under the causal arrow condition they are described by microcanonical distributions P(S) and P(S|F). Their structure supports a causal inference principle proposed recently in machine learning.

Web DOI [BibTex]

Allahverdyan, A., Janzing, D. Relating the Thermodynamic Arrow of Time to the Causal Arrow Journal of Statistical Mechanics, 2008(P04001):1-21, April 2008 (article)

Web DOI [BibTex]

no image
Generalization and Similarity in Exemplar Models of Categorization: Insights from Machine Learning

Jäkel, F., Schölkopf, B., Wichmann, F.

Psychonomic Bulletin and Review, 15(2):256-271, April 2008 (article)

Abstract
Exemplar theories of categorization depend on similarity for explaining subjects’ ability to generalize to new stimuli. A major criticism of exemplar theories concerns their lack of abstraction mechanisms and thus, seemingly, generalization ability. Here, we use insights from machine learning to demonstrate that exemplar models can actually generalize very well. Kernel methods in machine learning are akin to exemplar models and very successful in real-world applications. Their generalization performance depends crucially on the chosen similaritymeasure. While similarity plays an important role in describing generalization behavior it is not the only factor that controls generalization performance. In machine learning, kernel methods are often combined with regularization techniques to ensure good generalization. These same techniques are easily incorporated in exemplar models. We show that the Generalized Context Model (Nosofsky, 1986) and ALCOVE (Kruschke, 1992) are closely related to a statistical model called kernel logistic regression. We argue that generalization is central to the enterprise of understanding categorization behavior and suggest how insights from machine learning can offer some guidance. Keywords: kernel, similarity, regularization, generalization, categorization.

PDF Web DOI [BibTex]

no image
A Kernel Method for the Two-sample Problem

Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., Smola, A.

(157), Max-Planck-Institute for Biological Cybernetics Tübingen, April 2008 (techreport)

Abstract
We propose a framework for analyzing and comparing distributions, allowing us to design statistical tests to determine if two samples are drawn from different distributions. Our test statistic is the largest difference in expectations over functions in the unit ball of a reproducing kernel Hilbert space (RKHS). We present two tests based on large deviation bounds for the test statistic, while a third is based on the asymptotic distribution of this statistic. The test statistic can be computed in quadratic time, although efficient linear time approximations are available. Several classical metrics on distributions are recovered when the function space used to compute the difference in expectations is allowed to be more general (eg.~a Banach space). We apply our two-sample tests to a variety of problems, including attribute matching for databases using the Hungarian marriage method, where they perform strongly. Excellent performance is also obtained when comparing distributions over graphs, for which these are the first such tests.

PDF [BibTex]

Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., Smola, A. A Kernel Method for the Two-sample Problem (157), Max-Planck-Institute for Biological Cybernetics Tübingen, April 2008 (techreport)

PDF [BibTex]

no image
Data-driven efficient score tests for deconvolution hypotheses

Langovoy, M.

Inverse Problems, 24(2):1-17, April 2008 (article)

Abstract
We consider testing statistical hypotheses about densities of signals in deconvolution models. A new approach to this problem is proposed. We constructed score tests for the deconvolution density testing with the known noise density and efficient score tests for the case of unknown density. The tests are incorporated with model selection rules to choose reasonable model dimensions automatically by the data. Consistency of the tests is proved.

PDF DOI [BibTex]

no image
Manifold-valued Thin-plate Splines with Applications in Computer Graphics

Steinke, F., Hein, M., Peters, J., Schölkopf, B.

Computer Graphics Forum, 27(2):437-448, April 2008 (article)

Abstract
We present a generalization of thin-plate splines for interpolation and approximation of manifold-valued data, and demonstrate its usefulness in computer graphics with several applications from different fields. The cornerstone of our theoretical framework is an energy functional for mappings between two Riemannian manifolds which is independent of parametrization and respects the geometry of both manifolds. If the manifolds are Euclidean, the energy functional reduces to the classical thin-plate spline energy. We show how the resulting optimization problems can be solved efficiently in many cases. Our example applications range from orientation interpolation and motion planning in animation over geometric modelling tasks to color interpolation.

PDF AVI Web DOI [BibTex]

no image
The Metric Nearness Problem

Brickell, J., Dhillon, I., Sra, S., Tropp, J.

SIAM Journal on Matrix Analysis and Applications, 30(1):375-396, April 2008 (article)

Abstract
Metric nearness refers to the problem of optimally restoring metric properties to distance measurements that happen to be nonmetric due to measurement errors or otherwise. Metric data can be important in various settings, for example, in clustering, classification, metric-based indexing, query processing, and graph theoretic approximation algorithms. This paper formulates and solves the metric nearness problem: Given a set of pairwise dissimilarities, find a “nearest” set of distances that satisfy the properties of a metric—principally the triangle inequality. For solving this problem, the paper develops efficient triangle fixing algorithms that are based on an iterative projection method. An intriguing aspect of the metric nearness problem is that a special case turns out to be equivalent to the all pairs shortest paths problem. The paper exploits this equivalence and develops a new algorithm for the latter problem using a primal-dual method. Applications to graph clustering are provided as an illustratio n. We include experiments that demonstrate the computational superiority of triangle fixing over general purpose convex programming software. Finally, we conclude by suggesting various useful extensions and generalizations to metric nearness.

Web DOI [BibTex]

Brickell, J., Dhillon, I., Sra, S., Tropp, J. The Metric Nearness Problem SIAM Journal on Matrix Analysis and Applications, 30(1):375-396, April 2008 (article)

Web DOI [BibTex]

no image
Bayesian Inference and Optimal Design for the Sparse Linear Model

Seeger, MW.

Journal of Machine Learning Research, 9, pages: 759-813, April 2008 (article)

Abstract
The linear model with sparsity-favouring prior on the coefficients has important applications in many different domains. In machine learning, most methods to date search for maximum a posteriori sparse solutions and neglect to represent posterior uncertainties. In this paper, we address problems of Bayesian optimal design (or experiment planning), for which accurate estimates of uncertainty are essential. To this end, we employ expectation propagation approximate inference for the linear model with Laplace prior, giving new insight into numerical stability properties and proposing a robust algorithm. We also show how to estimate model hyperparameters by empirical Bayesian maximisation of the marginal likelihood, and propose ideas in order to scale up the method to very large underdetermined problems. We demonstrate the versatility of our framework on the application of gene regulatory network identification from micro-array expression data, where both the Laplace prior and the active experimental design approach are shown to result in significant improvements. We also address the problem of sparse coding of natural images, and show how our framework can be used for compressive sensing tasks.

PDF PDF [BibTex]

Seeger, MW. Bayesian Inference and Optimal Design for the Sparse Linear Model Journal of Machine Learning Research, 9, pages: 759-813, April 2008 (article)

PDF PDF [BibTex]

no image
Consistency of Spectral Clustering

von Luxburg, U., Belkin, M., Bousquet, O.

Annals of Statistics, 36(2):555-586, April 2008 (article)

Abstract
Consistency is a key property of statistical algorithms when the data is drawn from some underlying probability distribution. Surprisingly, despite decades of work, little is known about consistency of most clustering algorithms. In this paper we investigate consistency of the popular family of spectral clustering algorithms, which clusters the data with the help of eigenvectors of graph Laplacian matrices. We develop new methods to establish that for increasing sample size, those eigenvectors converge to the eigenvectors of certain limit operators. As a result we can prove that one of the two major classes of spectral clustering (normalized clustering) converges under very general conditions, while the other (unnormalized clustering) is only consistent under strong additional assumptions, which are not always satisfied in real data. We conclude that our analysis provides strong evidence for the superiority of normalized spectral clustering.

PDF DOI [BibTex]

von Luxburg, U., Belkin, M., Bousquet, O. Consistency of Spectral Clustering Annals of Statistics, 36(2):555-586, April 2008 (article)

PDF DOI [BibTex]

no image
Plant Classification from Bat-Like Echolocation Signals

Yovel, Y., Franz, MO., Stilz, P., Schnitzler, H-U.

PLoS Computational Biology, 4(3, e1000032):1-13, March 2008 (article)

Abstract
Classification of plants according to their echoes is an elementary component of bat behavior that plays an important role in spatial orientation and food acquisition. Vegetation echoes are, however, highly complex stochastic signals: from an acoustical point of view, a plant can be thought of as a three-dimensional array of leaves reflecting the emitted bat call. The received echo is therefore a superposition of many reflections. In this work we suggest that the classification of these echoes might not be such a troublesome routine for bats as formerly thought. We present a rather simple approach to classifying signals from a large database of plant echoes that were created by ensonifying plants with a frequency-modulated bat-like ultrasonic pulse. Our algorithm uses the spectrogram of a single echo from which it only uses features that are undoubtedly accessible to bats. We used a standard machine learning algorithm (SVM) to automatically extract suitable linear combinations of time and frequency cues from the spectrograms such that classification with high accuracy is enabled. This demonstrates that ultrasonic echoes are highly informative about the species membership of an ensonified plant, and that this information can be extracted with rather simple, biologically plausible analysis. Thus, our findings provide a new explanatory basis for the poorly understood observed abilities of bats in classifying vegetation and other complex objects.

Web DOI [BibTex]

Yovel, Y., Franz, MO., Stilz, P., Schnitzler, H-U. Plant Classification from Bat-Like Echolocation Signals PLoS Computational Biology, 4(3, e1000032):1-13, March 2008 (article)

Web DOI [BibTex]

no image
Causal Reasoning by Evaluating the Complexity of Conditional Densities with Kernel Methods

Sun, X., Janzing, D., Schölkopf, B.

Neurocomputing, 71(7-9):1248-1256, March 2008 (article)

Abstract
We propose a method to quantify the complexity of conditional probability measures by a Hilbert space seminorm of the logarithm of its density. The concept of reproducing kernel Hilbert spaces (RKHSs) is a flexible tool to define such a seminorm by choosing an appropriate kernel. We present several examples with artificial data sets where our kernel-based complexity measure is consistent with our intuitive understanding of complexity of densities. The intention behind the complexity measure is to provide a new approach to inferring causal directions. The idea is that the factorization of the joint probability measure P(effect, cause) into P(effect|cause)P(cause) leads typically to "simpler" and "smoother" terms than the factorization into P(cause|effect)P(effect). Since the conventional constraint-based approach of causal discovery is not able to determine the causal direction between only two variables, our inference principle can in particular be useful when combined with other existing methods. We provide several simple examples with real-world data where the true causal directions indeed lead to simpler (conditional) densities.

Web DOI [BibTex]

no image
Natural Actor-Critic

Peters, J., Schaal, S.

Neurocomputing, 71(7-9):1180-1190, March 2008 (article)

Abstract
In this paper, we suggest a novel reinforcement learning architecture, the Natural Actor-Critic. The actor updates are achieved using stochastic policy gradients em- ploying Amari’s natural gradient approach, while the critic obtains both the natural policy gradient and additional parameters of a value function simultaneously by lin- ear regression. We show that actor improvements with natural policy gradients are particularly appealing as these are independent of coordinate frame of the chosen policy representation, and can be estimated more efficiently than regular policy gra- dients. The critic makes use of a special basis function parameterization motivated by the policy-gradient compatible function approximation. We show that several well-known reinforcement learning methods such as the original Actor-Critic and Bradtke’s Linear Quadratic Q-Learning are in fact Natural Actor-Critic algorithms. Empirical evaluations illustrate the effectiveness of our techniques in comparison to previous methods, and also demonstrate their applicability for learning control on an anthropomorphic robot arm.

PDF PDF DOI [BibTex]

Peters, J., Schaal, S. Natural Actor-Critic Neurocomputing, 71(7-9):1180-1190, March 2008 (article)

PDF PDF DOI [BibTex]