Header logo is ei


2011


no image
Combined whole-body PET/MR imaging: MR contrast agents do not affect the quantitative accuracy of PET following attenuation correction

Lois, C., Kupferschläger, J., Bezrukov, I., Schmidt, H., Werner, M., Mannheim, J., Pichler, B., Schwenzer, N., Beyer, T.

(SST15-05 ), 97th Scientific Assemble and Annual Meeting of the Radiological Society of North America (RSNA), December 2011 (talk)

Abstract
PURPOSE Combined PET/MR imaging entails the use of MR contrast agents (MRCA) as part of integrated protocols. We assess additional attenuation of the PET emission signals in the presence of oral and intraveneous (iv) MRCA made up of iron oxide and Gd-chelates, respectively. METHOD AND MATERIALS Phantom scans were performed on a clinical PET/CT (Biograph HiRez16, Siemens) and integrated whole-body PET/MR (Biograph mMR, Siemens) using oral (Lumirem) and intraveneous (Gadovist) MRCA. Reference PET attenuation values were determined on a small-animal PET (Inveon, Siemens) using standard PET transmission imaging (TX). Seven syringes of 5mL were filled with (a) Water, (b) Lumirem_100 (100% conc.), (c) Gadovist_100 (100%), (d) Gadovist_18 (18%), (e) Gadovist_02 (0.2%), (f) Imeron-400 CT iv-contrast (100%) and (g) Imeron-400 (2.4%). The same set of syringes was scanned on CT (Sensation16, Siemens) at 120kVp and 160mAs. The effect of MRCA on the attenuation of PET emission data was evaluated using a 20cm cylinder filled uniformly with [18F]-FDG (FDG) in water (BGD). Three 4.5cm diameter cylinders were inserted into the phantom: (C1) Teflon, (C2) Water+FDG (2:1) and (C3) Lumirem_100+FDG (2:1). Two 50mL syringes filled with Gadovist_02+FDG (Sy1) and water+FDG (Sy2) were attached to the sides of (C1) to mimick the effects of iv-contrast in vessels near bone. Syringe-to-background activity ratio was 4-to-1. PET emission data were acquired for 10min each using the PET/CT and the PET/MR. Images were reconstructed using CT- and MR-based attenuation correction. RESULTS Mean linear PET attenuation (cm-1) on TX was (a) 0.098, (b) 0.098, (c) 0.300, (d) 0.134, (e) 0.095, (f) 0.397 and (g) 0.105. Corresponding CT attenuation (HU) was: (a) 5, (b) 14, (c) 3070, (d) 1040, (e) 13, (f) 3070 and (g) 347. Lumirem had little effect on PET attenuation with (C3) being 13% and 10% higher than (C2) on PET/CT and PET/MR, respectively. Gadovist_02 had even smaller effects with (Sy1) being 2.5% lower than (Sy2) on PET/CT and 1.2% higher than (Sy2) on PET/MR. CONCLUSION MRCA in high and clinically relevant concentrations have attenuation values similar to that of CT contrast and water, respectively. In clinical PET/MR scenarios MRCA are not expected to lead to significant attenuation of the PET emission signals.

Web [BibTex]

2011

Web [BibTex]


no image
Cooperative Cuts: a new use of submodularity in image segmentation

Jegelka, S.

Second I.S.T. Austria Symposium on Computer Vision and Machine Learning, October 2011 (talk)

Web [BibTex]

Web [BibTex]


no image
Effect of MR Contrast Agents on Quantitative Accuracy of PET in Combined Whole-Body PET/MR Imaging

Lois, C., Bezrukov, I., Schmidt, H., Schwenzer, N., Werner, M., Pichler, B., Kupferschläger, J., Beyer, T.

2011(MIC3-3), 2011 IEEE Nuclear Science Symposium, Medical Imaging Conference (NSS-MIC), October 2011 (talk)

Abstract
Combined whole-body PET/MR systems are being tested in clinical practice today. Integrated imaging protocols entail the use of MR contrast agents (MRCA) that could bias PET attenuation correction. In this work, we assess the effect of MRCA in PET/MR imaging. We analyze the effect of oral and intravenous MRCA on PET activity after attenuation correction. We conclude that in clinical scenarios, MRCA are not expected to lead to significant attenuation of PET signals, and that attenuation maps are not biased after the ingestion of adequate oral contrasts.

Web [BibTex]

Web [BibTex]


no image
First Results on Patients and Phantoms of a Fully Integrated Clinical Whole-Body PET/MRI

Schmidt, H., Schwenzer, N., Bezrukov, I., Kolb, A., Mantlik, F., Kupferschläger, J., Lois, C., Sauter, A., Brendle, C., Pfannenberg, C., Pichler, B.

2011(J2-8), 2011 IEEE Nuclear Science Symposium, Medical Imaging Conference (NSS-MIC), October 2011 (talk)

Abstract
First clinical fully integrated whole-body PET/MR scanners are just entering the field. Here, we present studies toward quantification accuracy and variation within the PET field of view of small lesions from our BrainPET/MRI, a dedicated clinical brain scanner which was installed three years ago in Tbingen. Also, we present first results for patient and phantom scans of a fully integral whole-body PET/MRI, which was installed two months ago at our department. The quantification accuracy and homogeneity of the BrainPET-Insert (Siemens Medical Solutions, Germany) installed inside the magnet bore of a clinical 3T MRI scanner (Magnetom TIM Trio, Siemens Medical Solutions, Germany) was evaluated by using eight hollow spheres with inner diameters from 3.95 to 7.86 mm placed at different positions inside a homogeneous cylinder phantom with an 9:1 and 6:1 sphere to background ratio. The quantification accuracy for small lesions at different positions in the PET FoV shows a standard deviation of up to 11% and is acceptable for quantitative brain studies where the homogeneity of quantification on the entire FoV is essental. Image quality and resolution of the new Siemens whole-body PET/MR system (Biograph mMR, Siemens Medical Solutions, Germany) was evaluated according to the NEMA NU2 2007 protocol using a body phantom containing six spheres with inner diameter from 10 to 37 mm at sphere to background ratios of 8:1 and 4:1 and the F-18 point sources located at different positions inside the PET FoV, respectively. The evaluation of the whole-body PET/MR system reveals a good PET image quality and resolution comparable to state-of-the-art clinical PET/CT scanners. First images of patient studies carried out at the whole-body PET/MR are presented highlighting the potency of combined PET/MR imaging.

Web [BibTex]

Web [BibTex]


no image
Effect of MR contrast agents on quantitative accuracy of PET in combined whole-body PET/MR imaging

Lois, C., Kupferschläger, J., Bezrukov, I., Schmidt, H., Werner, M., Mannheim, J., Pichler, B., Schwenzer, N., Beyer, T.

(OP314), Annual Congress of the European Association of Nuclear Medicine (EANM), October 2011 (talk)

Abstract
PURPOSE:Combined PET/MR imaging entails the use of MR contrast agents (MRCA) as part of integrated protocols. MRCA are made up of iron oxide and Gd-chelates for oral and intravenous (iv) application, respectively. We assess additional attenuation of the PET emission signals in the presence of oral and iv MRCA.MATERIALS AND METHODS:Phantom scans were performed on a clinical PET/CT (Biograph HiRez16, Siemens) and an integrated whole-body PET/MR (Biograph mMR, Siemens). Two common MRCA were evaluated: Lumirem (oral) and Gadovist (iv).Reference PET attenuation values were determined on a dedicated small-animal PET (Inveon, Siemens) using equivalent standard PET transmission source imaging (TX). Seven syringes of 5mL were filled with (a) Water, (b) Lumirem_100 (100% concentration), (c) Gadovist_100 (100%), (d) Gadovist_18 (18%), (e) Gadovist_02 (0.2%), (f) Imeron-400 CT iv-contrast (100%) and (g) Imeron-400 (2.4%). The same set of syringes was scanned on CT (Sensation16, Siemens) at 120kVp and 160mAs.The effect of MRCA on the attenuation of PET emission data was evaluated using a 20cm cylinder filled uniformly with [18F]-FDG (FDG) in water (BGD). Three 4.5cm diameter cylinders were inserted into the phantom: (C1) Teflon, (C2) Water+FDG (2:1) and (C3) Lumirem_100+FDG (2:1). Two 50mL syringes filled with Gadovist_02+FDG (Sy1) and water+FDG (Sy2) were attached to the sides of (C1) to mimick the effects of iv-contrast in vessels near bone. Syringe-to-background activity ratio was 4-to-1.PET emission data were acquired for 10min each using the PET/CT and the PET/MR. Images were reconstructed using CT- and MR-based attenuation correction (AC). Since Teflon is not correctly identified on MR, PET(/MR) data were reconstructed using MR-AC and CT-AC.RESULTS:Mean linear PET attenuation (cm-1) on TX was (a) 0.098, (b) 0.098, (c) 0.300, (d) 0.134, (e) 0.095, (f) 0.397 and (g) 0.105. Corresponding CT attenuation (HU) was: (a) 5, (b) 14, (c) 3070, (d) 1040, (e) 13, (f) 3070 and (g) 347.Lumirem had little effect on PET attenuation with (C3) being 13%, 10% and 11% higher than (C2) on PET/CT, PET/MR with MR-AC, and PET/MR with CT-AC, respectively. Gadovist_02 had even smaller effects with (Sy1) being 2.5% lower, 1.2% higher, and 3.5% lower than (Sy2) on PET/CT, PET/MR with MR-AC and PET/MR with CT-AC, respectively.CONCLUSION:MRCA in high and clinically relevant concentrations have attenuation values similar to that of CT contrast and water, respectively. In clinical PET/MR scenarios MRCA are not expected to lead to significant attenuation of the PET emission signals.

Web [BibTex]

Web [BibTex]


no image
Multi-parametric Tumor Characterization and Therapy Monitoring using Simultaneous PET/MRI: initial results for Lung Cancer and GvHD

Sauter, A., Schmidt, H., Gueckel, B., Brendle, C., Bezrukov, I., Mantlik, F., Kolb, A., Mueller, M., Reimold, M., Federmann, B., Hetzel, J., Claussen, C., Pfannenberg, C., Horger, M., Pichler, B., Schwenzer, N.

(T110), 2011 World Molecular Imaging Congress (WMIC), September 2011 (talk)

Abstract
Hybrid imaging modalities such as [18F]FDG-PET/CT are superior in staging of e.g. lung cancer disease compared with stand-alone modalities. Clinical PET/MRI systems are about to enter the field of hybrid imaging and offer potential advantages. One added value could be a deeper insight into the tumor metabolism and tumorigenesis due to the combination of PET and dedicated MR methods such as MRS and DWI. Additionally, therapy monitoring of diffucult to diagnose disease such as chronic sclerodermic GvHD (csGvHD) can potentially be improved by this combination. We have applied PET/MRI in 3 patients with lung cancer and 4 patients with csGvHD before and during therapy. All 3 patients had lung cancer confirmed by histology (2 adenocarcinoma, 1 carcinoid). First, a [18F]FDG-PET/CT was performed with the following parameters: injected dose 351.7±25.1 MBq, uptake time 59.0±2.6 min, 3 min/bed. Subsequently, patients were brought to the PET/MRI imaging facility. The whole-body PET/MRI Biograph mMR system comprises 56 detector cassettes with a 59.4 cm transaxial and 25.8 cm axial FoV. The MRI is a modified Verio system with a magnet bore of 60 cm. The following parameters for PET acquisition were applied: uptake time 121.3±2.3 min, 3 bed positions, 6 min/bed. T1w, T2w, and DWI MR images were recorded simultaneously for each bed. Acquired PET data were reconstructed with an iterative 3D OSEM algorithm using 3 iterations and 21 subsets, Gaussian filter of 3 mm. The 4 patients with GvHD were brought to the brainPET/MRI imaging facility 2:10h-2:28h after tracer injection. A 9 min brainPET-acquisition with simultaneous MRI of the lower extremities was accomplished. MRI examination included T1-weighted (pre and post gadolinium) and T2-weighted sequences. Attenuation correction was calculated based on manual bone segmentation and thresholds for soft tissue, fat and air. Soleus muscle (m), crural fascia (f1) and posterior crural intermuscular septum fascia (f2) were surrounded with ROIs based on the pre-treatment T1-weighted images and coregistered using IRW (Siemens). Fascia-to-muscle ratios for PET (f/m), T1 contrast uptake (T1_post-contrast_f-pre-contrast_f/post-contrast_m-pre-contrast_m) and T2 (T2_f/m) were calculated. Both patients with adenocarcinoma show a lower ADC value compared with the carcinoid patient suggesting a higher cellularity. This is also reflected in FDG-PET with higher SUV values. Our initial results reveal that PET/MRI can provide complementary information for a profound tumor characterization and therapy monitoring. The high soft tissue contrast provided by MRI is valuable for the assessment of the fascial inflammation. While in the first patient FDG and contrast uptake as well as edema, represented by T2 signals, decreased with ongoing therapy, all parameters remained comparatively stable in the second patient. Contrary to expectations, an increase in FDG uptake of patient 3 and 4 was accompanied by an increase of the T2 signals, but a decrease in contrast uptake. These initial results suggest that PET/MRI provides complementary information of the complex disease mechanisms in fibrosing disorders.

Web [BibTex]

Web [BibTex]


no image
Statistical Image Analysis and Percolation Theory

Langovoy, M., Habeck, M., Schölkopf, B.

2011 Joint Statistical Meetings (JSM), August 2011 (talk)

Abstract
We develop a novel method for detection of signals and reconstruction of images in the presence of random noise. The method uses results from percolation theory. We specifically address the problem of detection of multiple objects of unknown shapes in the case of nonparametric noise. The noise density is unknown and can be heavy-tailed. The objects of interest have unknown varying intensities. No boundary shape constraints are imposed on the objects, only a set of weak bulk conditions is required. We view the object detection problem as hypothesis testing for discrete statistical inverse problems. We present an algorithm that allows to detect greyscale objects of various shapes in noisy images. We prove results on consistency and algorithmic complexity of our procedures. Applications to cryo-electron microscopy are presented.

Web [BibTex]

Web [BibTex]


no image
JMLR Workshop and Conference Proceedings Volume 19: COLT 2011

Kakade, S., von Luxburg, U.

pages: 834, MIT Press, Cambridge, MA, USA, 24th Annual Conference on Learning Theory , June 2011 (proceedings)

Web [BibTex]

Web [BibTex]


no image
PAC-Bayesian Analysis of Martingales and Multiarmed Bandits

Seldin, Y., Laviolette, F., Shawe-Taylor, J., Peters, J., Auer, P.

Max Planck Institute for Biological Cybernetics, Tübingen, Germany, May 2011 (techreport)

Abstract
We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This lemma provides an alternative tool to Hoeffding-Azuma inequality to bound concentration of martingale values. Our second approach is based on integration of Hoeffding-Azuma inequality with PAC-Bayesian analysis. We also introduce a way to apply PAC-Bayesian analysis in situation of limited feedback. We combine the new tools to derive PAC-Bayesian generalization and regret bounds for the multiarmed bandit problem. Although our regret bound is not yet as tight as state-of-the-art regret bounds based on other well-established techniques, our results significantly expand the range of potential applications of PAC-Bayesian analysis and introduce a new analysis tool to reinforcement learning and many other fields, where martingales and limited feedback are encountered.

PDF Web [BibTex]

PDF Web [BibTex]


no image
Non-stationary Correction of Optical Aberrations

Schuler, C., Hirsch, M., Harmeling, S., Schölkopf, B.

(1), Max Planck Institute for Intelligent Systems, Tübingen, Germany, May 2011 (techreport)

Abstract
Taking a sharp photo at several megapixel resolution traditionally relies on high grade lenses. In this paper, we present an approach to alleviate image degradations caused by imperfect optics. We rely on a calibration step to encode the optical aberrations in a space-variant point spread function and obtain a corrected image by non-stationary deconvolution. By including the Bayer array in our image formation model, we can perform demosaicing as part of the deconvolution.

PDF [BibTex]

PDF [BibTex]


no image
Cooperative Cuts

Jegelka, S.

COSA Workshop: Combinatorial Optimization, Statistics, and Applications, March 2011 (talk)

Abstract
Combinatorial problems with submodular cost functions have recently drawn interest. In a standard combinatorial problem, the sum-of-weights cost is replaced by a submodular set function. The result is a powerful model that is though very hard. In this talk, I will introduce cooperative cuts, minimum cuts with submodular edge weights. I will outline methods to approximately solve this problem, and show an application in computer vision. If time permits, the talk will also sketch regret-minimizing online algorithms for submodular-cost combinatorial problems. This is joint work with Jeff Bilmes (University of Washington).

Web [BibTex]

Web [BibTex]


no image
Multiple Kernel Learning: A Unifying Probabilistic Viewpoint

Nickisch, H., Seeger, M.

Max Planck Institute for Biological Cybernetics, March 2011 (techreport)

Abstract
We present a probabilistic viewpoint to multiple kernel learning unifying well-known regularised risk approaches and recent advances in approximate Bayesian inference relaxations. The framework proposes a general objective function suitable for regression, robust regression and classification that is lower bound of the marginal likelihood and contains many regularised risk approaches as special cases. Furthermore, we derive an efficient and provably convergent optimisation algorithm.

Web [BibTex]

Web [BibTex]


no image
Multiple testing, uncertainty and realistic pictures

Langovoy, M., Wittich, O.

(2011-004), EURANDOM, Technische Universiteit Eindhoven, January 2011 (techreport)

Abstract
We study statistical detection of grayscale objects in noisy images. The object of interest is of unknown shape and has an unknown intensity, that can be varying over the object and can be negative. No boundary shape constraints are imposed on the object, only a weak bulk condition for the object's interior is required. We propose an algorithm that can be used to detect grayscale objects of unknown shapes in the presence of nonparametric noise of unknown level. Our algorithm is based on a nonparametric multiple testing procedure. We establish the limit of applicability of our method via an explicit, closed-form, non-asymptotic and nonparametric consistency bound. This bound is valid for a wide class of nonparametric noise distributions. We achieve this by proving an uncertainty principle for percolation on nite lattices.

PDF [BibTex]

PDF [BibTex]


no image
Nonconvex proximal splitting: batch and incremental algorithms

Sra, S.

(2), Max Planck Institute for Intelligent Systems, Tübingen, Germany, 2011 (techreport)

Abstract
Within the unmanageably large class of nonconvex optimization, we consider the rich subclass of nonsmooth problems having composite objectives (this includes the extensively studied convex, composite objective problems as a special case). For this subclass, we introduce a powerful, new framework that permits asymptotically non-vanishing perturbations. In particular, we develop perturbation-based batch and incremental (online like) nonconvex proximal splitting algorithms. To our knowledge, this is the rst time that such perturbation-based nonconvex splitting algorithms are being proposed and analyzed. While the main contribution of the paper is the theoretical framework, we complement our results by presenting some empirical results on matrix factorization.

PDF [BibTex]

PDF [BibTex]

2003


no image
Support Vector Channel Selection in BCI

Lal, T., Schröder, M., Hinterberger, T., Weston, J., Bogdan, M., Birbaumer, N., Schölkopf, B.

(120), Max Planck Institute for Biological Cybernetics, Tuebingen, Germany, December 2003 (techreport)

Abstract
Designing a Brain Computer Interface (BCI) system one can choose from a variety of features that may be useful for classifying brain activity during a mental task. For the special case of classifying EEG signals we propose the usage of the state of the art feature selection algorithms Recursive Feature Elimination [3] and Zero-Norm Optimization [13] which are based on the training of Support Vector Machines (SVM) [11]. These algorithms can provide more accurate solutions than standard filter methods for feature selection [14]. We adapt the methods for the purpose of selecting EEG channels. For a motor imagery paradigm we show that the number of used channels can be reduced significantly without increasing the classification error. The resulting best channels agree well with the expected underlying cortical activity patterns during the mental tasks. Furthermore we show how time dependent task specific information can be visualized.

PDF Web [BibTex]

2003

PDF Web [BibTex]


no image
Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop (COLT/Kernel 2003), LNCS Vol. 2777

Schölkopf, B., Warmuth, M.

Proceedings of the 16th Annual Conference on Learning Theory and 7th Kernel Workshop (COLT/Kernel 2003), COLT/Kernel 2003, pages: 746, Springer, Berlin, Germany, 16th Annual Conference on Learning Theory and 7th Kernel Workshop, November 2003, Lecture Notes in Computer Science ; 2777 (proceedings)

DOI [BibTex]

DOI [BibTex]


no image
Image Reconstruction by Linear Programming

Tsuda, K., Rätsch, G.

(118), Max Planck Institute for Biological Cybernetics, Tübingen, Germany, October 2003 (techreport)

PDF [BibTex]

PDF [BibTex]


no image
Statistical Learning Theory

Bousquet, O.

Machine Learning Summer School, August 2003 (talk)

PDF [BibTex]

PDF [BibTex]


no image
Remarks on Statistical Learning Theory

Bousquet, O.

Machine Learning Summer School, August 2003 (talk)

PDF [BibTex]

PDF [BibTex]


no image
Ranking on Data Manifolds

Zhou, D., Weston, J., Gretton, A., Bousquet, O., Schölkopf, B.

(113), Max Planck Institute for Biological Cybernetics, 72076 Tuebingen, Germany, June 2003 (techreport)

Abstract
The Google search engine has had a huge success with its PageRank web page ranking algorithm, which exploits global, rather than local, hyperlink structure of the World Wide Web using random walk. This algorithm can only be used for graph data, however. Here we propose a simple universal ranking algorithm for vectorial data, based on the exploration of the intrinsic global geometric structure revealed by a huge amount of data. Experimental results from image and text to bioinformatics illustrates the validity of our algorithm.

PDF [BibTex]

PDF [BibTex]


no image
Kernel Hebbian Algorithm for Iterative Kernel Principal Component Analysis

Kim, K., Franz, M., Schölkopf, B.

(109), MPI f. biologische Kybernetik, Tuebingen, June 2003 (techreport)

Abstract
A new method for performing a kernel principal component analysis is proposed. By kernelizing the generalized Hebbian algorithm, one can iteratively estimate the principal components in a reproducing kernel Hilbert space with only linear order memory complexity. The derivation of the method, a convergence proof, and preliminary applications in image hyperresolution are presented. In addition, we discuss the extension of the method to the online learning of kernel principal components.

PDF [BibTex]

PDF [BibTex]


no image
Learning with Local and Global Consistency

Zhou, D., Bousquet, O., Lal, T., Weston, J., Schölkopf, B.

(112), Max Planck Institute for Biological Cybernetics, Tuebingen, Germany, June 2003 (techreport)

Abstract
We consider the learning problem in the transductive setting. Given a set of points of which only some are labeled, the goal is to predict the label of the unlabeled points. A principled clue to solve such a learning problem is the consistency assumption that a classifying function should be sufficiently smooth with respect to the structure revealed by these known labeled and unlabeled points. We present a simple algorithm to obtain such a smooth solution. Our method yields encouraging experimental results on a number of classification problems and demonstrates effective use of unlabeled data.

[BibTex]

[BibTex]


no image
Implicit Wiener Series

Franz, M., Schölkopf, B.

(114), Max Planck Institute for Biological Cybernetics, June 2003 (techreport)

Abstract
The Wiener series is one of the standard methods to systematically characterize the nonlinearity of a neural system. The classical estimation method of the expansion coefficients via cross-correlation suffers from severe problems that prevent its application to high-dimensional and strongly nonlinear systems. We propose a new estimation method based on regression in a reproducing kernel Hilbert space that overcomes these problems. Numerical experiments show performance advantages in terms of convergence, interpretability and system size that can be handled.

PDF [BibTex]

PDF [BibTex]


no image
Machine Learning approaches to protein ranking: discriminative, semi-supervised, scalable algorithms

Weston, J., Leslie, C., Elisseeff, A., Noble, W.

(111), Max Planck Institute for Biological Cybernetics, Tübingen, Germany, June 2003 (techreport)

Abstract
A key tool in protein function discovery is the ability to rank databases of proteins given a query amino acid sequence. The most successful method so far is a web-based tool called PSI-BLAST which uses heuristic alignment of a profile built using the large unlabeled database. It has been shown that such use of global information via an unlabeled data improves over a local measure derived from a basic pairwise alignment such as performed by PSI-BLAST's predecessor, BLAST. In this article we look at ways of leveraging techniques from the field of machine learning for the problem of ranking. We show how clustering and semi-supervised learning techniques, which aim to capture global structure in data, can significantly improve over PSI-BLAST.

PDF [BibTex]

PDF [BibTex]


no image
The Geometry Of Kernel Canonical Correlation Analysis

Kuss, M., Graepel, T.

(108), Max Planck Institute for Biological Cybernetics, Tübingen, Germany, May 2003 (techreport)

Abstract
Canonical correlation analysis (CCA) is a classical multivariate method concerned with describing linear dependencies between sets of variables. After a short exposition of the linear sample CCA problem and its analytical solution, the article proceeds with a detailed characterization of its geometry. Projection operators are used to illustrate the relations between canonical vectors and variates. The article then addresses the problem of CCA between spaces spanned by objects mapped into kernel feature spaces. An exact solution for this kernel canonical correlation (KCCA) problem is derived from a geometric point of view. It shows that the expansion coefficients of the canonical vectors in their respective feature space can be found by linear CCA in the basis induced by kernel principal component analysis. The effect of mappings into higher dimensional feature spaces is considered critically since it simplifies the CCA problem in general. Then two regularized variants of KCCA are discussed. Relations to other methods are illustrated, e.g., multicategory kernel Fisher discriminant analysis, kernel principal component regression and possible applications thereof in blind source separation.

PDF [BibTex]

PDF [BibTex]


no image
The Kernel Mutual Information

Gretton, A., Herbrich, R., Smola, A.

Max Planck Institute for Biological Cybernetics, April 2003 (techreport)

Abstract
We introduce two new functions, the kernel covariance (KC) and the kernel mutual information (KMI), to measure the degree of independence of several continuous random variables. The former is guaranteed to be zero if and only if the random variables are pairwise independent; the latter shares this property, and is in addition an approximate upper bound on the mutual information, as measured near independence, and is based on a kernel density estimate. We show that Bach and Jordan‘s kernel generalised variance (KGV) is also an upper bound on the same kernel density estimate, but is looser. Finally, we suggest that the addition of a regularising term in the KGV causes it to approach the KMI, which motivates the introduction of this regularisation. The performance of the KC and KMI is verified in the context of instantaneous independent component analysis (ICA), by recovering both artificial and real (musical) signals following linear mixing.

PostScript [BibTex]

PostScript [BibTex]


no image
Rademacher and Gaussian averages in Learning Theory

Bousquet, O.

Universite de Marne-la-Vallee, March 2003 (talk)

PDF [BibTex]

PDF [BibTex]


no image
Introduction: Robots with Cognition?

Franz, MO.

6, pages: 38, (Editors: H.H. Bülthoff, K.R. Gegenfurtner, H.A. Mallot, R. Ulrich, F.A. Wichmann), 6. T{\"u}binger Wahrnehmungskonferenz (TWK), February 2003 (talk)

Abstract
Using robots as models of cognitive behaviour has a long tradition in robotics. Parallel to the historical development in cognitive science, one observes two major, subsequent waves in cognitive robotics. The first is based on ideas of classical, cognitivist Artificial Intelligence (AI). According to the AI view of cognition as rule-based symbol manipulation, these robots typically try to extract symbolic descriptions of the environment from their sensors that are used to update a common, global world representation from which, in turn, the next action of the robot is derived. The AI approach has been successful in strongly restricted and controlled environments requiring well-defined tasks, e.g. in industrial assembly lines. AI-based robots mostly failed, however, in the unpredictable and unstructured environments that have to be faced by mobile robots. This has provoked the second wave in cognitive robotics which tries to achieve cognitive behaviour as an emergent property from the interaction of simple, low-level modules. Robots of the second wave are called animats as their architecture is designed to closely model aspects of real animals. Using only simple reactive mechanisms and Hebbian-type or evolutionary learning, the resulting animats often outperformed the highly complex AI-based robots in tasks such as obstacle avoidance, corridor following etc. While successful in generating robust, insect-like behaviour, typical animats are limited to stereotyped, fixed stimulus-response associations. If one adopts the view that cognition requires a flexible, goal-dependent choice of behaviours and planning capabilities (H.A. Mallot, Kognitionswissenschaft, 1999, 40-48) then it appears that cognitive behaviour cannot emerge from a collection of purely reactive modules. It rather requires environmentally decoupled structures that work without directly engaging the actions that it is concerned with. This poses the current challenge to cognitive robotics: How can we build cognitive robots that show the robustness and the learning capabilities of animats without falling back into the representational paradigm of AI? The speakers of the symposium present their approaches to this question in the context of robot navigation and sensorimotor learning. In the first talk, Prof. Helge Ritter introduces a robot system for imitation learning capable of exploring various alternatives in simulation before actually performing a task. The second speaker, Angelo Arleo, develops a model of spatial memory in rat navigation based on his electrophysiological experiments. He validates the model on a mobile robot which, in some navigation tasks, shows a performance comparable to that of the real rat. A similar model of spatial memory is used to investigate the mechanisms of territory formation in a series of robot experiments presented by Prof. Hanspeter Mallot. In the last talk, we return to the domain of sensorimotor learning where Ralf M{\"o}ller introduces his approach to generate anticipatory behaviour by learning forward models of sensorimotor relationships.

Web [BibTex]

Web [BibTex]


no image
A Note on Parameter Tuning for On-Line Shifting Algorithms

Bousquet, O.

Max Planck Institute for Biological Cybernetics, Tübingen, Germany, 2003 (techreport)

Abstract
In this short note, building on ideas of M. Herbster [2] we propose a method for automatically tuning the parameter of the FIXED-SHARE algorithm proposed by Herbster and Warmuth [3] in the context of on-line learning with shifting experts. We show that this can be done with a memory requirement of $O(nT)$ and that the additional loss incurred by the tuning is the same as the loss incurred for estimating the parameter of a Bernoulli random variable.

PDF PostScript [BibTex]

PDF PostScript [BibTex]


no image
Interactive Images

Toyama, K., Schölkopf, B.

(MSR-TR-2003-64), Microsoft Research, Cambridge, UK, 2003 (techreport)

Abstract
Interactive Images are a natural extension of three recent developments: digital photography, interactive web pages, and browsable video. An interactive image is a multi-dimensional image, displayed two dimensions at a time (like a standard digital image), but with which a user can interact to browse through the other dimensions. One might consider a standard video sequence viewed with a video player as a simple interactive image with time as the third dimension. Interactive images are a generalization of this idea, in which the third (and greater) dimensions may be focus, exposure, white balance, saturation, and other parameters. Interaction is handled via a variety of modes including those we call ordinal, pixel-indexed, cumulative, and comprehensive. Through exploration of three novel forms of interactive images based on color, exposure, and focus, we will demonstrate the compelling nature of interactive images.

Web [BibTex]

Web [BibTex]

2002


no image
Kernel Dependency Estimation

Weston, J., Chapelle, O., Elisseeff, A., Schölkopf, B., Vapnik, V.

(98), Max Planck Institute for Biological Cybernetics, August 2002 (techreport)

Abstract
We consider the learning problem of finding a dependency between a general class of objects and another, possibly different, general class of objects. The objects can be for example: vectors, images, strings, trees or graphs. Such a task is made possible by employing similarity measures in both input and output spaces using kernel functions, thus embedding the objects into vector spaces. Output kernels also make it possible to encode prior information and/or invariances in the loss function in an elegant way. We experimentally validate our approach on several tasks: mapping strings to strings, pattern recognition, and reconstruction from partial images.

PDF [BibTex]

2002

PDF [BibTex]


no image
A compression approach to support vector model selection

von Luxburg, U., Bousquet, O., Schölkopf, B.

(101), Max Planck Institute for Biological Cybernetics, 2002, see more detailed JMLR version (techreport)

Abstract
In this paper we investigate connections between statistical learning theory and data compression on the basis of support vector machine (SVM) model selection. Inspired by several generalization bounds we construct ``compression coefficients'' for SVMs, which measure the amount by which the training labels can be compressed by some classification hypothesis. The main idea is to relate the coding precision of this hypothesis to the width of the margin of the SVM. The compression coefficients connect well known quantities such as the radius-margin ratio R^2/rho^2, the eigenvalues of the kernel matrix and the number of support vectors. To test whether they are useful in practice we ran model selection experiments on several real world datasets. As a result we found that compression coefficients can fairly accurately predict the parameters for which the test error is minimized.

[BibTex]

[BibTex]