Probabilistic deep learning methods have recently made great progress
for generative and discriminative modeling. I will give a brief overview of
recent developments and then present two contributions.
The first is on a generalization of generative adversarial networks (GAN),
extending their use considerably. GANs can be shown to approximately minimize
the Jensen-Shannon divergence between two distributions, the true sampling
distribution and the model distribution. We extend GANs to the class of
f-divergences which include popular divergences such as the Kullback-Leibler
divergence. This enables applications to variational inference and
likelihood-free maximum likelihood, as well as enables GAN models to become
basic building blocks in larger models.
The second contribution is to consider representation learning using
variational autoencoder models. To make learned representations of data
useful we need ground them in semantic concepts. We propose a generative
model that can decompose an observation into multiple separate latent factors,
each of which represents a separate concept. Such disentangled representation
is useful for recognition and for precise control in generative modeling. We
learn our representations using weak supervision in the form of groups of
observations where all samples within a group share the same value in a given
latent factor. To make such learning feasible we generalize recent methods
for amortized probabilistic inference to the dependent case.
Joint work with: Ryota Tomioka (MSR Cambridge), Botond Cseke (MSR
Cambridge), Diane Bouchacourt (Oxford)
Biography: Sebastian is an expert in Probabilistic Deep Learning and Structured Prediction. Among others, Sebastian's work on structured prediction models for pattern recognition tasks has shown that highly expressive probabilistic models can still be computationally tractable. More recently, his work on the theory and applications of deep generative models has advanced both the theory and practical aspects of the field. Sebastian is a former member of the Empirical Inference Department and is now a Principal Researcher at the Machine Intelligence and Perception Group at Microsoft Research Cambridge.