On-going research

Sample-efficient deep learning

Our current focus is on developing deep learning algorithms that can learn from little amount of data.

In our NIPS 2015 paper Semi-Supervised Learning with Ladder Networks, we intoduced a novel variant of autoencoder neural network that uses lateral links between encoding and decoding paths to learn rich features of the data. In contrast to traditional autoencoders, part of the information flows through the lateral links directly to the decoder relieving the pressure from the higher levels to represent all of information and allowing them to concentrate only on information that is not modeled by the lower layers. This is beneficial for two reasons. First, this kind of representation is more suitable to be used with supervised learning tasks where irrelevant information is often discarded on the way up. Second, this is more efficient use of neurons in the network allowing larger models and faster training times. The proposed Ladder network achieved world record results for some well-known benchmark tasks using semi-supervised learning with a small amount of known training data only.

Some of the features learned by the model from CIFAR10 image set grouped by similarity by the network.

In the NIPS 2016 paper Tagger: Deep Unsupervised Perceptual Grouping, we have developed together with researchers from IDSIA and Curious AI a method which can efficiently separate textured background from objects in digital images.

In the NIPS 2017 paper Recurrent Ladder Networks, we extended the Ladder architecture to modeling time series and learning iterative inference.

In the recent NIPS and ICLR workshop papers, we considered learning new classication tasks from few examples (few-shot learning) in the presence of unlabeled data.

Earlier research topics

You can find more information on the earlier research results of our group under its older name Bayesian learning of latent variable models: 2010-2011, 2008-2009, 2006-2007, 2004-2005, 2002-2003, and 2000-2001.

Some of our research results have been described under the activities of the Independent component analysis (ICA) group of HUT, which studies ICA, blind source separation (BSS), and their extensions. For more detailed information, see the research reports of our ICA group covering the years 2010-2011, 2008-2009, 2006-2007, 2004-2005, and 2002-2003, as well as theoretical ICA research in 2000-2001, and applications of ICA in 2000-2001.