On-going research

The focus of our current research is to improve sample efficiency of deep learning and deep reinforcement learning.

Some papers published on these topics:

• In the NIPS 2019 paper Regularizing Trajectory Optimization with Denoising Autoencoders (collaboration with Curious AI), we have proposed to regularize model-based planning with a denoising autoencoder. The proposed regularization resulted in fast initial learning in a set of motor-control tasks.


• In the ICML 2019 paper Manifold Mixup: Better Representations by Interpolating Hidden States (collaboration with MILA, Sharif University of Technology and FAIR), we propose Manifold Mixup, a simple regularizer that encourages neural networks to predict less confidently on interpolations of hidden representations.


• In the NIPS 2017 paper Recurrent Ladder Networks (collaboration with Curious AI), we have proposed an architecture for modeling time series and learning to perform iterative inference.


• In the NIPS 2016 paper Tagger: Deep Unsupervised Perceptual Grouping (collaboration with IDSIA and Curious AI), we have developed a method which can efficiently separate textured background from objects in digital images.


• In the NIPS 2015 paper Semi-Supervised Learning with Ladder Networks (collaboration with Curious AI), we introduced a novel variant of autoencoder neural network that uses lateral links between encoding and decoding paths to learn rich features of the data. The proposed Ladder network achieved world record results for some well-known benchmark tasks using semi-supervised learning with a small amount of known training data only.

Earlier research topics

• Deep learning: background
• Deep learning: theoretical results
• Deep learning: applications to image data
• Deep learning: applications to spech recognition
• Spatio-temporal Bayesian latent Gaussian models
• Bayesian modeling and variational learning
• Theoretical improvements in variational Bayes
• Building blocks for variational Bayesian learning
• Hierarchical modeling of variances
• Extensions of probabilistic PCA
• Gaussian process models for gene expression and regulation
• Nonlinear and non-negative blind source separation
• Dynamic modeling using nonlinear state-space models
• Relational models
• Time-series modelling in bioinformatics
• Applications in climate science
• Applications to astronomy
• Automated segmentation of brain MR images
• Oscillatory neural networks
• Other applications

You can find more information on the earlier research results of our group under its older name Bayesian learning of latent variable models: 2010-2011, 2008-2009, 2006-2007, 2004-2005, 2002-2003, and 2000-2001.

Some of our research results have been described under the activities of the Independent component analysis (ICA) group of HUT, which studies ICA, blind source separation (BSS), and their extensions. For more detailed information, see the research reports of our ICA group covering the years 2010-2011, 2008-2009, 2006-2007, 2004-2005, and 2002-2003, as well as theoretical ICA research in 2000-2001, and applications of ICA in 2000-2001.