We develop new methods for probabilistic modeling, Bayesian inference and machine learning. Our current focuses are:
We develop and improve a principled Bayesian data-analysis workflow. Our core research is on designing computational diagnostics, devising methods and putting together tools that assist domain-expert data analysts in an agile and iterative workflow. This is done by building modeling tools and methods independently of the actual implementation in domain-specific fields. This enables the research to cover a wide range of models, such as Bayesian hierarchical models and Gaussian processes. The research is disseminated as free and modular open-source software, including software for the most popular probabilistic programming framework Stan. Some of the projects we contribute towards include projpred (core contributions), loo (core contributions), ArviZ, and viabel.
Approximate Bayesian computation (ABC) and other likelihood-free methods have gained popularity across scientific fields, as they enable statistical inference with complex models that only exist as computer simulators. We focus on improving sample-efficiency, making likelihood-free inference (LFI) possible even on difficult problems where each single simulator run is computationally costly. In ongoing research, we harness this property to combine LFI and dynamical models. Important drivers for the theoretical work come from challenges we encounter in applying the methods in multiple fields, including cognitive science, population genetics, neuroscience, epidemiology, ecology and business-to-business marketing.
We develop the Engine for Likelihood-free Inference (ELFI, http://elfi.ai), an open-source Python software library that is designed to be modular and extensible. It provides a wide range of ABC algorithms, supports convenient syntax for defining the inference task and allows working with complex model hierarchies.
We augment traditional deep learning methods with uncertainty from Bayesian principles, using Gaussian process and Bayesian neural network families, and extending to continuous-time dynamics. For Gaussian processes we develop novel deep function spaces and flexible covariance structures under model marginalisation. We develop scalable Bayesian neural networks by studying functionally diverse posteriors. We propose novel approaches to deep learning as continuous-time flows, replacing the conventional sequence of discrete mappings in deep neural networks.
Trung Trinh, Samuel Kaski, Markus Heinonen. Scalable Bayesian neural networks by layer-wise input augmentation, (2020)
Simone Rossi, Markus Heinonen, Edwin Bonilla, Zheyang Shen, Maurizio Filippone. Sparse Gaussian Processes Revisited: Bayesian Approaches to Inducing-Variable Approximations (2020)
Zheyang Shen, Markus Heinonen, Samuel Kaski. Learning spectrograms with convolutional spectral kernels. AISTATS 2020
Pashupati Hegde, Markus Heinonen, Harri Lähdesmäki, Samuel Kaski. Deep learning with differential Gaussian process flows. AISTATS 2019, notable paper award
Cagatay Yildiz, Markus Heinonen, Harri Lähdesmäki. ODE2VAE: Deep generative second order ODEs with Bayesian neural networks. NeurIPS 2019
Figure : Using differential privacy (depicted as a
privacy wall), allows us to learn a predictive model y from sensitive data set D.
Careless use of increasingly refined personal data raises concerns for privacy. Recent advances in privacy-preserving machine learning have made it possible to learn from data in both centralised and decentralised settings, without violating the anonymity of individuals. We combine differential privacy, a de facto standard for privacy-preserving data analysis, with Bayesian inference to develop powerful techniques for probabilistic modelling with rigorous privacy guarantees. To advance the theory we are actively studying ways to introduce differential privacy into established inference algorithms for probabilistic models and improve on existing ones. For sharing data we have recently introduced methods for generating synthetic twins of sensitive data using differentially-private inference for probabilistic models (Jälkö et al., 2019), released in package Twinify which is developing towards an easy-to-use programming framework.
Tejas Kulkarni, Joonas Jälkö, Samuel Kaski, Antti Koskela, Antti Honkela (2020). Differentially Private Bayesian Inference for Generalized Linear Models [Preprint]
Mikko Heikkilä, Joonas Jälkö, Onur Dikmen, Antti Honkela (2019). Differentially private Markov chain Monte Carlo. Advances in Neural Information Processing Systems 32 [Paper]
Teppo Niinimäki, Mikko Heikkilä, Antti Honkela, Samuel Kaski (2019). Representation transfer for differentially private drug sensitivity prediction. Bioinformatics 35(14): i218-i224. [Paper]
Antti Honkela, Mrinal Das, Arttu Nieminen, Onur Dikmen, Samuel Kaski (2018). Efficient differentially private learning improves drug sensitivity prediction. Biology direct, 13(1):1-12. [Paper]
Most machine learning systems operate with us humans, to augment our skills and assist us in our tasks. In order for an AI assistant to help intelligent agents, in particular humans, it would need to understand the factors that affect their behaviour. These factors (goals, plans, biases,...) are latent and non-stationary, and it is a difficult challenge both to infer them and to use them in the best possible way. We develop the probabilistic interactive user models and inference techniques needed to understand human agents and how to assist them more efficiently. This research contributes directly to the FCAI Interactive AI program.
Related keywords: active learning, experimental design, knowledge elicitation, multi-agent learning, machine teaching.
Mikkola, P., Todorović, M., Järvi, J., Rinke, P. & Kaski, S.. (2020). Projective Preferential Bayesian Optimization. Proceedings of the 37th International Conference on Machine Learning, in PMLR 119:6884-6892. [Link]
Afrabandpey, H., Peltola, T., Piironen, J., Vehtari, A., & Kaski, S. (2019). Making Bayesian Predictive Models Interpretable: A Decision Theoretic Approach. arXiv preprint arXiv:1910.09358. [Link]
Tomi Peltola, Mustafa Mert Çelikok, Pedram Daee, Samuel Kaski (2019). Machine Teaching of Active Sequential Learners. 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada [Link]
Iiris Sundin, Peter Schulam, Eero Siivola, Aki Vehtari, Suchi Saria, Samuel Kaski (2019). Active Learning for Decision-Making from Imbalanced Observational Data. International Conference on Machine Learning, Proceedings of Machine Learning Research, 97. [Link]
Pedram Daee, Tomi Peltola, Aki Vehtari, Samuel Kaski (2018). User Modelling for Avoiding Overfitting in Interactive Knowledge Elicitation for Prediction. International Conference on Intelligent User Interfaces, IUI 2018 - Proceedings of the 23rd International Conference on Intelligent User Interfaces, p. 305-310. [Link]
We develop methods for pushing the boundaries of machine learning in many application fields, using e.g. interactive AI, active learning, exploratory data analysis, deep learning, and combining multiple data sources. In healthcare and medicine, machine learning can augment current medical procedures, and help to solve complex prediction problems, such as modeling treatment response, or predicting properties (such as toxicity) of chemical compounds from their molecular structure. In information retrieval and recommender systems, we aim at modeling users’ behaviors and decision-making in dynamic environments to infer their intents.
For example, our human-in-the-loop machine learning methods bring in domain experts’ knowledge to the models to improve individualized predictions of drug responses [Sundin et al., 2019, above], or to predict breaks in large-scale cable TV networks more robust. Recent works in other application domains include materials science [Mikkola et al., 2020, above], chemistry [Ghosh et al., 2019], neuroscience [Leppäaho et al., 2019], and genomics [Järvenpää et al., 2018].