Variational Bayesian Mixture of Gaussians
version 1.0, 2010-03-15

LEGALESE--------------------------------------------------------------

Copyright (C) 2008-2010 Mikael Kuusela.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License (included in file License.txt) for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.

DESCRIPTION-----------------------------------------------------------

This is a Matlab implementation of variational Bayesian mixture of Gaussians (VB-MoG) as described in [1,2]. The implementation was used to study different algorithms for learning the MoG model with a particular interest in the NCG algorithm. The scripts starting with the prefix "MoG_VB_" are the different algorithms for learning the model. The NCG algorithm is implemented in the script called "MoG_VB_m_gamma_nat_conj_grad_PR_Riemann.m".

USAGE-----------------------------------------------------------------

Please refer to the following four demonstration scripts:
MoG_comparison_experiment_1.m Comparison of gradient descent, CG, NG, NCG and L-BFGS using the cluster data
MoG_comparison_experiment_2.m Comparison of VB EM, pattern search, NG and NCG as a function of R using the cluster data
MoG_comparison_experiment_3.m Comparison of VB EM, pattern search and NCG as a function of N using the cluster data
MoG_comparison_experiment_4.m Comparison of VB EM, pattern search and NCG using the swan image data

The Netlab library (http://www.ncrg.aston.ac.uk/netlab/index.php) and the L-BFGS-B implementation by Peter Carbonetto (http://people.cs.ubc.ca/~pcarbo/lbfgsb-for-matlab.html) are required to be installed.

CONTACT INFORMATION---------------------------------------------------

The WWW home page of this package is:

  http://www.cis.hut.fi/projects/bayes/

The author can be reached by email:

  mjkuusel@cis.hut.fi

REFERENCES------------------------------------------------------------

[1] M. Kuusela, T. Raiko, A. Honkela, and J. Karhunen. A gradient-based algorithm competitive with variational Bayesian EM for mixture of Gaussians. In Proceedings of the International Joint Conference on Neural Networks, IJCNN 2009, pages 1688–1695, Atlanta, Georgia, June 2009.
[2] M. Kuusela. Algorithms for variational learning of mixture of Gaussians, 2009. Bachelor's thesis.
