The doctoral dissertations of the former Helsinki University of Technology (TKK) and Aalto University Schools of Technology (CHEM, ELEC, ENG, SCI) published in electronic format are available in the electronic publications archive of Aalto University - Aaltodoc.
Aalto

Algorithms for Approximate Bayesian Inference with Applications to Astronomical Data Analysis

Markus Harva

Dissertation for the degree of Doctor of Science in Technology to be presented with due permission of the Faculty of Information and Natural Sciences for public examination and debate in Auditorium T2 at Helsinki University of Technology (Espoo, Finland) on the 9th of May, 2008, at 12 noon.

Overview in PDF format (ISBN 978-951-22-9348-3)   [2836 KB]
Dissertation is also available in print (ISBN 978-951-22-9347-6)

Abstract

Bayesian inference is a theoretically well-founded and conceptually simple approach to data analysis. The computations in practical problems are anything but simple though, and thus approximations are almost always a necessity. The topic of this thesis is approximate Bayesian inference and its applications in three intertwined problem domains.

Variational Bayesian learning is one type of approximate inference. Its main advantage is its computational efficiency compared to the much applied sampling based methods. Its main disadvantage, on the other hand, is the large amount of analytical work required to derive the necessary components for the algorithm. One part of this thesis reports on an effort to automate variational Bayesian learning of a certain class of models.

The second part of the thesis is concerned with heteroscedastic modelling which is synonymous to variance modelling. Heteroscedastic models are particularly suitable for the Bayesian treatment as many of the traditional estimation methods do not produce satisfactory results for them. In the thesis, variance models and algorithms for estimating them are studied in two different contexts: in source separation and in regression.

Astronomical applications constitute the third part of the thesis. Two problems are posed. One is concerned with the separation of stellar subpopulation spectra from observed galaxy spectra; the other is concerned with estimating the time-delays in gravitational lensing. Solutions to both of these problems are presented, which heavily rely on the machinery of approximate inference.

This thesis consists of an overview and of the following 7 publications:

  1. Tapani Raiko, Harri Valpola, Markus Harva, and Juha Karhunen. 2007. Building blocks for variational Bayesian learning of latent variable models. Journal of Machine Learning Research, volume 8, pages 155-201.
  2. Markus Harva, Tapani Raiko, Antti Honkela, Harri Valpola, and Juha Karhunen. 2005. Bayes Blocks: An implementation of the variational Bayesian building blocks framework. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI 2005), Edinburgh, Scotland, 26-29 July 2005, pages 259-266.
  3. Harri Valpola, Markus Harva, and Juha Karhunen. 2004. Hierarchical models of variance sources. Signal Processing, volume 84, number 2, pages 267-282.
  4. Markus Harva. 2007. A variational EM approach to predictive uncertainty. Neural Networks, volume 20, number 4, pages 550-558.
  5. Markus Harva and Ata Kabán. 2007. Variational learning for rectified factor analysis. Signal Processing, volume 87, number 3, pages 509-527.
  6. Louisa A. Nolan, Markus O. Harva, Ata Kabán, and Somak Raychaudhury. 2006. A data-driven Bayesian approach for finding young stellar populations in early-type galaxies from their ultraviolet–optical spectra. Monthly Notices of the Royal Astronomical Society, volume 366, number 1, pages 321-338.
  7. Markus Harva and Somak Raychaudhury. 2008. Bayesian estimation of time delays between unevenly sampled signals. Neurocomputing, to appear.

Keywords: machine learning, data analysis, Bayesian inference, variational methods, blind source separation, nonnegative factor analysis, heteroscedasticity, predictive uncertainty, delay estimation, elliptical galaxies, gravitational lenses

This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.

© 2008 Helsinki University of Technology

Last update 2011-05-26