The doctoral dissertations of the former Helsinki University of Technology (TKK) and Aalto University Schools of Technology (CHEM, ELEC, ENG, SCI) published in electronic format are available in the electronic publications archive of Aalto University - Aaltodoc.
Aalto

Advanced Source Separation Methods with Applications to Spatio-Temporal Datasets

Alexander Ilin

Dissertation for the degree of Doctor of Science in Technology to be presented with due permission of the Department of Computer Science and Engineering for public examination and debate in Auditorium T2 at Helsinki University of Technology (Espoo, Finland) on the 3rd of November, 2006, at 12 o'clock noon.

Overview in PDF format (ISBN 951-22-8425-1)   [5570 KB]
Dissertation is also available in print (ISBN 951-22-8424-3)

Abstract

Latent variable models are useful tools for statistical data analysis in many applications. Examples of popular models include factor analysis, state-space models and independent component analysis. These types of models can be used for solving the source separation problem in which the latent variables should have a meaningful interpretation and represent the actual sources generating data. Source separation methods is the main focus of this work.

Bayesian statistical theory provides a principled way to learn latent variable models and therefore to solve the source separation problem. The first part of this work studies variational Bayesian methods and their application to different latent variable models. The properties of variational Bayesian methods are investigated both theoretically and experimentally using linear source separation models. A new nonlinear factor analysis model which restricts the generative mapping to the practically important case of post-nonlinear mixtures is presented. The variational Bayesian approach to learning nonlinear state-space models is studied as well. This method is applied to the practical problem of detecting changes in the dynamics of complex nonlinear processes.

The main drawback of Bayesian methods is their high computational burden. This complicates their use for exploratory data analysis in which observed data regularities often suggest what kind of models could be tried. Therefore, the second part of this work proposes several faster source separation algorithms implemented in a common algorithmic framework. The proposed approaches separate the sources by analyzing their spectral contents, decoupling their dynamic models or by optimizing their prominent variance structures. These algorithms are applied to spatio-temporal datasets containing global climate measurements from a long period of time.

This thesis consists of an overview and of the following 9 publications:

  1. A. Ilin and H. Valpola. On the effect of the form of the posterior approximation in variational learning of ICA models. Neural Processing Letters, Vol. 22, No. 2, pages 183-204, October 2005. © 2005 Springer Science+Business Media. By permission.
  2. A. Ilin, H. Valpola, and E. Oja. Nonlinear dynamical factor analysis for state change detection. IEEE Transactions on Neural Networks, Vol. 15, No. 3, pages 559-575, May 2004. © 2004 IEEE. By permission.
  3. A. Ilin, S. Achard, and C. Jutten. Bayesian versus constrained structure approaches for source separation in post-nonlinear mixtures. In Proceedings of the International Joint Conference on Neural Networks (IJCNN 2004), pages 2181-2186, Budapest, Hungary, July 2004. © 2004 IEEE. By permission.
  4. A. Ilin and A. Honkela. Post-nonlinear independent component analysis by variational Bayesian learning. In Proceedings of the Fifth International Conference on Independent Component Analysis and Blind Signal Separation (ICA 2004), pages 766-773, Granada, Spain, September 2004. © 2004 Springer Science+Business Media. By permission.
  5. A. Ilin, H. Valpola, and E. Oja. Semiblind source separation of climate data detects El Niño as the component with the highest interannual variability. In Proceedings of the International Joint Conference on Neural Networks (IJCNN 2005), pages 1722-1727, Montréal, Québec, Canada, August 2005. © 2005 IEEE. By permission.
  6. A. Ilin and H. Valpola. Frequency-based separation of climate signals. In Proceedings of the 9th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD 2005), pages 519-526, Porto, Portugal, October 2005. © 2005 Springer Science+Business Media. By permission.
  7. A. Ilin, H. Valpola, and E. Oja. Exploratory analysis of climate data using source separation methods. Neural Networks, Vol. 19, No. 2, pages 155-167, March 2006. © 2006 Elsevier Science. By permission.
  8. A. Ilin. Independent dynamics subspace analysis. In Proceedings of the 14th European Symposium on Artificial Neural Networks (ESANN 2006), pages 345-350, Bruges, Belgium, April 2006. © 2006 d-side publications. By permission.
  9. A. Ilin, H. Valpola, and E. Oja. Extraction of components with structured variance. In Proceedings of the IEEE World Congress on Computational Intelligence (WCCI 2006), pages 10528-10535, Vancouver, BC, Canada, July 2006. © 2006 IEEE. By permission.

Keywords: Bayesian learning, blind source separation, global climate, denoising source separation, frequency-based separation, independent component analysis, independent subspace analysis, latent variable models, nonstationarity of variance, post-nonlinear mixing, unsupervised learning, variational methods

This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.

© 2006 Helsinki University of Technology

Last update 2011-05-26