The doctoral dissertations of the former Helsinki University of Technology (TKK) and Aalto University Schools of Technology (CHEM, ELEC, ENG, SCI) published in electronic format are available in the electronic publications archive of Aalto University - Aaltodoc.
Aalto

Advances in Independent Component Analysis and Nonnegative Matrix Factorization

Zhijian Yuan

Dissertation for the degree of Doctor of Science in Technology to be presented with due permission of the Faculty of Information and Natural Sciences for public examination and debate in Auditorium T2 at Helsinki University of Technology (Espoo, Finland) on the 24th of April, 2009, at 12 noon.

Overview in PDF format (ISBN 978-951-22-9831-0)   [759 KB]
Dissertation is also available in print (ISBN 978-951-22-9830-3)

Abstract

A fundamental problem in machine learning research, as well as in many other disciplines, is finding a suitable representation of multivariate data, i.e. random vectors. For reasons of computational and conceptual simplicity, the representation is often sought as a linear transformation of the original data. In other words, each component of the representation is a linear combination of the original variables. Well-known linear transformation methods include principal component analysis (PCA), factor analysis, and projection pursuit. In this thesis, we consider two popular and widely used techniques: independent component analysis (ICA) and nonnegative matrix factorization (NMF).

ICA is a statistical method in which the goal is to find a linear representation of nongaussian data so that the components are statistically independent, or as independent as possible. Such a representation seems to capture the essential structure of the data in many applications, including feature extraction and signal separation. Starting from ICA, several methods of estimating the latent structure in different problem settings are derived and presented in this thesis. FastICA as one of most efficient and popular ICA algorithms has been reviewed and discussed. Its local and global convergence and statistical behavior have been further studied. A nonnegative FastICA algorithm is also given in this thesis.

Nonnegative matrix factorization is a recently developed technique for finding parts-based, linear representations of non-negative data. It is a method for dimensionality reduction that respects the nonnegativity of the input data while constructing a low-dimensional approximation. The non-negativity constraints make the representation purely additive (allowing no subtractions), in contrast to many other linear representations such as principal component analysis and independent component analysis. A literature survey of Nonnegative matrix factorization is given in this thesis, and a novel method called Projective Nonnegative matrix factorization (P-NMF) and its applications are provided.

This thesis consists of an overview and of the following 7 publications:

  1. Zhijian Yuan and Erkki Oja. 2004. A FastICA algorithm for non-negative independent component analysis. In: Carlos G. Puntonet and Alberto Prieto (editors). Proceedings of the 5th International Conference on Independent Component Analysis and Blind Signal Separation (ICA 2004). Granada, Spain. 22-24 September 2004. Springer. Lecture Notes in Computer Science, volume 3195, pages 1-8.
  2. Scott C. Douglas, Zhijian Yuan, and Erkki Oja. 2006. Average convergence behavior of the FastICA algorithm for blind source separation. In: Justinian Rosca, Deniz Erdogmus, José C. Príncipe, and Simon Haykin (editors). Proceedings of the 6th International Conference on Independent Component Analysis and Blind Signal Separation (ICA 2006). Charleston, SC, USA. 5-8 March 2006. Springer. Lecture Notes in Computer Science, volume 3889, pages 790-798.
  3. Erkki Oja and Zhijian Yuan. 2006. The FastICA algorithm revisited: convergence analysis. IEEE Transactions on Neural Networks, volume 17, number 6, pages 1370-1381.
  4. Zhijian Yuan and Erkki Oja. 2005. Projective nonnegative matrix factorization for image compression and feature extraction. In: Heikki Kalviainen, Jussi Parkkinen, and Arto Kaarna (editors). Proceedings of the 14th Scandinavian Conference on Image Analysis (SCIA 2005). Joensuu, Finland. 19-22 June 2005. Springer. Lecture Notes in Computer Science, volume 3540, pages 333-342.
  5. Zhirong Yang, Zhijian Yuan, and Jorma Laaksonen. 2007. Projective non-negative matrix factorization with applications to facial image processing. International Journal of Pattern Recognition and Artificial Intelligence, volume 21, number 8, pages 1353-1362.
  6. Zhijian Yuan and Erkki Oja. 2007. A family of modified projective nonnegative matrix factorization algorithms. In: Mohammed Al-Mualla (editor). Proceedings of the 9th International Symposium on Signal Processing and Its Applications (ISSPA 2007). Sharjah, United Arab Emirates. 12-15 February 2007, pages 1-4.
  7. Zhijian Yuan, Zhirong Yang, and Erkki Oja. Projective nonnegative matrix factorization: sparseness, orthogonality, and clustering. Submitted to a journal.

Keywords: independent component analysis, FastICA algorithms, nonnegative matrix factorization

This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.

© 2009 Helsinki University of Technology

Last update 2011-05-26