The doctoral dissertations of the former Helsinki University of Technology (TKK) and Aalto University Schools of Technology (CHEM, ELEC, ENG, SCI) published in electronic format are available in the electronic publications archive of Aalto University - Aaltodoc.
Aalto

Probabilistic Models of Early Vision

Patrik O. Hoyer

Dissertation for the degree of Doctor of Science in Technology to be presented with due permission of the Department of Computer Science and Engineering for public examination and debate in Auditorium T2 at Helsinki University of Technology (Espoo, Finland) on the 15th of November, 2002, at 12 o'clock noon.

Overview in PDF format (ISBN 951-22-6086-7)   [1217 KB]
Dissertation is also available in print (ISBN 951-666-613-2)

Abstract

How do our brains transform patterns of light striking the retina into useful knowledge about objects and events of the external world? Thanks to intense research into the mechanisms of vision, much is now known about this process. However, we do not yet have anything close to a complete picture, and many questions remain unanswered. In addition to its clinical relevance and purely academic significance, research on vision is important because a thorough understanding of biological vision would probably help solve many major problems in computer vision.

A major framework for investigating the computational basis of vision is what might be called the probabilistic view of vision. This approach emphasizes the general importance of uncertainty and probabilities in perception and, in particular, suggests that perception is tightly linked to the statistical structure of the natural environment. This thesis investigates this link by building statistical models of natural images, and relating these to what is known of the information processing performed by the early stages of the primate visual system.

Recently, it was suggested that the response properties of simple cells in the primary visual cortex could be interpreted as the result of the cells performing an independent component analysis of the natural visual sensory input. This thesis provides some further support for that proposal, and, more importantly, extends the theory to also account for complex cell properties and the columnar organization of the primary visual cortex. Finally, the application of these methods to predicting neural response properties further along the visual pathway is considered.

Although the models considered account for only a relatively small part of known facts concerning early visual information processing, it is nonetheless a rather impressive amount considering the simplicity of the models. This is encouraging, and suggests that many of the intricacies of visual information processing might be understood using fairly simple probabilistic models of natural sensory input.

This thesis consists of an overview and of the following 6 publications:

  1. P. O. Hoyer and A. Hyvärinen, Independent component analysis applied to feature extraction from colour and stereo images, Network: Computation in Neural Systems, vol. 11, no. 3, pp. 191-210, 2000.
  2. A. Hyvärinen and P. O. Hoyer, Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces, Neural Computation, vol. 12, no. 7, pp. 1705-1720, 2000.
  3. A. Hyvärinen and P. O. Hoyer, A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images, Vision Research, vol. 41, no. 18, pp. 2413-2423, 2001.
  4. P. O. Hoyer, Non-negative sparse coding, in Neural Networks for Signal Processing XII (Proc. IEEE Workshop on Neural Networks for Signal Processing 2002, Martigny, Switzerland), pp. 557-565, 2002.
  5. P. O. Hoyer, Modeling receptive fields with non-negative sparse coding, in Computational Neuroscience: Trends in Research 2003, Elsevier, Amsterdam, 2003. In press.
  6. P. O. Hoyer and A. Hyvärinen, A multi-layer sparse coding network learns contour coding from natural images, Vision Research, vol. 42, no. 12, pp. 1593-1605, 2002.

Keywords: natural images, independent component analysis, latent variable models, unsupervised learning, neural networks, early vision, visual cortex

This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.

© 2002 Helsinki University of Technology

Last update 2011-05-26