The doctoral dissertations of the former Helsinki University of Technology (TKK) and Aalto University Schools of Technology (CHEM, ELEC, ENG, SCI) published in electronic format are available in the electronic publications archive of Aalto University - Aaltodoc.
Aalto

Adaptive Combinations of Classifiers with Application to On-Line Handwritten Character Recognition

Matti Aksela

Dissertation for the degree of Doctor of Science in Technology to be presented with due permission of the Department of Computer Science and Engineering for public examination and debate in Auditorium TU2 at Helsinki University of Technology (Espoo, Finland) on the 29th of March, 2007, at 12 noon.

Overview in PDF format (ISBN 978-951-22-8690-4)   [1135 KB]
Dissertation is also available in print (ISBN 978-951-22-8689-8)

Abstract

Classifier combining is an effective way of improving classification performance. User adaptation is clearly another valid approach for improving performance in a user-dependent system, and even though adaptation is usually performed on the classifier level, also adaptive committees can be very effective. Adaptive committees have the distinct ability of performing adaptation without detailed knowledge of the classifiers. Adaptation can therefore be used even with classification systems that intrinsically are not suited for adaptation, whether that be due to lack of access to the workings of the classifier or simply a classification scheme not suitable for continuous learning.

This thesis proposes methods for adaptive combination of classifiers in the setting of on-line handwritten character recognition. The focal part of the work introduces adaptive classifier combination schemes, of which the two most prominent ones are the Dynamically Expanding Context (DEC) committee and the Class-Confidence Critic Combining (CCCC) committee. Both have been shown to be capable of successful adaptation to the user in the task of on-line handwritten character recognition. Particularly the highly modular CCCC framework has shown impressive performance also in a doubly-adaptive setting of combining adaptive classifiers by using an adaptive committee.

In support of this main topic of the thesis, some discussion on a methodology for deducing correct character labeling from user actions is presented. Proper labeling is paramount for effective adaptation, and deducing the labels from the user's actions is necessary to perform adaptation transparently to the user. In that way, the user does not need to give explicit feedback on the correctness of the recognition results.

Also, an overview is presented of adaptive classification methods for single-classifier adaptation in handwritten character recognition developed at the Laboratory of Computer and Information Science of the Helsinki University of Technology, CIS-HCR. Classifiers based on the CIS-HCR system have been used in the adaptive committee experiments as both member classifiers and to provide a reference level.

Finally, two distinct approaches for improving the performance of committee classifiers further are discussed. Firstly, methods for committee rejection are presented and evaluated. Secondly, measures of classifier diversity for classifier selection, based on the concept of diversity of errors, are presented and evaluated.

The topic of this thesis hence covers three important aspects of pattern recognition: on-line adaptation, combining classifiers, and a practical evaluation setting of handwritten character recognition. A novel approach combining these three core ideas has been developed and is presented in the introductory text and the included publications.

To reiterate, the main contributions of this thesis are: 1) introduction of novel adaptive committee classification methods, 2) introduction of novel methods for measuring classifier diversity, 3) presentation of some methods for implementing committee rejection, 4) discussion and introduction of a method for effective label deduction from on-line user actions, and as a side-product, 5) an overview of the CIS-HCR adaptive on-line handwritten character recognition system.

This thesis consists of an overview and of the following 8 publications:

  1. V. Vuori, M. Aksela, J. Laaksonen, E. Oja, and J. Kangas. Adaptive character recognizer for a hand-held device: Implementation and evaluation setup. In Proceedings of the 7th International Workshop on Frontiers in Handwriting Recognition, pages 13-22, Amsterdam, Netherlands, September 11-13 2000. © 2000 International Unipen Foundation. By permission.
  2. M. Aksela, J. Laaksonen, E. Oja, and J. Kangas. Application of adaptive committee classifiers in on-line character recognition. In Proceedings of the Second International Conference on Advances in Pattern Recognition, Lecture Notes in Computer Science, 2013: 270-279, Rio de Janeiro, Brazil, March 11-14 2001.
  3. M. Aksela, J. Laaksonen, E. Oja, and J. Kangas. Rejection methods for an adaptive committee classifier. In Proceedings of the Sixth International Conference on Document Analysis and Recognition, pages 982-986, Seattle, Washington, USA, September 10-13 2001. © 2001 IEEE. By permission.
  4. M. Aksela, R. Girdziušas, J. Laaksonen, E. Oja, and J. Kangas. Class-confidence critic combining. In Proceedings of the 8th International Workshop on Frontiers in Handwriting Recognition, pages 201-206, Niagara-on-the-Lake, Canada, August 6-8 2002. © 2002 IEEE. By permission.
  5. M. Aksela, R. Girdziušas, J. Laaksonen, E. Oja, and J. Kangas. Methods for adaptive combination of classifiers with application to recognition of handwritten characters. International Journal on Document Analysis and Recognition, 6 (1): 23-41, 2003.
  6. M. Aksela and J. Laaksonen. On adaptive confidences for critic-driven classifier combining. In Proceedings of the Third International Conference on Advances in Pattern Recognition, Lecture Notes in Computer Science, 3686: 71-80, Bath, United Kingdom, August 22-25 2005.
  7. M. Aksela and J. Laaksonen. Using diversity of errors for selecting members of a committee classifier. Pattern Recognition, 39 (4): 608-623, 2006.
  8. M. Aksela and J. Laaksonen. Adaptive combination of adaptive classifiers for handwritten character recognition. Pattern Recognition Letters, 28 (1): 136-143, 2007.

Keywords: classifier combining, adaptive classifier, adaptive committee, on-line handwritten character recognition, pattern recognition

This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.

© 2007 Helsinki University of Technology

Last update 2011-05-26