The doctoral dissertations of the former Helsinki University of Technology (TKK) and Aalto University Schools of Technology (CHEM, ELEC, ENG, SCI) published in electronic format are available in the electronic publications archive of Aalto University - Aaltodoc.
Aalto

Induction of the Morphology of Natural Language: Unsupervised Morpheme Segmentation with Application to Automatic Speech Recognition

Mathias Creutz

Dissertation for the degree of Doctor of Science in Technology to be presented with due permission of the Department of Computer Science and Engineering for public examination and debate in Auditorium T2 at Helsinki University of Technology (Espoo, Finland) on the 15th of June, 2006, at 12 o'clock noon.

Overview in PDF format (ISBN 951-22-8211-9)   [786 KB]
Dissertation is also available in print (ISBN 951-22-8210-0)

Abstract

In order to develop computer applications that successfully process natural language data (text and speech), one needs good models of the vocabulary and grammar of as many languages as possible. According to standard linguistic theory, words consist of morphemes, which are the smallest individually meaningful elements in a language. Since an immense number of word forms can be constructed by combining a limited set of morphemes, the capability of understanding and producing new word forms depends on knowing which morphemes are involved (e.g., "water, water+s, water+y, water+less, water+less+ness, sea+water").

Morpheme boundaries are not normally marked in text unless they coincide with word boundaries. The main objective of this thesis is to devise a method that discovers the likely locations of the morpheme boundaries in words of any language. The method proposed, called Morfessor, learns a simple model of concatenative morphology (word forming) in an unsupervised manner from plain text. Morfessor is formulated as a Bayesian, probabilistic model. That is, it does not rely on predefined grammatical rules of the language, but makes use of statistical properties of the input text.

Morfessor situates itself between two types of existing unsupervised methods: morphology learning vs. word segmentation algorithms. In contrast to existing morphology learning algorithms, Morfessor can handle words consisting of a varying and possibly high number of morphemes. This is a requirement for coping with highly-inflecting and compounding languages, such as Finnish. In contrast to existing word segmentation methods, Morfessor learns a simple grammar that takes into account sequential dependencies, which improves the quality of the proposed segmentations.

Morfessor is evaluated in two complementary ways in this work: directly by comparing to linguistic reference morpheme segmentations of Finnish and English words and indirectly as a component of a large (or virtually unlimited) vocabulary Finnish speech recognition system. In both cases, Morfessor is shown to outperform state-of-the-art solutions.

The linguistic reference segmentations were produced as part of the current work, based on existing linguistic resources. This has resulted in a morphological gold standard, called Hutmegs, containing analyses of a large number of Finnish and English word forms.

This thesis consists of an overview and of the following 8 publications:

  1. Mathias Creutz and Krista Lagus. Unsupervised Discovery of Morphemes. In: Proceedings of the 6th Meeting of the ACL Special Interest Group in Computational Phonology in cooperation with the ACL Special Interest Group in Natural Language Learning: Workshop on Morphological and Phonological Learning, held in conjunction with the 40th Annual Meeting of the Association for Computational Linguistics (ACL-02), pages 21-30, Philadelphia, Pennsylvania, USA, July 2002. © 2002 Association for Computational Linguistics. By permission.
  2. Mathias Creutz. Unsupervised Segmentation of Words Using Prior Distributions of Morph Length and Frequency. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL-03), pages 280-287, Sapporo, Japan, July 2003. © 2003 Association for Computational Linguistics. By permission.
  3. Mathias Creutz and Krista Lagus. Induction of a Simple Morphology for Highly-Inflecting Languages. In: Proceedings of the 7th Meeting of the ACL Special Interest Group in Computational Phonology: Workshop on Current Themes in Computational Phonology and Morphology, held in conjunction with the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 43-51, Barcelona, Spain, July 2004. © 2004 Association for Computational Linguistics. By permission.
  4. Mathias Creutz and Krista Lagus. Inducing the Morphological Lexicon of a Natural Language from Unannotated Text. In: Proceedings of the International and Interdisciplinary Conference on Adaptive Knowledge Representation and Reasoning (AKRR05), pages 106-113, Espoo, Finland, June 2005. © 2005 by authors.
  5. Mathias Creutz and Krista Lagus. Unsupervised Models for Morpheme Segmentation and Morphology Learning. ACM Transactions on Speech and Language Processing, accepted for publication, 2006.
  6. Vesa Siivola, Teemu Hirsimäki, Mathias Creutz, and Mikko Kurimo. Unlimited Vocabulary Speech Recognition Based on Morphs Discovered in an Unsupervised Manner. In: Proceedings of the 8th European Conference on Speech Communication and Technology (EUROSPEECH 2003), pages 2293-2296, Geneva, Switzerland, September 2003. © 2003 International Speech Communication Association (ISCA). By permission.
  7. Teemu Hirsimäki, Mathias Creutz, Vesa Siivola, Mikko Kurimo, Sami Virpioja, and Janne Pylkkönen. Unlimited Vocabulary Speech Recognition with Morph Language Models Applied to Finnish. Computer Speech and Language, in press, 2006. © 2006 Elsevier Science. By permission.
  8. Mathias Creutz and Krister Lindén. Morpheme Segmentation Gold Standards for Finnish and English. Helsinki University of Technology, Publications in Computer and Information Science, Report A77, October 2004. © 2004 by authors.

Keywords: morpheme segmentation, morphology induction, unsupervised learning, probabilistic models, concatenative morphology, agglutinative languages, unlimited vocabulary speech recognition, Finnish, English

This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.

© 2006 Helsinki University of Technology

Last update 2011-05-26