Abstract
In this paper we propose a new method to extract from a corpus the histories for which a given language model is better than another one. The decision is based on a measure stemmed from perplexity. This measure allows, for a given history, to compare two language models, and then to choose the best one for this history. Using this principle, and with a 20K vocabulary words, we combined two language models: a bigram and a distant bigram. The contribution of a distant bigram is significant and outperforms a bigram model by 7.5%. Moreover, the performance in Shannon game are improved. We show through this article that we proposed a cheaper framework in comparison to the maximum entropy principle, for combining language models. In addition, the selected histories for which a model is better than another one, have been collected and studied. Almost, all of them are beginnings of very frequently used French phrases. Finally, by using this principle, we achieve a better trigram model in terms of parameters and perplexity. This model is a combination of a bigram and a trigram based on a selected history.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ciprian Chelba and Frederick Jelinek. Structured language modeling. Computer Speech and Language, 14(4):283–332, October 2000.
Renato de Mori and Marcello Federico. Language model adaptation. In K. Ponting, editor, Computational models of speech pattern processing, volume 169, pages 280–303. NATO ASI, 1999.
A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum-likelihood from incomplete data via the em algorithm. Journal of Royal Statistic Society, pages 1–38, 1977.
Marcello Federico and Renato de Mori. Spoken dialogues with computers, chapter 7, pages 199–230. Academic Press, 1997.
Dominique Fohr, Jean-Paul Haton, Jean-François Mari, Kamel Smaýli, and Imedu Zitouni. Towards an oral interface of data entry: The maud system. In Proceedings of the 3rd European Research Consortium for Informatics and Mathematics Workshop on ”User Interface for All”, pages 233–234, 1997.
E. Giachin. Phrase bigrams for continuous speech recognition. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, pages 225–228, 1995.
Zhou Guo Dong and Lua Kim Teng. Interpolation of n-gram and mutual-information based trigger pair language models for mandarin speech recognition. Computer Speech and Language, 13:125–141, 1999.
M. Jardino, F. Bimbot, S. Igounet, K. Smaïli, and M. El-Beze. A first evaluation campaign for language models. In Proceedings of the 1st International Conference on Language Resources and Evaluation, volume 2, pages 801–805, Granada, Spain, 1998.
Frederic Jelinek. Self-organized language modelling for speech recognition. In A. Waibel and K.-F. Lee, editors, Readings in Speech Recognition, pages 450–506. Kaufmann Publishers, San Mateo, CA, 1990.
L. Lamel, J.-L. Gauvain, and M. Eskenazi. Bref, a large vocabulary spoken corpus for french. In Proceeding of European Conference on Speech Communication and Technology, volume 2, pages 505–508, Gênes, 1991.
David Langlois and Kamel Smaïli. A new distance language model for a dictation machine: application to maud. In Proceeding of European Conference on Speech Communication and Technology, volume 4, pages 1779–1782, Budapest, Hungary, September 1999.
David Langlois, Kamel Smaïli, and Jean-Paul Haton. Dealing with distant relationships in natural language modelling for automatic speech recognition. In Proceedings of the SCI2000 conference, volume 6, pages 400–405. International Institute of Informatics and Systemics, 2000.
Ronald Rosenfeld. A maximum entropy approach to adaptative statistical language modelling. Computer Speech and Language, 10:187–228, 1996.
K. Smaïli, A. Brun, I. Zitouni, and J.-P. Haton. Automatic and manual clustering for large vocabulary speech recognition: a comparative study. In Proceeding of European Conference on Speech Communication and Technology, volume 4, pages 1795–1798, September 1999.
I. Zitouni, J.-F. Mari, K. Smaïli, and J.-P. Haton. Variable-length sequence language model for large vocabulary continuous dictation machine: the n-seqgram approach. In Proceeding of the European Conference on Speech and Technology, volume 4, pages 1811–1814, Budapest, Hungary, September 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Langlois, D., Smaïli, K., Haton, JP. (2001). A New Method Based on Context for Combining Statistical Language Models. In: Akman, V., Bouquet, P., Thomason, R., Young, R. (eds) Modeling and Using Context. CONTEXT 2001. Lecture Notes in Computer Science(), vol 2116. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44607-9_18
Download citation
DOI: https://doi.org/10.1007/3-540-44607-9_18
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42379-9
Online ISBN: 978-3-540-44607-1
eBook Packages: Springer Book Archive