Off-campus access
Using PhilPapers from home?
Click here to configure this browser for off-campus access.
- Alex Clark & Shalom Lappin, Unsupervised Learning and Grammar Induction.In this chapter we consider unsupervised learning from two perspectives. First, we briefly look at its advantages and disadvantages as an engineering technique applied to large corpora in natural language processing. While supervised learning generally achieves greater accuracy with less data, unsupervised learning offers significant savings in the intensive labour required for annotating text. Second, we discuss the possible relevance of unsupervised learning to debates on the cognitive basis of human language acquisition. In this context we explore the implications of recent work on grammar induction for poverty of stimulus arguments that purport to motivate a strong bias model of language learning, commonly formulated as a theory of Universal Grammar (UG). We examine the second issue both as a problem in computational learning theory, and with reference to empirical work on unsupervised Machine Learning (ML) of syntactic structure. We compare two models of learning theory and the place of unsupervised learning within each of them. Looking at recent work on part of speech tagging and the recognition of syntactic structure, we see how far unsupervised ML methods have come in acquiring different kinds of grammatical knowledge from raw text.
Similar books and articles
To learn a visual code in an unsupervised manner, one may attempt to capture those features of the stimulus set that would contribute significantly to a statistically efficient representation (as dictated, e.g., by the Minimum Description Length principle). Paradoxically, all the candidate features in this approach need to be known before statistics over them can be computed. This paradox may be circumvented by confining the repertoire of candidate features to actual scene fragments, which resemble the “what+where” receptive fields found in the ventral visual stream in primates. We describe a single-layer network that learns such fragments from unsegmented raw images of structured objects. The learning method combines fast imprinting in the feedforward stream with lateral interactions to achieve single-epoch unsupervised acquisition of spatially localized features that can support systematic treatment of structured objects [1].
No categories
Reading list
| Discuss | Edit | Categorize | My bibliography
| Export citation | Other links: kybele.psych.cornell.edu | Scholar | More options ...
| | Other links: kybele.psych.cornell.edu | Scholar | More options ... In this paper, we explore the possibility that machine learning approaches to naturallanguage processing being developed in engineering-oriented computational linguistics may be able to provide specific scientific insights into the nature of human language. We argue that, in principle, machine learning results could inform basic debates about language, in one area at least, and that in practice, existing results may offer initial tentative support for this prospect. Further, results from computational learning theory can inform arguments carried on within linguistic theory as well.
Reading list
| Discuss | Edit | Categorize | My bibliography | Export citation | Other links: hdl.handle.net | Scholar | More options ...
| | Other links: hdl.handle.net | Scholar | More options ... We compare our model of unsupervised learning of linguistic structures, ADIOS [1, 2, 3], to some recent work in computational linguistics and in grammar theory. Our approach resembles the Construction Grammar in its general philosophy (e.g., in its reliance on structural generalizations rather than on syntax projected by the lexicon, as in the current generative theories), and the Tree Adjoining Grammar in its computational characteristics (e.g., in its apparent affinity with Mildly Context Sensitive Languages). The representations learned by our algorithm are truly emergent from the (unannotated) corpus data, whereas those found in published works on cognitive and construction grammars and on TAGs are hand-tailored. Thus, our results complement and extend both the computational and the more linguistically oriented research into language acquisition. We conclude by suggesting how empirical and formal study of language can be best integrated.
No categories
Reading list
| Discuss | Edit | Categorize | My bibliography | Export citation | Other links: kybele.psych.cornell.edu | Scholar | More options ...
| | Other links: kybele.psych.cornell.edu | Scholar | More options ... We report a quantitative analysis of the cross-utterance coordination observed in child-directed language, where successive utterances often overlap in a manner that makes their constituent structure more prominent, and describe the application of a recently published unsupervised algorithm for grammar induction to the largest available corpus of such language, producing a grammar capable of accepting and generating novel wellformed sentences. We also introduce a new corpus-based method for assessing the precision and recall of an automatically acquired generative grammar without recourse to human judgment. The present work sets the stage for the eventual development of more powerful unsupervised algorithms for language acquisition, which would make use of the coordination structures present in natural child-directed speech.
No categories
Reading list
| Discuss | Edit | Categorize | My bibliography | Export citation | Other links: kybele.psych.cornell.edu | Scholar | More options ...
| | Other links: kybele.psych.cornell.edu | Scholar | More options ... We present a generative distributional model for the unsupervised induction of natural language syntax which explicitly models constituent yields and contexts. Parameter search with EM produces higher quality analyses than previously exhibited by unsupervised systems, giving the best published unsupervised parsing results on the ATIS corpus. Experiments on Penn treebank sentences of comparable length show an even higher F1 of 71% on nontrivial brackets. We compare distributionally induced and actual part-of-speech tags as input data, and examine extensions to the basic model. We discuss errors made by the system, compare the system to previous models, and discuss upper bounds, lower bounds, and stability for this task.
No categories
Reading list
| Discuss | Edit | Categorize | My bibliography | Export citation | Scholar | More options ...
| | Scholar | More options ... This paper presents a novel approach to the unsupervised learning of syntactic analyses of natural language text. Most previous work has focused on maximizing likelihood according to generative PCFG models. In contrast, we employ a simpler probabilistic model over trees based directly on constituent identity and linear context, and use an EM-like iterative procedure to induce structure. This method produces much higher quality analyses, giving the best published results on the ATIS dataset.
No categories
Reading list
| Discuss | Edit | Categorize | My bibliography | Export citation | Scholar | More options ...
| | Scholar | More options ... This paper presents a novel approach to the unsupervised learning of syntactic analyses of natural language text. Most previous work has focused on maximizing likelihood according to generative PCFG models. In contrast, we employ a simpler probabilistic model over trees based directly on constituent identity and linear context, and use an EM-like iterative procedure to induce structure. This method produces much higher quality analyses, giving the best published results on the ATIS dataset.
No categories
Reading list
| Discuss | Edit | Categorize | My bibliography | Export citation | Scholar | More options ...
| | Scholar | More options ... Unsupervised statistical learning is the standard setting for the development of the only advanced visual system that is both highly sophisticated and versatile, and extensively studied: that of monkeys and humans. In this extended abstract, we invoke philosophical observations, computational arguments, behavioral data and neurobiological findings to explain why computer vision researchers should care about (1) unsupervised learning, (2) statistical inference, and (3) the visual brain. We then outline a neuromorphic approach to structural primitive learning motivated by these considerations, survey a range of neurobiological findings and behavioral data consistent with it, and conclude by mentioning some of the more challenging directions for future research.
No categories
Reading list
| Discuss | Edit | Categorize | My bibliography | Export citation | Other links: kybele.psych.cornell.edu | Scholar | More options ...
| | Other links: kybele.psych.cornell.edu | Scholar | More options ... We compare our model of unsupervised learning of linguistic structures, ADIOS [1], to some recent work in computational linguistics and in grammar theory. Our approach resembles the Construction Grammar in its general philosophy (e.g., in its reliance on structural generalizations rather than on syntax projected by the lexicon, as in the current generative theories), and the Tree Adjoining Grammar in its computational characteristics (e.g., in its apparent affinity with Mildly Context Sensitive Languages). The representations learned by our algorithm are truly emergent from the (unannotated) corpus data, whereas those found in published works on cognitive and construction grammars and on TAGs are hand-tailored. Thus, our results complement and extend both the computational and the more linguistically oriented research into language acquisition. We conclude by suggesting how empirical and formal study of language can be best integrated.
No categories
Reading list
| Discuss | Edit | Categorize | My bibliography | Export citation | Other links: kybele.psych.cornell.edu | Scholar | More options ...
| | Other links: kybele.psych.cornell.edu | Scholar | More options ... Machine learning and statistical methods have yielded impressive results in a wide variety of natural language processing tasks. These advances have generally been regarded as engineering achievements. In fact it is possible to argue that the success of machine learning methods is significant for our understanding of the cognitive basis of language acquisition and processing. Recent work in unsupervised grammar induction is particularly relevant to this issue. It suggests that knowledge of language can be achieved through general learning procedures, and that a richly articulated language faculty is not required to explain its acquisition.
Reading list
| Discuss | Edit | Categorize | My bibliography | Export citation | Other links: dcs.kcl.ac.uk | Scholar | More options ...
| | Other links: dcs.kcl.ac.uk | Scholar | More options ... Discussion of Alex Clark & Shalom Lappin, Unsupervised learning and grammar induction
 Back
All discussions
Start a new thread
|
There are no threads in this forum |
Nothing in this forum yet.
