This paper presents a novel approach to the unsupervised learning of syntactic analyses of natural language text. Most previous work has focused on maximizing likelihood according to generative PCFG models. In contrast, we employ a simpler probabilistic model over trees based directly on constituent identity and linear context, and use an EM-like iterative procedure to induce structure. This method produces much higher quality analyses, giving the best published results on the ATIS dataset.

This paper presents a novel approach to the unsupervised learning of syntactic analyses of natural language text. Most previous work has focused on maximizing likelihood according to generative PCFG models. In contrast, we employ a simpler probabilistic model over trees based directly on constituent identity and linear context, and use an EM-like iterative procedure to induce structure. This method produces much higher quality analyses, giving the best published results on the ATIS dataset.

In this important and pioneering book Frederick Newmeyer takes on the question of language variety. He considers why some language types are impossible and why some grammatical features are more common than others. The task of trying to explain typological variation among languages has been mainly undertaken by functionally-oriented linguists. Generative grammarians entering the field of typology in the 1980s put forward the idea that cross-linguistic differences could be explained by linguistic parameters within Universal Grammar, whose operation (...) might vary from language to language. Unfortunately, this way of looking at variation turned out to be much less successful than had been hoped for. Professor Newmeyer's alternative to parameters combines leading ideas from functionalist and formalist approaches which in the past have been considered incompatible. He throws fresh light on language typology and variation, and provides new insights into the principles of Universal Grammar. The book is written in a clear, readable style and will be readily understood by anyone with a couple of years' study of linguistics. It will interest a wide range of scholars and students of language, including typologists, historical linguists, and theorists of every shade. (shrink)

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed (...) in this manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed (...) in this manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

A study of the contribution semantics makes to the syntactic patterns of English: an intepretive theory of grammar.

The theory of language must predict the possible thought—signal (or meaning—sound or sign) pairings of a language. We argue for a Meaning First architecture of language where a thought structure is generated first. The thought structure is then realized using language to communicate the thought, to memorize it, or perhaps with another purpose. Our view contrasts with the T-model architecture of mainstream generative grammar, according to which distinct phrase-structural representations—Phonetic Form (PF) for articulation, Logical Form (...) (LF) for interpretation—are generated within the grammar. At the same time, our view differs from early transformational grammar and generative semantics: We view the relationship between the thought structure and the corresponding signal as one of compression. We specify a formal sketch of compression as a choice between multiple possible pronounciations balancing the desire to transmit information against the effort of pronounciation. The Meaning First architecture allows a greater degree of independence between thought structures and the linguistic signal. We present three arguments favoring this type of independence. First we argue that scopal properties can be better explained if we only compare thought structures independent of the their realization as a sentence. Secondly, we argue that Meaning First architecture allows contentful late insertion, an idea that has been argued for in Distributed Morphology already, but as we argue is also motivated by the division of the logical and socio-emotive meaning content of language. Finally, we show that only the Meaning First architecture provides a satisfying account of the mixing of multiple languages by multilingual speakers, especially for cases of simultaneous articulation across two modalities in bimodal speakers. Our view of the structure of grammar leads to a reassessment of priorities in linguistic analyses: while current mainstream work is often focused on establishing one-to-one relationships between concepts and morphemes, our view makes it plausible that primitive concepts are frequently marked indirectly or unpronounced entirely. Our view therefore assigns great value to the understanding of logical primitives and of compression. (shrink)

We present a generative distributional model for the unsupervised induction of natural language syntax which explicitly models constituent yields and contexts. Parameter search with EM produces higher quality analyses than previously exhibited by unsupervised systems, giving the best published unsupervised parsing results on the ATIS corpus. Experiments on Penn treebank sentences of comparable length show an even higher F1 of 71% on nontrivial brackets. We compare distributionally induced and actual part-of-speech tags as input data, and examine extensions to (...) the basic model. We discuss errors made by the system, compare the system to previous models, and discuss upper bounds, lower bounds, and stability for this task. (shrink)

The generative grammar approach to language seeks a fully explicit account of the modular systems of knowledge (competence) that underlies the human language capacity. Similarly, the relevance-theoretic approach to pragmatics attempts an explicit characterisation of the sub-personal systems involved in utterance interpretation. As an on-line performance system, however, it is subject to certain additional constraints; this is demonstrated by the way in which matters of computational (processing effort) economy are currently employed in the two types of theory. (...) A sub-module of “discourse competence” is shown to be compatible with and complementary to the wider system of pragmatic processes. (shrink)

Like Einstein's general theory of relativity is concerned with explaining the basics of an observable experience – i.e., gravity – most people take for granted that Chomsky's theory of generative grammar (GG) is concerned with the basic nature of language. This chapter highlights a mere subset of central constructs in GG, showing how they have featured prominently and thus shaped formal linguistic studies in multilingualism. Because multilingualism includes a wide range of nonmonolingual populations, the constructs are divided across (...) child bilingualism and adult third language for greater coverage. In the case of the former, the chapter examines how poverty of the stimulus has been investigated. Using the nascent field of L3/Ln acquisition as the backdrop, it discusses how the GG constructs of I‐language versus E‐language sit at the core of debates regarding the very notion of what linguistic transfer and mental representations should be taken to be. (shrink)

In this paper, I address the issue of scientific modelling in contemporary linguistics, focusing on the generative tradition. In so doing, I identify two common varieties of linguistic idealisation, which I call determination and isolation respectively. I argue that these distinct types of idealisation can both be described within the remit of Weisberg’s :639–659, 2007) minimalist idealisation strategy in the sciences. Following a line set by Blutner :27–35, 2011), I propose this minimalist idealisation analysis for a broad construal of (...) the generative linguistic programme and thus cite examples from a wide range of linguistic frameworks including early generative syntax, Minimalism, the parallel architecture and optimality theory. Lastly, I claim that from a modelling perspective, the dynamic turn in syntax can be explained as a continuation, as opposed to a marked shift, of the generative modelling paradigm. Seen in this light, my proposal is an even broader construal of the generative tradition, along scientific modelling lines. Thus, I offer a lens through which to appreciate the scientific contribution of generative grammar, amid an increased resistance to some of its core theoretical posits, in terms of a brand of structural realism in the philosophy of science and specifically scientific modelling. (shrink)

A remarkable philosophical affinity may be observed between the intuitionistic conception of mathematics and the transformational generative approach to the study of language: both disciplines profess a mentalistic ontology, both posit an idealized subject, and both insist on their autonomy with respect to other disciplines. This philosophical parallel is formalized in terms of a generalization of the intuitionistic notion of creative subject; resulting are the foundations of a unified theory of mental acts based on intuitionistic logic — capturing, (...) inter alia, similarities between proof acts and speech acts. As an application of the theory, it is then shown how the notion of mental act may provide for an insightful formalization of various hypotheses pertaining to the linguistic dependence or relativity of mathematics. (shrink)

A familiar argument goes as follows: natural languages have infinitely many sentences, finite representation of infinite sets requires recursion; therefore any adequate account of linguistic competence will require some kind of recursive device. The first part of this paper argues that this argument is not convincing. The second part argues that it was not the original reason recursive devices were introduced into generative linguistics. The real basis for the use of recursive devices stems from a deeper philosophical concern; a (...) grammar that functions merely as a metalanguage would not be explanatorily adequate as it would merely push the problem of explaining linguistic competence back to another level. The paper traces this concern from Zellig Harris and Chomsky’s early work in generative linguistics and presents some implications. (shrink)

Already hailed as a masterpiece, Foundations of Language offers a brilliant overhaul of the last thirty-five years of research in generative linguistics and related fields. "Few books really deserve the cliché 'this should be read by every researcher in the field'," writes Steven Pinker, author of The Language Instinct, "but Ray Jackendoff's Foundations of Language does." Foundations of Language offers a radically new understanding of how language, the brain, and perception intermesh. The book renews (...) the promise of early generative linguistics: that language can be a valuable entrée into understanding the human mind and brain. The approach is remarkably interdisciplinary. Behind its innovations is Jackendoff's fundamental proposal that the creativity of language derives from multiple parallel generative systems linked by interface components. This shift in basic architecture makes possible a radical reconception of mental grammar and how it is learned. As a consequence, Jackendoff is able to reintegrate linguistics with philosophy of mind, cognitive and developmental psychology, evolutionary biology, neuroscience, and computational linguistics. Among the major topics treated are language processing, the relation of language to perception, the innateness of language, and the evolution of the language capacity, as well as more standard issues in linguistic theory such as the roles of syntax and the lexicon. In addition, Jackendoff offers a sophisticated theory of semantics that incorporates insights from philosophy of language, logic and formal semantics, lexical semantics of various stripes, cognitive grammar, psycholinguistic and neurolinguistic approaches, and the author's own conceptual semantics. (shrink)

The goal of this study is to reintegrate the theory of generative grammar into the cognitive sciences. Generative grammar was right to focus on the child's acquisition of language as its central problem, leading to the hypothesis of an innate Universal Grammar. However, generative grammar was mistaken in assuming that the syntactic component is the sole course of combinatoriality, and that everything else is “interpretive.” The proper approach is a parallel architecture, in which phonology, syntax, and (...) semantics are autonomous generative systems linked by interface components. The parallel architecture leads to an integration within linguistics, and to a far better integration with the rest of cognitive neuroscience. It fits naturally into the larger architecture of the mind/brain and permits a properly mentalistic theory of semantics. It results in a view of linguistic performance in which the rules of grammar are directly involved in processing. Finally, it leads to a natural account of the incremental evolution of the language capacity. Key Words: evolution of language; generative grammar; parallel architecture; semantics; syntax. (shrink)

Name der Zeitschrift: Cognitive Linguistics Jahrgang: 26 Heft: 4 Seiten: 613-632.

A generative grammar for a language L generates one or more syntactic structures for each sentence of L and interprets those structures both phonologically and semantically. A widely accepted assumption in generative linguistics dating from the mid-60s, the Generative Grammar Hypothesis , is that the ability of a speaker to understand sentences of her language requires her to have tacit knowledge of a generative grammar of it, and the task of linguistic semantics in those (...) early days was taken to be that of specifying the form that the semantic component of a generative grammar must take. Then in the 70s linguistic semantics took a curious turn. Without rejecting GGH, linguists turned away from the task of characterizing the semantic component of a generative grammar to pursue instead the Montague-inspired project of providing for natural languages the same kind of model-theoretic semantics that logicians devise for the artificial languages of formal systems of logic, and “formal semantics” continues to dominate semantics in linguistics. This essay argues that the sort of compositional meaning theory that would verify GGH would not only be quite different from the theories formal semanticists construct, but would be a more fundamental theory that supersedes those theories in that it would explain why they are true when they are true, but their truth wouldn’t explain its truth. Formal semantics has undoubtedly made important contributions to our understanding of such phenomena as anaphora and quantification, but semantics in linguistics is supposed to be the study of meaning. This means that the formal semanticist can’t be unconcerned that the kind of semantic theory for a natural language that interests her has no place in a theory of linguistic competence; for if GGH is correct, then the more fundamental semantic theory is the compositional meaning theory that is the semantic component of the internally represented generative grammar, and if that is so, then linguistic semantics has so far ignored what really ought to be its primary concern. (shrink)

Various restrictions on transformational grammars have been investigated in order to reduce their generative power from recursively enumerable languages to recursive languages.It will be shown that any restriction on transformational grammars defining a recursively enumerable subset of the set of all transformational grammars, is either too weak (in the sense that there does not exist a general decision procedure for all languages generated under such a restriction) or too strong (in the sense that there exists a recursive language (...) that cannot be generated by any transformational grammar thus restricted). In addition, some related problems will be discussed. (shrink)

Predictive Processing (PP) is an increasingly influential neurocognitive-computational framework. PP research has so far focused predominantly on lower level perceptual, motor, and various psychological phenomena. But PP seems to face a “scale-up challenge”: How can it be extended to conceptual thought, language, and other higher cognitive competencies? Compositionality, arguably a central feature of conceptual thought, cannot easily be accounted for in PP because it is not couched in terms of classical symbol processing. I argue, using the example of (...) class='Hi'>language, that there is no strong reason to think that PP cannot be scaled up to higher cognition. I suggest that the tacitly assumed common-sense conception of language as Generative Grammar (“folk linguistics”) and its notion of composition leads to the scale-up concerns. Fodor’s Language of Thought Hypothesis (LOTH) plays the role of a cognitive computational paradigm for folk linguistics. Therefore, we do not take LOTH as facing problems with higher cognition, at least with regard to compositionality. But PP can plausibly play the role of a cognitive-computational paradigm for an alternative conception of language, namely Construction Grammar. If Construction Grammar is a plausible alternative to folk linguistics, then PP is not in a worse position than LOTH. (shrink)

Recent defenders of the cognitive significance of religious language have had to face opponents from two directions; from those who demand that religious language be capable of some form of empirical verification and from those who demand that for religious language to be meaningful it must be capable of being understood in ordinary language. Apologists who have taken the first challenge seriously have strained to show that religious statements can be verified by ‘religious experience’, or by (...) an ‘odd discernment’ or by an ‘eschatological verification’. Each of these responses raises further problems for the defender of religion, but in general they all are subject to their failure to provide an adequate criterion or standard by which they could be inter-subjectively tested. Facing in the other direction, theological apologists have attempted to justify religious expressions by showing that they could be subsumed under special categories of ordinary language; namely those of ‘convictional language’ and ‘performative language’. Although these defenders have shown that religious language has an ‘ordinary usage’, they have not shown that the cognitive elements of this usage can be reduced to the ‘use of ordinary language’. (shrink)

The target article advanced the null, unified and widely misinterpreted generative hypothesis regarding second language (L2) acquisition. Postulating that UG (Universal Grammar) constrains L2 knowledge growth does not entail identical developmental trajectories for L2 and first language (LI) acquisition; nor does it preclude a role for the L1. In embracing this hypothesis, we maintain a distinction between competence and performance. Those who conflate the two repeat fundamental and by no means unprecedented misconstruals of the generative enterprise, (...) and more specifically, of the empirical content of the null hypothesis regarding L2 linguistic knowledge growth. We hope to have identified certain common goals, the adoption of which might constitute a firm foundation for continued productive interdisciplinary development of contemporary theoretical and experimental L2 acquisition research. (shrink)

Three long papers are collected here which constitute Chomsky’s major theoretical work on syntax and semantics subsequent to the "standard theory" of Aspects of the Theory of Syntax. Since 1965, transformational-generative linguists have suggested various changes in "standard theory," centering on the relationship between the syntactic and semantic components in natural language grammars. In these papers Chomsky explains several specific problems that require the extension of standard theory and he criticizes the proposals and arguments of the generative (...) semanticists, attempting to show that many of the specific features of language that are claimed to support the generativist position are as well or better explained by extended standard theory, and emphasizing that most of the generativist’s proposals turn out to be mere "notational variants," evidentially indistinguishable from extended standard theory. Aside from appealing to particular data favoring his "lexicalist" defense of syntactic deep structure, Chomsky makes a Popperean appeal that his position is preferable because it is more constraining and more falsifiable. (shrink)

A generative grammar for a language L generates one or more syntactic structures for each sentence of L and interprets those structures both phonologically and semantically. A widely accepted assumption in generative linguistics dating from the mid-60s, the Generative Grammar Hypothesis, is that the ability of a speaker to understand sentences of her language requires her to have tacit knowledge of a generative grammar of it, and the task of linguistic semantics in those early (...) days was taken to be that of specifying the form that the semantic component of a generative grammar must take. Then in the 70s linguistic semantics took a curious turn. Without rejecting GGH, linguists turned away from the task of characterizing the semantic component of a generative grammar to pursue instead the Montague-inspired project of providing for natural languages the same kind of model-theoretic semantics that logicians devise for the artificial languages of formal systems of logic, and “formal semantics” continues to dominate semantics in linguistics. This essay argues that the sort of compositional meaning theory that would verify GGH would not only be quite different from the theories formal semanticists construct, but would be a more fundamental theory that supersedes those theories in that it would explain why they are true when they are true, but their truth wouldn’t explain its truth. Formal semantics has undoubtedly made important contributions to our understanding of such phenomena as anaphora and quantification, but semantics in linguistics is supposed to be the study of meaning. This means that the formal semanticist can’t be unconcerned that the kind of semantic theory for a natural language that interests her has no place in a theory of linguistic competence; for if GGH is correct, then the more fundamental semantic theory is the compositional meaning theory that is the semantic component of the internally represented generative grammar, and if that is so, then linguistic semantics has so far ignored what really ought to be its primary concern. (shrink)

Although there is a broad consensus that both the procedural and declarative memory systems play a crucial role in language learning, use, and knowledge, the mapping between linguistic types and memory structures remains underspecified: by default, a dual-route mapping of language systems to memory systems is assumed, with declarative memory handling idiosyncratic lexical knowledge and procedural memory handling rule-governed knowledge of grammar.We experimentally contrast the processing of morphology (case and aspect), syntax (subordination), and lexical semantics (collocations) in a (...) healthy L1 population of Polish, a language rich in form distinctions. We study the processing of these four types under two conditions: a single task condition in which the grammaticality of stimuli was judged and a concurrent task condition in which grammaticality judgments were combined with a digit span task. Dividing attention impedes access to declarative memory while leaving procedural memory unaffected and hence constitutes a test that dissociates which types of linguistic information each long-term memory construct subserves.Our findings confirm the existence of a distinction between lexicon and grammar as a generative, dual-route model would predict, but the distinction is graded, as usage-based models assume: the hypothesized grammar–lexicon opposition appears as a continuum on which grammatical phenomena can be placed as being more or less “ruly” or “idiosyncratic.” However, usage-based models, too, need adjusting as not all types of linguistic knowledge are proceduralized to the same extent. This move away from a simple dichotomy fundamentally changes how we think about memory for language, and hence how we design and interpret behavioral and neuroimaging studies that probe into the nature of language cognition. (shrink)