Connectionism and Compositionality

In his famous 1982 paper, Allen Newell [22, 23] introduced the notion of knowledge level to indicate a level of analysis, and prediction, of the rational behavior of a cognitive arti cial agent. This analysis concerns the investigation about the availability of the agent knowledge, in order to pursue its own goals, and is based on the so-called Rationality Principle (an assumption according to which "an agent will use the knowledge it has of its environment to achieve its goals" [22, (...) p. 17]. By using the Newell's own words: "To treat a system at the knowledge level is to treat it as having some knowledge, some goals, and believing it will do whatever is within its power to attain its goals, in so far as its knowledge indicates" [22, p. 13]. In the last decades, the importance of the knowledge level has been historically and system- atically downsized by the research area in cognitive architectures (CAs), whose interests have been mainly focused on the analysis and the development of mechanisms and the processes governing human and (arti cial) cognition. The knowledge level in CAs, however, represents a crucial level of analysis for the development of such arti cial general systems and therefore deserves greater research attention [17]. In the following, we will discuss areas of broad agree- ment and outline the main problematic aspects that should be faced within a Common Model of Cognition [12]. Such aspects, departing from an analysis at the knowledge level, also clearly impact both lower (e.g. representational) and higher (e.g. social) levels. (shrink)

The standard approach to model how human beings understand natural languages is the symbolic, compositional approach according to which the meaning of a complex expression is a function of the meanings of its constituents. In other words, meaning plays a fundamental role in the model. In this work, because of the polysemous, flexible, dynamic, and contextual structure of natural languages, this approach is rejected. Instead, a connectionist model which eliminates the concept of meaning is proposed.

This paper aims to clarify certain features of the systematicity arguments by a review of some of the largely underexamined background in Chomsky’s and Fodor’s early work on transformational grammar.

The systematicity argument only challenges connectionism if systematicity is a general property of cognition. I examine this thesis in terms of properties of concepts. First, I propose that Evans's Generality Constraint only applies to attributions of belief. Then I defend a variety of conceptual pluralism, arguing that concepts share two fundamental properties related to centrality and belief-attribution, and contending that there are two kinds of concepts that differ in their compositional properties. Finally, I rely on Dual Systems Theory and on (...) differences between animal and human cognition to suggest a scenario of two processing systems that work on different kinds of concepts, with only one of them supporting full systematicity. I sketch a non-classical systematicity argument that rules out classicism as the basis of one of those systems given that it would wrongly entail that both systems are fully systematic. (shrink)

Instead of using low-level neurophysiology mimicking and exploratory programming methods commonly used in the machine consciousness field, the hierarchical Operational Architectonics (OA) framework of brain and mind functioning proposes an alternative conceptual-theoretical framework as a new direction in the area of model-driven machine (robot) consciousness engineering. The unified brain-mind theoretical OA model explicitly captures (though in an informal way) the basic essence of brain functional architecture, which indeed constitutes a theory of consciousness. The OA describes the neurophysiological basis of the (...) phenomenal level of brain organization. In this context the problem of producing man-made “machine” consciousness and “artificial” thought is a matter of duplicating all levels of the operational architectonics hierarchy (with its inherent rules and mechanisms) found in the brain electromagnetic field. We hope that the conceptual-theoretical framework described in this paper will stimulate the interest of mathematicians and/or computer scientists to abstract and formalize principles of hierarchy of brain operations which are the building blocks for phenomenal consciousness and thought. (shrink)

The “systematicity argument” has been used to argue for a classical cognitive architecture (Fodor in The Language of Thought. Harvester Press, London, 1975, Why there still has to be a language of thought? In Psychosemantics, appendix. MIT Press, Cambridge, pp 135–154, 1987; Fodor and Pylyshyn in Cognition 28:3–71, 1988; Aizawa in The systematicity arguments. Kluwer Academic Press, Dordrecht, 2003). From the premises that cognition is systematic and that the best/only explanation of systematicity is compositional structure, it concludes that cognition is (...) to be explained in terms of symbols (in a language of thought) and formal rules. The debate, with connectionism, has mostly centered on the second premise-whether an explanation of systematicity requires compositional structure, which neural networks do not to exhibit (for example, Hadley and Hayward, in Minds and Machines, 7:1–37). In this paper, I will take issue with the first premise. Several arguments will be deployed that show that cognition is not systematic in general; that, in fact, systematicity seems to be related to language. I will argue that it is just verbal minds that are systematic, and they are so because of the structuring role of language in cognition. A dual-process theory of cognition will be defended as the best explanation of the facts. (shrink)

Leading linguists and philosophers report on all aspects of compositionality, the notion that the meaning of an expression can be derived from its parts. This book explores every dimension of this field, reporting critically on different lines of research, revealing connections between them, and highlighting current problems and opportunities.

Hayek s philosophical psychology as set out in his The Sensory Order (1952) has, for the most part, been neglected. Despite being lauded by computer scientist grandee Frank Rosenblatt and by Nobel prize-winning biologist Gerald Edelman, cognitive scientists -- with a few exceptions -- have yet to discover Hayek s philosophical psychology. On the other hand, social theorists, Hayek s traditional disciplinary constituency, have only recently begun to take note and examine the importance of psychology in the complete Hayek corpus. (...) This volume brings together for the first time state-of-the-art contributions from neuroscientists and philosophers of mind as well as economists and social theorists, all critically engaging in many aspects of Hayek's philosophical psychology. (shrink)

A comprehensive introduction to the Language of Though Hypothesis (LOTH) accessible to general audiences. LOTH is an empirical thesis about thought and thinking. For their explication, it postulates a physically realized system of representations that have a combinatorial syntax (and semantics) such that operations on representations are causally sensitive only to the syntactic properties of representations. According to LOTH, thought is, roughly, the tokening of a representation that has a syntactic (constituent) structure with an appropriate semantics. Thinking thus consists in (...) syntactic operations defined over representations. Most of the arguments for LOTH derive their strength from their ability to explain certain empirical phenomena like productivity, systematicity of thought and thinking. (shrink)

Robert Cummins has argued that Jerry Fodor’s well-known systematicity argument begs the question. I show that the systematicity argument for thought structure does not beg the question, nor run in either explanatory nor inferential circles, nor illegitimately project sentence structure onto thoughts. Because the evidence does not presuppose that thought has structure, connectionist explanations of the same interconnections between thoughts are at least possibilities. Butthey are likely to be ad hoc.

The Systematicity Arguments is the only book-length treatment of the systematicity and productivity arguments.

While neuroscientists garner success in identifying brain regions and in analyzing individual neurons, ground is still being broken at the intermediate scale of understanding how neurons combine to encode information. This book proposes a method of representing information in a computer that would be suited for modeling the brain's methods of processing information. Holographic Reduced Representations (HRRs) are introduced here to model how the brain distributes each piece of information among thousands of neurons. It had been previously thought that the (...) grammatical structure of a language cannot be encoded practically in a distributed representation, but HRRs can overcome the problems of earlier proposals. Thus this work has implications for psychology, neuroscience, linguistics, and computer science, and engineering. (shrink)

One of the hallmarks of human cognition is the capacity to generalize over arbitrary constituents. Recently, Marcus (1998, 1998a, b; Cognition 66, p. 153; Cognitive Psychology 37, p. 243) argued that this capacity, called universal generalization (universality), is not supported by Connectionist models. Instead, universality is best explained by Classical symbol systems, with Connectionism as its implementation. Here it is argued that universality is also a problem for Classicism in that the syntax-sensitive rules that are supposed to provide causal explanations (...) of mental processes are either too strict, precluding possible generalizations; or too lax, providing no information as to the appropriate alternative. Consequently, universality is not explained by a Classical theory. (shrink)

Running head: Functional neuroimaging Abstract Several recently developed techniques enable the investigation of the neural basis of cognitive function in the human brain. Two of these, PET and fMRI, yield whole-brain images reflecting regional neural activity associated with the performance of specific tasks. This article explores the spatial and temporal capabilities and limitations of these techniques, and discusses technical, biological, and cognitive issues relevant to understanding the goals and methods of neuroimaging studies. The types of advances in understanding cognitive and (...) brain function made possible with these methods are illustrated with examples from the neuroimaging literature. (shrink)

This paper investigates connectionism's potential to solve the frame problem. The frame problem arises in the context of modelling the human ability to see the relevant consequences of events in a situation. It has been claimed to be unsolvable for classical cognitive science, but easily manageable for connectionism. We will focus on a representational approach to the frame problem which advocates the use of intrinsic representations. We argue that although connectionism's distributed representations may look promising from this perspective, doubts can (...) be raised about the potential of distributed representations to allow large amounts of complexly structured information to be adequately encoded and processed. It is questionable whether connectionist models that are claimed to effectively represent structured information can be scaled up to a realistic extent. We conclude that the frame problem provides a difficulty to connectionism that is no less serious than the obstacle it constitutes for classical cognitive science. (shrink)

Martin Davies' criterion for the knowledge of implicit rules, viz. the causal systematicity of cognitive processes, is first exposed. Then the inference from causal systematicity of a process to syntactic properties of the input states is examined. It is argued that Davies' notion of a syntactic property is too weak to bear the conclusion that causal systematicity implies a language of thought as far as the input states are concerned. Next, it is shown that Davies' criterion leads to a counterintuitive (...) consequence: it groups together distributed connectionist systems with look-up tables. To avoid this consequence, a modified construal of causal systematicity is proposed and Davies' argument for the causal systematicity of thought is shown to be question-begging. It is briefly sketched how the modified construal links up with multiple dispositions of the same categorical base. Finally, the question of the causal efficacy of single rules is distinguished from the question of their psychological reality: implicit rules might be psychologically real without being causally efficacious. (shrink)

Despite the considerable attention that the systematicity argument has enjoyed, it is worthwhile examining the argument within the context of similar explanatory arguments from the history of science. This kind of analysis helps show that Connectionism, qua Connectionism, really does not have an explanation of systematicity. Second, and more surprisingly, one finds that the systematicity argument sets such a high explanatory standard that not even Classicism can explain the systematicity of thought.

Despite the considerable attention that the systematicity argument has enjoyed, it is worthwhile examining the argument within the context of similar explanatory arguments from the history of science. This kind of analysis helps show that Connectionism, qua Connectionism, really does not have an explanation of systematicity. Second, and more surprisingly, one finds that the systematicity argument sets such a high explanatory standard that not even Classicism can explain the systematicity of thought.

Despite the prominence of the systematicity argument in the debate between Classicists and Connectionists, there is extremely widespread misunderstanding of the nature of the argument. For example, Matthews (1994), has argued that the systematicity argument is a kind of trick, where Niklasson and van Gelder (1994), have claimed that it is obscure. More surprisingly, once one examines the argument carefully, one finds that Fodor, Pylyshyn, and McLaughlin, themselves have not fully understood it. 1 In part as a result of this, (...) many Connectionists who have tried to meet the challenge of explaining the systematicity of thought have been misled about what this challenge involves (e.g. Pollack 1990; Smolensky 1990; Niklasson and van Gelder 1994). I have five principal objectives in this paper. First, I want to respond to those who believe that the systematicity argument is mere obscurity by providing a clear presentation of it. Second, and at the same time, I want to respond to those who believe the argument is mere trickery by showing it to be an instance of a rather familiar form of legitimate scientific reasoning. Third, having seen what sort of reasoning is involved in the argument, I wish to indicate how certain attempts to meet the challenge of systematicity are inadequate. Fourth, having seen the explanatory standard set by the systematicity argument, I want to indicate how even Fodor, Pylyshyn, and McLaughlin, fail to see how Classicism fails to meet the standard. Fifth, and finally, I wish to indicate what consequences the foregoing considerations have for the prospects of Classicism and Connectionism. (shrink)

Fodor and Pylyshyn's critique of connectionism has posed a challenge to connectionists: Adequately explain such nomological regularities as systematicity and productivity without postulating a "language of thought" (LOT). Some connectionists like Smolensky took the challenge very seriously, and attempted to meet it by developing models that were supposed to be non-classical. At the core of these attempts lies the claim that connectionist models can provide a representational system with a combinatorial syntax and processes sensitive to syntactic structure. They are not (...) implementation models because, it is claimed, the way they obtain syntax and structure sensitivity is not "concatenative," hence "radically different" from the way classicists handle them. In this paper, I offer an analysis of what it is to physically satisfy/realize a formal system. In this context, I examine the minimal truth-conditions of LOT Hypothesis. From my analysis it will follow that concatenative realization of formal systems is irrelevant to LOTH since the very notion of LOT is indifferent to such an implementation level issue as concatenation. I will conclude that to the extent to which they can explain the law-like cognitive regularities, a certain class of connectionist models proposed as radical alternatives to the classical LOT paradigm will in fact turn out to be LOT models, even though new and potentially very exciting ones. (shrink)

Classical computational modellers of mind urge that the mind is something like a von Neumann computer operating over a system of symbols constituting a language of thought. Such an architecture, they argue, presents us with the best explanation of the compositionality, systematicity and productivity of thought. The language of thought hypothesis is supported by additional independent arguments made popular by Jerry Fodor. Paul Smolensky has developed a connectionist architecture he claims adequately explains compositionality, systematicity and productivity without positing any language (...) of thought, and without positing any operations over a set of symbols. This architecture encodes the information represented in linguistic trees without explicitly representing those trees or their constituents, and indeed without employing any representational vehicles with constituent structure. In a recent article, Fodor (1997; Connectionism and systematicity, Cognition , 62, 109-119) argues that Smolensky's proposal does not work. I defend Smolensky against Fodor's attack, and use this interchange as a vehicle for exploring and criticising the “Language of Thought” hypothesis more generally and the arguments Fodor adduces on its behalf. (shrink)

In his discussion of results which I (with Michael Hayward) recently reported in this journal, Kenneth Aizawa takes issue with two of our conclusions, which are: (a) that our connectionist model provides a basis for explaining systematicity within the realm of sentence comprehension, and subject to a limited range of syntax (b) that the model does not employ structure-sensitive processing, and that this is clearly true in the early stages of the network''s training. Ultimately, Aizawa rejects both (a) and (b) (...) for reasons which I think are ill-founded. In what follows, I offer a defense of our position. In particular, I argue (1) that Aizawa adopts a standard of explanation that many accepted scientific explanations could not meet, and (2) that Aizawa misconstrues the relevant meaning of structure-sensitive process. (shrink)

Fodor's and Pylyshyn's stand on systematicity in thought and language has been debated and criticized. Van Gelder and Niklasson, among others, have argued that Fodor and Pylyshyn offer no precise definition of systematicity. However, our concern here is with a learning based formulation of that concept. In particular, Hadley has proposed that a network exhibits strong semantic systematicity when, as a result of training, it can assign appropriate meaning representations to novel sentences (both simple and embedded) which contain words in (...) syntactic positions they did not occupy during training. The experience of researchers indicates that strong systematicity in any form is difficult to achieve in connectionist systems.Herein we describe a network which displays strong semantic systematicity in response to Hebbian, connectionist training. During training, two-thirds of all nouns are presented only in a single syntactic position (either as grammatical subject or object). Yet, during testing, the network correctly interprets thousands of sentences containing those nouns in novel positions. In addition, the network generalizes to novel levels of embedding. Successful training requires a, corpus of about 1000 sentences, and network training is quite rapid. The architecture and learning algorithms are purely connectionist, but classical insights are discernible in one respect, viz, that complex semantic representations spatially contain their semantic constituents. However, in other important respects, the architecture is distinctly non-classical. (shrink)

Figure 1: A pr ototyp ical exa mple of a three-layer feed forward network, used by Plunkett and M archm an (1 991 ) to simulate learning the past-tense of En glish verbs. The inpu t units encode representations of the three phonemes of the present tense of the artificial words used in this simulation. Th e netwo rk is trained to produce a representation of the phonemes employed in the past tense form and the suffix (/d/, /ed/, or /t/) (...) used on regular verbs. To run the network, each input unit is assigned an activation value o f 0 or 1 , dep ending on whethe r the featu re is present or not. Eac h input unit is conne cted to each of the 30 hidden units by a we ighted conn ection and p rovid es an inp ut to each hidden unit equal to the product of the input unit's activation and the weight. Each hidd en unit's activation is then determined by summing ov er the va lues co ming fro m each inp ut unit to deter mine a netinput, and then applying a non-linear function (e.g., the logistic function 1/(1+e^netinput). Th is whole proced ure is. (shrink)

The paper examines an alleged distinction claimed to exist by Van Gelder between two different, but equally acceptable ways of accounting for the systematicity of cognitive output (two varieties of compositionality): concatenative compositionality vs. functional compositionality. The second is supposed to provide an explanation alternative to the Language of Thought Hypothesis. I contend that, if the definition of concatenative compositionality is taken in a different way from the official one given by Van Gelder (but one suggested by some of his (...) formulations) then there is indeed a different sort of compositionality; however, the second variety is not an alternative to the language of thought in that case. On the other hand, if the concept of concatenative compositionality is taken in a different way, along the lines of Van Gelder's explicit definition, then there is no reason to think that there is an alternative way of explaining systematicity. (shrink)

This paper responds to criticisms levelled by Fodor, Pylyshyn, and McLaughlin against connectionism. Specifically, I will rebut the charge that connectionists cannot account for representational systematicity without implementing a classical architecture. This will be accomplished by drawing on Paul Smolensky's Tensor Product model of representation and on his insights about split-level architectures.

Fodor and Pylyshyn (1988), Fodor and McLaughlin (1990) and McLaughlin (1993) challenge connectionists to explain systematicity without simply implementing a classical architecture. In this paper I argue that what makes the challenge difficult for connectionists to meet has less to do with what is to be explained than with what is to count as an explanation. Fodor et al. are prepared to admit as explanatory, accounts of a sort that only classical models can provide. If connectionists are to meet the (...) challenge, they are going to have to insist on the propriety of changing what counts as an explanation of systematicity. Once that is done, there would seem to be as yet no reason to suppose that connectionists are unable to explain systematicity. (shrink)

In 1988 Fodor and Pylyshyn issued a challenge to the newly-popular connectionism: explain the systematicity of cognition without merely implementing a so-called classical architecture. Since that time quite a number of connectionist models have been put forward, either by their designers or by others, as in some measure demonstrating that the challenge can be met (e.g., Pollack, 1988, 1990; Smolensky, 1990; Chalmers, 1990; Niklasson and Sharkey, 1992; Brousse, 1993). Unfortu- nately, it has generally been unclear whether these models actually do (...) have this implication (see, for instance, the extensive philosophical debate in Smolensky, 1988; Fodor and McLaughlin, 1990; van Gelder, 1990, 1991; McLaughlin, 1993a, 1993b; Clark, 1993). Indeed, we know of no major supporter of classical orthodoxy who has felt compelled, by connectionist models and argu- ments, to concede in print that connectionists have in fact delivered a non-classical explanation of systematicity. (shrink)

In an influential critique, Jerry Fodor and Zenon Pylyshyn point to the existence of a potentially devastating dilemma for connectionism (Fodor and Pylyshyn [1988]). Either connectionist models consist in mere associations of unstructured representations, or they consist in processes involving complex representations. If the former, connectionism is mere associationism, and will not be capable of accounting for very much of cognition. If the latter, then connectionist models concern only the implementation of cognitive processes, and are, therefore, not informative at the (...) level of cognition. I shall argue that Fodor and Pylyshyn's argument is based on a crucial misunderstanding, the same misunderstanding which motivates the entire language of thought hypothesis. (shrink)

Department of Computer Science Philosophy Program, Research School of Social Sciences University of Skövde, S-54128, SWEDEN Australian National University, Canberra ACT 0200.

This paper discusses the relation between cognitive and implementational levels of analysis. Chalmers (1990, 1993) argues that a connectionist implementation of a classical cognitive architecture possesses a compositional semantics, and therefore undercuts Fodor and Pylyshyn's (1988) argument that connectionist networks cannot possess a compositional semantics. I argue that Chalmers argument misconstrues the relation between cognitive and implementational levels of analysis. This paper clarifies the distinction, and shows that while Fodor and Pylyshyn's argument survives Chalmers' critique, it cannot be used to (...) establish the irrelevance of neurophysiological implementation to cognitive modeling; some aspects of Chater and Oaksford's (1990) response to Fodor and Pylyshyn, though not all, are therefore cogent. (shrink)

This paper offers both a theoretical and an experimental perspective on the relationship between connectionist and Classical (symbol-processing) models. Firstly, a serious flaw in Fodor and Pylyshyn’s argument against connectionism is pointed out: if, in fact, a part of their argument is valid, then it establishes a conclusion quite different from that which they intend, a conclusion which is demonstrably false. The source of this flaw is traced to an underestimation of the differences between localist and distributed representation. It has (...) been claimed that distributed representations cannot support systematic operations, or that if they can, then they will be mere implementations of traditional ideas. This paper presents experimental evidence against this conclusion: distributed representations can be used to support direct structure-sensitive operations, in a man- ner quite unlike the Classical approach. Finally, it is argued that even if Fodor and Pylyshyn’s argument that connectionist models of compositionality must be mere implementations were correct, then this would still not be a serious argument against connectionism as a theory of mind. (shrink)

Connectionism is currently one of the most flourishing and interdisciplinary areas of cognitive science. Drawing on research in neural computation and networks it has found applications in areas such as psychology and animal intelligence. By using types of network which attempt to mirror our own cognitive architecture, connectionism is making breakthroughs in the understanding of the human mind a real possibility.