Connectionism and Compositionality

In his famous 1982 paper, Allen Newell [22, 23] introduced the notion of knowledge level to indicate a level of analysis, and prediction, of the rational behavior of a cognitive arti cial agent. This analysis concerns the investigation about the availability of the agent knowledge, in order to pursue its own goals, and is based on the so-called Rationality Principle (an assumption according to which "an agent will use the knowledge it has of its environment to achieve its goals" [22, (...) p. 17]. By using the Newell's own words: "To treat a system at the knowledge level is to treat it as having some knowledge, some goals, and believing it will do whatever is within its power to attain its goals, in so far as its knowledge indicates" [22, p. 13]. In the last decades, the importance of the knowledge level has been historically and system- atically downsized by the research area in cognitive architectures (CAs), whose interests have been mainly focused on the analysis and the development of mechanisms and the processes governing human and (arti cial) cognition. The knowledge level in CAs, however, represents a crucial level of analysis for the development of such arti cial general systems and therefore deserves greater research attention [17]. In the following, we will discuss areas of broad agree- ment and outline the main problematic aspects that should be faced within a Common Model of Cognition [12]. Such aspects, departing from an analysis at the knowledge level, also clearly impact both lower (e.g. representational) and higher (e.g. social) levels. (shrink)

We describe and try to motivate our project to build systems using both a knowledge based and a neural network approach. These two approaches are used at different stages in the solution of a problem, instead of using knowledge bases exclusively on some problems, and neural nets exclusively on others. The knowledge base (KB) is defined first in a declarative, symbolic language that is easy to use. It is then compiled into an efficient neural network (NN) representation, run, and the (...) results from run time and (eventually) from learning are decompiled to a symbolic description of the knowledge contained in the network. After inspecting this recovered knowledge, a designer would be able to modify the KB and go through the whole cycle of compiling, running, and decompiling again. The central question with which this project is concerned is, therefore, How do we go from a KB to an NN, and back again? We are investigating this question by building tools consisting of a repertoire of language/translation/network types, and trying them on problems in a variety of domains. (shrink)

The systematicity argument only challenges connectionism if systematicity is a general property of cognition. I examine this thesis in terms of properties of concepts. First, I propose that Evans's Generality Constraint only applies to attributions of belief. Then I defend a variety of conceptual pluralism, arguing that concepts share two fundamental properties related to centrality and belief-attribution, and contending that there are two kinds of concepts that differ in their compositional properties. Finally, I rely on Dual Systems Theory and on (...) differences between animal and human cognition to suggest a scenario of two processing systems that work on different kinds of concepts, with only one of them supporting full systematicity. I sketch a non-classical systematicity argument that rules out classicism as the basis of one of those systems given that it would wrongly entail that both systems are fully systematic. (shrink)

Martin Davies' criterion for the knowledge of implicit rules, viz. the causal systematicity of cognitive processes, is first exposed. Then the inference from causal systematicity of a process to syntactic properties of the input states is examined. It is argued that Davies' notion of a syntactic property is too weak to bear the conclusion that causal systematicity implies a language of thought as far as the input states are concerned. Next, it is shown that Davies' criterion leads to a counterintuitive (...) consequence: it groups together distributed connectionist systems with look-up tables. To avoid this consequence, a modified construal of causal systematicity is proposed and Davies' argument for the causal systematicity of thought is shown to be question-begging. It is briefly sketched how the modified construal links up with multiple dispositions of the same categorical base. Finally, the question of the causal efficacy of single rules is distinguished from the question of their psychological reality: implicit rules might be psychologically real without being causally efficacious. (shrink)

A comprehensive introduction to the Language of Though Hypothesis (LOTH) accessible to general audiences. LOTH is an empirical thesis about thought and thinking. For their explication, it postulates a physically realized system of representations that have a combinatorial syntax (and semantics) such that operations on representations are causally sensitive only to the syntactic properties of representations. According to LOTH, thought is, roughly, the tokening of a representation that has a syntactic (constituent) structure with an appropriate semantics. Thinking thus consists in (...) syntactic operations defined over representations. Most of the arguments for LOTH derive their strength from their ability to explain certain empirical phenomena like productivity, systematicity of thought and thinking. (shrink)

The paper examines an alleged distinction claimed to exist by Van Gelder between two different, but equally acceptable ways of accounting for the systematicity of cognitive output (two varieties of compositionality): concatenative compositionality vs. functional compositionality. The second is supposed to provide an explanation alternative to the Language of Thought Hypothesis. I contend that, if the definition of concatenative compositionality is taken in a different way from the official one given by Van Gelder (but one suggested by some of his (...) formulations) then there is indeed a different sort of compositionality; however, the second variety is not an alternative to the language of thought in that case. On the other hand, if the concept of concatenative compositionality is taken in a different way, along the lines of Van Gelder's explicit definition, then there is no reason to think that there is an alternative way of explaining systematicity. (shrink)

Smolensky's (1995) proposal for a connectionist explanation of systematicity doesn't work.

In his discussion of results which I (with Michael Hayward) recently reported in this journal, Kenneth Aizawa takes issue with two of our conclusions, which are: (a) that our connectionist model provides a basis for explaining systematicity within the realm of sentence comprehension, and subject to a limited range of syntax (b) that the model does not employ structure-sensitive processing, and that this is clearly true in the early stages of the network''s training. Ultimately, Aizawa rejects both (a) and (b) (...) for reasons which I think are ill-founded. In what follows, I offer a defense of our position. In particular, I argue (1) that Aizawa adopts a standard of explanation that many accepted scientific explanations could not meet, and (2) that Aizawa misconstrues the relevant meaning of structure-sensitive process. (shrink)

Figure 1: A pr ototyp ical exa mple of a three-layer feed forward network, used by Plunkett and M archm an (1 991 ) to simulate learning the past-tense of En glish verbs. The inpu t units encode representations of the three phonemes of the present tense of the artificial words used in this simulation. Th e netwo rk is trained to produce a representation of the phonemes employed in the past tense form and the suffix (/d/, /ed/, or /t/) (...) used on regular verbs. To run the network, each input unit is assigned an activation value o f 0 or 1 , dep ending on whethe r the featu re is present or not. Eac h input unit is conne cted to each of the 30 hidden units by a we ighted conn ection and p rovid es an inp ut to each hidden unit equal to the product of the input unit's activation and the weight. Each hidd en unit's activation is then determined by summing ov er the va lues co ming fro m each inp ut unit to deter mine a netinput, and then applying a non-linear function (e.g., the logistic function 1/(1+e^netinput). Th is whole proced ure is. (shrink)

Fodor and Pylyshyn's critique of connectionism has posed a challenge to connectionists: Adequately explain such nomological regularities as systematicity and productivity without postulating a "language of thought" (LOT). Some connectionists like Smolensky took the challenge very seriously, and attempted to meet it by developing models that were supposed to be non-classical. At the core of these attempts lies the claim that connectionist models can provide a representational system with a combinatorial syntax and processes sensitive to syntactic structure. They are not (...) implementation models because, it is claimed, the way they obtain syntax and structure sensitivity is not "concatenative," hence "radically different" from the way classicists handle them. In this paper, I offer an analysis of what it is to physically satisfy/realize a formal system. In this context, I examine the minimal truth-conditions of LOT Hypothesis. From my analysis it will follow that concatenative realization of formal systems is irrelevant to LOTH since the very notion of LOT is indifferent to such an implementation level issue as concatenation. I will conclude that to the extent to which they can explain the law-like cognitive regularities, a certain class of connectionist models proposed as radical alternatives to the classical LOT paradigm will in fact turn out to be LOT models, even though new and potentially very exciting ones. (shrink)

Fodor's and Pylyshyn's stand on systematicity in thought and language has been debated and criticized. Van Gelder and Niklasson, among others, have argued that Fodor and Pylyshyn offer no precise definition of systematicity. However, our concern here is with a learning based formulation of that concept. In particular, Hadley has proposed that a network exhibits strong semantic systematicity when, as a result of training, it can assign appropriate meaning representations to novel sentences (both simple and embedded) which contain words in (...) syntactic positions they did not occupy during training. The experience of researchers indicates that strong systematicity in any form is difficult to achieve in connectionist systems.Herein we describe a network which displays strong semantic systematicity in response to Hebbian, connectionist training. During training, two-thirds of all nouns are presented only in a single syntactic position (either as grammatical subject or object). Yet, during testing, the network correctly interprets thousands of sentences containing those nouns in novel positions. In addition, the network generalizes to novel levels of embedding. Successful training requires a, corpus of about 1000 sentences, and network training is quite rapid. The architecture and learning algorithms are purely connectionist, but classical insights are discernible in one respect, viz, that complex semantic representations spatially contain their semantic constituents. However, in other important respects, the architecture is distinctly non-classical. (shrink)

Fodor and Pylyshyn (1988), Fodor and McLaughlin (1990) and McLaughlin (1993) challenge connectionists to explain systematicity without simply implementing a classical architecture. In this paper I argue that what makes the challenge difficult for connectionists to meet has less to do with what is to be explained than with what is to count as an explanation. Fodor et al. are prepared to admit as explanatory, accounts of a sort that only classical models can provide. If connectionists are to meet the (...) challenge, they are going to have to insist on the propriety of changing what counts as an explanation of systematicity. Once that is done, there would seem to be as yet no reason to suppose that connectionists are unable to explain systematicity. (shrink)

This paper investigates connectionism's potential to solve the frame problem. The frame problem arises in the context of modelling the human ability to see the relevant consequences of events in a situation. It has been claimed to be unsolvable for classical cognitive science, but easily manageable for connectionism. We will focus on a representational approach to the frame problem which advocates the use of intrinsic representations. We argue that although connectionism's distributed representations may look promising from this perspective, doubts can (...) be raised about the potential of distributed representations to allow large amounts of complexly structured information to be adequately encoded and processed. It is questionable whether connectionist models that are claimed to effectively represent structured information can be scaled up to a realistic extent. We conclude that the frame problem provides a difficulty to connectionism that is no less serious than the obstacle it constitutes for classical cognitive science. (shrink)

In an influential critique, Jerry Fodor and Zenon Pylyshyn point to the existence of a potentially devastating dilemma for connectionism (Fodor and Pylyshyn [1988]). Either connectionist models consist in mere associations of unstructured representations, or they consist in processes involving complex representations. If the former, connectionism is mere associationism, and will not be capable of accounting for very much of cognition. If the latter, then connectionist models concern only the implementation of cognitive processes, and are, therefore, not informative at the (...) level of cognition. I shall argue that Fodor and Pylyshyn's argument is based on a crucial misunderstanding, the same misunderstanding which motivates the entire language of thought hypothesis. (shrink)

Classical computational modellers of mind urge that the mind is something like a von Neumann computer operating over a system of symbols constituting a language of thought. Such an architecture, they argue, presents us with the best explanation of the compositionality, systematicity and productivity of thought. The language of thought hypothesis is supported by additional independent arguments made popular by Jerry Fodor. Paul Smolensky has developed a connectionist architecture he claims adequately explains compositionality, systematicity and productivity without positing any language (...) of thought, and without positing any operations over a set of symbols. This architecture encodes the information represented in linguistic trees without explicitly representing those trees or their constituents, and indeed without employing any representational vehicles with constituent structure. In a recent article, Fodor (1997; Connectionism and systematicity, Cognition , 62, 109-119) argues that Smolensky's proposal does not work. I defend Smolensky against Fodor's attack, and use this interchange as a vehicle for exploring and criticising the “Language of Thought” hypothesis more generally and the arguments Fodor adduces on its behalf. (shrink)

This paper responds to criticisms levelled by Fodor, Pylyshyn, and McLaughlin against connectionism. Specifically, I will rebut the charge that connectionists cannot account for representational systematicity without implementing a classical architecture. This will be accomplished by drawing on Paul Smolensky's Tensor Product model of representation and on his insights about split-level architectures.

One of the hallmarks of human cognition is the capacity to generalize over arbitrary constituents. Recently, Marcus (1998, 1998a, b; Cognition 66, p. 153; Cognitive Psychology 37, p. 243) argued that this capacity, called universal generalization (universality), is not supported by Connectionist models. Instead, universality is best explained by Classical symbol systems, with Connectionism as its implementation. Here it is argued that universality is also a problem for Classicism in that the syntax-sensitive rules that are supposed to provide causal explanations (...) of mental processes are either too strict, precluding possible generalizations; or too lax, providing no information as to the appropriate alternative. Consequently, universality is not explained by a Classical theory. (shrink)

Robert Cummins has argued that Jerry Fodor’s well-known systematicity argument begs the question. I show that the systematicity argument for thought structure does not beg the question, nor run in either explanatory nor inferential circles, nor illegitimately project sentence structure onto thoughts. Because the evidence does not presuppose that thought has structure, connectionist explanations of the same interconnections between thoughts are at least possibilities. Butthey are likely to be ad hoc.

This volume provides an introduction to and review of key contemporary debates concerning connectionism, and the nature of explanation and methodology in cognitive psychology. The first debate centers on the question of whether human cognition is best modeled by classical or by connectionist architectures. The second centres on the question of the compatibility between folk, or commonsense, psychological explanation and explanations based on connectionist models of cognition. Each of the two sections includes a classic reading along with important responses, and (...) concludes with a specially commissioned reply by the main contributor. The editorial introductions provide a comprehensive survey and map through the debates. (shrink)

Leading linguists and philosophers report on all aspects of compositionality, the notion that the meaning of an expression can be derived from its parts. This book explores every dimension of this field, reporting critically on different lines of research, revealing connections between them, and highlighting current problems and opportunities.

The “systematicity argument” has been used to argue for a classical cognitive architecture (Fodor in The Language of Thought. Harvester Press, London, 1975, Why there still has to be a language of thought? In Psychosemantics, appendix. MIT Press, Cambridge, pp 135–154, 1987; Fodor and Pylyshyn in Cognition 28:3–71, 1988; Aizawa in The systematicity arguments. Kluwer Academic Press, Dordrecht, 2003). From the premises that cognition is systematic and that the best/only explanation of systematicity is compositional structure, it concludes that cognition is (...) to be explained in terms of symbols (in a language of thought) and formal rules. The debate, with connectionism, has mostly centered on the second premise-whether an explanation of systematicity requires compositional structure, which neural networks do not to exhibit (for example, Hadley and Hayward, in Minds and Machines, 7:1–37). In this paper, I will take issue with the first premise. Several arguments will be deployed that show that cognition is not systematic in general; that, in fact, systematicity seems to be related to language. I will argue that it is just verbal minds that are systematic, and they are so because of the structuring role of language in cognition. A dual-process theory of cognition will be defended as the best explanation of the facts. (shrink)

Hayek s philosophical psychology as set out in his The Sensory Order (1952) has, for the most part, been neglected. Despite being lauded by computer scientist grandee Frank Rosenblatt and by Nobel prize-winning biologist Gerald Edelman, cognitive scientists -- with a few exceptions -- have yet to discover Hayek s philosophical psychology. On the other hand, social theorists, Hayek s traditional disciplinary constituency, have only recently begun to take note and examine the importance of psychology in the complete Hayek corpus. (...) This volume brings together for the first time state-of-the-art contributions from neuroscientists and philosophers of mind as well as economists and social theorists, all critically engaging in many aspects of Hayek's philosophical psychology. (shrink)

Instead of using low-level neurophysiology mimicking and exploratory programming methods commonly used in the machine consciousness field, the hierarchical Operational Architectonics (OA) framework of brain and mind functioning proposes an alternative conceptual-theoretical framework as a new direction in the area of model-driven machine (robot) consciousness engineering. The unified brain-mind theoretical OA model explicitly captures (though in an informal way) the basic essence of brain functional architecture, which indeed constitutes a theory of consciousness. The OA describes the neurophysiological basis of the (...) phenomenal level of brain organization. In this context the problem of producing man-made “machine” consciousness and “artificial” thought is a matter of duplicating all levels of the operational architectonics hierarchy (with its inherent rules and mechanisms) found in the brain electromagnetic field. We hope that the conceptual-theoretical framework described in this paper will stimulate the interest of mathematicians and/or computer scientists to abstract and formalize principles of hierarchy of brain operations which are the building blocks for phenomenal consciousness and thought. (shrink)

The Systematicity Arguments is the only book-length treatment of the systematicity and productivity arguments.

Running head: Functional neuroimaging Abstract Several recently developed techniques enable the investigation of the neural basis of cognitive function in the human brain. Two of these, PET and fMRI, yield whole-brain images reflecting regional neural activity associated with the performance of specific tasks. This article explores the spatial and temporal capabilities and limitations of these techniques, and discusses technical, biological, and cognitive issues relevant to understanding the goals and methods of neuroimaging studies. The types of advances in understanding cognitive and (...) brain function made possible with these methods are illustrated with examples from the neuroimaging literature. (shrink)

Department of Computer Science Philosophy Program, Research School of Social Sciences University of Skövde, S-54128, SWEDEN Australian National University, Canberra ACT 0200.

In 1988 Fodor and Pylyshyn issued a challenge to the newly-popular connectionism: explain the systematicity of cognition without merely implementing a so-called classical architecture. Since that time quite a number of connectionist models have been put forward, either by their designers or by others, as in some measure demonstrating that the challenge can be met (e.g., Pollack, 1988, 1990; Smolensky, 1990; Chalmers, 1990; Niklasson and Sharkey, 1992; Brousse, 1993). Unfortu- nately, it has generally been unclear whether these models actually do (...) have this implication (see, for instance, the extensive philosophical debate in Smolensky, 1988; Fodor and McLaughlin, 1990; van Gelder, 1990, 1991; McLaughlin, 1993a, 1993b; Clark, 1993). Indeed, we know of no major supporter of classical orthodoxy who has felt compelled, by connectionist models and argu- ments, to concede in print that connectionists have in fact delivered a non-classical explanation of systematicity. (shrink)

This paper discusses the relation between cognitive and implementational levels of analysis. Chalmers (1990, 1993) argues that a connectionist implementation of a classical cognitive architecture possesses a compositional semantics, and therefore undercuts Fodor and Pylyshyn's (1988) argument that connectionist networks cannot possess a compositional semantics. I argue that Chalmers argument misconstrues the relation between cognitive and implementational levels of analysis. This paper clarifies the distinction, and shows that while Fodor and Pylyshyn's argument survives Chalmers' critique, it cannot be used to (...) establish the irrelevance of neurophysiological implementation to cognitive modeling; some aspects of Chater and Oaksford's (1990) response to Fodor and Pylyshyn, though not all, are therefore cogent. (shrink)

This paper offers both a theoretical and an experimental perspective on the relationship between connectionist and Classical (symbol-processing) models. Firstly, a serious flaw in Fodor and Pylyshyn’s argument against connectionism is pointed out: if, in fact, a part of their argument is valid, then it establishes a conclusion quite different from that which they intend, a conclusion which is demonstrably false. The source of this flaw is traced to an underestimation of the differences between localist and distributed representation. It has (...) been claimed that distributed representations cannot support systematic operations, or that if they can, then they will be mere implementations of traditional ideas. This paper presents experimental evidence against this conclusion: distributed representations can be used to support direct structure-sensitive operations, in a man- ner quite unlike the Classical approach. Finally, it is argued that even if Fodor and Pylyshyn’s argument that connectionist models of compositionality must be mere implementations were correct, then this would still not be a serious argument against connectionism as a theory of mind. (shrink)