The commentaries on our article encourage us to believe that researchers are beginning to take seriously the goal of achieving the broad adequacy that Newell aspired to. The commentators offer useful elaborations to the criteria we suggested for the Newell Test. We agree with many of the commentators that classical connectionism is too restrictive to achieve this broad adequacy, and that other connectionist approaches are not so limited and can deal with the symbolic components of thought. All these approaches, including (...) ACT-R, need to accept the idea that progress in science is a matter of better approximating these goals, and it is premature to be making judgments of true or false. (shrink)

Newell (1980; 1990) proposed that cognitive theories be developed in an effort to satisfy multiple criteria and to avoid theoretical myopia. He provided two overlapping lists of 13 criteria that the human cognitive architecture would have to satisfy in order to be functional. We have distilled these into 12 criteria: flexible behavior, real-time performance, adaptive behavior, vast knowledge base, dynamic behavior, knowledge integration, natural language, learning, development, evolution, and brain realization. There would be greater theoretical progress if we evaluated theories (...) by a broad set of criteria such as these and attended to the weaknesses such evaluations revealed. To illustrate how theories can be evaluated we apply these criteria to both classical connectionism (McClelland & Rumelhart 1986; Rumelhart & McClelland 1986b) and the ACT-R theory (Anderson & Lebiere 1998). The strengths of classical connectionism on this test derive from its intense effort in addressing empirical phenomena in such domains as language and cognitive development. Its weaknesses derive from its failure to acknowledge a symbolic level to thought. In contrast, ACT-R includes both symbolic and subsymbolic components. The strengths of the ACT-R theory derive from its tight integration of the symbolic component with the subsymbolic component. Its weaknesses largely derive from its failure, as yet, to adequately engage in intensive analyses of issues related to certain criteria on Newell's list. Key Words: cognitive architecture; connectionism; hybrid systems; language; learning; symbolic systems. (shrink)

We present an account of processing capacity in the ACT-R theory. At the symbolic level, the number of chunks in the current goal provides a measure of relational complexity. At the subsymbolic level, limits on spreading activation, measured by the attentional parameter W, provide a theory of processing capacity, which has been applied to performance, learning, and individual differences data.

Abstract Stephen Schiffer, in his recent book, Remnants of Meaning, argues against the possibility of any compositional theory of meaning for natural language. Because the argument depends on the premise that there is no possible naturalistic reduction of the intentional to the physical, Schiffer's attack on theories of meaning is of central importance for theorists of mind. I respond to Schiffer's argument by showing that there is at least one reductive account of the mental that he has neglected to consider?the (...) computationalist account known as the Representational Theory of Mind. Not only is this view immune from the criticisms Schiffer mounts against other reductivist theories, but it solves problems that arise on Schiffer's own non?reductive account of the relation between the intentional and the physical. (shrink)

Accounts of mechanistic explanation have emphasized the importance of looking down—decomposing a mechanism into its parts and operations. Using research on visual processing as an exemplar, I illustrate how productive such research has been. But once multiple components of a mechanism have been identified, researchers also need to figure out how it is organized—they must look around and determine how to recompose the mechanism. Although researchers often begin by trying to recompose the mechanism in terms of sequential operations, they frequently (...) find that the components of a mechanism interact in complex ways involving positive and negative feedback and that the organization often exhibits highly interactive local networks linked by a few long-range connections (small-worlds organization) and power law distributions of connections. The mechanisms are themselves active systems that are perturbed by inputs but not set in motion by them. Researchers also need to look up —situate a mechanism in its context, which may be a larger mechanism that modulates its behavior. When looking down is combined with looking around and up, mechanistic research results in an integrated, multi-level perspective. (shrink)

The notion of a ‘symbol’ plays an important role in the disciplines of Philosophy, Psychology, Computer Science, and Cognitive Science. However, there is comparatively little agreement on how this notion is to be understood, either between disciplines, or even within particular disciplines. This paper does not attempt to defend some putatively ‘correct’ version of the concept of a ‘symbol.’ Rather, some terminological conventions are suggested, some constraints are proposed and a taxonomy of the kinds of issue that give (...) rise to disagreement is articulated. The goal here is to provide something like a ‘geography’ of the various notions of ‘symbol’ that have appeared in the various literatures, so as to highlight the key issues and to permit the focusing of attention upon the important dimensions. In particular, the relationship between ‘tokens’ and ‘symbols’ is addressed. The issue of designation is discussed in some detail. The distinction between simple and complex symbols is clarified and an apparently necessary condition for a system to be potentially symbol, or token bearing, is introduced. (shrink)

The relationship of mind and matter is approached in a new way in this article. This approach is based on the causal interpretation of the quantum theory, in which an electron, for example, is regarded as an inseparable union of a particle and afield. This field has, however, some new properties that can be seen to be the main sources of the differences between the quantum theory and the classical (Newtonian) theory. These new properties suggest that the field may be (...) regarded as containing objective and active information, and that the activity of this information is similar in certain key ways to the activity of information in our ordinary subjective experience. The analogy between mind and matter is thus fairly close. This analogy leads to the proposal of the general outlines of a new theory of mind, matter, and their relationship, in which the basic notion is participation rather than interaction. Although the theory can be developed mathematically in more detail, the main emphasis here is to show qualitatively how it provides a way of thinking that does not divide mind from matter, and thus leads to a more coherent understanding of such questions than is possible in the common dualistic and reductionistic approaches. These ideas may be relevant to connectionist theories and might perhaps suggest new directions for their development. (shrink)

Cognitive developmental disorders cannot be properly understood without due attention to the developmental process, and we commend the authors’simulations in this regard. We note the contribution of these simulations to the nascent field of connectionist modeling of developmental disorders and outline a set of criteria for assessing individual models in the hope of furthering future modeling efforts.

The ability to process events in their temporal and sequential context is a fundamental skill made mandatory by constant interaction with a dynamic environment. Sequence learning studies have demonstrated that subjects exhibit detailed — and often implicit — sensitivity to the sequential structure of streams of stimuli. Current connectionist models of performance in the so-called Serial Reaction Time Task (SRT), however, fail to capture the fact that sequence learning can be based not only on sensitivity to the sequential associations between (...) successive stimuli, but also on sensitivity to the associations between successive responses, and on the predictive relationships that exist between these sequences of responses and their effects in the environment. In this paper, we offer an initial exploration of an alternative architecture for sequence learning, based on the principles of Forward Models. (shrink)

Holistically ignited Hebbian models are fundamentally different from the serially organized connectionist implementations of language. This may be important for the recovery of language after injury, because connectionist models have provided useful insights into recovery of some cognitive functions. I ask whether cell assembly modelling can make an important contribution and whether the apparent incompatibility with successful connectionist modelling is a problem.

In this paper I put forward a reconstruction of the evolution of certain explanatory hypotheses on the neural basis of association and learning that are the premises of connectionism in the cybernetic age and of present-day connectionism. The main point of my reconstruction is based on two little-known case studies. The first is the project, published in 1913, of a hydraulic machine through which its author believed it was possible to simulate certain essential elements of the plasticity of nervous connections. (...) The author, S. Bent Russell, was an engineer deeply influenced by the neurological hypotheses on nervous conduction of Herbert Spencer, Max Meyer and Edward L. Thorndike. The second is the project, published in 1929, of an electromechanical machine in which the author, the psychologist J.M. Stephens, believed it was possible to embody Thorndike's law of effect. Thus both Bent Russell and Stephens referred to the principles of learning that Thorndike defined as connectionist . Their attempt was that of simulating by machines at least certain simple aspects of inhibition, association and habit formation that are typical of living organisms. I propose to situate their projects within the frame of thediscovery of a simulative (modelling) methodology which I believe might be considered an important topic of the Culture of the Artificial . Certain more recent steps toward such a methodology made by both connectionism of the 1950s and present-day connectionism are briefly pointed out in the paper. (shrink)

Although the scientific study of the mind as a distinct discipline has been around for only a short time, there are already rumblings of a fundamental change in view about what the mind is like. At the center of this controversy is a cluster of approaches that are together variously called connectionism, neural network modeling, parallel distributed processing (PDP), or dynamic systems theory. This course is an intensive introduction to the connectionist's proposals for thinking about the mind and understanding its (...) achievements. At the same time, it is an introduction to one way of conducting an interdisciplinary science where the boundaries between psychology, philosophy, neuroscience, computer science and mathematics are not set. (shrink)

This paper examines whether a classical model could be translated into a PDP network using a standard connectionist training technique called extra output learning. In Study 1, standard machine learning techniques were used to create a decision tree that could be used to classify 8124 different mushrooms as being edible or poisonous on the basis of 21 different Features (Schlimmer, 1987). In Study 2, extra output learning was used to insert this decision tree into a PDP network being trained on (...) the identical problem. An interpretation of the trained network revealed a perfect mapping from its internal structure to the decision tree, representing a precise translation of the classical theory to the connectionist model. In Study 3, a second network was trained on the mushroom problem without using extra output learning. An interpretation of this second network revealed a different algorithm for solving the mushroom problem, demonstrating that the Study 2 network was indeed a proper theory translation. (shrink)

One new tradition that has emerged from early research on autonomous robots is embodied cognitive science. This paper describes the relationship between embodied cognitive science and a related tradition, synthetic psychology. It is argued that while both are synthetic, embodied cognitive science is antirepresentational while synthetic psychology still appeals to representations. It is further argued that modern connectionism offers a medium for conducting synthetic psychology, provided that researchers analyze the internal representations that their networks develop. The paper then provides a (...) detailed example of the synthetic approach by showing how the construction (and subsequent analysis) of a connectionist network can be used to contribute to a theory of how humans solve Piaget's classic balance scale task. (shrink)

Connectionism can provide useful theories in which consciousness is the exclusive vehicle of explicit representation. The theories may not, however, handle some phenomena adequately: sense of agency, modes and contents of awareness, propositional and deliberative thought, metacognitive awareness and consciousness of self. They should, however, be useful in describing automatic, activational relations among nonpropositional conscious contents.

Research in attitudes and social cognition exemplifies van Gelder's distinction between the computational and dynamical approaches. The former emphasizes linear measurement and rational decision-making. The latter considers processes of associative memory and self-organization in attitude formation and social influence. The study of dynamical processes in social cognition has been facilitated by connectionist approaches to computation.

The theory of fast and frugal heuristics, developed in a new book called Simple Heuristics that make Us Smart (Gigerenzer, Todd, and the ABC Research Group, in press), includes two requirements for rational decision making. One is that decision rules are bounded in their rationality –- that rules are frugal in what they take into account, and therefore fast in their operation. The second is that the rules are ecologically adapted to the environment, which means that they `fit to reality.' (...) The main purpose of this article is to apply these ideas to learning rules–-methods for constructing, selecting, or evaluating competing hypotheses in science, and to the methodology of machine learning, of which connectionist learning is a special case. The bad news is that ecological validity is particularly difficult to implement and difficult to understand. The good news is that it builds an important bridge from normative psychology and machine learning to recent work in the philosophy of science, which considers predictive accuracy to be a primary goal of science. (shrink)

Ramsey, Stick and Garon (1991) argue that if the correct theory of mind is some parallel distributed processing theory, then folk psychology must be false. Their idea is that if the nodes and connections that encode one representation are causally active then all representations encoded by the same set of nodes and connections are also causally active. We present a clear, and concrete, counterexample to RSG's argument. In conclusion, we suggest that folk psychology and connectionism are best understood as complementary (...) theories. Each has different limitations, yet each will co-evolve with the other in an overlapping domain of 'normal' psychology. (shrink)

Uses connectionism (neural networks) to extract the "gist" of a story in order to represent a context going forward for the disambiguation of incoming words as a text is processed.

The importance of the Stability Problem in neurocomputing is discussed, as well as the need for the study of infinite networks. Stability must be the key ingredient in the solution of a problem by a neural network without external intervention. Infinite discrete networks seem to be the proper objects of study for a theory of neural computability which aims at characterizing problems solvable, in principle, by a neural network. Precise definitions of such problems and their solutions are given. Some consequences (...) are explored, in particular, the neural unsolvability of the Stability Problem for neural networks. (shrink)

Two categorization arguments pose particular problems for localist connectionist models. The internal representations of localist networks do not reflect the variability within categories in the environment, whereas networks with distributed internal representations do reflect this essential feature of categories. We provide a real biological example of perceptual categorization in the monkey that seems to require population coding (i.e., distributed internal representations).

In this commentary I explore nonclassical connectionism (NCC) as a coherent framework for evaluation in the spirit of the Newell Test. Focusing on knowledge integration, development, real-time performance, and flexible behavior, I argue that NCC's “within-theory rank ordering” would place subsymbolic modeling in a better position. Failure to adopt a symbolic level of thought cannot be interpreted as a weakness.

The authors, on the basis of brief arguments, have dismissed tensor networks as a viable response to Jackendoff's challenges. However, there are reasons to believe that connectionist approaches descended from tensor networks are actually very well suited to answering Jackendoff's challenges. I rebut their arguments for dismissing tensor networks and briefly compare the approaches.

I have three types of interrelated comments. First, on the choice of the proposed criteria, I argue against any list and for a system of criteria. Second, on grading, I suggest modifications with respect to consciousness and development. Finally, on the choice of “theories” for evaluation, I argue for Edelman's theory of neuronal group selection instead of connectionism (classical or not).

The employment of a particular class of computer programs known as "connectionist networks" to model mental processes is a widespread approach to research in cognitive science these days. Little has been written, however, on the precise connection that is thought to hold between such programs and actual in vivo cognitive processes such that the former can be said to "model" the latter in a scientific sense. What is more, this relation can be shown to be problematic. In this paper I (...) give a brief overview of the use of connectionist models in cognitive science, and then explore some of the statements connectionists have made about the nature of the "modeling relation" thought to hold between them and cognitive processes. Finally I show that these accounts are inadequate and that more work is necessary if connectionist networks are to be seriously regarded as scientific models of cognitive processes. (shrink)

ACT is compared with a particular type of connectionist model that cannot handle symbols and use nonbiological operations which do not learn in real time. This focus continues an unfortunate trend of straw man debates in cognitive science. Adaptive Resonance Theory, or ART-neural models of cognition can handle both symbols and subsymbolic representations, and meet the Newell criteria at least as well as connectionist models.

0. Introduction The past decade has seen Cognitive Linguistics (CL) emerge as an important, exciting and promising theoretical alternative to Chomskyan approaches to the study of language. Even so, sheer numbers and institutional inertia make it the case that most current neurolinguistic research either assumes that the Chomskyan formalist story is more or less correct (and thus that the task of neurolinguistics is to determine how the brain implements GB, for instance), or that the there are two possibilities, Chomskyanism or (...) associationism/connectionism, and that the task of neurolinguistics is to discover which is really the way the brain does it. In either case, the theoretical apparatus of CL is not being explored by neurolinguistics, and hence the promise CL holds for making genuine fruitful contact with theoretical neurobiology is not materializing as quickly as one might hope. This paper is an attempt to make some initial steps at fulfilling this promise. (shrink)

1.1 The predominant approach to cognitive modeling is still what has come to be called "computationalism" (Dietrich 1990, Harnad 1990b), the hypothesis that cognition is computation. The more recent rival approach is "connectionism" (Hanson & Burr 1990, McClelland & Rumelhart 1986), the hypothesis that cognition is a dynamic pattern of connections and activations in a "neural net." Are computationalism and connectionism really deeply different from one another, and if so, should they compete for cognitive hegemony, or should they collaborate? These (...) questions will be addressed here, in the context of an obstacle that is faced by computationalism (as well as by connectionism if it is either computational or seeks cognitive hegemony on its own): The symbol grounding problem (Harnad 1990). (shrink)

We provide the first demonstration of an artificial neural network encoding the perceptual templates that form an important component of the high level strategic understanding developed by experts. Experts have a highly refined sense of knowing where to look, what information is important and what information to ignore. The conclusions these experts reach are of a higher quality and typically made in a shorter amount of time than those of non-experts. Understanding the manifestation of such abilities in terms of both (...) the psychology of expert performance and the underlying neural mechanisms constitutes one of the most challenging problems in the cognitive sciences. Using perceptual templates we show how the amount of contextual information can change significantly even within a given task, the relationship between local and non-local contexts and finally why there is very little correlation between measures of intelligence and level of expertise in many of the most complex tasks performed by humans. (shrink)

What van Gelder calls the dynamical hypothesis is only a special case of what we here dub the general dynamical hypothesis. His terminology makes it easy to overlook important alternative dynamical approaches in cognitive science. Connectionist models typically conform to the general dynamical hypothesis, but not to van Gelder's.

A central component in the E-Z Reader model is a two-stage word processing mechanism made responsible for both the triggering of eye movements and sequential shifts of attention. We point to problems with both the verbal description of this mechanism and its computational implementation in the simulation. As an alternative, we consider the use of a connectionist processing module in combination with a more indirect form of cognitive eye-movement control.

Ramscar and colleagues (2010, this volume) describe the “feature-label-order” (FLO) effect on category learning and characterize it as a constraint on symbolic learning. I argue that FLO is neither a constraint on symbolic learning in the sense of “learning elements of a symbol system” (instead, it is an effect on nonsymbolic, association learning) nor is it, more than any other constraint on category learning, a constraint on symbolic learning in the sense of “solving the symbol grounding problem.”.

To assess O'Brien & Opie's connectionist vehicle theory of consciousness, (1) it is not enough to point to the methodological weakness of certain experiments (dichotic listening, etc.). Successful cognitive theories postulating explicit unconscious representations have to be taken into account as well. (2) The distinction between vehicle and process theories cannot be drawn in the way envisaged by the authors because a representation's explicitness depends not only on its structural but also on its processing properties. (3) The stability of an (...) activation vector is not very suitable for implementing the explicitness of a representation. (shrink)

The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the visual context impinge minutely on a (...) dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP. (shrink)

Some empirical evidence in the artificial language acquisition literature has been taken to suggest that statistical learning mechanisms are insufficient for extracting structural information from an artificial language. According to the more than one mechanism (MOM) hypothesis, at least two mechanisms are required in order to acquire language from speech: (a) a statistical mechanism for speech segmentation; and (b) an additional rule-following mechanism in order to induce grammatical regularities. In this article, we present a set of neural network studies demonstrating (...) that a single statistical mechanism can mimic the apparent discovery of structural regularities, beyond the segmentation of speech. We argue that our results undermine one argument for the MOM hypothesis. (shrink)

A rule-based expert system is demonstrated to have both a symbolic computational network representation and a sub-symbolic connectionist representation. These alternate views enhance the usefulness of the original system by facilitating introduction of connectionist learning methods into the symbolic domain. The connectionist representation learns and stores metaknowledge in highly connected subnetworks and domain knowledge in a sparsely connected expert network superstructure. The total connectivity of the neural network representation approximates that of real neural systems and hence avoids scaling and memory (...) stability problems associated with other connectionist models. (shrink)

There is a broad consensus about the fundamental role of thehippocampal system (hippocampus and its adjacent areas) in theencoding and retrieval of episodic memories. This paper presents afunctional model of this system. Although memory is not asingle-unit cognitive function, we took the view that the wholesystem of the smooth, interrelated memory processes may have acommon basis. That is why we follow the Ockham's razor principleand minimize the size or complexity of our model assumption set.The fundamental assumption is the requirement of (...) solving the socalled ``homunculus fallacy'', which addresses the issue ofinterpreting the input. Generative autoassociators seem to offer aresolution of the paradox. Learning to represent and to recallinformation, in these generative networks, imply maximization ofinformation transfer, sparse representation and noveltyrecognition. A connectionist architecture, which integrates theseaspects as model constraints, is derived. Numerical studiesdemonstrate the novelty recognition and noise filtering propertiesof the architecture. Finally, we conclude that the derivedconnectionist architecture can be related to the neurobiologicalsubstrate. (shrink)

Research into consciousness has now become respectable, and much has been written about it. Is consciousness the exclusive property of human beings, or can it be found also in animals? Can machines become conscious? Is consciousness an illusion, and are all mental states ultimately reducible to the movement of molecules? If consciousness is other than matter, what connection does it have with matter? These and others like them are now serious scientific questions in the West. This article discusses consciousness within (...) the frame of the following assertions: Consciousness has evolved from earlier states of awareness to be found in lower forms of life. The current scientific method is too restrictive for the study of conscience and its evolution. In particular classical logic leads scientists to ignore or reject consciousness as a legitimate field of study. Mind and matter, generalized as knowing and being, have equal status. (shrink)

This article critically examines the contemporary resurgence of empiricism (or “neo-empiricism”) in philosophy, psychology, neuropsychology, and artificial intelligence. This resurgence is an important and positive development. It is the first time that this centuries-old empiricist approach to cognition is precisely formulated in the context of cognitive science and neuroscience. Moreover, neo-empiricists have made several findings that challenge amodal theories of concepts and higher cognition. It is argued, however, that the theoretical foundations of and the empirical evidence for neo-empiricism are not (...) as strong as is usually claimed by its proponents. The empirical evidence for and against neo-empiricism is discussed in detail. (shrink)

O'Brien & Opie's connectionist vehicle theory of consciousness is heavily dependent on their notion of explicitness as (1) structural and (2) necessary and sufficient for consciousness. These assumptions unnecessarily constrain their position: the authors are forced to find an intrinsic property of patterns that accounts for the distinction between conscious and unconscious states. Their candidate property, stability, does not capture this distinction. Yet, I show that we can drop assumptions (1) and (2) and still develop a vehicle theory of consciousness. (...) This alternative is better served by models that incorporate both connectionist and symbolic representations. (shrink)

Some controversies in cognitive science, such as arguments about whether classical or distributed connectionist architectures best model the human cognitive system, reenact long-standing debates in the philosophy of science. For millennia philosophers have pondered whether mentality can submit to scientific explanation generally and to physical explanation particularly. Recently, positive answers have gained popularity. The question remains, though, as to the analytical level at which mentality is best explained. Is there a level of analysis that is peculiarly appropriate for the explanation (...) of either consciousness or mental contents? Are human consciousness, cognition, and conduct best understood in terms of talk about neurons and networks or schemas and scripts or intentions and inferences? If our best accounts make no appeal to our hopes or beliefs or desires, how do we square those views with our conception of ourselves as rational beings? Moreover, can models of physical processes explain our mental lives? Does mentality require a special level of rational or cognitive explanation or is it best understood in terms of overall brain functioning or neuronal or molecular or even quantum activities--or any of a dozen levels of physical explanation in between? Also, regardless of how they compare with explanations cast at physical levels, what is the status of psychological explanations that appeal fundamentally to mental contents? As a means for beginning to address such questions, proposals about cognitive architecture concern which kind of explanation best characterizes primitive psychological activities. Although, technically, approaches to modeling those activities are unlimited, two 1 strategies have enjoyed most of the attention. The prominence of the classical account and the distributed connectionist (or parallel distributed processing (PDP)) account, notwithstanding, nothing bars the development of additional proposals. Classicism employs rules that apply to symbolic representations to explain cognitive processing.. (shrink)

O'Brien & Opie's theory fails to address the issue of consciousness and introspection. They take for granted that once something is experienced, it can be commented on. But introspection requires neural structures that, according to their theory, have nothing to do with experience as such. That makes the tight coupling between the two in humans a mystery.

Connectionist models offer concretemechanisms for cognitive processes. When these modelsmimic the performance of human subjects theycan offer insights into the computationswhich might underlie human cognition. We illustratethis with the performance of a recurrentconnectionist network which produces the meaningof words in response to their spellingpattern. It mimics a paradoxical pattern oferrors produced by people trying to read degradedwords. The reason why the network produces thesurprising error pattern lies in the nature ofthe attractors which it develops as it learns tomap spelling patterns (...) to semantics. The keyrole of attractor structure in the successfulsimulation suggests that the normal adult semanticreading route may involve attractor dynamics, andthus the paradoxical error pattern isexplained. (shrink)

Exploratory analysis is an area of increasing interest in the computational linguistics arena. Pragmatically speaking, exploratory analysis may be paraphrased as natural language processing by means of analyzing large corpora of text. Concerning the analysis, appropriate means are statistics, on the one hand, and artificial neural networks, on the other hand. As a challenging application area for exploratory analysis of text corpora we may certainly identify text databases, be it information retrieval or information filtering systems. With this paper we present (...) recent findings of exploratory analysis based on both statistical and neural models applied to legal text corpora. Concerning the artificial neural networks, we rely on a model adhering to the unsupervised learning paradigm. This choice appears naturally when taking into account the specific properties of large text corpora where one is faced with the fact that input-output-mappings as required by supervised learning models cannot be provided beforehand to a satisfying extent. This is due to the fact of the highly changing contents of text archives. In a nutshell, artificial neural networks count for their highly robust behavior regarding the parameters for model optimization. In particular, we found statistical classification techniques much more susceptible to minor parameter variations than unsupervised artificial neural networks. In this paper we describe two different lines of research in exploratory analysis. First, we use the classification methods for concept analysis. The general goal is to uncover different meanings of one and the same natural language concept. A task that, obviously, is of specific importance during the creation of thesauri. As a convenient environment to present the results we selected the legal term of neutrality, which is a perfect representative of a concept having a number of highly divergent meanings. Second, we describe the classification methods in the setting of document classification. The ultimate goal in such an application is to uncover semantic similarities of various text documents in order to increase the efficiency of an information retrieval system. In this sense, document classification has its fixed position in information retrieval research from the very beginning. Nowadays renewed massive interest in document classification may be witnessed due to the appearance of large-scale digital libraries. (shrink)

Folk psychology is under threat - that is to say - our everyday conception that human beings are agents who experience the world in terms of sights, sounds, tastes, smells and feelings and who deliberate, make plans, and generally execute actions on the basis of their beliefs, needs and wants - is under threat. This threat is evidenced in intellectual circles by the growing attitude amongst some cognitive scientists that our common sense categories are in competition with connectionist theories and (...) modern neuroscience. It is often thought that either folk psychology or modern cognitive science must go. It is in these terms that the battle lines of today's philosophy of mind are drawn. (shrink)

The probabilistic analysis of functional questions is maturing into a rigorous and coherent research paradigm that may unify the cognitive sciences, from the study of single neurons in the brain to the study of high level cognitive processes and distributed cognition. Endless debates about undecidable structural issues (modularity vs. interactivity, serial vs. parallel processing, iconic vs. propositional representations, symbolic vs. connectionist models) may be put aside in favor of a rigorous understanding of the problems solved by organisms in their natural (...) environments. [Shepard; Tenenbaum & Griffiths]. (shrink)

In Connectionism and the Philosophy of Psychology, Horgan and Tienson (1996) argue that cognitive processes, pace classicism, are not governed by exceptionless, “representation-level” rules; they are instead the work of defeasible cognitive tendencies subserved by the non-linear dynamics of the brain’s neural networks. Many theorists are sympathetic with the dynamical characterisation of connectionism and the general (re)conception of cognition that it affords. But in all the excitement surrounding the connectionist revolution in cognitive science, it has largely gone unnoticed that connectionism (...) adds to the traditional focus on computational processes, a new focus – one on the vehicles of mental representation, on the entities that carry content through the mind. Indeed, if Horgan and Tienson’s dynamical characterisation of connectionism is on the right track, then so intimate is the relationship between computational processes and representational vehicles, that connectionist cognitive science is committed to a resemblance theory of mental content. (shrink)

This commentary questions the applicability of the Newell Test for evaluating the utility of connectionism. Rather than being a specific theory of cognition (because connectionism can be used to model nativist, behaviorist, or constructivist theories), connectionism, we argue, offers researchers a collection of computational and conceptual tools that are particularly useful for investigating and rendering specific fundamental issues of human development. These benefits of connectionism are not well captured by evaluating it against Newell's criteria for a unified theory of cognition.

Over the last decade, fully distributed models have become dominant in connectionist psychological modelling, whereas the virtues of localist models have been underestimated. This target article illustrates some of the benefits of localist modelling. Localist models are characterized by the presence of localist representations rather than the absence of distributed representations. A generalized localist model is proposed that exhibits many of the properties of fully distributed models. It can be applied to a number of problems that are difficult for fully (...) distributed models, and its applicability can be extended through comparisons with a number of classic mathematical models of behaviour. There are reasons why localist models have been underused, though these often misconstrue the localist position. In particular, many conclusions about connectionist representation, based on neuroscientific observation, can be called into question. There are still some problems inherent in the application of fully distributed systems and some inadequacies in proposed solutions to these problems. In the domain of psychological modelling, localist modelling is to be preferred. Key Words: choice; competition; connectionist modelling; consolidation; distributed; localist; neural networks; reaction-time. (shrink)

To deal with sequential decision tasks we present a learning model Clarion which is a hybrid connectionist model consisting of both localist and distributed represen tations based on the two level approach proposed in Sun The model learns and utilizes procedural and declarative knowledge tapping into the synergy of the two types of processes It uni es neural reinforcement and symbolic methods to perform on line bottom up learning Experiments in various situations are reported that shed light on the working (...) of the model.. (shrink)

Forthcoming in Cognitive Architecture: from bio-politics to noo-politics, eds. Deborah Hauptmann, Warren Neidich and Abdul-Karim Mustapha INTRODUCTION The cognitive and affective sciences have benefitted in the last twenty years from a rethinking of the long-dominant computer model of the mind espoused by the standard approaches of computationalism and connectionism. The development of this alternative, often named the “embodied mind” approach or the “4EA” approach (embodied, embedded, enactive, extended, affective), has relied on a trio of classical 20th century phenomenologists for its (...) philosophical framework: Husserl, Heidegger, and Merleau-Ponty.[1] In this essay I propose that the French thinker Gilles Deleuze can provide the conceptual framework that will enable us to thematize some unstated presuppositions of the 4EA School, as well as to sharpen, extend and / or radicalize some of their explicit presuppositions. I highlight three areas here: 1) an ontology of distributed and differential systems, using Deleuze’s notion of the virtual; 2) a thought of multiple subjectification practices rather than a thought of “the” subject, even if it be seen as embodied and embedded; and 3) a rethinking of the notion of affect in order to thematize a notion of “political affect.”[2] I will develop this proposal with reference to Bruce Wexler’s Brain and Culture,[3] a work which resonates superbly with the Deleuzean approach. (shrink)

The target article is based upon the principle that complex mental phenomena result from the interactions among some elementary entities. Connectionist nodes and ACT-R's production rules can be considered as such entities. However, before testing against Newell's macro-criteria, self-organizing models must be tested against criteria relating to the properties of their elementary entities. When such micro-criteria are considered, they separate connectionism from ACT-R and the comparison of these theories against Newell's Tests is hardly correct.

In her Behavioral and Brain Sciences target article, Greenfield (1991) proposed that early in a child's development Broca's area may serve the dual function of coordinating object assembly and organizing the production of structured utterances. As development progresses, the upper and lower regions of Broca's area become increasingly specialized for motor coordination and speech, respectively. This commentary presents a connectionist simulation of aspects of this proposal. The results of the simulation confirm the main thrust of Greenfield's argument and suggest that (...) an important impetus for the developmental differentiation in Broca's area may be the increasing complexity of the computational demands made upon it. (shrink)

I distinguish between being cognisant and being able to perform intelligent operations. The former, but not the latter, minimally involves the capacity to make adequate judgements about one's relation to objects in the environment. The referential nature of cognisance entails that the mental states of cognisant systems must be inter-related holistically, such that an individual thought becomes possible because of its relation to a system of potential thoughts. I use Gareth Evans' 'Generality Constraint' as a means of describing how the (...) reference and holism of mental states in cognisant systems are mutually dependent. Next, I describe attempts to deny the relevance of holism and reference by positing a mentalese. These attempts fail because the meanings of symbols are under determined, with there being no principled means of distinguishing between the mental tokening of a symbol and its disambiguation. I argue that the connectionist meta-theory does not encounter this problem because it is able to encompass the holism of the mental. Recent attempts to show that symbol processing theories of thought must be preferred to connectionist theories do not work. Despite appearances to the contrary, the Generality Constraint favours connectionist not symbol-processing theories. (shrink)

In Representation Reconsidered , William Ramsey suggests that the notion of structural representation is posited by classical theories of cognition, but not by the ‘newer accounts’ (e.g. connectionist modeling). I challenge the assertion about the newer accounts. I argue that the newer accounts also posit structural representations; in fact, the notion plays a key theoretical role in the current computational approaches in cognitive neuroscience. The argument rests on a close examination of computational work on the oculomotor system.

Cognitive function certainly poses the biggest challenge for computational neuroscience. As we argue, past efforts to build neural models of cognition (the target article included) had too narrow a focus on implementing rule-based language processing. The problem with these models is that they sacrifice the advantages of connectionism rather than building on them. Recent and more promising approaches for modeling cognition build on the mathematical properties of distributed neural representations. These approaches truly exploit the key advantages of connectionism, that is, (...) the high representational power of distributed neural codes and similarity-based pattern recognition. The architectures for cognitive computing that emerge from these approaches are neural associative memories endowed with additional mapping operations to handle invariances and to form reduced representations of combinatorial structures. (shrink)

I propose a new argument showing that conscious vision sometimes depends constitutively on conscious attention. I criticise traditional arguments for this constitutive connection, on the basis that they fail adequately to dissociate evidence about visual consciousness from evidence about attention. On the same basis, I criticise Ned Block's recent counterargument that conscious vision is independent of one sort of attention (‘cognitive access'). Block appears to achieve the dissociation only because he underestimates the indeterminacy of visual consciousness. I then appeal to (...) empirical work on the interaction between visual indeterminacy and attention, to argue for the constitutive connection. (shrink)

Current artificial neural network or connectionist models of music cognition embody feature-extraction and feature-weighting principles. This paper reports two experiments which seek evidence for similar processes mediating recognition of short musical compositions by musically trained and untrained listeners. The experiments are cast within a pattern recognition framework based on the vision-audition analogue wherein music is considered an auditory pattern consisting of local and global features. Local features such as inter-note interval, and global features such as melodic contour, are derived from (...) a two-dimensional matrix in which music is represented as a series of frequencies plotted over time.Manipulation of inter-note interval affected accuracy and reaction time measures in a discrimination task, whereas the same variables were affected by manipulation of melodic contour in a classification task. Musical training is thought of as a form of practice in musical pattern recognition and, as predicted, accuracy and reaction time measures of musically trained subjects were significantly better than those of untrained subjects. Given the evidence for feature-extraction and weighting processes in music recognition tasks, two connectionist models are discussed. The first is a single-layer perceptron which has been trained to discriminate between compositions according to inter-note interval. A second network, using the back-propagation algorithm and sequential input of patterns, is also discussed. (shrink)

The performance of symbolic inference tasks has long been a challenge to connectionists. In this paper, we present an extended survey of this area. Existing connectionist inference systems are reviewed, with particular reference to how they perform variable binding and rule- based reasoning and whether they involve distributed or localist representations. The bene®ts and disadvantages of different representations and systems are outlined, and conclusions drawn regarding the capabilities of connectionist inference systems when compared with symbolic inference systems or when used (...) for cognitive modelling. q 2001 Elsevier Science Ltd. All rights reserved. (shrink)

This article proposes a unified framework for understanding creative problem solving, namely, the explicit–implicit interaction theory. This new theory of creative problem solving constitutes an attempt at providing a more unified explanation of relevant phenomena (in part by reinterpreting/integrating various fragmentary existing theories of incubation and insight). The explicit–implicit interaction theory relies mainly on 5 basic principles, namely, (a) the coexistence of and the difference between explicit and implicit knowledge, (b) the simultaneous involvement of implicit and explicit processes in most (...) tasks, (c) the redundant representation of explicit and implicit knowledge, (d) the integration of the results of explicit and implicit processing, and (e) the iterative (and possibly bidirectional) processing. A computational implementation of the theory is developed based on the CLARION cognitive architecture and applied to the simulation of relevant human data. This work represents an initial step in the development of process-based theories of creativity encompassing incubation, insight, and various other related.. (shrink)

This commentary examines one aspect of the target article – the comparison of ACT-R with connectionist models. It argues that conceptions of connectionist models should be broadened to cover the whole spectrum of work in this area, especially the so-called hybrid models. Doing so may change drastically ratings of connectionist models, and consequently shed more light on the developing field of cognitive architectures.

We present a skill learning model CLARION. Different from existing models of high-level skill learning that use a topdown approach (that is, turning declarative knowledge into procedural knowledge), we adopt a bottom-up approach toward low-level skill learning, where procedural knowledge develops first and declarative knowledge develops later. CLAR- ION is formed by integrating connectionist, reinforcement, and symbolic learning methods to perform on-line learning. We compare the model with human data in a minefield navigation task. A match between the model and (...) human data is found in several respects. (shrink)

This paper introduces a hybrid model that unifies connectionist, symbolic, and reinforcement learning into an integrated architecture for bottom-up skill learning in reactive sequential decision tasks. The model is designed for an agent to learn continuously from on-going experience in the world, without the use of preconceived concepts and knowledge. Both procedural skills and high-level knowledge are acquired through an agent’s experience interacting with the world. Computational experiments with the model in two domains are reported.

We believe that the distinction between procedural and declarative knowledge unnecessarily confounds two issues: action-centeredness and accessibility, and can be made clearer through separating the two aspects. The work presents an integrated model of skill learning that takes into account both implicit and explicit processes and both action-centered and non-action-centered knowledge. We examine and simulate human data in the Letter Counting task. The work shows how the data may be captured using either the action-centered knowledge alone or the combined action-centered (...) and non-action-centered knowledge. The results provide a new perspective on skill learning. (shrink)

This paper describes how meta-cognitive processes (i.e., the self monitoring and regulating of cognitive processes) may be captured within a cognitive architecture Clarion. Some currently popular cognitive architectures lack sufficiently complex built-in meta-cognitive mechanisms. However, a sufficiently complex meta-cognitive mechanism is important, in that it is an essential part of cognition and without it, human cognition may not function properly. We contend that such a meta-cognitive mechanism should be an integral part of a cognitive architecture. Thus such a mechanism has (...) been developed within the Clarion cognitive architecture. The paper demonstrates how human data of two meta-cognitive experiments are simulated using Clarion. The simulations show that the meta-cognitive processes represented by the experimental data (and beyond) can be adequately captured within the Clarion framework. (shrink)

Clark & Thornton present the well-established principle that recoding inputs can make learning easier. A useful goal would be to make such recoding automatic. We discuss some ways in which incrementality and transfer in connectionist networks could attain this goal.

The basic postulate that consciousness arises from stable states of recurrent activity is shown to need considerable modification from our current knowledge of the neural networks of the brain. Some of these modifications are outlined.

Using connectionist modelling, Thomas & Karmiloff-Smith (T&K-S) claim that developmental disorders in children are characterised by atypical trajectories and an ultimate functional architecture that is fundamentally different from normal. We argue that there is no empirical evidence for these claims in any developmental disorder and that the available evidence provides support for Residual Normality in both developmental and acquired disorders. We also refute the claim that modular accounts cannot encompass developmental trajectories in children with developmental disorders.

It is often assumed that similar domain-specific behavioural impairments found in cases of adult brain damage and developmental disorders correspond to similar underlying causes, and can serve as convergent evidence for the modular structure of the normal adult cognitive system. We argue that this correspondence is contingent on an unsupported assumption that atypical development can produce selective deficits while the rest of the system develops normally (Residual Normality), and that this assumption tends to bias data collection in the field. Based (...) on a review of connectionist models of acquired and developmental disorders in the domains of reading and past tense, as well as on new simulations, we explore the computational viability of Residual Normality and the potential role of development in producing behavioural deficits. Simulations demonstrate that damage to a developmental model can produce very different effects depending on whether it occurs prior to or following the training process. Because developmental disorders typically involve damage prior to learning, we conclude that the developmental process is a key component of the explanation of endstate impairments in such disorders. Further simulations demonstrate that in simple connectionist learning systems, the assumption of Residual Normality is undermined by processes of compensation or alteration elsewhere in the system. We outline the precise computational conditions required for Residual Normality to hold in development, and suggest that in many cases it is an unlikely hypothesis. We conclude that in developmental disorders, inferences from behavioural deficits to underlying structure crucially depend on developmental conditions, and that the process of ontogenetic development cannot be ignored in constructing models of developmental disorders. Key Words: Acquired and developmental disorders; connectionist models; modularity; past tense; reading. (shrink)

Solipsism, Individualism and Cognitive Science [1] "Artificial Intelligence cannot ignore philosophy" - John McCarthy (McCarthy 1988) I shall challenge the claim that Good Old-Fashioned Artificial Intelligence, or GOFAI (Haugeland 1985) is solipsistic while more recent neural or "brain-style" approaches to AI are not. (Rumelhart et. al. 1986) After distinguishing GOFAI from connectionism, I will first show that GOFAI is not committed to solipsism but rather to what is more properly called individualism.

The Newell Test is an important step in advancing our understanding of cognition. One critical constraint is missing from this test: A cognitive architecture must be self-contained. ACT-R and connectionism fail on this account. I present an alternative proposal, called Distributed Adaptive Control (DAC), and expose it to the Newell Test with the goal of achieving a clearer specification of the different constraints and their relationships, as proposed by Anderson & Lebiere (A&L).

Barsalou's theory rightly emphasizes the perceptual basis of cognition. However, the perceptual symbols that he proposes seem ill suited to carry the representational burden entailed by the architecture in which they function, given that Barsalou accepts the requirement for productivity. A more radical proposal is needed in which symbols are largely external to the cognizer and linked to internal states via perception.

This paper explores some of the constructive dimensions and specifics of human theoretic cognition, combining perspectives from (Husserlian) genetic phenomenology and distributed cognition approaches. I further consult recent psychological research concerning spatial and numerical cognition. The focus is on the nexus between the theoretic development of abstract, idealized geometrical and mathematical notions of space and the development and effective use of environmental cognitive support systems. In my discussion, I show that the evolution of the theoretic cognition of space apparently follows (...) two opposing, but in truth, intrinsically aligned trajectories. On the epistemic plane, which is the main focus of Husserl’s genetic phenomenological investigations, theoretic conceptions of space are progressively constituted by way of an idealizing emancipation of spatial cognition from the concrete, embodied intentionality underlying the human organism’s perception of space. As a result of this emancipation, it ultimately becomes possible for the human mind to theoretically conceive of and posit space as an ideal entity that is universally geometrical and mathematical. At the same time, by synthesizing a range of literature on spatial and mathematical cognition, I illustrate that for the theoretic mind to undertake precisely this emancipating process successfully, and further, for an ideal and objective notion of geometrical and mathematical space to first of all become fully scientifically operative, the cognitive support provided by a range of specific symbolic technologies is central. These include lettered diagrams, notation systems, and more generally, the technique of formalization and require for their functioning various cognitively efficacious types of embodiment. Ultimately, this paper endeavors to understand the specific symbolic-technological dimensions that have been instrumental to major shifts in the development of idealized, scientific conceptions of space. The epistemic characteristics of these shifts have been previously discussed in genetic phenomenology, but without devoting sufficient attention to the constructive role of symbolic technologies. At the same time, this paper identifies some of the irreducible phenomenological and epistemic dimensions that characterize the functioning of the historically situated, embodied and distributed theoretic mind. (shrink)

Dual-mechanism models of language maintain a distinction between a lexicon and a computational system of linguistic rules. In his target article, Clahsen provides support for such a distinction, presenting evidence from German inflections. He argues for a structured lexicon, going beyond the strict lexicon versus rules dichotomy. We agree with the author in assuming a dual mechanism; however, we argue that a next step must be taken, going beyond the notion of the computational system as specific rules applying to a (...) linguistic domain. By assuming a richer lexicon, the computational system can be conceived as a more general binding process that applies to different linguistic levels: syntax, morphology, reading, and spelling. (shrink)

The Systematicity Arguments is the only book-length treatment of the systematicity and productivity arguments.

Fodor and Pylyshyn (1988) have argued that the cognitive architecture is not Connectionist. Their argument takes the following form: (1) the cognitive architecture is Classical; (2) Classicalism and Connectionism are incompatible; (3) therefore the cognitive architecture is not Connectionist. In this essay I argue that Fodor and Pylyshyn's defenses of (1) and (2) are inadequate. Their argument for (1), based on their claim that Classicalism best explains the systematicity of cognitive capacities, is an invalid instance of inference to the best (...) explanation. And their argument for (2) turns out to be question-begging. The upshot is that, while Fodor and Pylyshyn have presented Connectionists with the important empirical challenge of explaining systematicity, they have failed to provide sufficient reason for inferring that the cognitive architecture is Classical and not Connectionist. (shrink)

Fodor and Pylyshyn's critique of connectionism has posed a challenge to connectionists: Adequately explain such nomological regularities as systematicity and productivity without postulating a "language of thought" (LOT). Some connectionists like Smolensky took the challenge very seriously, and attempted to meet it by developing models that were supposed to be non-classical. At the core of these attempts lies the claim that connectionist models can provide a representational system with a combinatorial syntax and processes sensitive to syntactic structure. They are not (...) implementation models because, it is claimed, the way they obtain syntax and structure sensitivity is not "concatenative," hence "radically different" from the way classicists handle them. In this paper, I offer an analysis of what it is to physically satisfy/realize a formal system. In this context, I examine the minimal truth-conditions of LOT Hypothesis. From my analysis it will follow that concatenative realization of formal systems is irrelevant to LOTH since the very notion of LOT is indifferent to such an implementation level issue as concatenation. I will conclude that to the extent to which they can explain the law-like cognitive regularities, a certain class of connectionist models proposed as radical alternatives to the classical LOT paradigm will in fact turn out to be LOT models, even though new and potentially very exciting ones. (shrink)

This paper discusses the relation between cognitive and implementational levels of analysis. Chalmers (1990, 1993) argues that a connectionist implementation of a classical cognitive architecture possesses a compositional semantics, and therefore undercuts Fodor and Pylyshyn's (1988) argument that connectionist networks cannot possess a compositional semantics. I argue that Chalmers argument misconstrues the relation between cognitive and implementational levels of analysis. This paper clarifies the distinction, and shows that while Fodor and Pylyshyn's argument survives Chalmers' critique, it cannot be used to (...) establish the irrelevance of neurophysiological implementation to cognitive modeling; some aspects of Chater and Oaksford's (1990) response to Fodor and Pylyshyn, though not all, are therefore cogent. (shrink)

This paper offers both a theoretical and an experimental perspective on the relationship between connectionist and Classical (symbol-processing) models. Firstly, a serious flaw in Fodor and Pylyshyn’s argument against connectionism is pointed out: if, in fact, a part of their argument is valid, then it establishes a conclusion quite different from that which they intend, a conclusion which is demonstrably false. The source of this flaw is traced to an underestimation of the differences between localist and distributed representation. It has (...) been claimed that distributed representations cannot support systematic operations, or that if they can, then they will be mere implementations of traditional ideas. This paper presents experimental evidence against this conclusion: distributed representations can be used to support direct structure-sensitive operations, in a man- ner quite unlike the Classical approach. Finally, it is argued that even if Fodor and Pylyshyn’s argument that connectionist models of compositionality must be mere implementations were correct, then this would still not be a serious argument against connectionism as a theory of mind. (shrink)

1. I think that by emphasizing theoretical spaces of representations, Andy has put his finger on an issue that is key to connectionism's success, and whose investigation will be a key determinant of the field's further progress. I also think that if we look at representational spaces in the right way, we can see that they are deeply related to classical phenomenon of systematicity in representation. I want to argue that the key to understanding representational spaces, and in particular their (...) ability to capture the deep organization underlying various problems, lies in the idea of what I will call. (shrink)

There has been much interest in the possibility of connectionist models whose representations can be endowed with compositional structure, and a variety of such models have been proposed. These models typically use distributed representations that arise from the functional composition of constituent parts. Functional composition and decomposition alone, however, yield only an implementation of classical symbolic theories. This paper explores the possibility of moving beyond implementation by exploiting holistic structure-sensitive operations on distributed representations. An experiment is performed using Pollack’s Recursive (...) Auto-Associative Memory. RAAM is used to construct distributed representations of syntactically structured sentences. A feed-forward network is then trained to operate directly on these representations, modeling syn- tactic transformations of the represented sentences. Successful training and generalization is obtained, demonstrating that the implicit structure present in these representations can be used for a kind of structure-sensitive processing unique to the connectionist domain. (shrink)