Bifurcation analysis of a real-time implementation of an ART network, which is functionally similar to the generalized localist model discussed in Page's manifesto shows that it yields a phase transition from local to distributed representation owing to continuous variation of the range of inhibitory connections. Hence there appears to be a qualitative dichotomy between local and distributed representations at the level of connectionistic networks conceived of as instances of nonlinear dynamical systems.

A number of examples are given of how localist models may incorporate distributed representations, without the types of nonlocal interactions that often render distributed models implausible. The need to analyze the information that is encoded by these representations is also emphasized as a metatheoretical constraint on model plausibility.

A dynamic threshold, which controls the nature and course of learning, is a pivotal concept in Page's general localist framework. This commentary addresses various issues surrounding biologically plausible implementations for such thresholds. Relevant previous research is noted and the particular difficulties relating to the creation of so-called instance representations are highlighted. It is stressed that these issues also apply to distributed models.

We focus on two components of Page's argument in favour of localist representations in connectionist networks: First, we take issue with the claim that localist representations can give rise to generalisation and show that whenever generalisation occurs, distributed representations are involved. Second, we counter the alleged shortcomings of distributed representations and show that their properties are preferable to those of localist approaches.

Almost all representations have both distributed and localist aspects, depending upon what properties of the data are being considered. With noisy data, features represented in a localist way can be detected very efficiently, and in binary representations they can be counted more efficiently than those represented in a distributed way. Brains operate in noisy environments, so the localist representation of behaviourally important events is advantageous, and fits what has been found experimentally. Distributed (...) representations require more neurons to perform as efficiently, but they do have greater versatility. (shrink)

In the Localist Manifesto, Page enumerated several computational advantages that localist representations have over distributed representations, but the most important difference between such networks concerns their theoretical clarity. Distributed representations are normally closed to theoretical interpretation and, for that reason, contribute little to psychology, whereas the meaning of the information processing in networks using localist representations can be transparent.

In order to benefit from the advantages of localist coding, neural models that feature winner-take-all representations at the top level of a network hierarchy must still solve the computational problems inherent in distributed representations at the lower levels.

This paper offers both a theoretical and an experimental perspective on the relationship between connectionist and Classical (symbol-processing) models. Firstly, a serious flaw in Fodor and Pylyshyn’s argument against connectionism is pointed out: if, in fact, a part of their argument is valid, then it establishes a conclusion quite different from that which they intend, a conclusion which is demonstrably false. The source of this flaw is traced to an underestimation of the differences between localist and distributed representation. (...) It has been claimed that distributed representations cannot support systematic operations, or that if they can, then they will be mere implementations of traditional ideas. This paper presents experimental evidence against this conclusion: distributed representations can be used to support direct structure-sensitive operations, in a man- ner quite unlike the Classical approach. Finally, it is argued that even if Fodor and Pylyshyn’s argument that connectionist models of compositionality must be mere implementations were correct, then this would still not be a serious argument against connectionism as a theory of mind. (shrink)

Page has done connectionist researchers a valuable service in this target article. He points out that connectionist models using localized representations often work as well or better than models using distributed representations. I point out that models using distributed representations are difficult to understand and often lack parsimony and plausibility. In conclusion, I give an example – the case of the missing fundamental in music – that can easily be explained by a model using (...) class='Hi'>localist representations but can be explained only with great difficulty and implausibility by a model using distributed representations. (shrink)

We contend that if efficiency and reliability are important factors in neural information processing then distributed, not localist, representations are “evolution's best bet.” We note that distributed codes are the most efficient method for representing information, and that this efficiency minimizes metabolic costs, providing adaptive advantage to an organism.

Over the last decade, fully distributed models have become dominant in connectionist psychological modelling, whereas the virtues of localist models have been underestimated. This target article illustrates some of the benefits of localist modelling. Localist models are characterized by the presence of localist representations rather than the absence of distributed representations. A generalized localist model is proposed that exhibits many of the properties of fully distributed models. It can be applied (...) to a number of problems that are difficult for fully distributed models, and its applicability can be extended through comparisons with a number of classic mathematical models of behaviour. There are reasons why localist models have been underused, though these often misconstrue the localist position. In particular, many conclusions about connectionist representation, based on neuroscientific observation, can be called into question. There are still some problems inherent in the application of fully distributed systems and some inadequacies in proposed solutions to these problems. In the domain of psychological modelling, localist modelling is to be preferred. Key Words: choice; competition; connectionist modelling; consolidation; distributed; localist; neural networks; reaction-time. (shrink)

This paper presents a skill learning model CLARION. Different from existing models of mostly high-level skill learning that use a top-down approach (that is, turning declarative knowledge into procedural knowledge through practice), we adopt a bottom-up approach toward low-level skill learning, where procedural knowledge develops first and declarative knowledge develops later. Our model is formed by integrating connectionist, reinforcement, and symbolic learning methods to perform on-line reactive learning. It adopts a two-level dual-representation framework (Sun, 1995), with a combination of (...) class='Hi'>localist and distributed representation. We compare the model with human data in a minefield navigation task, demonstrating some match between the model and human data in several respects. (shrink)

This paper aims to explore mechanistic and teleological explanations of consciousness. In terms of mechanistic explanations, it critiques various existing views, especially those embodied by existing computational cognitive models. In this regard, the paper argues in favor of the explanation based on the distinction between localist (symbolic) representation and distributed representation (as formulated in the connectionist literature), which reduces the phenomenological difference to a mechanistic difference. Furthermore, to establish a teleological explanation of consciousness, the paper discusses the issue (...) of the functional role of consciousness on the basis of the aforementioned mechanistic explanation. A proposal based on synergistic interaction between the conscious and the unconscious is advanced that encompasses various existing views concerning the functional role of consciousness. This two-step deepening explanation has some empirical support, in the form of a cognitive model and various cognitive data that it captures. © 2001 Elsevier Science B.V. All rights reserved. (shrink)

In developing autonomous agents, one usually emphasizes only (situated) procedural knowledge, ignoring more explicit declarative knowledge. On the other hand, in developing symbolic reasoning models, one usually emphasizes only declarative knowledge, ignoring procedural knowledge. In contrast, we have developed a learning model Clarion, which is a hybrid connectionist model consisting of both localist and distributed representations, based on the two-level approach proposed in Sun (1995). learns and utilizes both procedural and declarative knowledge, tapping into the synergy of..

To deal with reactive sequential decision tasks we present a learning model which is a hybrid connectionist model consisting of both localist and distributed representations based on the two level approach proposed in..

The distinction made by Page between localist and distributed representations seems confounded by the distinction between competitive and associative learning. His manifesto can also be read as a plea for competitive learning. The power of competitive models can even be extended further, by simulating similarity effects in forced-choice perceptual identification (Ratcliff & McKoon 1997) that have defied explanation by most memory models.

A computational model of inference during story comprehension is presented, in which story situations are represented distributively as points in a high‐dimensional “situation‐state space.” This state space organizes itself on the basis of a constructed microworld description. From the same description, causal/temporal world knowledge is extracted. The distributed representation of story situations is more flexible than Golden and Rumelhart's [Discourse Proc 16 (1993) 203] localist representation.A story taking place in the microworld corresponds to a trajectory through situation‐state space. (...) During the inference process, world knowledge is applied to the story trajectory. This results in an adjusted trajectory, reflecting the inference of propositions that are likely to be the case. Although inferences do not result from a search for coherence, they do cause story coherence to increase. The results of simulations correspond to empirical data concerning inference, reading time, and depth of processing.An extension of the model for simulating story retention shows how coherence is preserved during retention without controlling the retention process. Simulation results correspond to empirical data concerning story recall and intrusion. (shrink)

This paper argues for an explanation of the mechanistic (computational) basis of consciousness that is based on the distinction between localist (symbolic) representation and distributed representation, the ideas of which have been put forth in the connectionist literature. A model is developed to substantiate and test this approach. The paper also explores the issue of the functional roles of consciousness, in relation to the proposed mechanistic explanation of consciousness. The model, embodying the representational difference, is able to account (...) for the functional role of consciousness, in the form of the synergy between the conscious and the unconscious. The fit between the model and various cognitive phenomena and data (documented in the psychological literatures) is discussed to accentuate the plausibility of the model and its explanation of consciousness. Comparisons with existing models of consciousness are made in the end. (shrink)

Two categorization arguments pose particular problems for localist connectionist models. The internal representations of localist networks do not reflect the variability within categories in the environment, whereas networks with distributed internal representations do reflect this essential feature of categories. We provide a real biological example of perceptual categorization in the monkey that seems to require population coding (i.e., distributed internal representations).

Two additional sources of evidence are provided in support of localist coding within connectionist networks. First, only models with localist codes can currently represent multiple pieces of information simultaneously or represent order among a set of items on-line. Second, recent priming data appear problematic for theories that rely on distributed representations. However, a faulty argument advanced by Page is also pointed out.

A computational model of inference during story comprehension is presented, in which story situations are represented distributively as points in a high‐dimensional “situation‐state space.” This state space organizes itself on the basis of a constructed microworld description. From the same description, causal/temporal world knowledge is extracted. The distributed representation of story situations is more flexible than Golden and Rumelhart's [Discourse Proc 16 (1993) 203] localist representation.A story taking place in the microworld corresponds to a trajectory through situation‐state space. (...) During the inference process, world knowledge is applied to the story trajectory. This results in an adjusted trajectory, reflecting the inference of propositions that are likely to be the case. Although inferences do not result from a search for coherence, they do cause story coherence to increase. The results of simulations correspond to empirical data concerning inference, reading time, and depth of processing.An extension of the model for simulating story retention shows how coherence is preserved during retention without controlling the retention process. Simulation results correspond to empirical data concerning story recall and intrusion. (shrink)

In “Connectionism and the fats of folk psychology”, Forster and Saidel argue that the central claim of Ramsey, Stich and Garon (1991)—that distributed connectionist models are incompatible with the causal discreteness of folk psychology—is mistaken. To establish their claim, they offer an intriguing model which allegedly shows how distributed representations can function in a causally discrete manner. They also challenge our position regarding projectibility of folk psychology. In this essay, I offer a response to their account and (...) show how their model fails to demonstrate that our original argument was mistaken. While I will discuss several difficulties with their model, my primary criticism will be that the features of their model that are causally discrete are not truly distributed, while the features that are distributed are not really discrete. Concerning the issue of projectibility, I am more inclined to agree with Forster and Saidel and I offer a revised account of what we should have said originally. (shrink)

A fundamental claim associated with parallel distributed processing theories of cognition is that knowledge is coded in a distributed manner in mind and brain. This approach rejects the claim that knowledge is coded in a localist fashion, with words, objects, and simple concepts, that is, coded with their own dedicated representations. One of the putative advantages of this approach is that the theories are biologically plausible. Indeed, advocates of the PDP approach often highlight the close parallels (...) between distributed representations learned in connectionist models and neural coding in brain and often dismiss localist theories as biologically implausible. The author reviews a range a data that strongly challenge this claim and shows that localist models provide a better account of single-cell recording studies. The author also contrast local and alternative distributed coding schemes and argues that common rejection of grandmother cell theories in neuroscience is due to a misunderstanding about how localist models behave. The author concludes that the localist representations embedded in theories of perception and cognition are consistent with neuroscience; biology only calls into question the distributed representations often learned in PDP models. (shrink)

Variable binding has long been a challenge to connectionists. Attempts to perform variable binding using localist and distributed connectionist representations are discussed, and problems inherent in each type of representation are outlined.

Localist models of spreading activation (SA) and models assuming distributed representations offer very different takes on semantic priming, a widely investigated paradigm in word recognition and semantic memory research. In this study, we implemented SA in an attractor neural network model with distributed representations and created a unified framework for the two approaches. Our models assume a synaptic depression mechanism leading to autonomous transitions between encoded memory patterns (latching dynamics), which account for the major characteristics (...) of automatic semantic priming in humans. Using computer simulations, we demonstrated how findings that challenged attractor‐based networks in the past, such as mediated and asymmetric priming, are a natural consequence of our present model’s dynamics. Puzzling results regarding backward priming were also given a straightforward explanation. In addition, the current model addresses some of the differences between semantic and associative relatedness and explains how these differences interact with stimulus onset asynchrony in priming experiments. (shrink)

In this article, we apply a special case of holographic representations to letter position coding. We translate different well-known schemes into this format, which uses distributed representations and supports constituent structure. We show that in addition to these brain-like characteristics, performances on a standard benchmark of behavioral effects are improved in the holographic format relative to the standard localist one. This notably occurs because of emerging properties in holographic codes, like transposition and edge effects, for which (...) we give formal demonstrations. Finally, we outline the limits of the approach as well as its possible future extensions. (shrink)

This paper is a revised and extended version of a keynote contribution to a recent conference on Cognitive Informatics. It offers a brief summary of some of the core concerns of other contributions to the conference, highlighting the range of issues under discussion; and argues that many of the central concepts and preoccupations of cognitive informatics as understood by participants--and others in the general field of computation--rely on ill-founded realist assumptions, and what has been termed the functionalist view of representation. (...) Even if such ideas--albeit in a revised form -- can be defended, there must be a more extensive engagement with the literature and issues outside the confines of the computing and computational orthodoxy. (shrink)

There has been much interest in the possibility of connectionist models whose representations can be endowed with compositional structure, and a variety of such models have been proposed. These models typically use distributed representations that arise from the functional composition of constituent parts. Functional composition and decomposition alone, however, yield only an implementation of classical symbolic theories. This paper explores the possibility of moving beyond implementation by exploiting holistic structure-sensitive operations on distributed representations. An experiment (...) is performed using Pollack’s Recursive Auto-Associative Memory. RAAM is used to construct distributed representations of syntactically structured sentences. A feed-forward network is then trained to operate directly on these representations, modeling syn- tactic transformations of the represented sentences. Successful training and generalization is obtained, demonstrating that the implicit structure present in these representations can be used for a kind of structure-sensitive processing unique to the connectionist domain. (shrink)

There are many conflicting views concerning the nature of distributed representation, its compatibility or otherwise with symbolic representation, and its importance in characterizing the nature of connectionist models and their relationship to more traditional symbolic approaches to understanding cognition. Many have simply assumed that distribution is merely an implementation issue, and that symbolic mechanisms can be designed to take advantage of the virtues of distribution if so desired. Others, meanwhile, see the use of distributed representation as marking a (...) fundamental difference between the two approaches. One reason for this diversity of opinion is the fact that the relevant notions - especially that of. (shrink)

This paper is a revised and extended version of a keynote contribution to a recent conference on Cognitive Informatics. It offers a brief summary of some of the core concerns of other contributions to the conference, highlighting the range of issues under discussion; and argues that many of the central concepts and preoccupations of cognitive informatics as understood by participants--and others in the general field of computation--rely on ill-founded realist assumptions, and what has been termed the functionalist view of representation. (...) Even if such ideas--albeit in a revised form -- can be defended, there must be a more extensive engagement with the literature and issues outside the confines of the computing and computational orthodoxy. (shrink)

We continue work of our earlier paper :219–241, 2009) where abstract logics and particularly intuitionistic abstract logics are studied.logics can be topologized in a direct and natural way. This facilitates a topological study of classes of concrete logics whenever they are given in abstract form. Moreover, such a direct topological approach avoids the often complex algebraic and lattice-theoretic machinery usually applied to represent logics. Motivated by that point of view, we define in this paper the category of intuitionistic abstract logics (...) with stable logic maps as morphisms, and the category of implicative spectral spaces with spectral maps as morphisms. We show the equivalence of these categories and conclude that the larger categories of distributive abstract logics and distributive sober spaces are equivalent, too. (shrink)

Abstract Since their inception, distributional models of semantics have been criticized as inadequate cognitive theories of human semantic learning and representation. A principal challenge is that the representations derived by distributional models are purely symbolic and are not grounded in perception and action; this challenge has led many to favor feature-based models of semantic representation. We argue that the amount of perceptual and other semantic information that can be learned from purely distributional statistics has been underappreciated. We compare the (...) representations of three feature-based and nine distributional models using a semantic clustering task. Several distributional models demonstrated semantic clustering comparable with clustering-based on feature-based representations. Furthermore, when trained on child-directed speech, the same distributional models perform as well as sensorimotor-based feature representations of children’s lexical semantic knowledge. These results suggest that, to a large extent, information relevant for extracting semantic categories is redundantly coded in perceptual and linguistic experience. Detailed analyses of the semantic clusters of the feature-based and distributional models also reveal that the models make use of complementary cues to semantic organization from the two data streams. Rather than conceptualizing feature-based and distributional models as competing theories, we argue that future focus should be on understanding the cognitive mechanisms humans use to integrate the two sources. (shrink)

This article describes a representation-based framework of distributed cognition. This framework considers distributed cognition as a cognitive system whose structures and processes are distributed between internal and external representations, across a group of individuals, and across space and time. The major issue for distributed research, under this framework, are the distribution, transformation, and propagation of information across the components of the distributed cognitive system and how they affect the performance of the system as a (...) whole. To demonstrate the value of this representation-based approach, the framework was used to describe and explain an important, challenging, and controversial issue — the concept of affordance. (shrink)

This article describes a representation-based framework of distributed cognition. This framework considers distributed cognition as a cognitive system whose structures and processes are distributed between internal and external representations, across a group of individuals, and across space and time. The major issue for distributed research, under this framework, are the distribution, transformation, and propagation of information across the components of the distributed cognitive system and how they affect the performance of the system as a (...) whole. To demonstrate the value of this representation-based approach, the framework was used to describe and explain an important, challenging, and controversial issue — the concept of affordance. (shrink)

This article describes various ways actors in Kashmiri carpet weaving practice deploy a range of artifacts, from symbolic, to material, to hybrid, in order to achieve diverse cognitive accomplishments in their particular task domains: information representation, inter and intra-domain communication, distribution of cognitive labor across people and time, coordination of team activities, and carrying of cultural heritage. In this repertoire, some artifacts position themselves as naïve tools in the actors’ environment to the point of being ignored; however, their usage-in-context unfolds (...) their cognitive involvement in the tasks. These usages-in-context are shown through artifact analysis of their routine, improvised, and opportunistic uses, where cognitive artifacts like talim—the central artifact of this practice—are shown to play not only multifunctional roles beyond representation, but are also complemented by trade-specific skills bearing strong cognitive implications in a task. (shrink)

Distributed Cognition and Integrational Linguistics have much in common. Both approaches see communicative activity and intelligent behaviour in general as strongly con- text-dependent and action-oriented, and brains as permeated by history. But there is some ten- sion between the two frameworks on three important issues. The majority of theorists of distributed cognition want to maintain some notions of mental representation and computa- tion, and to seek generalizations and patterns in the various ways in which creatures like us couple (...) with technologies, media, and other agents; many also want to offer explanations at subpersonal levels which may undercut the autonomy of personal-level accounts. In contrast, dominant views in integrational linguistics reject all invocation of representation, resist the explanatory search for similarity across contexts and moments, and see linguistics as a lay dis- cipline which should not offer explanations in terms alien to ordinary agents. On each of these issues, I argue that integrationists could move closer to the distributed cognition frame- work without losing the most important aspects of their view: integrationist criticisms of mainstream or classical theories can be respected while alliances with revised cognitivist views about representation, context, and explanation are developed. Ó 2004 Elsevier Ltd. All rights reserved. (shrink)

For more than a century the paradigm inspiringcognitive neuroscience has been modular and localist.Contemporary research in functional brain imaginggenerally relies on methods favorable to localizingparticular functions in one or more specific brainregions. Meanwhile, connectionist cognitive scientistshave celebrated the computational powers ofdistributed processing, and pioneered methods forinterpreting distributed representations. This papertakes a connectionist approach to functionalneuroimaging. A tabulation of 35 PET (positronemission tomography) experiments strongly indicatesdistributed function for at least the ''medium sized''anatomical units, the cortical Brodmann areas. Moreimportant, (...) when these PET experiments were interpretedas distributed representations, multidimensionalscaling revealed a ''brain activation space'' with asalient structure organized primarily by the sensorymodality of the stimulus, and secondarily by the typeof motor response. These results suggest that currentanalytical techniques in functional neuroimagingshould be augmented by distributed processinganalyses, and that these analyses may lead to manydiscoveries about the structure of ''inner space.''. (shrink)

Page's definition of localism is inspired by the instance-based paradigm. However, the locality of representations is not necessary for a model to be instance-based and, on the other hand, explicit featural representations are generally considered local. The important distinction is between instance-based and noninstance-based paradigms and not between distributed and local representations as Page claims.

Over the past 15 years, there have been two increasingly popular approaches to the study of meaning in cognitive science. One, based on theories of embodied cognition, treats meaning as a simulation of perceptual and motor states. An alternative approach treats meaning as a consequence of the statistical distribution of words across spoken and written language. On the surface, these appear to be opposing scientific paradigms. In this review, we aim to show how recent cross-disciplinary developments have done much to (...) reconcile these two approaches. The foundation to these developments has been the recognition that intralinguistic distributional and sensory-motor data are interdependent. We describe recent work in philosophy, psychology, cognitive neuroscience, and computational modeling that are all based on or consistent with this conclusion. We conclude by considering some possible directions for future research that arise as a consequence of these developments. (shrink)

Distributed representations can be distributed in very many ways. The specific choice of representation for a specific model is based on considerations unique to the area of study. General statements about the effectiveness of distributed models are therefore of little value. The popularity of these models is discussed, particularly with respect to reporting conventions.

Some aspects of the theory of Boolean algebras and distributive lattices–in particular, the Stone Representation Theorems and the properties of filters and ideals–are analyzed in a constructive setting.

In this article we propose a theoretical framework of distributed representations and a methodology of representational analysis for the study of distributed cognitive tasks—tasks that require the processing of information distributed across the internal mind and the external environment. The basic principle of distributed representations Is that the representational system of a distributed cognitive task is a set of internal and external representations, which together represent the abstract structure of the task. The (...) basic strategy of representational analysis is to decompose the representation of a hierarchical task into its component levels so that the representational properties at each level can be independently examined. The theoretical framework and the methodology are used to analyze the hierarchical structure of the Tower of Hanoi problem. Based on this analysis, four experiments are designed to examine the representational properties of the Tower of Hanoi. Finally, the nature of external representations is discussed. (shrink)