Subsymbolic Computation

Connectionist attention to variables has been too restricted in two ways. First, it has not exploited certain ways of doing without variables in the symbolic arena. One variable-avoidance method, that of logical combinators, is particularly well established there. Secondly, the attention has been largely restricted to variables in long-term rules embodied in connection weight patterns. However, short-lived bodies of information, such as sentence interpretations or inference products, may involve quantification. Therefore short-lived activation patterns may need to achieve the effect of (...) variables. The paper is mainly a theoretical analysis of some benefits and drawbacks of using logical combinators to avoid variables in short-lived connectionist encodings without loss of expressive power. The paper also includes a brief survey of some possible methods for avoiding variables other than by using combinators. (shrink)

The notion of a ‘symbol’ plays an important role in the disciplines of Philosophy, Psychology, Computer Science, and Cognitive Science. However, there is comparatively little agreement on how this notion is to be understood, either between disciplines, or even within particular disciplines. This paper does not attempt to defend some putatively ‘correct’ version of the concept of a ‘symbol.’ Rather, some terminological conventions are suggested, some constraints are proposed and a taxonomy of the kinds of issue that give rise to (...) disagreement is articulated. The goal here is to provide something like a ‘geography’ of the various notions of ‘symbol’ that have appeared in the various literatures, so as to highlight the key issues and to permit the focusing of attention upon the important dimensions. In particular, the relationship between ‘tokens’ and ‘symbols’ is addressed. The issue of designation is discussed in some detail. The distinction between simple and complex symbols is clarified and an apparently necessary condition for a system to be potentially symbol, or token bearing, is introduced. (shrink)

Berkeley [Minds Machines 10 (2000) 1] described a methodology that showed the subsymbolic nature of an artificial neural network system that had been trained on a logic problem, originally described by Bechtel and Abrahamsen [Connectionism and the mind. Blackwells, Cambridge, MA, 1991]. It was also claimed in the conclusion of this paper that the evidence was suggestive that the network might, in fact, count as a symbolic system. Dawson and Piercey [Minds Machines 11 (2001) 197] took issue with this latter (...) claim. They described some lesioning studies that they argued showed that Berkeley’s (2000) conclusions were premature. In this paper, these lesioning studies are replicated and it is shown that the effects that Dawson and Piercey rely upon for their argument are merely an artifact of a threshold function they chose to employ. When a threshold function much closer to that deployed in the original studies is used, the significant effects disappear. (shrink)

Psychoneural reduction is under attack again, only this time from a former ally: cognitive neuroscience. It has become popular to think of the brain as a complex system whose theoretically important properties emerge from dynamic, non-linear interactions between its component parts. ``Emergence'' is supposed to replace reduction: the latter is thought to be incapable of explaining the brain qua complex system. Rather than engage this issue at the level of theories of reduction versus theories of emergence, I here emphasize a (...) role that reductionism plays – and will continue to play – even if neuroscience adopts this ``complex systems'' view. In detailed investigations into the function of complex neural circuits, certain questions can only be addressed by moving down levels and scales. This role for reduction also finds a place for approaches that dominate mainstream neuroscience, like the widespread use of experimental techniques and theories from molecular biology and biochemistry. These are difficult to reconcile with the anti-reductionist sentiments of the ``complex systems'' branch of cognitive neuroscience. (shrink)

In a reflective and richly entertaining piece from 1979, Doug Hofstadter playfully imagined a conversation between ‘Achilles’ and an anthill (the eponymous ‘Aunt Hillary’), in which he famously explored many ideas and themes related to cognition and consciousness. For Hofstadter, the anthill is able to carry on a conversation because the ants that compose it play roughly the same role that neurons play in human languaging; unfortunately, Hofstadter’s work is notably short on detail suggesting how this magic might be achieved1. (...) Conversely in this paper - finally reifying Hofstadter’s imagination - we demonstrate how populations of simple ant-like creatures can be organised to solve complex problems; problems that involve the use of forward planning and strategy. Specifically we will demonstrate that populations of such creatures can be configured to play a strategically strong - though tactically weak - game of HeX (a complex strategic game).We subsequently demonstrate how tactical play can be improved by introducing a form of forward planning instantiated via multiple populations of agents; a technique that can be compared to the dynamics of interacting populations of social insects via the concept of meta-population. In this way although, pace Hofstadter, we do not establish that a meta-population of ants could actually hold a conversation with Achilles, we do successfully introduce Aunt Hillary to the complex, seductive charms of HeX. (shrink)

We argue that neural networks for semantic cognition, as proposed by Rogers & McClelland (R&M), do not acquire semantics and therefore cannot be the basis for a theory of semantic cognition. The reason is that the neural networks simply perform statistical categorization procedures, and these do not require any semantics for their successful operation. We conclude that this has severe consequences for the semantic cognition views of R&M.

More than a decade ago, philosopher John Searle started a long-running controversy with his paper “Minds, Brains, and Programs” (Searle, 1980a), an attack on the ambitious claims of artificial intelligence (AI). With his now famous _Chinese Room_ argument, Searle claimed to show that despite the best efforts of AI researchers, a computer could never recreate such vital properties of human mentality as intentionality, subjectivity, and understanding. The AI research program is based on the underlying assumption that all important aspects of (...) human cognition may in principle be captured in a computational model. This assumption stems from the belief that beyond a certain level, implementational details are irrelevant to cognition. According to this belief, neurons, and biological wetware in general, have no preferred status as the substrate for a mind. As it happens, the best examples of minds we have at present have arisen from a carbon-based substrate, but this is due to constraints of evolution and possibly historical accidents, rather than to an absolute metaphysical necessity. As a result of this belief, many cognitive scientists have chosen to focus not on the biological substrate of the mind, but instead on the abstract causal structure_ _that the mind embodies (at an appropriate level of abstraction). The view that it is abstract causal structure that is essential to mentality has been an implicit assumption of the AI research program since Turing (1950), but was first articulated explicitly, in various forms, by Putnam (1960), Armstrong (1970) and Lewis (1970), and has become known as _functionalism_. From here, it is a very short step to _computationalism_, the view that computational structure is what is important in capturing the essence of mentality. This step follows from a belief that any abstract causal structure can be captured computationally: a belief made plausible by the Church–Turing Thesis, which articulates the power. (shrink)

Marinov''s critique I argue, is vitiated by its failure to recognize the distinctive role of superposition within the distributed connectionist paradigm. The use of so-called subsymbolic distributed encodings alone is not, I agree, enough to justify treating distributed connectionism as a distinctive approach. It has always been clear that microfeatural decomposition is both possible and actual within the confines of recognizably classical approaches. When such approaches also involve statistically-driven learning algorithms — as in the case of ID3 — the fundamental (...) differences become even harder to spot. To see them, it is necessary to consider not just the nature of an acquired input-output function but the nature of the representational scheme underlying it. Differences between such schemes make themselves best felt outside the domain of immediate problem solving. It is in the more extended contexts of performance DURING learning and cognitive change as a result of SUBSEQUENT training on new tasks (or simultaneous training on several tasks) that the effects of superpositional storage techniques come to the fore. I conclude that subsymbols, distribution and statistically driven learning alone are indeed not of the essence. But connectionism is not just about subsymbols and distribution. It is about the generation of whole subsymbol SYSTEMS in which multiple distributed representations are created and superposed. (shrink)

The constructivist notion that features are purely functional is incompatible with the classical computational metaphor of mind. I suggest that the discontent expressed by Schyns, Goldstone and Thibaut about fixed-features theories of categorization reflects the growing impact of connectionism, and show how their perspective is similar to recent research on implicit learning, consciousness, and development. A hard problem remains, however: How to bridge the gap between subsymbolic and symbolic cognition.

This book records the thoughts of researchers -- from both computer science and philosophy -- on resolving the debate between the symbolic and connectionist...

Dietmar Heinke and Eirini Mavritsaki (eds): Computational Modelling in Behavioural Neuroscience Content Type Journal Article Category Book Review Pages 57-60 DOI 10.1007/s11023-011-9265-8 Authors Juan Felipe Martinez Florez, Institute of Psychology, Universidad del Valle, Campus Universitario Melndez, Ed. 388, Of. 4017, Cali, Colombia Journal Minds and Machines Online ISSN 1572-8641 Print ISSN 0924-6495 Journal Volume Volume 22 Journal Issue Volume 22, Number 1.

The dominating models of information processes have been based on symbolic representations of information and knowledge. During the last decades, a variety of non-symbolic models have been proposed as superior. The prime examples of models within the non-symbolic approach are neural networks. However, to a large extent they lack a higher-level theory of representation. In this paper, conceptual spaces are suggested as an appropriate framework for non- symbolic models. Conceptual spaces consist of a number of 'quality dimensions' that often are (...) derived from perceptual mechanisms. It will be outlined how conceptual spaces can represent various kind of information and how they can be used to describe concept learning. The connections to prototype theory will also be presented. (shrink)

Not long ago the standard view in cognitive science was that representations are symbols in an internal representational system or language of thought and that psychological processes are computations defined over such representations. This orthodoxy has been challenged by adherents of functional analysis and by connectionists. Functional analysis as practiced by Marr is consistent with an analysis of representation that grants primacy to a stands for conception of representation. Connectionism is also compatible with this notion of representation; when conjoined with (...) functional analysis, it provides a means of analyzing psychological systems in term of rules and representations without becoming committed to symbolism. Direct theorists, who rejected the orthodox symbolist conception of representation because it violated their strictures against cognitive mediational mechanisms, should find it possible to accept rules-and-representations and information-processing analyses of the mechanisms of information pickup couched in terms of functional analysis. (shrink)

During the last decades, many cognitive architectures (CAs) have been realized adopting different assumptions about the organization and the representation of their knowledge level. Some of them (e.g. SOAR [35]) adopt a classical symbolic approach, some (e.g. LEABRA[ 48]) are based on a purely connectionist model, while others (e.g. CLARION [59]) adopt a hybrid approach combining connectionist and symbolic representational levels. Additionally, some attempts (e.g. biSOAR) trying to extend the representational capacities of CAs by integrating diagrammatical representations and reasoning are (...) also available [34]. In this paper we propose a reflection on the role that Conceptual Spaces, a framework developed by Peter G¨ardenfors [24] more than fifteen years ago, can play in the current development of the Knowledge Level in Cognitive Systems and Architectures. In particular, we claim that Conceptual Spaces offer a lingua franca that allows to unify and generalize many aspects of the symbolic, sub-symbolic and diagrammatic approaches (by overcoming some of their typical problems) and to integrate them on a common ground. In doing so we extend and detail some of the arguments explored by G¨ardenfors [23] for defending the need of a conceptual, intermediate, representation level between the symbolic and the sub-symbolic one. In particular we focus on the advantages offered by Conceptual Spaces (w.r.t. symbolic and sub-symbolic approaches) in dealing with the problem of compositionality of representations based on typicality traits. Additionally, we argue that Conceptual Spaces could offer a unifying framework for interpreting many kinds of diagrammatic and analogical representations. As a consequence, their adoption could also favor the integration of diagrammatical representation and reasoning in CAs. (shrink)

In this paper we identify and characterize an analysis of two problematic aspects affecting the representational level of cognitive architectures (CAs), namely: the limited size and the homogeneous typology of the encoded and processed knowledge. We argue that such aspects may constitute not only a technological problem that, in our opinion, should be addressed in order to build arti cial agents able to exhibit intelligent behaviours in general scenarios, but also an epistemological one, since they limit the plausibility of the (...) comparison of the CAs' knowledge representation and processing mechanisms with those executed by humans in their everyday activities. In the fi nal part of the paper further directions of research will be explored, trying to address current limitations and future challenges. (shrink)

The article criticises the attempt to establish connectionism as an alternative theory of human cognitive architecture through the introduction of thesymbolic/subsymbolic distinction (Smolensky, 1988). The reasons for the introduction of this distinction are discussed and found to be unconvincing. It is shown that thebrittleness problem has been solved for a large class ofsymbolic learning systems, e.g. the class oftop-down induction of decision-trees (TDIDT) learning systems. Also, the process of articulating expert knowledge in rules seems quite practical for many important domains, (...) including common sense knowledge.The article discusses several experimental comparisons betweenTDIDT systems and artificial neural networks using the error backpropagation algorithm (ANNs usingBP). The properties of one of theTDIDT systemsID3 (Quinlan, 1986a) are examined in detail. It is argued that the differences in performance betweenANNs usingBP andTDIDT systems reflect slightly different inductive biases but are not systematic; these differences do not support the view that symbolic and subsymbolic systems are fundamentally incompatible. It is concluded, that thesymbolic/subsymbolic distinction is spurious. It cannot establish connectionism as an alternative cognitive architecture. (shrink)

Representation is a central part of models in cognitive science, but recently this idea has come under attack. Researchers advocating perceptual symbol systems, situated action, embodied cognition, and dynamical systems have argued against central assumptions of the classical representational approach to mind. We review the core assumptions of the dominant view of representation and the four suggested alternatives. We argue that representation should remain a core part of cognitive science, but that the insights from these alternative approaches must be incorporated (...) into models of cognitive processing. (shrink)

This paper investigates the prospects of Rodney Brooks’ proposal for AI without representation. It turns out that the supposedly characteristic features of “new AI” (embodiment, situatedness, absence of reasoning, and absence of representation) are all present in conventional systems: “New AI” is just like old AI. Brooks proposal boils down to the architectural rejection of central control in intelligent agents—Which, however, turns out to be crucial. Some of more recent cognitive science suggests that we might do well to dispose of (...) the image of intelligent agents as central representation processors. If this paradigm shift is achieved, Brooks’ proposal for cognition without representation appears promising for full-blown intelligent agents—Though not for conscious agents. (shrink)

Although connectionism is advocated by its proponents as an alternative to the classical computational theory of mind, doubts persist about its _computational_ credentials. Our aim is to dispel these doubts by explaining how connectionist networks compute. We first develop a generic account of computation—no easy task, because computation, like almost every other foundational concept in cognitive science, has resisted canonical definition. We opt for a characterisation that does justice to the explanatory role of computation in cognitive science. Next we examine (...) what might be regarded as the “conventional” account of connectionist computation. We show why this account is inadequate and hence fosters the suspicion that connectionist networks aren’t genuinely computational. Lastly, we turn to the principal task of the paper: the development of a more robust portrait of connectionist computation. The basis of this portrait is an explanation of the representational capacities of connection weights, supported by an analysis of the weight configurations of a series of simulated neural networks. (shrink)

To deal with reactive sequential decision tasks we present a learning model Clarion which is a hybrid connectionist model consisting of both localist and dis tributed representations based on the two level ap proach proposed in Sun The model learns and utilizes procedural and declarative knowledge tapping into the synergy of the two types of processes It uni es neural reinforcement and symbolic methods to perform on line bottom up learning Experiments in various situations are reported that shed light on (...) the working of the model.. (shrink)

In this small book logician and mathematician Jens Erik Fenstad addresses some of the most important foundational questions of linguistics: What should a theory of meaning look like and how might we provide the missing link between meaning theory and our knowledge of how the brain works? The author’s answer is twofold. On the one hand, he suggests that logical semantics in the Montague tradition and other broadly conceived symbolic approaches do not suffice. On the other hand, he does not (...) argue that the logical approach should be discarded; instead, he opts for a methodological pluralism in which symbolic approaches to meaning are combined with geo- metric ones such as Conceptual Spaces [9] and discusses ways in which these geometric accounts could be hooked up with connectionist frameworks and dynamic systems approaches in neurophysiology. (shrink)

A set of hypotheses is formulated for a connectionist approach to cognitive modeling. These hypotheses are shown to be incompatible with the hypotheses underlying traditional cognitive models. The connectionist models considered are massively parallel numerical computational systems that are a kind of continuous dynamical system. The numerical variables in the system correspond semantically to fine-grained features below the level of the concepts consciously used to describe the task domain. The level of analysis is intermediate between those of symbolic cognitive models (...) and neural models. The explanations of behavior provided are like those traditional in the physical sciences, unlike the explanations provided by symbolic models. (shrink)

The commentators expressed concerns regarding the relevance and value of non-computational non-symbolic explanations of cognitive performance. But what counts as an “explanation” depends on the pre-theoretical assumptions behind the scenes of empirical science regarding the kinds of variables and relationships that are sought out in the first place, and some of the present disagreements stem from incommensurate assumptions. Traditional cognitive science presumes cognition to be a decomposable system of components interacting according to computational rules to generate cognitive performances (i.e., component-dominant (...) dynamics). We assign primacy to interaction-dominant dynamics among components. Though either choice can be a good guess before the fact, the primacy of interactions is now supported by much recent empirical work in cognitive science. Consequently, in the main, the commentators have failed so far to address the growing evidence corroborating the theory-driven predictions of complexity science. (shrink)