Online research in philosophy

The constructivist notion that features are purely functional is incompatible with the classical computational metaphor of mind. I suggest that the discontent expressed by Schyns, Goldstone and Thibaut about fixed-features theories of categorization reflects the growing impact of connectionism, and show how their perspective is similar to recent research on implicit learning, consciousness, and development. A hard problem remains, however: How to bridge the gap between subsymbolic and symbolic cognition.

Connectionism and computationalism are currently vying for hegemony in cognitive modeling. At first glance the opposition seems incoherent, because connectionism is itself computational, but the form of computationalism that has been the prime candidate for encoding the "language of thought" has been symbolic computationalism (Dietrich 1990, Fodor 1975, Harnad 1990c; Newell 1980; Pylyshyn 1984), whereas connectionism is nonsymbolic (Fodor & Pylyshyn 1988, or, as some have hopefully dubbed it, "subsymbolic" Smolensky 1988). This paper will examine what is and is not a symbol system. A hybrid nonsymbolic/symbolic system will be sketched in which the meanings of the symbols are grounded bottom-up in the system's capacity to discriminate and identify the objects they refer to. Neural nets are one possible mechanism for learning the invariants in the analog sensory projection on which successful categorization is based. "Categorical perception" (Harnad 1987a), in which similarity space is "warped" in the service of categorization, turns out to be exhibited by both people and nets, and may mediate the constraints exerted by the analog world of objects on the formal world of symbols.

Abstra,ct— This paper will discuss learning in hybrid models that goes beyond simple rule extraction from backpropagation networks. Although simple rule extraction has received a lot of research attention, to further develop hybrid learning models that include both symbolic and subsymbolic knowledge and that learn autonomously, it is necessary to study autonomous learning of both subsymbolic and symbolic knowledge in integrated architectures. This paper will describe knowledge extraction from neural reinforcement learning. It includes two approaches towards extracting plan knowledge: the extraction of explicit, symbolic rules from neural reinforcement learning, and the extraction of complete plans. This work points to the creation of a general framework for achieving the subsymbolic to symbolic transition in an integrated autonomous learning framework.

Newell (1980; 1990) proposed that cognitive theories be developed in an effort to satisfy multiple criteria and to avoid theoretical myopia. He provided two overlapping lists of 13 criteria that the human cognitive architecture would have to satisfy in order to be functional. We have distilled these into 12 criteria: flexible behavior, real-time performance, adaptive behavior, vast knowledge base, dynamic behavior, knowledge integration, natural language, learning, development, evolution, and brain realization. There would be greater theoretical progress if we evaluated theories by a broad set of criteria such as these and attended to the weaknesses such evaluations revealed. To illustrate how theories can be evaluated we apply these criteria to both classical connectionism (McClelland & Rumelhart 1986; Rumelhart & McClelland 1986b) and the ACT-R theory (Anderson & Lebiere 1998). The strengths of classical connectionism on this test derive from its intense effort in addressing empirical phenomena in such domains as language and cognitive development. Its weaknesses derive from its failure to acknowledge a symbolic level to thought. In contrast, ACT-R includes both symbolic and subsymbolic components. The strengths of the ACT-R theory derive from its tight integration of the symbolic component with the subsymbolic component. Its weaknesses largely derive from its failure, as yet, to adequately engage in intensive analyses of issues related to certain criteria on Newell's list. Key Words: cognitive architecture; connectionism; hybrid systems; language; learning; symbolic systems.

Marinov''s critique I argue, is vitiated by its failure to recognize the distinctive role of superposition within the distributed connectionist paradigm. The use of so-called subsymbolic distributed encodings alone is not, I agree, enough to justify treating distributed connectionism as a distinctive approach. It has always been clear that microfeatural decomposition is both possible and actual within the confines of recognizably classical approaches. When such approaches also involve statistically-driven learning algorithms — as in the case of ID3 — the fundamental differences become even harder to spot. To see them, it is necessary to consider not just the nature of an acquired input-output function but the nature of the representational scheme underlying it. Differences between such schemes make themselves best felt outside the domain of immediate problem solving. It is in the more extended contexts of performance DURING learning and cognitive change as a result of SUBSEQUENT training on new tasks (or simultaneous training on several tasks) that the effects of superpositional storage techniques come to the fore. I conclude that subsymbols, distribution and statistically driven learning alone are indeed not of the essence. But connectionism is not just about subsymbols and distribution. It is about the generation of whole subsymbol SYSTEMS in which multiple distributed representations are created and superposed.

In 1988, Smolensky proposed that connectionist processing systems should be understood as operating at what he termed the `subsymbolic'' level. Subsymbolic systems should be understood by comparing them to symbolic systems, in Smolensky''s view. Up until recently, there have been real problems with analyzing and interpreting the operation of connectionist systems which have undergone training. However, recently published work on a network trained on a set of logic problems originally studied by Bechtel and Abrahamsen (1991) seems to offer the potential to provide a detailed, empirically based answer to questions about the nature of subsymbols. In this paper, a network analysis procedure and the results obtained using it are discussed. This provides the basis for an insight into the nature of subsymbols, which is surprising.