Online research in philosophy

The goal of philosophy of information is to understand what information is, how it operates, and how to put it to work. But unlike ‘information’ in the technical sense of information theory, what we are interested in is meaningful information. To understand the nature and dynamics of information in this sense we have to understand meaning. What we offer here are simple computational models that show emergence of meaning and information transfer in randomized arrays of neural nets. These we take to be formal instantiations of a tradition of theories of meaning as use. What they offer, we propose, is a glimpse into the origin and dynamics of at least simple forms of meaning and information transfer as properties inherent in behavioral coordination across a community.

It is unlikely that the systematic, compositional properties of formal symbol systems -- i.e., of computation -- play no role at all in cognition. However, it is equally unlikely that cognition is just computation, because of the symbol grounding problem (Harnad 1990): The symbols in a symbol system are systematically interpretable, by external interpreters, as meaning something, and that is a remarkable and powerful property of symbol systems. Cognition (i.e., thinking), has this property too: Our thoughts are systematically interpretable by external interpreters as meaning something. However, unlike symbols in symbol systems, thoughts mean what they mean autonomously: Their meaning does not consist of or depend on anyone making or being able to make any external interpretations of them at all. When I think "the cat is on the mat," the meaning of that thought is autonomous; it does not depend on YOUR being able to interpret it as meaning that (even though you could interpret it that way, and you would be right).

Stevan Harnad correctly perceives a deep problem in computationalism, the hypothesis that cognition is computation, namely, that the symbols manipulated by a computational entity do not automatically mean anything. Perhaps, he proposes, transducers and neural nets will not have this problem. His analysis goes wrong from the start, because computationalism is not as rigid a set of theories as he thinks. Transducers and neural nets are just two kinds of computational system, among many, and any solution to the semantic problem that works for them will work for most other computational systems.

Connectionism and computationalism are currently vying for hegemony in cognitive modeling. At first glance the opposition seems incoherent, because connectionism is itself computational, but the form of computationalism that has been the prime candidate for encoding the "language of thought" has been symbolic computationalism (Dietrich 1990, Fodor 1975, Harnad 1990c; Newell 1980; Pylyshyn 1984), whereas connectionism is nonsymbolic (Fodor & Pylyshyn 1988, or, as some have hopefully dubbed it, "subsymbolic" Smolensky 1988). This paper will examine what is and is not a symbol system. A hybrid nonsymbolic/symbolic system will be sketched in which the meanings of the symbols are grounded bottom-up in the system's capacity to discriminate and identify the objects they refer to. Neural nets are one possible mechanism for learning the invariants in the analog sensory projection on which successful categorization is based. "Categorical perception" (Harnad 1987a), in which similarity space is "warped" in the service of categorization, turns out to be exhibited by both people and nets, and may mediate the constraints exerted by the analog world of objects on the formal world of symbols.

I argue that neural activity, strictly speaking, is not computation. This is because computation, strictly speaking, is the processing of strings of symbols, and neuroscience shows that there are no neural strings of symbols. This has two consequences. On the one hand, the following widely held consequences of computationalism must either be abandoned or supported on grounds independent of computationalism: (i) that in principle we can capture what is functionally relevant to neural processes in terms of some formalism taken from computability theory (such as Turing Machines), (ii) that it is possible to design computer programs that are functionally equivalent to neural processes in the same sense in which it is possible to design computer programs that are functionally equivalent to each other, (iii) that the study of neural (or mental) computation is independent of the study of neural implementation, (iv) that the Church-Turing thesis applies to neural activity in the sense in which it applies to digital computers. On the other hand, we need to gradually reinterpret or replace computational theories in psychology in terms of theoretical constructs that can be realized by known neural processes, such as the spike trains of neuronal ensembles.Â.

I argue that neural activity, strictly speaking, is not computation. This is because computation, strictly speaking, is the processing of strings of symbols, and neuroscience shows that there are no neural strings of symbols. This has two consequences. On the one hand, the following widely held consequences of computationalism must either be abandoned or supported on grounds independent of computationalism: (i) that in principle we can capture what is functionally relevant to neural processes in terms of some formalism taken from computability theory (such as Turing Machines), (ii) that it is possible to design computer programs that are functionally equivalent to neural processes in the same sense in which it is possible to design computer programs that are functionally equivalent to each other, (iii) that the study of neural (or mental) computation is independent of the study of neural implementation, (iv) that the Church-Turing thesis applies to neural activity in the sense in which it applies to digital computers. On the other hand, we need to gradually reinterpret or replace computational theories in psychology in terms of theoretical constructs that can be realized by known neural processes, such as the spike trains of neuronal ensembles.

I address whether neural networks perform computations in the sense of computability theory and computer science. I explicate and defend
the following theses. (1) Many neural networks compute—they perform computations. (2) Some neural networks compute in a classical way.
Ordinary digital computers, which are very large networks of logic gates, belong in this class of neural networks. (3) Other neural networks
compute in a non-classical way. (4) Yet other neural networks do not perform computations. Brains may well fall into this last class.

"Symbol Grounding" is beginning to mean too many things to too many people. My own construal has always been simple: Cognition cannot be just computation, because computation is just the systematically interpretable manipulation of meaningless symbols, whereas the meanings of my thoughts don't depend on their interpretability or interpretation by someone else. On pain of infinite regress, then, symbol meanings must be grounded in something other than just their interpretability if they are to be candidates for what is going on in our heads. Neural nets may be one way to ground the names of concrete objects and events in the capacity to categorize them (by learning the invariants in their sensorimotor projections). These grounded elementary symbols could then be combined into symbol strings expressing propositions about more abstract categories. Grounding does not equal meaning, however, and does not solve any philosophical problems.

Harnad's main argument can be roughly summarised as follows: due to Searle's Chinese Room argument, symbol systems by themselves are insufficient to exhibit cognition, because the symbols are not grounded in the real world, hence without meaning. However, a symbol system that is connected to the real world through transducers receiving sensory data, with neural nets translating these data into sensory categories, would not be subject to the Chinese Room argument. Harnad's article is not only the starting point for the present debate, but is also a contribution to a longlasting discussion about such questions as: Can a computer think? If yes, would this be solely by virtue of its program? Is the Turing Test appropriate for deciding whether a computer thinks?

Some of the features of animal and human categorical perception (CP) for color, pitch and speech are exhibited by neural net simulations of CP with one-dimensional inputs: When a backprop net is trained to discriminate and then categorize a set of stimuli, the second task is accomplished by "warping" the similarity space (compressing within-category distances and expanding between-category distances). This natural side-effect also occurs in humans and animals. Such CP categories, consisting of named, bounded regions of similarity space, may be the ground level out of which higher-order categories are constructed; nets are one possible candidate for the mechanism that learns the sensorimotor invariants that connect arbitrary names (elementary symbols?) to the nonarbitrary shapes of objects. This paper examines how and why such compression/expansion effects occur in neural nets.