There has been much discussion recently about the scope and limits of purely symbolic models of the mind and about the proper role of connectionism in cognitive modeling. This paper describes the symbol grounding problem : How can the semantic interpretation of a formal symbol system be made intrinsic to the system, rather than just parasitic on the meanings in our heads? How can the meanings of the meaningless symbol tokens, manipulated solely on the basis of (...) their shapes, be grounded in anything but other meaningless symbols? The problem is analogous to trying to learn Chinese from a Chinese/Chinese dictionary alone. A candidate solution is sketched: Symbolic representations must be grounded bottom-up in nonsymbolic representations of two kinds: iconic representations, which are analogs of the proximal sensory projections of distal objects and events, and categorical representations, which are learned and innate feature-detectors that pick out the invariant features of object and event categories from their sensory projections. Elementary symbols are the names of these object and event categories, assigned on the basis of their categorical representations. Higher-order symbolic representations, grounded in these elementary symbols, consist of symbol strings describing category membership relations. Connectionism is one natural candidate for the mechanism that learns the invariant features underlying categorical representations, thereby connecting names to the proximal projections of the distal objects they stand for. In this way connectionism can be seen as a complementary component in a hybrid nonsymbolic/symbolic model of the mind, rather than a rival to purely symbolic modeling. Such a hybrid model would not have an autonomous symbolic module, however; the symbolic functions would emerge as an intrinsically dedicated symbol system as a consequence of the bottom-up grounding of categories ' names in their sensory representations. Symbol manipulation would be governed not just by the arbitrary shapes of the symbol tokens, but by the nonarbitrary shapes of the icons and category invariants in which they are grounded. (shrink)

The paper presents a paradoxical feature of computational systems that suggests that computationalism cannot explain symbol grounding. If the mind is a digital computer, as computationalism claims, then it can be computing either over meaningful symbols or over meaningless symbols. If it is computing over meaningful symbols its functioning presupposes the existence of meaningful symbols in the system, i.e. it implies semantic nativism. If the mind is computing over meaningless symbols, no intentional cognitive processes are available prior to (...) symbol grounding. In this case, no symbol grounding could take place since any grounding presupposes intentional cognitive processes. So, whether computing in the mind is over meaningless or over meaningful symbols, computationalism implies semantic nativism. (shrink)

Floridi and Taddeo propose a condition of “zero semantic commitment” for solutions to the grounding problem, and a solution to it. I argue briefly that their condition cannot be fulfilled, not even by their own solution. After a look at Luc Steels' very different competing suggestion, I suggest that we need to re-think what the problem is and what role the ‘goals’ in a system play in formulating the problem. On the basis of a proper understanding of computing, I (...) come to the conclusion that the only sensible ground-ing problem is how we can explain and re-produce the behavioral ability and function of meaning in artificial computational agents. (shrink)

The symbol grounding problem is concerned with the question of how the knowledge used in AI programs, expressed as tokens in one form or another or simply symbols, could be grounded to the outside world. By grounding the symbols, it is meant that the system will know the actual objects, events, or states of affairs in the world to which each symbol refers and thus be worldly-wise. Solving this problem, it was hoped, would enable the program (...) to understand its own action and hence be truly intelligent ). The problem becomes more acute after a challenge posed by Searle in his now famous Chinese Room Gedanken experiment. Searle argued that no AI programs can be said to understand or have other cognitive states, if all they do is formal symbol manipulation. (shrink)

Symbols should be grounded, as has been argued before. But we insist that they should be grounded not only in subsymbolic activities, but also in the interaction between the agent and the world. The point is that concepts are not formed in isolation (from the world), in abstraction, or "objectively." They are formed in relation to the experience of agents, through their perceptual/motor apparatuses, in their world and linked to their goals and actions. This paper takes a detailed look at (...) this relatively old issue, with a new perspective, aided by our work of computational cognitive model development. To further our understanding, we also go back in time to link up with earlier philosophical theories related to this issue. The result is an account that extends from computational mechanisms to philosophical abstractions. (shrink)

Deacon speculates on the origin of interpretation of signs using autocatalytic origin of life models and Peircean terminology. I explain why interpretation evolved only later as a triadic intervention between symbols and actions. In all organisms the passive one-dimensional genetic informational symbol sequences are converted to active functional proteins or nucleic acids by three-dimensional folding. This symbol grounding is a direct symbol-to-action conversion. It is universal throughout all evolution. Folding is entirely a lawful physical process, leaving (...) neither freedom nor necessity for interpretation. Similarly, the initial converse action-to-symbol conversion of sensory inputs also leaves no freedom for interpretation until after the action-to-symbol conversion. (shrink)

Scholars studying the origins and evolution of language are also interested in the general issue of the evolution of cognition. Language is not an isolated capability of the individual, but has intrinsic relationships with many other behavioral, cognitive, and social abilities. By understanding the mechanisms underlying the evolution of linguistic abilities, it is possible to understand the evolution of cognitive abilities. Cognitivism, one of the current approaches in psychology and cognitive science, proposes that symbol systems capture mental phenomena, and (...) attributes cognitive validity to them. Therefore, in the same way that language is considered the prototype of cognitive abilities, a symbol system has become the prototype for studying language and cognitive systems. Symbol systems are advantageous as they are easily studied through computer simulation (a computer program is a symbol system itself), and this is why language is often studied using computational models. (shrink)

What language allows us to do is to "steal" categories quickly and effortlessly through hearsay instead of having to earn them the hard way, through risky and time-consuming sensorimotor "toil" (trial-and-error learning, guided by corrective feedback from the consequences of miscategorisation). To make such linguistic "theft" possible, however, some, at least, of the denoting symbols of language must first be grounded in categories that have been earned through sensorimotor toil (or else in categories that have already been "prepared" for us (...) through Darwinian theft by the genes of our ancestors); it cannot be linguistic theft all the way down. The symbols that denote categories must be grounded in the capacity to sort, label and interact with the proximal sensorimotor projections of their distal category-members in a way that coheres systematically with their semantic interpretations, both for individual symbols, and for symbols strung together to express truth-value-bearing propositions. (shrink)

This paper illustrates how external symbol grounding can be studied in simulations with large populations. We discuss how we can simulate language evolution in a relatively complex environment which has been developed in the context of the New Ties project. This project has the objective of evolving a cultural society and, in doing so, the agents have to evolve a communication system that is grounded in their interactions with their virtual environment and with other individuals. A preliminary experiment (...) is presented in which we investigate the effect of a number of learning mechanisms. The results show that the social symbol grounding problem is a particularly hard one; however, we provide an ideal platform to study this problem. (shrink)

This paper illustrates how external symbol grounding can be studied in simulations with large populations. We discuss how we can simulate language evolution in a relatively complex environment which has been developed in the context of the New Ties project. This project has the objective of evolving a cultural society and, in doing so, the agents have to evolve a communication system that is grounded in their interactions with their virtual environment and with other individuals. A preliminary experiment (...) is presented in which we investigate the effect of a number of learning mechanisms. The results show that the social symbol grounding problem is a particularly hard one; however, we provide an ideal platform to study this problem. (shrink)

Theories of embodied cognition assume that concepts are grounded in non-linguistic, sensorimotor experience. In support of this assumption, previous studies have shown that upwards response movements are faster than downwards movements after participants have been presented with words whose referents are typically located in the upper vertical space. This is taken as evidence that processing these words reactivates sensorimotor experiential traces. This congruency effect was also found for novel words, after participants learned these words as labels for novel objects that (...) they encountered either in their upper or lower visual field. While this indicates that direct experience with a word's referent is sufficient to evoke said congruency effects, the present study investigates whether this direct experience is also a necessary condition. To this end, we conducted five experiments in which participants learned novel words from purely linguistic input: Novel words were presented in pairs with real up- or down-words ; they were presented in natural sentences where they replaced these real words ; they were presented as new labels for these real words ; and they were presented as labels for novel combined concepts based on these real words. In all five experiments, we did not find any congruency effects elicited by the novel words; however, participants were always able to make correct explicit judgements about the vertical dimension associated to the novel words. These results suggest that direct experience is necessary for reactivating experiential traces, but this reactivation is not a necessary condition for understanding the corresponding aspects of word meaning. (shrink)

The symbol grounding problem (SGP) proposed by Stevan Harnad in 1990, originates from Searle’s “Chinese Room Argument” and refers to the problem of how a pure symbolic system acquires its meaning. While many solutions to this problem have been proposed, all of them have encountered inconsistencies to different extents. A recent approach for resolving the problem is to divide the SGP into hard and easy problems echoing the distinction between hard and easy problems for resolving the enigma of (...) consciousness. This however turns out not to be an ideal strategy: Everything related to consciousness that cannot be well-explained by present theories can be categorized as a hard problem which as a consequence would doom the SGP to irresolvability. We therefore argue that the SGP can be regarded as a general problem of how an AI system can have intentionality, and develop a theoretical direction for its solution. (shrink)

This paper develops a bridge from AL issues about the symbol–matter relation to AI issues about symbol-grounding by focusing on the concepts of formality and syntactic interpretability. Using the DNA triplet-amino acid specification relation as a paradigm, it is argued that syntactic properties can be grounded as high-level features of the non-syntactic interactions in a physical dynamical system. This argu- ment provides the basis for a rebuttal of John Searle’s recent assertion that syntax is observer-relative (1990, 1992). (...) But the argument as developed also challenges the classic symbol-processing theory of mind against which Searle is arguing, as well as the strong AL thesis that life is realizable in a purely computational medium. Finally, it provides a new line of support for the autonomous systems approach in AL and AI (Varela & Bourgine 1992a, 1992b). © 1997 Academic Press. (shrink)

This article reviews eight proposed strategies for solving the Symbol Grounding Problem (SGP), which was given its classic formulation in Harnad (1990). After a concise introduction, we provide an analysis of the requirement that must be satisfied by any hypothesis seeking to solve the SGP, the zero semantical commitment condition. We then use it to assess the eight strategies, which are organised into three main approaches: representationalism, semi-representationalism and non-representationalism. The conclusion is that all the strategies are semantically (...) committed and hence that none of them provides a valid solution to the SGP, which remains an open problem. (shrink)

The aim of the paper is to present the underlying reason of the unsolved symbol grounding problem. The Church-Turing Thesis states that a physical problem, for which there is an algorithm of solution, can be solved by a Turing machine, but machine operations neglect the semantic relationship between symbols and their meaning. Symbols are objects that are manipulated on rules based on their shapes. The computations are independent of the context, mental states, emotions, or feelings. The symbol (...) processing operations are interpreted by the machine in a way quite different from the cognitive processes. Cognitive activities of living organisms and computation differ from each other, because of the way they act in the real word. The result is the problem of mutual understanding of symbol grounding. (shrink)

This article is the second step in our research into the Symbol Grounding Problem (SGP). In a previous work, we defined the main condition that must be satisfied by any strategy in order to provide a valid solution to the SGP, namely the zero semantic commitment condition (Z condition). We then showed that all the main strategies proposed so far fail to satisfy the Z condition, although they provide several important lessons to be followed by any new proposal. (...) Here, we develop a new solution of the SGP. It is called praxical in order to stress the key role played by the interactions between the agents and their environment. It is based on a new theory of meaning—Action-based Semantics (AbS)—and on a new kind of artificial agents, called two-machine artificial agents (AM²). Thanks to their architecture, AM2s implement AbS, and this allows them to ground their symbols semantically and to develop some fairly advanced semantic abilities, including the development of semantically grounded communication and the elaboration of representations, while still respecting the Z condition. (shrink)

The Chinese room argument has presented a persistent headache in the search for Artificial Intelligence. Since it first appeared in the literature, various interpretations have been made, attempting to understand the problems posed by this thought experiment. Throughout all this time, some researchers in the Artificial Intelligence community have seen Symbol Grounding as proposed by Harnad as a solution to the Chinese room argument. The main thesis in this paper is that although related, these two issues present different (...) problems in the framework presented by Harnad himself. The work presented here attempts to shed some light on the relationship between John Searle’s intentionality notion and Harnad’s Symbol Grounding Problem. (shrink)

Taking a distributed view of language, this paper naturalizes symbol grounding. Learning to talk is traced to — not categorizing speech sounds — but events that shape the rise of human-style autonomy. On the extended symbol hypothesis, this happens as babies integrate micro-activity with slow and deliberate adult action. As they discover social norms, intrinsic motive formation enables them to reshape co-action. Because infants link affect to contingencies, dyads develop norm-referenced routines. Over time, infant doings become analysis (...) amenable. The caregiver of a nine-month-old may, for example, prompt the baby to fetch objects. Once she concludes that the baby uses ‘words’ to understand what she says, the infant can use this belief in orienting to more abstract contingencies. New cognitive powers will develop as the baby learns to act in ways that are consistent with a caregiver’s false belief that her baby uses ‘words.’. (shrink)

Cognitive theories based on a fixed feature set suffer from frame and symbol grounding problems. Flexible features and other empirically acquired constraints (e.g., analog-to-analog mappings) provide a framework for letting extrinsic relations influence symbol manipulation. By offering a biologically plausible basis for feature learning, nonorthogonal multiresolution analysis and dimensionality reduction, informed by functional constraints, may contribute to a solution to the symbol grounding problem.

After people learn to sort objects into categories they see them differently. Members of the same category look more alike and members of different categories look more different. This phenomenon of within-category compression and between-category separation in similarity space is called categorical perception (CP). It is exhibited by human subjects, animals and neural net models. In backpropagation nets trained first to auto-associate 12 stimuli varying along a onedimensional continuum and then to sort them into 3 categories, CP arises as a (...) natural side-effect because of four factors: (1) Maximal interstimulus separation in hidden-unit space during autoassociation learning, (2) movement toward linear separability during categorization learning, (3) inverse-distance repulsive force exerted by the between-category boundary, and (4) the modulating effects of input iconicity, especially in interpolating CP to untrained regions of the continuum. Once similarity space has been "warped" in this way, the compressed and separated "chunks" have symbolic labels which could then be combined into symbol strings that constitute propositions about objects. The meanings of such symbolic representations would be "grounded" in the system's capacity to pick out from their sensory projections the object categories that the propositions were about. (shrink)

As the accompanying articles in the Special Issue on Semiotic Scaffolding will attest, my colleagues in biosemiotics have done an exemplary job in showing us how to think about the critically generative role that semiotic scaffolding plays “vertically” – i.e., in evolutionary and developmental terms – by “allowing access to the upper floors” of biological complexity, cognition and evolution.In addition to such diachronic considerations of semiotic scaffolding, I wish to offer here a consideration of semiotic scaffolding’s synchronic power, as well (...) – and in particular the ability that it can afford its users to access new and other sign relations “horizontally” as a function of the way that multiple semiotically scaffolded relations intertwine to result in a “definite semantic topology that determines the ways that symbols modify each other’s referential functions in different combinations”.Taking up, in turn, Terrence Deacon’s later challenge that what the sciences of cognition – and biology more generally – needs to “come to grips with [is] the process of semiosis; the dynamic of interpretive activity by which semiotic relationships emerge from other semiotic relationships [as] intrinsically dynamic phases in a generative process”, I attempt here to show how Deacon’s own Peirce-inspired matrix of referential sign relations as presented in The Symbolic Species, when viewed as a semiotic scaffold of interactional constraints and possibility biases, provides the key to understanding the essentially thirdness-manifesting nature of symbol reference, formation and growth. (shrink)

The double function of language, as a social/communicative means, and as an individual/cognitive capability, derives from its fundamental property that allows us to internally re-represent the world we live in. This is possible through the mechanism of symbol grounding, i.e., the ability to associate entities and states in the external and internal world with internal categorical representations. The symbol grounding mechanism, as language, has both an individual and a social component. The individual component, called the “Physical (...) Symbol Grounding“, refers to the ability of each individual to create an intrinsic link between world entities and internal categorical representations. The social component, called “Social Symbol Grounding“, refers to the collective negotiation for the selection of shared symbols and their grounded meanings. The paper discusses these two aspects of symbol grounding in relation to distributed cognition, using examples from cognitive modeling research on grounded agents and robots. (shrink)

1.1 The predominant approach to cognitive modeling is still what has come to be called "computationalism" (Dietrich 1990, Harnad 1990b), the hypothesis that cognition is computation. The more recent rival approach is "connectionism" (Hanson & Burr 1990, McClelland & Rumelhart 1986), the hypothesis that cognition is a dynamic pattern of connections and activations in a "neural net." Are computationalism and connectionism really deeply different from one another, and if so, should they compete for cognitive hegemony, or should they collaborate? These (...) questions will be addressed here, in the context of an obstacle that is faced by computationalism (as well as by connectionism if it is either computational or seeks cognitive hegemony on its own): The symbol grounding problem (Harnad 1990). (shrink)

The traditional view of symbol grounding seeks to connect an a priori internal representation or ‘form’ to its external referent. But such a ‘form’ is usually itself systematically composed out of more primitive parts, so this view ignores its grounding in the physics of the world. Some previous work simulating multiple talking/listening agents has effectively taken this stance, and shown how a shared discrete speech code can emerge. Taking the earlier work of Oudeyer, we have extended his (...) model to include a dispersive force intended to account broadly for a speaker’s motivation to increase auditory distinctiveness. New simulations show that vowel systems result that are more representative of the range seen in human languages. These simulations make many profound abstractions and assumptions. Relaxing these by including more physically and physiologically realistic mechanisms for talking and listening is seen as the key to replicating more complex and dynamic aspects of speech, such as consonant-vowel patterning. (shrink)

The traditional view of symbol grounding seeks to connect an a priori internal representation or ‘form’ to its external referent. But such a ‘form’ is usually itself systematically composed out of more primitive parts, so this view ignores its grounding in the physics of the world. Some previous work simulating multiple talking/listening agents has effectively taken this stance, and shown how a shared discrete speech code can emerge. Taking the earlier work of Oudeyer, we have extended his (...) model to include a dispersive force intended to account broadly for a speaker’s motivation to increase auditory distinctiveness. New simulations show that vowel systems result that are more representative of the range seen in human languages. These simulations make many profound abstractions and assumptions. Relaxing these by including more physically and physiologically realistic mechanisms for talking and listening is seen as the key to replicating more complex and dynamic aspects of speech, such as consonant-vowel patterning. (shrink)

According to Lee and Schwarz, the sensorimotor experience of cleansing involves separating one physical entity from another and grounds mental separation of one psychological entity from another. We propose that cleansing effects may result from symbolic cognition. Instead of viewing abstract meanings as emerging from concrete physical acts of cleansing, this physical act may be appended with pre-existing, symbolic meaning.

According to "computationalism" (Newell, 1980; Pylyshyn 1984; Dietrich 1990), mental states are computational states, so if one wishes to build a mind, one is actually looking for the right program to run on a digital computer. A computer program is a semantically interpretable formal symbol system consisting of rules for manipulating symbols on the basis of their shapes, which are arbitrary in relation to what they can be systematically interpreted as meaning. According to computationalism, every physical implementation of the (...) right symbol system will have mental states. (shrink)

The proposed theory is broad enough to accommodate the reduction or elimination of prior influences by a variety of acts symbolizing separation. However, it does not account for stability in psychological variables, and is contradicted by widely documented stability in people's actual attitudes and behavior over time, in multiple domains, despite people's pervasive everyday acts of symbolic separation.

Amodality is not incompatible with being originally derived from sensory experience. The transformation of perceptual symbols into amodal abstractions could take place spontaneously through self-organizing processes such as autocatalysis. The organizational role played by “simulators” happens implicitly in a neural network, and quite possibly, in the brain as well.

It was once common to regard ASL as less than a full-fledged language, as a mere combination of miming, pointing and a few primitive gestures. That conception of ASL was laid to rest by William Stokoe’s landmark work [22] and much careful research that has come in its wake. This work..

Ramscar and colleagues (2010, this volume) describe the “feature-label-order” (FLO) effect on category learning and characterize it as a constraint on symbolic learning. I argue that FLO is neither a constraint on symbolic learning in the sense of “learning elements of a symbol system” (instead, it is an effect on nonsymbolic, association learning) nor is it, more than any other constraint on category learning, a constraint on symbolic learning in the sense of “solving the symbol grounding problem.”.

We are reminded of relevant stories, tales, or symbols from long-term memory when facing a novel problem our daily lives. Visual cues are 1 tool known to facilitate reminding. In 2 experiments, Chinese students, who had experienced a folk tale many years ago during childhood, were asked to solve an analogous problem. We tested the hypothesis that a visual cue can help bridge the gap between a novel problem and a source analogy experienced in the distant past. Different types of (...) cues proved to have distinct influences on components of analogical problem solving. This research suggests that analogical representational features of a story or problem can be stored in long-term memory and can be activated by visual cues. The implications of these findings for theories that point to perceptual or grounded, rather than decontextualized structures in knowledge representation are discussed. (shrink)

This article proposes a number of models to examine through which mechanisms a population of autonomous agents could arrive at a repertoire of perceptually grounded categories that is sufficiently shared to allow successful communication. The models are inspired by the main approaches to human categorisation being discussed in the literature: nativism, empiricism, and culturalism. Colour is taken as a case study. Although we take no stance on which position is to be accepted as final truth with respect to human categorisation (...) and naming, we do point to theoretical constraints that make each position more or less likely and we make clear suggestions on what the best engineering solution would be. Specifically, we argue that the collective choice of a shared repertoire must integrate multiple constraints, including constraints coming from communication. Key Words: autonomous agents; colour categorisation; colour naming; connectionism; cultural evolution; genetic evolution; memes; origins of language; self-organisation; semiotic dynamics; symbol grounding. (shrink)

I see four symbol grounding problems: 1) How can a purely computational mind acquire meaningful symbols? 2) How can we get a computational robot to show the right linguistic behavior? These two are misleading. I suggest an 'easy' and a 'hard' problem: 3) How can we explain and re-produce the behavioral ability and function of meaning in artificial computational agents?4) How does physics give rise to meaning?

This edited volume covers new ground by bringing together perspectives from symbolic legislation theory on the one hand, and from biolaw and bioethics on the other hand. Symbolic legislation has a bad name. It usually refers to instances of legislation which are ineffective and that serve other political and social goals than the goals officially stated. Recently, a more positive notion of symbolic legislation has emerged in legislative theory. From this perspective, symbolic legislation is regarded as a positive alternative to (...) the more traditional, top-down legislative approach. The legislature no longer merely issues commands backed up with severe sanctions, as in instrumental legislation. Instead, lawmakers provide open and aspirational norms that are meant to change behavior not by means of threat, but indirectly, through debate and social interaction. Since the 1990s, biomedical developments have revived discussions on symbolic legislation. One of the reasons is that biomedical legislation touches on deep-rooted, symbolic-cultural representations of the biological aspects of human life. Moreover, as it is often impossible to reach consensus on these controversial questions, legislators have sought alternative ways to develop legal frameworks. Consequently, communicative and interactive approaches to legislation are prominent within the governance of medical biotechnology. The symbolic dimensions of biolaw are often overlooked. Yet, it is clear that the symbolic is at the heart of many legal-political debates on bioethical questions. Since the rise of biomedical technologies, human body materials have acquired a scientific, medical and even commercial value. These new approaches, which radically question existing legal symbolizations of the human body, raise the question whether and how the law should continue to reflect symbolic values and meanings. Moreover, how can we decide what these symbolic values are, given the fact that we live in a pluralistic society? (shrink)

Thirty years ago, grounded cognition had roots in philosophy, perception, cognitive linguistics, psycholinguistics, cognitive psychology, and cognitive neuropsychology. During the next 20 years, grounded cognition continued developing in these areas, and it also took new forms in robotics, cognitive ecology, cognitive neuroscience, and developmental psychology. In the past 10 years, research on grounded cognition has grown rapidly, especially in cognitive neuroscience, social neuroscience, cognitive psychology, social psychology, and developmental psychology. Currently, grounded cognition appears to be achieving increased acceptance throughout cognitive (...) science, shifting from relatively minor status to increasing importance. Nevertheless, researchers wonder whether grounded mechanisms lie at the heart of the cognitive system or are peripheral to classic symbolic mechanisms. Although grounded cognition is currently dominated by demonstration experiments in the absence of well-developed theories, the area is likely to become increasingly theory driven over the next 30 years. Another likely development is the increased incorporation of grounding mechanisms into cognitive architectures and into accounts of classic cognitive phenomena. As this incorporation occurs, much functionality of these architectures and phenomena is likely to remain, along with many original mechanisms. Future theories of grounded cognition are likely to be heavily influenced by both cognitive neuroscience and social neuroscience, and also by developmental science and robotics. Aspects from the three major perspectives in cognitive science—classic symbolic architectures, statistical/dynamical systems, and grounded cognition—will probably be integrated increasingly in future theories, each capturing indispensable aspects of intelligence. (shrink)