Psychological research on emotion perception anchors heavily on an object perception analogy. We present static “cues,” such as facial expressions, as objects for perceivers to categorize. Yet in the real world, emotions play out as dynamic multidimensional events. Current theoretical approaches and research methods are limited in their ability to capture this complexity. We draw on insights from a predictive coding account of neural activity and a grounded cognition account of concept representation to conceive of emotion perception as a stream (...) of synchronized conceptualizations between two individuals, which is supported and shaped by language. We articulate how this framework can illuminate the fundamental need to study culture, as well as other sources of conceptual variation, in unpacking conceptual synchrony in emotion. We close by suggesting that the conceptual system provides the necessary flexibility to overcome gaps in emotional synchrony. (shrink)

Over a century ago, William James outlined the first psychological constructionist model of emotion, arguing that emotions are phenomena constructed of more basic psychological parts. In this article, I outline a modern psychological constructionist model of emotion. I first explore the history of psychological construction to demonstrate that psychological constructionist models have historically emerged in an attempt to explain variability in emotion that cannot be accounted for by other approaches. I next discuss the modern psychological constructionist model of emotion that (...) I take in my own research, outlining its hypotheses, existing empirical support, and areas of future research. I conclude by arguing that psychological constructionist models can help scientists better understand the human mind. (shrink)

Emotional experiences are often described in metaphoric language. A major question in linguistics and cognitive science is whether such metaphoric linguistic expressions reflect a deeper principle of cognition. Are abstract concepts structured by the embodied, sensorimotor domains that we use to describe them? This review presents the argument for conceptual metaphors of affect and summarizes recent findings from empirical studies. These findings show that, consistent with the conceptual metaphor account, the associations between affect and physical domains such as spatial position, (...) musical pitch, brightness, and size which are captured in linguistic metaphors also influence performance on attention, memory and judgment tasks. Despite this evidence, a number of concerns with metaphor as an account of affect representation are considered. (shrink)

According to simulation or shared-substrates models of emotion recognition, our ability to recognize the emotions expressed by other individuals relies, at least in part, on processes that internally simulate the same emotional state in ourselves. The term “emotional expressions” is nearly synonymous, in many people's minds, with facial expressions of emotion. However, vocal prosody and whole-body cues also convey emotional information. What is the relationship between these various channels of emotional communication? We first briefly review simulation models of emotion recognition, (...) and then discuss neuroscientific evidence related to these models, including studies using facial expressions, whole-body cues, and vocal prosody. We conclude by discussing these data in the context of simulation and shared-substrates models of emotion recognition. (shrink)

Embodied cognitive theories predict that linguistic conceptual representations are grounded and continually represented in real world, sensorimotor experiences. However, there is an on-going debate on whether this also holds for abstract concepts. Grammar is the archetype of abstract knowledge, and therefore constitutes a test case against embodied theories of language representation. Former studies have largely focussed on lexical-level embodied representations. In the present study we take the grounding-by-modality idea a step further by using reaction time (RT) data from the linguistic (...) processing of nominal classifiers in Chinese. We take advantage of an independent body of research, which shows that attention in hand space is biased. Specifically, objects near the hand consistently yield shorter RTs as a function of readiness for action on graspable objects within reaching space, and the same biased attention inhibits attentional disengagement. We predicted that this attention bias would equally apply to the graspable object classifier but not to the big object classifier. Chinese speakers (N = 22) judged grammatical congruency of classifier-noun combinations in two conditions: graspable object classifier and big object classifier. We found that RTs for the graspable object classifier were significantly faster in congruent combinations, and significantly slower in incongruent combinations, than the big object classifier. There was no main effect on grammatical violations, but rather an interaction effect of classifier type. Thus, we demonstrate here grammatical category- specific effects pertaining to the semantic content and by extension the visual and tactile modality of acquisition underlying the acquisition of these categories. We conclude that abstract grammatical categories are subjected to the same mechanisms as general cognitive and neurophysiological processes and may therefore be grounded. (shrink)

How many words—and which ones—are sufficient to define all other words? When dictionaries are analyzed as directed graphs with links from defining words to defined words, they reveal a latent structure. Recursively removing all words that are reachable by definition but that do not define any further words reduces the dictionary to a Kernel of about 10% of its size. This is still not the smallest number of words that can define all the rest. About 75% of the Kernel turns (...) out to be its Core, a “Strongly Connected Subset” of words with a definitional path to and from any pair of its words and no word's definition depending on a word outside the set. But the Core cannot define all the rest of the dictionary. The 25% of the Kernel surrounding the Core consists of small strongly connected subsets of words: the Satellites. The size of the smallest set of words that can define all the rest—the graph's “minimum feedback vertex set” or MinSet—is about 1% of the dictionary, about 15% of the Kernel, and part-Core/part-Satellite. But every dictionary has a huge number of MinSets. The Core words are learned earlier, more frequent, and less concrete than the Satellites, which are in turn learned earlier, more frequent, but more concrete than the rest of the Dictionary. In principle, only one MinSet's words would need to be grounded through the sensorimotor capacity to recognize and categorize their referents. In a dual-code sensorimotor/symbolic model of the mental lexicon, the symbolic code could do all the rest through recombinatory definition. (shrink)

For humans, the ability to communicate and use language is instantiated not only in the vocal modality but also in the visual modality. The main examples of this are sign languages and gestures. Sign languages, the natural languages of Deaf communities, use systematic and conventionalized movements of the hands, face, and body for linguistic expression. Co-speech gestures, though non-linguistic, are produced in tight semantic and temporal integration with speech and constitute an integral part of language together with speech. The articles (...) in this issue explore and document how gestures and sign languages are similar or different and how communicative expression in the visual modality can change from being gestural to grammatical in nature through processes of conventionalization. As such, this issue contributes to our understanding of how the visual modality shapes language and the emergence of linguistic structure in newly developing systems. Studying the relationship between signs and gestures provides a new window onto the human ability to recruit multiple levels of representation in the service of using or creating conventionalized communicative systems. (shrink)

To explain how abstract concepts are grounded in sensory-motor experiences, several theories have been proposed. I will discuss two of these proposals, Conceptual Metaphor Theory and Situated Cognition, and argue why they do not fully explain grounding. A central idea in Conceptual Metaphor Theory is that image schemas ground abstract concepts in concrete experiences. Image schemas might themselves be abstractions, however, and therefore do not solve the grounding problem. Moreover, image schemas are too simple to explain the full richness of (...) abstract concepts. Situated cognition might provide such richness. Research in our laboratory, however, has shown that even for concrete concepts, sensory-motor grounding is task dependent. Therefore, it is questionable whether abstract concepts can be significantly grounded in sensory-motor processing. (shrink)

Traditional views separate cognitive processes from sensory–motor processes, seeing cognition as amodal, propositional, and compositional, and thus fundamentally different from the processes that underlie perceiving and acting. These were the ideas on which cognitive science was founded 30 years ago. However, advancing discoveries in neuroscience, cognitive neuroscience, and psychology suggests that cognition may be inseparable from processes of perceiving and acting. From this perspective, this study considers the future of cognitive science with respect to the study of cognitive development.

In relation to semantics, “grounding” has two relevant meanings. “Symbol grounding” is the process of connecting symbols to perception and the world. “Communicative grounding” is the process of interactively adding to common ground in dialog. Strategies for grounding in human communication include, crucially, strategies for resolving troubles caused by various kinds of miscommunication. As it happens, these two processes of grounding are closely related. As a side-effect of grounding an utterance, dialog participants may adjust the meanings they assign to linguistic (...) expressions, in a process of semantic coordination. Meanings of at least some expressions include perceptual aspects which enable DPs to classify entities as falling under the expression or not based on their perception of those entities. We show how perceptual grounding of symbols can be achieved in a process of interactively adding to common ground. This requires that perceptual aspects of meaning can be updated as a result of participating in linguistic interaction, thereby enabling fine-grained semantic coordination of perceptually grounded linguistic meanings. A formal semantics for low-level perceptual aspects of meaning is presented, tying these together with the logical-inferential aspects of meaning traditionally studied in formal semantics. The key idea is to model perceptual meanings as classifiers of perceptual input. This requires a framework where intensions are represented independently of extensions, and structured objects which can be modified as a result of learning. We use Type Theory with Records, a formal semantics framework which starts from the idea that information and meaning are founded on our ability to perceive and classify the world, that is, to perceive objects and situations as being of types. As an example of our approach, we show how a simple classifier of spatial information based on the Perceptron can be cast in TTR. (shrink)

The literature contains a disconnect between accounts of how humans learn lexical semantic representations for words. Theories generally propose that lexical semantics are learned either through perceptual experience or through exposure to regularities in language. We propose here a model to integrate these two information sources. Specifically, the model uses the global structure of memory to exploit the redundancy between language and perception in order to generate inferred perceptual representations for words with which the model has no perceptual experience. We (...) test the model on a variety of different datasets from grounded cognition experiments and demonstrate that this diverse set of results can be explained as perceptual simulation (cf. Barsalou, Simmons, Barbey, & Wilson, 2003) within a global memory model. (shrink)

The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the visual context impinge minutely on a (...) dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP. (shrink)

Previous studies show that reading sentences about actions leads to specific motor activity associated with actually performing those actions. We investigate how sign language input may modulate motor activation, using British Sign Language sentences, some of which explicitly encode direction of motion, versus written English, where motion is only implied. We find no evidence of action simulation in BSL comprehension, but we find effects of action simulation in comprehension of written English sentences by deaf native BSL signers. These results provide (...) constraints on the nature of mental simulations involved in comprehending action sentences referring to transfer events, suggesting that the richer contextual information provided by BSL sentences versus written or spoken English may reduce the need for action simulation in comprehension, at least when the event described does not map completely onto the signer's own body. (shrink)

Many kinds of creativity result from combination of mental representations. This paper provides a computational account of how creative thinking can arise from combining neural patterns into ones that are potentially novel and useful. We defend the hypothesis that such combinations arise from mechanisms that bind together neural activity by a process of convolution, a mathematical operation that interweaves structures. We describe computer simulations that show the feasibility of using convolution to produce emergent patterns of neural activity that can support (...) cognitive and emotional processes underlying human creativity. (shrink)

The reconciliation of theories of concepts based on prototypes, exemplars, and theory-like structures is a longstanding problem in cognitive science. In response to this problem, researchers have recently tended to adopt either hybrid theories that combine various kinds of representational structure, or eliminative theories that replace concepts with a more finely grained taxonomy of mental representations. In this paper, we describe an alternative approach involving a single class of mental representations called “semantic pointers.” Semantic pointers are symbol-like representations that result (...) from the compression and recursive binding of perceptual, lexical, and motor representations, effectively integrating traditional connectionist and symbolic approaches. We present a computational model using semantic pointers that replicates experimental data from categorization studies involving each prior paradigm. We argue that a framework involving semantic pointers can provide a unified account of conceptual phenomena, and we compare our framework to existing alternatives in accounting for the scope, content, recursive combination, and neural implementation of concepts. (shrink)

Although cognitive scientists have learned a lot about concepts, their findings have yet to be organized in a coherent theoretical framework. In addition, after twenty years of controversy, there is little sign that philosophers and psychologists are converging toward an agreement about the very nature of concepts.Doing without Concepts attempts to remedy this state of affairs. In this article, I review the main points and arguments developed at greater length inDoing without Concepts.

An eye-tracking while listening study based on the blank screen paradigm was conducted to investigate the processing of literal and metaphorical verbs of motion. The study was based on two assumpti...

Using empirical evidence to attack intuitions can be epistemically dangerous, because various of the complaints that one might raise against them (e.g., that they are fallible; that we possess no non-circular defense of their reliability) can be raised just as easily against perception itself. But the opponents of intuition wish to challenge intuitions without at the same time challenging the rest of our epistemic apparatus. How might this be done? Let us use the term “hopefulness” to refer to the extent (...) to which we possess a good capacity for the detection and correction of the errors of any fallible source of evidence. I argue that we should not trust putative sources of evidence that are substantially lacking in hopefulness (even if they are basically reliable), and that we are indeed already operating under such a norm in our ordinary and scientific practices. I argue further that the philosophical practice of the appeal to intuitions is, in these terms, badly hopeless... (shrink)

The phenomenal character of conscious experience has long been regarded as the major problem for physicalist accounts of consciousness. In recent years, defenders of physicalism have typically been relying on the so-called Phenomenal Concept Strategy to avoid dualism. In this paper, we argue with PCS that cognitive-physicalistic explanations can account for the peculiarities of phenomenal character. However, we think that the conceptual features PCS investigates are not the genuine causes of the special characteristics of phenomenal consciousness but only symptoms, which (...) can themselves be explained in terms of the features of the sensory-perceptual representations underlying conscious experiences, namely that some, but not all, of these states are representationally unstructured. (shrink)

Traditionally, the “Imagery Debate” has opposed two main camps: depictivism and descriptivism. This debate has essentially focused on the nature of the internal representations thought to be involved in imagery, without addressing at all the question of action. More recently, a third, “embodied” view is moving the debate into a new phase. The embodied approach focuses on the interdependence of perception, cognition and action, and in its more radical line this approach promotes the idea that perception is not a process (...) involving internal world-models. The anti-representationalist version of the embodied paradigm covers, among others that we shall not discuss here, two quite different positions, namely the enactive approach and sensorimotor theory. Up to now these two anti-representationalist accounts have generally been confounded. In this paper we will argue that despite some important commonalities, enactive and sensorimotor accounts come with distinctive theoretical traits with regard to their approach to imagery. These become manifest when critically examining the role they assign to sensorimotor engagements with the world. We shall argue that enactive and sensorimotor approaches are different in their understanding of the role of embodied action, and these different notions of embodiment lead to different explanatory accounts of perception and imagery. We propose that, due to existing ambiguities in enactivism, the sensorimotor theory is a better framework for a skill-based approach to imagery. (shrink)