An action-oriented perspective changes the role of an individual from a passive observer to an actively engaged agent interacting in a closed loop with the world as well as with others. Cognition exists to serve action within a landscape that contains both. This chapter surveys this landscape and addresses the status of the pragmatic turn. Its potential influence on science and the study of cognition are considered (including perception, social cognition, social interaction, sensorimotor entrainment, and language acquisition) and its impact (...) on how neuroscience is studied is also investigated (with the notion that brains do not passively build models, but instead support the guidance of action). A review of its implications in robotics and engineering includes a discussion of the application of enactive control principles to couple action and perception in robotics as well as the conceptualization of system design in a more holistic, less modular manner. Practical applications that can impact the human condition are reviewed (e.g., educational applications, treatment possibilities for developmental and psychopathological disorders, the development of neural prostheses). All of this foreshadows the potential societal implications of the pragmatic turn. The chapter concludes that an action-oriented approach emphasizes a continuum of interaction between technical aspects of cognitive systems and robotics, biology, psychology, the social sciences, and the humanities, where the individual is part of a grounded cultural system. (shrink)

In robotics research with language-based interaction, simplifications are made, such that a given event can be described in a unique manner, where there is a direct mapping between event representations and sentences that can describe these events. However, common experience tells us that the same physical event can be described in multiple ways, depending on the perspective of the speaker. The current research develops methods for representing events from multiple perspectives, and for choosing the perspective that will be used for (...) generating a linguistic construal, based on attentional processes in the system. The multiple perspectives are based on the principle that events can be considered in terms of the force driving the event, and the result obtained from the event, based on the theory of Gärdenfors. In addition, within these perspectives a further refinement can be made with respect to the agent, object, and recipient perspectives. We develop a system for generating appropriate construals of meaning, and demonstrate how this can be used in a realistic dialogic interaction between a behaving robot and a human interlocutor. (shrink)

In the context of Hurford's claim that “some feature of language structure maps onto a feature of primitive mental representations,” I will argue that Hurford's focus on 1-place predicates as the basis of the “mental representations of situations in the world” is problematic, particularly with respect to spatiotemporal events. A solution is proposed.

The development of reasoning systems exploiting expert knowledge from interactions with humans is a non-trivial problem, particularly when considering how the information can be coded in the knowledge representation. For example, in human development, the acquisition of knowledge at one level requires the consolidation of knowledge from lower levels. How is the accumulated experience structured to allow the individual to apply knowledge to new situations, allowing reasoning and adaptation? We investigate how this can be done automatically by an iCub that (...) interacts with humans to acquire knowledge via demonstration. Once consolidated, this knowledge is used in further acquisitions of experience concerning preconditions and consequences of actions. Finally, this knowledge is translated into rules that allow reasoning and planning for novel problem solving, including a Tower of Hanoi scenario. We thus demonstrate proof of concept for an interaction system that uses knowledge acquired from human interactions to reason about new situations. (shrink)

How do people learn to talk about the causal and temporal relations between events, and the motivation behind why people do what they do? The narrative practice hypothesis of Hutto and Gallagher holds that children are exposed to narratives that provide training for understanding and expressing reasons for why people behave as they do. In this context, we have recently developed a model of narrative processing where a structured model of the developing situation is built up from experienced events, and (...) enriched by sentences in a narrative that describe event meanings. The main interest is to develop a proof of concept for how narrative can be used to structure, organize and describe experience. Narrative sentences describe events, and they also define temporal and causal relations between events. These relations are specified by a class of narrative function words, including “because, before, after, first, finally.” The current research develops a proof of concept that by observing how people describe social events, a developmental robotic system can begin to acquire early knowledge of how to explain the reasons for events. We collect data from naïve subjects who use narrative function words to describe simple scenes of human-robot interaction, and then employ algorithms for extracting the statistical structure of how narrative function words link events in the situation model. By using these statistical regularities, the robot can thus learn from human experience about how to properly employ in question-answering dialogues with the human, and in generating canonical narratives for new experiences. The behavior of the system is demonstrated over several behavioral interactions, and associated narrative interaction sessions, while a more formal extended evaluation and user study will be the subject of future research. Clearly this is far removed from the power of the full blown narrative practice capability, but it provides a first step in the development of an experimental infrastructure for the study of socially situated narrative practice in human-robot interaction. (shrink)

Deictic pointers allow the nervous system to exploit information in a frame that is centered on the object of interest. This processing may take place in visual or haptic space, but the information processing advantages of deictic pointing can also be applied in abstract spaces, providing the basis for analogical transfer. Simulation and behavioral results illustrating this progression from embodiment to abstraction are discussed.

Pickering & Garrod (P&G) describe a mechanism by which the situation models of dialog participants become progressively aligned via priming at different levels. This commentary attempts to characterize how alignment and routinization can be extended into the language acquisition domain by establishing links between alignment and joint attention, and between routinization and grammatical construction learning.

In Carruthers’ formulation, cross-domain thinking requires translation of domain specific data into a common format, and linguistic LF thus plays the role of the common medium of exchange. Alternatively, I propose a process-oriented characterization, in which there is no common representation and cross-domain thinking is rather the process of establishing mappings across domains, as in the process of analogical reasoning.

Phrasal semantics is concerned with how the meaning of a sentence is composed both from the meaning of the constituent words, and from extra meaning contained within the structural organization of the sentence itself. In this context, grammatical constructions correspond to form-meaning mappings that essentially capture this “extra” meaning and allow its representation. The current research examines how a computational model of language processing based on a construction grammar approach can account for aspects of descriptive, referential and information content of (...) phrasal semantics. (shrink)

As there is in the neuroscience of individuals engaged in dynamic interactions, similar dark matter is present in the domain of interaction between humans and cognitive robots. Progress in second-person neuroscience will contribute to the development of robotic cognitive systems, and such developed robotic systems will be used to test the validity of the underlying theories.

Heyes does well to argue that some of the apparently innate human capabilities for cultural learning can be considered in terms of more general-purpose mechanisms. In the application of this to language, she overlooks some of its most interesting properties. I review three, and then illustrate how mindreading can come from general-purpose mechanism via language.

In arguing for a rules-similarity continuum, Pothos should demonstrate that a single process or mechanism (a neural network model, for example) can handle the entire continuum. Pothos deliberately avoids this exercise as beyond the scope of the current research. In this context, I will present simulation, neuropsychological, neurophysiological, and experimental psychological results, arguing against the continuity hypothesis.

Grodzinsky's characterization of the syntactic function of Broca's area is convincing, but his argument that this transformation processing capability is specific to language is less so. Based on predictions from simulation studies of sequence learning, we report a correlation between agrammatic patients' impairments in (a) syntactic comprehension, and (b) nonlinguistic sequence transformation processing, indicating the existence of a nonlinguistic correlate of agrammatic aphasia.

Flexible categorization clearly requires an adaptive component, but at what level of representation? We have investigated categorization in sequence learning that requires the extraction of abstract rules, but no modification of sensory primitives. This motivates the need to make explicit the distinction between sensory-level “atomic” features as opposed to concept-level “abstract” features, and the proposal that flexible categorization probably relies on learning at the abstract feature level.

Recognizing limitations of the “syntactocentric” perspective, Jackendoff proposes a model in which phonology, syntax, and conceptual systems are each independently combinatorial. We can ask, however, whether he has taken this issue to its logical conclusion. The fundamental question that is not fully addressed is whether the combinatorial aspect of syntax originated in, and derives from, the indeed “far richer” conceptual system, a question to be discussed.

Neural organization achieves its stated goal to “show how theory and experiment can supplement each other in an integrated, evolving account of structure, function, and dynamics” (p. ix), showing in a variety of contexts – from olfactory processing to spatial navigation, motor learning and more – how function may be realized in the neural tissue, with explanatory and predictive neural network models providing a cornerstone in this approach.

Analogical transfer in sequence learning is presented as an example of how the type-2 problem of learning an unbounded number of isomorphic sequences is reduced to the type-1 problem of learning a small finite set of sequences. The commentary illustrates how the difficult problem of appropriate analogical filter creation and selection is addressed while avoiding the trap of strong nativism, and it provides theoretical and experimental evidence for the existence of dissociable mechanisms for type-1 learning and type-2 recoding.