The detection of a human’s intended behavior is one of the most important skills that a social robot should have in order to become acceptable as a part of human society, because humans are used to understand the actions of other humans in a goal-directed manner and they will expect the social robot to behave similarly. A breakthrough in this area can advance several research branches related to social intelligence such as learning by imitation and mutual adaptation. To achieve this (...) goal the robot needs to integrate all possible evidence of intention and neglect the unintended behavior, and a complete solution should use low-level signal processing and high-level reasoning. This work explores the low-level signal processing part of the solution by proposing an interactive adaptive perception scheme that uses four important features of human behavior to amplify the signals originating from intended behavior with respect to signals originating from unintended behavior and other noise sources such as instrumental noise. This work follows the vision that intelligence is not only a function of a centralized sophisticated artificial brain, but can be presented in different forms in the entire robot including its perception and motion systems (the mind is not contained in the brain, but distributed in every cell in the body). A simple example of using the proposed scheme was implemented and the results of two experiments with it are also presented. (shrink)

In the near future, our life will normally be surrounded with fairly complicated artifacts, enabled by the autonomous robot and brain–machine interface technologies. In this paper, we argue that what we call the responsibility flaw problem and the inappropriate use problem need to be overcome in order for us to benefit from complicated artifacts. In order to solve these problems, we propose an approach to endowing artifacts with an ability of socially communicating with other agents based on the artifact-as-a-half-mirror metaphor. (...) The idea is to have future artifacts behave according to the hybrid intention composed of the owner’s intention and the social rules. We outline the approach and discuss its feasibility together with preliminary work. (shrink)

We have developed an approach to implementing a system for managing situated knowledge for complex instruments. Our aim is to develop a system that guides a user through the steps for operating complex scientific instruments. A user manual is often inadequate support for a community of users, so direct communication with an expert is often required. One reason for this is that not all of the author’s expert knowledge was included in the manual, thus limiting the contents to explicit knowledge. (...) This is a main concern of modern knowledge management practitioners who are attempting to design systems that consider both explicit and tacit knowledge. The key is to distribute explicit knowledge through interaction with the real world so that users can develop tacit knowledge as well as acquire explicit knowledge. We describe technical difficulties related to referencing the real world, which is required for interaction, and describe a novel approach to building a low-cost three-dimensional pointer for obtaining the required knowledge, which constitutes our preliminary result. (shrink)

A robot that is easy to teach not only has to be able to adapt to humans but also has to be easily adaptable to. In order to develop a robot with mutual adaptation ability, we believe that it will be beneficial to first observe the mutual adaptation behaviors that occur in human–human communication. In this paper, we propose a human–human WOZ (Wizard-of-Oz) experiment setting that can help us to observe and understand how the mutual adaptation procedure occurs between human (...) beings in nonverbal communication. By analyzing the experimental results, we obtained three important findings: alignment-based action, symbol-emergent learning, and environmental learning. (shrink)

For a robot to cohabit with people, it should be able to learn people’s nonverbal social behavior from experience. In this paper, we propose a novel machine learning method for recognizing gestures used in interaction and communication. Our method enables robots to learn gestures incrementally during human–robot interaction in an unsupervised manner. It allows the user to leave the number and types of gestures undefined prior to the learning. The proposed method (HB-SOINN) is based on a self-organizing incremental neural network (...) and the hidden Markov model. We have added an interactive learning mechanism to HB-SOINN to prevent a single cluster from running into a failure as a result of polysemy of being assigned more than one meaning. For example, a sentence: “Keep on going left slowly” has three meanings such as, “Keep on (1)”, “going left (2)”, “slowly (3)”. We experimentally tested the clustering performance of the proposed method against data obtained from measuring gestures using a motion capture device. The results show that the classification performance of HB-SOINN exceeds that of conventional clustering approaches. In addition, we have found that the interactive learning function improves the learning performance of HB-SOINN. (shrink)

We have developed a broadcasting agent system, public opinion channel (POC) caster, which generates understandable conversational form from text-based documents. The POC caster circulates the opinions of community members by using conversational form in a broadcasting system on the Internet. We evaluated its transformation rules in two experiments. In experiment 1, we examined our transformation rules for conversational form in relation to sentence length. Twenty-four participants listened to two types of sentence (long sentences and short sentences) with conversational form or (...) with single speech. In experiment 2, we investigated the relationship between conversational form and the user’s knowledge level. Forty-two participants (21 with a high knowledge level and 21 with a low knowledge level) were selected for a knowledge task and listened to two kinds of sentence (sentences about a well-known topic or sentences about an unfamiliar topic). Our results indicate that the conversational form aided comprehension, especially for long sentences and when users had little knowledge about the topic. We explore possible explanations and implications of these results with regard to human cognition and text comprehension. (shrink)

The success of social robots in achieving natural coexistence with humans depends on both their level of autonomy and their interactive abilities. Although a lot of robotic architectures have been suggested and many researchers have focused on human–robot interaction, a robotic architecture that can effectively combine interactivity and autonomy is still unavailable. This paper contributes to the research efforts toward this architecture in the following ways. First a theoretical analysis is provided that leads to the notion of co-evolution between the (...) agent and its environment and with other agents as the condition needed to combine both autonomy and interactivity. The analysis also shows that the basic competencies needed to achieve the required level of autonomy and the envisioned level of interactivity are similar but not the same. Secondly nine specific requirements are then formalized that should be achieved by the architecture. Thirdly a robotic architecture that tries to achieve those requirements by utilizing two main theoretical hypothesis and several insights from social science, developmental psychology and neuroscience is detailed. Lastly two experiments with a humanoid robot and a simulated agent are reported to show the potential of the proposed architecture. (shrink)

A system for analyzing concerns of people from Weblog articles is proposed. The system called KANSHIN analyzes concerns of people by collecting Japanese, Chinese, and Korean Weblog articles. Users can find concerns of people in each language. Users can also compare differences of concerns between Japanese, Chinese, and Korean language communities. We describe several analysis results: (1) patterns of social concerns, (2) change of focuses on a problem along with the time, (3) differences of concerns on a problem between Japanese, (...) Chinese, and Korean Weblog sites, and (4) relation between words in Weblog articles and real world natural phenomenon. (shrink)

This paper presents an approach to managing knowledge specific to a particular location for complex instruments. The goal is to improve the knowledge communication between experts and end-users of scientific instruments. We propose a computational framework that integrates augmented reality and augmented virtuality as interface for manipulating knowledge. The augmented virtuality-based interface can be produced and distributed without extra costs. It allows knowledge dissemination at a larger scale. The prominent feature of our model is that the knowledge representation is independent (...) from those interfaces. A preliminary version of our framework has been implemented and deployed in customers’ environments. (shrink)

This paper addresses the problem of human–computer interactions when the computer can interpret and express a kind of human-like behavior, offering natural communication. A conceptual framework for incorporating emotions with rationality is proposed. A model of affective social interactions is described. The model utilizes the SAIBA framework, which distinguishes among several stages of processing of information. The SAIBA framework is extended, and a model is realized in human behavior detection, human behavior interpretation, intention planning, attention tracking behavior planning, and behavior (...) realization components. Two models of incorporating emotions with rationality into a virtual artifact are presented. The first one uses an implicit implementation of emotions. The second one has an explicit realization of a three-layered model of emotions, which is highly interconnected with other components of the system. Details of the model with implicit implementation of emotional behavior are shown as well as evaluation methodology and results. Discussions about the extended model of an agent are given in the final part of the paper. (shrink)

The paper reports our research aiming at utilization of human interactive communication modeling principles in application to a novel interaction paradigm designed for brain–computer/machine-interfacing (BCI/BMI) technologies as well as for socially aware intelligent environments or communication support systems. Automatic procedures for human affective responses or emotional states estimation are still a hot topic of contemporary research. We propose to utilize human brain and bodily physiological responses for affective/emotional as well as communicative interactivity estimation, which potentially could be used in the (...) future for human–machine/environment interaction design. As a test platform for such an intelligent human–machine communication application, an emotional stimuli paradigm was chosen to evaluate brain responses to various affective stimuli in an emotional empathy mode. Videos with moving faces expressing various emotional displays as well as speech stimuli with similarly emotionally articulated sentences are presented to the subjects in order to further analyze different affective responses. From information processing point of view, several challenges with multimodal signal conditioning and stimuli dynamic response extraction in time frequency domain are addressed. Emotions play an important role in human daily life and human-to-human communication. This is why involvement of affective stimuli principles to human–machine communication or machine-mediated communication with utilization of multichannel neurophysiological and periphery physiological signals monitoring techniques, allowing real-time subjective brain responses evaluation, is discussed. We present our preliminary results and discuss potential applications of brain/body affective responses estimation for future interactive/smart environments. (shrink)

In this article we present a parameterized model for generating multimodal behavior based on cultural heuristics. To this end, a multimodal corpus analysis of human interactions in two cultures serves as the empirical basis for the modeling endeavor. Integrating the results from this empirical study with a well-established theory of cultural dimensions, it becomes feasible to generate culture-specific multimodal behavior in embodied agents by giving evidence for the cultural background of the agent. Two sample applications are presented that make use (...) of the model and are designed to be applied in the area of coaching intercultural communication. (shrink)

Technology explosion induced by information explosion will eventually change artifacts into intelligent autonomous agents consisting of surrogates and mediators from which humans can receive services without special training. Four potential problems might arise as a result of the paradigm shift: technology abuse, responsibility flaw, moral in crisis, and overdependence on artifacts. Although the first and second might be resolved in principle by introduction of public mediators, the rest seems beyond technical solution. Under the circumstances, a reasonable goal might be to (...) create a mutual dependency between empathy and technology: Using technology to help people cultivate empathy among people, so empathy in the society may allow people to help each other to recover from disasters beyond the scope of the assumption underlying the society of mediators. I highlight an immersive collaborative interaction environment for helping people share first-person view and its application to building empathic agents. (shrink)

The principle behind anthropomorphic robots is that the appearance and behaviours enable the pre-defined social skills that people use with each other each day to be used as a means of interaction. One of the problems with this approach is that there are many attributes of such a robot which can influence a user’s behaviour, potentially causing undesirable effects. This paper aims to identify and discuss a series of the most salient behaviour influencing factors in the literature, related to a (...) number of robot attributes. A particularly novel aspect of this work is the study of robots as monitoring or data collection devices, one of several behaviour influencing factors which has received insufficient attention. As a first step towards this, the PSA matrix is produced, which visualises the relationships between system attributes and user perceptions, grounded in empirical evidence in the literature. This matrix highlights gaps in the literature, brings together a series of salient behaviour influencing factors for the first time and gives a clear view of the state of the art. (shrink)

A key issue in social intelligence design is the realization of artifacts that can fluently communicate with people. Thus, we proposed a two-layered approach to enhance a robot’s capacity of involvement and engagement. The upper layer flexibly controls social interaction by dynamic Bayesian networks (DBN) representing social interaction patterns. The lower layer improves the robustness of the system by detecting rhythmic and repetitive gestures. We designed a listener robot that can follow and record humans’ explanation on how to assemble and/or (...) disassemble a bicycle. The implementation of this system is described by assembling the key algorithms presented in this paper. (shrink)

This comment discusses work by Aylett and Paiva (2012) which describes a synthetic approach to building a virtual world inhabited by synthetic characters where the user can experience subjective culture, that is, the experience of social reality, and learn how to empathetically communicate with people in other cultures. It provides a computational theory for integrating recent findings on emotion and cultural sensitivities into an interactive drama played by interacting characters with varying personalities. The FAtiMA-PSI, the implementation of their theory, has (...) advanced a critical step towards basing affective intelligence on a firmer ground. Future challenges include building an empathic agent on the simulated physiology level that can be recognized as an independent ego with its own personality and presence. (shrink)