E sposito et al. (E ds.) To w ard A u to n o m o u s, A d ap tive, an d C o n text-A w are M u ltim o d al In terfaces LNCS 6456 Anna Esposito Antonietta M. Esposito Raffaele Martone Vincent C. Müller Gaetano Scarpetta (Eds.) 123 LN CS 6 45 6 Third COST 2102 International Training School Caserta, Italy, March 2010 Revised Selected Papers Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces Theoretical and Practical Issues COST 2102 2010 ISSN 0302-9743 › springer.com Lecture Notes in Computer Science The LNCS series reports state-of-the-art results in computer science re search, development, and education, at a high level and in both printed and electronic form. Enjoying tight cooperation with the R&D community, with numerous individuals, as well as with prestigious organizations and societies, LNCS has grown into the most comprehensive computer science research forum available. The scope of LNCS, including its subseries LNAI and LNBI, spans the whole range of computer science and information technology including interdisciplinary topics in a variety of application fields. The type of material published traditionally includes – proceedings (published in time for the respective conference) – post-proceedings (consisting of thoroughly revised final full papers) – research monographs (which may be based on outstanding PhD work, research projects, technical reports, etc.) More recently,several color-cover sublines have been added featuring, beyond a collection of papers, various added-value components; these sublines in clude – tutorials (textbook-like monographs or collections of lectures given at advanced courses) – state-of-the-art surveys (offering complete and mediated coverage of a topic) – hot topics (introducing emergent topics to the broader community) In parallel to the printed book, each new volume is published electronically in LNCS Online. Detailed information on LNCS can be found at www.springer.com/lncs Proposals for publication should be sent to LNCS Editorial, Tiergartenstr. 17, 69121 Heidelberg, Germany E-mail: lncs@springer.com ISBN 978-3-642-18183-2 9 7 8 3 6 4 2 1 8 1 8 3 2 Lecture Notes in Computer Science 6456 Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max Planck Institute for Informatics, Saarbruecken, Germany Anna Esposito Antonietta M. Esposito Raffaele Martone Vincent C. Müller Gaetano Scarpetta (Eds.) Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces Theoretical and Practical Issues Third COST 2102 International Training School Caserta, Italy, March 15-19, 2010 Revised Selected Papers 13 Volume Editors Anna Esposito Second University of Naples and IIASS, International Institute for Advanced Scientific Studies Via Pellegrino 19, 84019 Vietri sul Mare (SA), Italy E-mail: iiass.annaesp@tin.it Antonietta M. Esposito Istituto Nazionale di Geofisica e Vulcanologia, Osservatorio Vesuviano Via Diocleziano 328, 80124 Napoli, Italy E-mail: aesposito@ov.ingv.it Raffaele Martone Second University of Naples, Department of Engineering and Informatics Via Roma 29, 81031 Aversa (CE), Italy E-mail: raffaele.martone@unina2.it Vincent C. Müller Anatolia College/ACT, Department of Humanities and Social Sciences Kennedy Street, 55510 Pylaia, Greece E-mail: vmueller@act.edu Gaetano Scarpetta University of Salerno and IIASS, International Institute for Advanced Scientific Studies 84081 Baronissi (SA), Italy E-mail: scarpetta@fisica.unisa.it ISSN 0302-9743 e-ISSN 1611-3349 ISBN 978-3-642-18183-2 e-ISBN 978-3-642-18184-9 DOI 10.1007/978-3-642-18184-9 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2010941945 CR Subject Classification (1998): H.5, H.5.2, I.2.7, H.3-4, I.2.10, J.1 LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Web and HCI © Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) This book is dedicated to our needs, that may overcome our goals and intentions one day, though we resist. Preface This volume brings together the advanced research results obtained by the European COST Action 2102: "Cross Modal Analysis of Verbal and Nonverbal Communication." The research published in this book was discussed at the Third EUCOGII-COST 2102 International Training School entitled "Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces: Theoretical and Practical Issues," held in Caserta, Italy, during March 15–19, 2010. The school was jointly sponsored by: a) COST (European Cooperation in Science and Technology, www.cost.eu) in the domain of Information and Communication Technologies (ICT) for disseminating the advances of the research activities developed within the COST Action 2102 (cost2102.cs.stir.ac.uk) b) EUCogII: 2nd European Network for the Advancement of Artificial Cognitive Systems, Interaction and Robotics (http://www.eucognition.org/). The school a!orded a change of perspective in verbal and nonverbal communication, where the research focus moved from "communicative tools" to "communicative instances" and asked for investigations that, in modeling interaction, will take into account not only the verbal and nonverbal signals but also the internal and external environment, the context, and the cultural specificity in which communicative acts take place. The consequences in information communication technology (ICT) research should result in the development of autonomous, adaptive, and context-aware multimodal interfaces able to act by exploiting instantiated contextual and environmental signals and process them by combining previous experience (memory) adapted to the communicative instance. This new approach will foster artificial cognitive research by creating a bridge between the most recent research in multimodal communication (taking into account gestures, emotions, social signal processing etc.) and computation models that exploit these signals being aware of the context in which these signals are instantiated and of the internal and external environmental background. Human behavior exploits this information and adapts. Artificial cognitive systems must take account of this human ability for implementing a friendly and emotionally colored human machine interaction. In order to do this, investigations in cognitive computation must move from purely data-driven systems to behavioral systems able to "interact with human to achieve their goals,"which may require ability to manifest intentions and goals through "resistance" to other intentions and goals (Müller argues in Chap. 1). In summary, cognitive models must be developed such that the current interactive dialogue systems, robots, and intelligent virtual avatars graphically embodied in VIII Preface a 2D and/or 3D interactive virtual world, are able to interact intelligently with the environment, other avatars, and particularly with human users. The themes of the papers presented in this book emphasize theoretical and practical issues for modelling cognitive behavioral systems, ranging from the attempts to describe brain computer interface (BCI) applications, a contextbased approach to the interpretation and generation of dialogue acts, close synchronization among both speakers and listeners, mutual ratification, interaction and resistance, embodiment, language and multimodal cognition, timing e!ects on perception, action, and behaviors. The book is arranged in two scientific sections according to a rough thematic classification, even though both sections are closely connected and both provide fundamental insights for the cross-fertilization of di!erent disciplines. The first section, "Human–Computer Interaction: Cognitive and Computational Issues," deals with conjectural and processing issues of defining models, algorithms, and strategies for implementing cognitive behavioral systems. The second section, "Synchrony Through Verbal and Nonverbal Signals," presents original studies that provide theoretical and practical solutions to the modelling of timing synchronization between linguistic and paralinguistic expressions, actions, body movements, activities in human interaction and on their assistance for an e!ective communication. The papers included in this book benefited from the live interactions among the many participants of the successful meeting in Caserta. Over 150 established and apprenticing researchers converged for the event. The editors would like to thank the Coordination Council of EUCogII and the ESF COSTICT Programme for the support in the realization of the school and the publication of this volume. Acknowledgements go in particular to the COST Science O"cers Matteo Razzanelli, Aranzazu Sanchez, Jamsheed Shorish, and the COST 2102 rapporteur Guntar Balodis for their constant help, guidance, and encouragement. Appreciation goes to the COST Publication O"ce for supporting and guiding the publication e!ort. The event owes its success to more individuals than can be named, but notably the members of the local Steering Committee Alida Labella, Olimpia Matarazzo, Nicola Melone, Giovanna Nigro, Augusto Parente, Paolo Pedone, Francesco Piazza, and Luigi Trojano who supported and encouraged the initiative as well as to the sta! members Hicham Atassi, Ivana Baldassarre, Domenico Carbone, Vincenzo Capuano, Francesco Alessandro Conventi, Mauro De Vito, Davide Esposito, Paolo Fioretto, Marco Grassi, and Gianluigi Ombrato who were actively involved in the success of the event. Special appreciation goes to the International Institute for Advanced Scientific Studies (IIASS), with a special mention to the memory of Maria Marinaro, and to the IIASS team Tina Marcella Nappi, Michele Donnarumma, and Antonio Natale who provided precious technical support in the organization of the school. The editors are deeply indebted to Maria Teresa Riviello for the wonderful work done in taking care of the local organization. Preface IX In addition, the editors are grateful to the contributors for making this book a scientifically stimulating compilation of new and original ideas. Finally, the editors would like to express their greatest appreciation to all the members of the COST 2102 International Scientific Committee for their rigorous and invaluable scientific revisions, their dedication, and their priceless selection process. Anna Esposito Antonietta M. Esposito Ra!aele Martone Vincent C. Müller Gaetano Scarpetta Organization International Steering Committee Anna Esposito Second University of Naples and IIASS, Italy Marcos Faundez-Zanuy Escola Universitaria Politecnica de Mataro, Spain Amir Hussain University of Stirling, UK Ra!aele Martone Second University of Naples and IIASS, Italy Nicola Melone Second University of Naples and IIASS, Italy COST 2102 International Scientific Committee Samer Al Moubayed Royal Institute of Technology, Sweden Uwe Altmann Technische Universität Dresden, Germany Sigrún Maŕıa Ammendrup School of Computer Science, Reykjavik, Iceland Hicham Atassi Brno University of Technology, Czech Republic Nikos Avouris University of Patras, Greece Ruth Bahr University of South Florida, USA Gérard Bailly GIPSA, GRENOBLE, France Marena Balinova University of Applied Sciences, Vienna, Austria Marian Bartlett University of California, San Diego, USA Dominik Bauer RWTH Aachen University, Germany Štefan Beňuš Constantine the Philosopher University, Nitra, Slovakia Niels Ole Bernsen University of Southern Denmark, Denmark Jonas Beskow Royal Institute of Technology, Sweden Peter Birkholz RWTH Aachen University, Germany Horst Bishof Technical University Graz, Austria Jean-Francois Bonastre Universitè d'Avignon, France Nikolaos Bourbakis ITRI, Wright State University, Dayton, USA Maja Bratanić University of Zagreb, Croatia Antonio Calabrese Istituto di Cibernetica CNR, Naples, Italy Erik Cambria University of Stirling, UK Paola Campadelli Università di Milano, Italy Nick Campbell University of Dublin, Ireland Valentın Cardeñoso Payo Universidad de Valladolid, Spain Antonio Castro-Fonseca Universidade de Coimbra, Portugal Aleksandra Cerekovic Faculty of Electrical Engineering , Croatia Josef Chaloupka Technical University of Liberec, Czech Republic XII Organization Mohamed Chetouani Universitè Pierre et Marie Curie, France Gérard Chollet CNRS URA-820, ENST, France Simone Cifani Università Politecnica delle Marche, Italy Muzeyyen Ciyiltepe Gulhane Askeri Tip Academisi, Ankara, Turkey Anton Cizmar Technical University of Kosice, Slovakia Nicholas Costen Manchester Metropolitan University, UK Francesca D'Olimpio Second University of Naples, Italy Vlado Delić University of Novi Sad, Serbia Giuseppe Di Maio Second University of Naples, Italy Marion Dohen ICP, Grenoble, France Thierry Dutoit Faculté Polytechnique de Mons, Belgium Laila DybkjAEr University of Southern Denmark, Denmark Jens Edlund Royal Institute of Technology, Sweden Matthias Eichner Technische Universität Dresden, Germany Aly El-Bahrawy Ain Shams University, Cairo, Egypt Engin Erzin Koc University, Istanbul, Turkey Anna Esposito Second University of Naples, Italy Joan Fàbregas Peinado Escola Universitaria de Mataro, Spain Sascha Fagel Technische Universität Berlin, Germany Nikos Fakotakis University of Patras, Greece Manuela Farinosi University of Udine, Italy Marcos Faúndez-Zanuy Universidad Politécnica de Cataluña, Spain Dilek Fidan Ankara Universitesi, Turkey Leopoldina Fortunati Università di Udine, Italy Todor Ganchev University of Patras, Greece Carmen Garćıa-Mateo University of Vigo, Spain Augusto Gnisci Second University of Naples, Italy Milan Gnjatović University of Novi Sad, Serbia Bjorn Granstrom Royal Institute of Technology, Sweden Marco Grassi Università Politecnica delle Marche, Italy Maurice Grinberg New Bulgarian University, Bulgaria Jorge Gurlekian LIS CONICET, Buenos Aires, Argentina Mohand-Said Hacid Universite Claude Bernard Lyon 1, France Jaakko Hakulinen University of Tampere, Finland Ioannis Hatzilygeroudis University of Patras, Greece Immaculada Hernaez University of the Basque Country, Spain Javier Hernando Technical University of Catalonia, Spain Wolfgang Hess Universität Bonn, Germany Dirk Heylen University of Twente, The Netherlands Daniel Hládek Technical University of Košice, Slovak Republic Rüdiger Ho!mann Technische Universität Dresden, Germany Hendri Hondorp University of Twente, The Netherlands Organization XIII David House Royal Institute of Technology, Sweden Evgenia Hristova New Bulgarian University, Sofia, Bulgaria Stephan Hübler Dresden University of Technology, Germany Isabelle Hupont Aragon Institute of Technology, Zaragoza, Spain Amir Hussain University of Stirling, UK Ewa Jarmolowicz Adam Mickiewicz University, Poznan, Poland Kristiina Jokinen University of Helsinki, Finland Jozef Juhár Technical University Košice, Slovak Republic Zdravko Kacic University of Maribor, Slovenia Bridget Kane Trinity College Dublin, Ireland Jim Kannampuzha RWTH Aachen University, Germany Maciej Karpinski Adam Mickiewicz University, Poznan, Poland Eric Keller Université de Lausanne, Switzeland Adam Kendon University of Pennsylvania, USA Stefan Kopp University of Bielefeld, Germany Jacques Koreman University of Science and Technology, Norway Theodoros Kostoulas University of Patras, Greece Maria Koutsombogera Inst. for Language and Speech Processing, Greece Robert Krauss Columbia University, New York, USA Bernd Kröger RWTH Aachen University, Germany Gernot Kubin Graz University of Technology, Austria Olga Kulyk University of Twente, The Netherlands Alida Labella Second University of Naples, Italy Emilian Lalev New Bulgarian University, Bulgaria Yiannis Laouris Cyprus Neuroscience and Technology Institute, Cyprus Anne-Maria Laukkanen University of Tampere, Finland Borge Lindberg Aalborg University, Denmark Saturnino Luz Trinity College Dublin, Ireland Wojciech Majewski Wroclaw University of Technology, Poland Pantelis Makris Neuroscience and Technology Institute, Cyprus Ra!aele Martone Second University of Naples, Italy Rytis Maskeliunas Kaunas University of Technology, Lithuania Dominic Massaro University of California Santa Cruz, USA Olimpia Matarazzo Second University of Naples, Italy Christoph Mayer Technische Universität München, Germany David McNeill University of Chicago, USA Nicola Melone Second University of Naples, Italy Katya Mihaylova University of National and World Economy, Sofia Bulgaria Michal Mirilovič Technical University of Košice, Slovakia Helena Moniz INESC-ID, Lisbon, Portugal XIV Organization Peter Murphy University of Limerick, Ireland Antonio Natale University of Salerno and IIASS, Italy Eva Navas Escuela Superior de Ingenieros, Bilbao, Spain Delroy Nelson University College London, UK Géza Németh University of Technology and Economics, Budapest, Hungary Friedrich Neubarth Austrian Research Inst. Artificial Intelligence, Austria Christiane Neuschaefer-Rube RWTH Aachen University, Germany Giovanna Nigro Second University of Naples, Italy Anton Nijholt Universiteit Twente, The Netherlands Jan Nouza Technical University of Liberec, Czech Republic Stanislav Ondáš Technical University of Košice, Slovak Republic Rieks Op den Akker University of Twente, The Netherlands Igor Pandzic Faculty of Electrical Engineering, Zagreb, Croatia Harris Papageorgiou Institute for Language and Speech Processing, Greece Kinga Papay University of Debrecen, Hungary Paolo Parmeggiani Università degli Studi di Udine, Italy Ana Pavia Spoken Language Systems Laboratory, Portugal Paolo Pedone Second University of Naples, Italy Tomislav Pejsa University of Zagreb, Croatia Catherine Pelachaud Université de Paris, France Bojan Petek University of Ljubljana, Slovenia Harmut R. Pfitzinger University of Munich, Germany Francesco Piazza Università degli Studi di Ancona, Italy Neda Pintaric University of Zagreb, Croatia Matúš Pleva Technical University of Košice, Slovak Republic Isabella Poggi Università di Roma 3, Italy Ken Prepin Telecom-ParisTech, France Jǐŕı Přibil Academy of Sciences, Czech Republic Anna Přibilová Slovak University of Technology, Slovakia Emanuele Principi Università Politecnica delle Marche, Italy Michael Pucher Telecommunications Research Center Vienna, Austria Jurate Puniene Kaunas University of Technology, Lithuania Kari-Jouko Räihä University of Tampere, Finland Roxanne Raine University of Twente, The Netherlands Giuliana Ramella Istituto di Cibernetica CNR, Naples, Italy Fabian Ramseyer University Hospital of Psychiatry, Bern, Switzerland Josè Rebelo Universidade de Coimbra, Portugal Organization XV Peter Reichl FTW Telecommunications Research Center, Austria Luigi Maria Ricciardi Università di Napoli "Federico II", Italy Maria Teresa Riviello Second University of Naples and IIASS, Italy Matej Rojc University of Maribor, Slovenia Nicla Rossini Università del Piemonte Orientale, Italy Rudi Rotili Università Politecnica delle Marche, Italy Algimantas Rudzionis Kaunas university of Technology, Lithuania Vytautas Rudzionis Kaunas University of Technology, Lithuania Hugo L. Rufiner Universidad Nacional de Entre Rıos, Argentina Milan Rusko Slovak Academy of Sciences, Slovak Republic Zsófia Ruttkay Pazmany Peter Catholic University, Hungary Yoshinori Sagisaka Waseda University, Tokyo, Japan Bartolomeo Sapio Fondazione Ugo Bordoni, Rome, Italy Mauro Sarrica University of Padova, Italy Gaetano Scarpetta University of Salerno and IIASS, Italy Silvia Scarpetta Salerno University, Italy Ralph Schnitker Aachen University, Germany Jean Schoentgen Université Libre de Bruxelles, Belgium Milan Sečujski University of Novi Sad, Serbia Stefanie Shattuck-Hufnagel MIT, Research Laboratory of Electronics, USA Marcin Skowron Austrian Research Institute for Artificial Intelligence, Austria Zdenek Smékal Brno University of Technology, Czech Republic Stefano Squartini Università Politecnica delle Marche, Italy Piotr Staroniewicz Wroclaw University of Technology, Poland Vojtěch Stejskal Brno University of Technology, Czech Republic Marian Stewart-Bartlett University of California, San Diego, USA Jin Su Trinity College Dublin, Ireland Dávid Sztahó Budapest University of Technology and Economics, Hungary Jianhua Tao Chinese Academy of Sciences, P. R. China Jure F. Tasič University of Ljubljana, Slovenia Murat Tekalp Koc University, Istanbul, Turkey Kristinn Thórisson Reykjavık University, Iceland Isabel Trancoso Spoken Language Systems Laboratory, Portugal Luigi Trojano Second University of Naples, Italy Wolfgang Tschacher University of Bern, Switzerland Markku Turunen University of Tampere, Finland Henk Van den Heuvel Radboud University Nijmegen, The Netherlands Leticia Vicente-Rasoamalala Alchi Prefectural University, Japan XVI Organization Robert Vich Academy of Sciences, Czech Republic Klára Vicsi Budapest University, Hungary Hannes Högni Vilhjálmsson Reykjavık University, Iceland Jane Vincent University of Surrey, Guildford, UK Carl Vogel University of Dublin, Ireland Jan Voĺın Charles University, Czech Republic Rosa Volpe Université De Perpignan, France Martin Vondra Academy of Sciences, Czech Republic Yorick Wilks University of She"eld, UK Matthias Wimmer Institute for Informatics, Munich, Germany Matthias Wolf Technische Universität Dresden, Germany Bencie Woll University College London, UK Bayya Yegnanarayana International Institute of Information Technology, India Jerneja Žganec Gros Alpineon, Development and Research, Slovenia Goranka Zoric Faculty of Electrical Engineering, Zagreb, Croatia Sponsors The following organizations sponsored and supported the International Training School: • European COST Action 2102 "Cross-Modal Analysis of Verbal and Nonverbal Communication" (cost2102.cs.stir.ac.uk) COST-the acronym for European Cooperation in Science and Technology-is the oldest and widest European intergovernmental network for cooperation in research. Established by the Ministerial Conference in November 1971, COST is presently used by the scientific communities of 36 European countries to cooperate in common research projects supported by national funds. The funds provided by COST-less than 1 of the total value of the projects- support the COST cooperation networks (COST Actions) through which, with EUR 30 million per year, more than 30,000 European scientists are involved in research having a total value which exceeds EUR 2 billion per year. This is the financial worth of the European added value which COST achieves. A"bottom – Up approach"(the initiative of launching a COST Action comes from the European scientists themselves), "à la carte participation" (only countries interested in the Action participate), "equality of access" (participation is open also to the scientific communities of countries not belonging to the European Union) and"flexible structure"(easy implementation and light management of the research initiatives) are the main characteristics of COST. As precursor of advanced multidisciplinary research, COST has a very important role for the realization of the European Research Area (ERA) anticipating and complementing the activities of the Framework Programmes, constituting a "bridge" towards the scientific communities of emerging countries, increasing the mobility of researchers across Europe and fostering the establishment of "Networks of Excellence" in many key scientific domains such as: biomedicine and molecular biosciences; food and agriculture; forests, their products and services; materials, physical and nanosciences; chemistry and molecular sciences and technologies; earth system science and environmental management; information and communication technologies; transport and urban development; individuals, societies cultures and health. It covers basic and more applied research and also addresses issues of pre-normative nature or of societal importance. Web: http://www.cost.eu • EUCogII: 2nd European Network for the Advancement of Artificial Cognitive Systems, Interaction and Robotics (http://www.eucognition.org/) • Second University of Naples (www.unina2.it/), Faculty and Department of Psychology, Faculty of Science, Mathematics, and Physics and Department of Computer Engineering & Informatics, Italy XVIII Sponsors • International Institute for Advanced Scientific Studies "E.R. Caianiello" (IIASS, www.iiassvietri.it/), Italy • Società Italiana Reti Neuroniche (SIREN, www.associazionesiren.org/) • Regione Campania, Italy • Provincia di Salerno, Italy ESF Provide the COST Office through and EC contract COST is supported by the EU RTD Framework programme Table of Contents I Human-Computer Interaction: Cognitive and Computational Issues Interaction and Resistance: The Recognition of Intentions in New Human-Computer Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Vincent C. Müller Speaking without Thinking: Embodiment, Speech Technology and Social Signal Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Tim Rohrer More Than Words: Inference of Socially Relevant Information from Nonverbal Vocal Cues in Speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Alessandro Vinciarelli, Hugues Salamin, Gelareh Mohammadi, and Khiet Truong A Timely Endeavor: Theoretical, Behavioral, Bioimaging, and Clinical Perspectives on Time Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Argiro Vatakis and Georgios Papadelis Instruction and Belief E!ects on Sentential Reasoning . . . . . . . . . . . . . . . . 39 Olimpia Matarazzo and Ivana Baldassarre Audio-Visual Prosody: Perception, Detection, and Synthesis of Prominence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Samer Al Moubayed, Jonas Beskow, Björn Granström, and David House Adaptation in Turn-Initiations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Štefan Beňuš Sentic Avatar: Multimodal A!ective Conversational Agent with Common Sense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Erik Cambria, Isabelle Hupont, Amir Hussain, Eva Cerezo, and Sandra Baldassarri Patterns of Synchronization of Non-verbal Cues and Speech in ECAs: Towards a More "Natural" Conversational Agent . . . . . . . . . . . . . . . . . . . . . 96 Nicla Rossini Top-Down Influences on Eye-Movements during Painting Perception: The E!ect of Task and Titles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Evgenia Hristova, Severina Georgieva, and Maurice Grinberg XX Table of Contents Visual Sociology in the Classroom: Fostering Interaction Awareness Using Video . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Paolo Parmeggiani Analysis of Interrogatives in Di!erent Domains . . . . . . . . . . . . . . . . . . . . . . 134 Helena Moniz, Fernando Batista, Isabel Trancoso, and Ana Isabel Mata Interviewers' Use of Coercive Questioning during a Midterm Period Favorable to the Opposition Party . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Angiola Di Conza, Augusto Gnisci, and Angelo Caputo Emotions and Speech Disorders: Do Developmental Stutters Recognize Emotional Vocal Expressions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Anna Esposito and Alda Troncone Representing Meaning in Mind: When Predicate Argument Structures Meet Mental Representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Rosa Volpe Beyond the Panopticon Framework: Privacy, Control and User Generated Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Manuela Farinosi Micro and Macro Spatial Dimensions of New Media in Five European Countries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Leopoldina Fortunati and Federico de Luca Nonverbal Expressions of Turn Management in TV Interviews: A Cross-Cultural Study between Greek and Icelandic . . . . . . . . . . . . . . . . . 207 Maria Koutsombogera, Sigrún Maŕıa Ammendrup, Hannes Högni Vilhjálmsson, and Harris Papageorgiou II Synchrony through Verbal and Nonverbal Signals Interpretation and Generation of Dialogue with Multidimensional Context Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 Harry Bunt Coordination, Not Control, Is Central to Movement . . . . . . . . . . . . . . . . . . 243 Fred Cummins Speech, Gaze and Head Motion in a Face-to-Face Collaborative Task . . . 256 Sascha Fagel and Gérard Bailly Neuroelectric Methodologies for the Study of the Economic Decisions in Humans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 Giovanni Vecchiato and Fabio Babiloni Table of Contents XXI An Evaluation Study on Speech Feature Densities for Bayesian Estimation in Robust ASR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Simone Cifani, Emanuele Principi, Rudy Rotili, Stefano Squartini, and Francesco Piazza Naturalness, Adaptation and Cooperativeness in Spoken Dialogue Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 Milan Gnjatović, Darko Pekar, and Vlado Delić Towards Semantic Multimodal Video Annotation . . . . . . . . . . . . . . . . . . . . 305 Marco Grassi, Christian Morbidoni, and Francesco Piazza The E!ect of Subharmonic Stimuli on Singing Voices . . . . . . . . . . . . . . . . . 317 Marena Balinova, Peter Reichl, Inma Hernáez Rioja, and Ibon Saratxaga Speech Modeling Using the Complex Cepstrum . . . . . . . . . . . . . . . . . . . . . . 324 Martin Vondra and Robert Vıch Problems of the Automatic Emotion Recognitions in Spontaneous Speech; An Example for the Recognition in a Dispatcher Center . . . . . . . 331 Klára Vicsi and Dávid Sztahó Slovak Language Model from Internet Text Data . . . . . . . . . . . . . . . . . . . . . 340 Ján Staš, Daniel Hládek, Matúš Pleva, and Jozef Juhár Automatic Recognition of Emotional State in Polish Speech . . . . . . . . . . . 347 Piotr Staroniewicz Categorical Perception of Consonants and Vowels: Evidence from a Neurophonetic Model of Speech Production and Perception . . . . . . . . . . . 354 Bernd J. Kröger, Peter Birkholz, Jim Kannampuzha, and Christiane Neuschaefer-Rube The MultiLis Corpus – Dealing with Individual Di!erences in Nonverbal Listening Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 Iwan de Kok and Dirk Heylen Comparing the Rhythmical Characteristics of Speech and Music – Theoretical and Practical Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 Stephan Hübler and Rüdiger Ho!mann The Ability of Children with Mild Learning Disabilities to Encode Emotions through Facial Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 Christiane El-Haddad and Yiannis Laouris Designing a Hungarian Multimodal Database – Speech Recording and Annotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 Kinga Papay XXII Table of Contents Conveying Directional Gaze Cues to Support Remote Participation in Hybrid Meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412 Betsy van Dijk, Job Zwiers, Rieks op den Akker, Olga Kulyk, Hendri Hondorp, Dennis Hofs, and Anton Nijholt A!ect Recognition in Real Life Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 Theodoros Kostoulas, Todor Ganchev, and Nikos Fakotakis Understanding Parent-Infant Behaviors Using Non-negative Matrix Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436 Ammar Mahdhaoui and Mohamed Chetouani Learning and Knowledge-Based Sentiment Analysis in Movie Review Key Excerpts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448 Björn Schuller and Tobias Knaup Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .