Abstract
The way we see the objects around us determines speech and gestures we use to refer to them. The gestures we produce structure our visual perception. The words we use have an influence on the way we see. In this manner, visual perception, language and gesture present multiple interactions between each other. The problem is global and has to be tackled as a whole in order to understand the complexity of reference phenomena and to deduce a formal model. This model may be useful for any kind of man-machine dialogue system that focuses on deep comprehension. We show how a referring act takes place into a contextual subset of objects, called ‘reference domain,’ and we present the ‘multimodal reference domain’ model that can be exploited in a dialogue system when interpreting.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Beun, R.-J., Cremers, A.H.M.: Object Reference in a Shared Domain of Conversation. Pragmatics and Cognition 6(1/2), 121–152 (1998)
Bolt, R.A.: Put-That-There: Voice and Gesture at the Graphics Interface. Computer Graphics 14(3), 262–270 (1980)
Cosnier, J., Vaysse, J.: Sémiotique des gestes communicatifs. Nouveaux actes sémiotiques (geste, cognition et communication) 52, 7–28 (1997)
Dekker, P.: Speaker’s Reference, Descriptions and Information Structure. Journal of Semantics 15(4), 305–334 (1998)
Edmonds, P.G.: A Computational Model of Collaboration on Reference in Direction-Giving Dialogues. Ms. Thesis, University of Toronto, Canada (1993)
Efron, D.: Gesture, Race and Culture. Mouton, The Hague (1972)
Fitts, M.: The Information Capacity of the Human Motor System in Controlling Amplitude of Movement. Journal of Experimental Psychology 47, 381–391 (1954)
Grosz, B.J., Sidner, C.L.: Attention, Intentions and the Structure of Discourse. Computational Linguistics 12(3), 175–204 (1986)
Itten, J.: The Art of Color. Reinhold Publishing Corp., New York (1961)
Jöhsson, A., Dählback, N.: Talking to a Computer is not like Talking to your Best Friend. In: Proceedings of the Scandinavian Conference on Artificial Intelligence, Tromsø (1988)
Kamp, H., Reyle, U.: From Discourse to Logic. Kluwer, Dordrecht (1993)
Karmiloff-Smith, A.: A Functional Approach to Child Language. Cambridge University Press, Cambridge (1979)
Kehler, A.: Cognitive Status and Form of Reference in Multimodal Human-Computer Interaction. In: Proceedings of the 17th National Conference on Artificial Intelligence, Austin (2000)
Kievit, L., Piwek, P., Beun, R.-J., Bunt, H.: Multimodal Cooperative Resolution of Referential Expressions in the DenK System. In: Bunt, H., Beun, R.-J. (eds.) CMC 1998. LNCS (LNAI), vol. 2155, pp. 197–214. Springer, Heidelberg (2001)
Kleiber, G.: Anaphores et pronoms. Duculot, Louvain-la-Neuve (1994)
Landragin, F.: Dialogue homme-machine multimodal. Hermes Science Publishing, Paris (2004)
McNeill, D.: Psycholinguistics: A New Approach. Harper and Row, New York (1987)
Moeschler, J., Reboul, A.: Dictionnaire encyclopédique de pragmatique. Seuil, Paris (1994)
Olson, D.R.: Language and Thought: Aspects of a Cognitive Theory of Semantics. Psychological Review 77, 257–273 (1970)
Reboul, A., Moeschler, J.: Pragmatique du discours. Armand Colin, Paris (1998)
Rock, I., Brosgole, L.: Grouping Based on Phenomenal Proximity. Journal of Experimental Psychology 67 (1964)
Salmon-Alt, S.: Reference Resolution within the Framework of Cognitive Grammar. In: Proceedings of the International Colloquium on Cognitive Science, San Sebastian, Spain (2001)
Sperber, D., Wilson, D.: Relevance. In: Communication and Cognition, 2nd edn. Blackwell, Oxford (1995)
Thórisson, K.R.: Simulated Perceptual Grouping: An Application to Human-Computer Interaction. In: Proceedings of the 16th Annual Conference of the Cognitive Science Society, Atlanta, Georgia (1994)
Wertheimer, M.: Untersuchungen zur Lehre von der Gestalt II. Psychologische Forschung 4 (1923)
Wolff, F., De Angeli, A., Romary, L.: Acting on a Visual World: The Role of Perception in Multimodal HCI. In: Proceedings of AAAI 1998 Workshop: Representations for Multi-modal Human-Computer Interaction, Madison, Wisconsin (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Landragin, F. (2005). Modeling Context for Referring in Multimodal Dialogue Systems. In: Dey, A., Kokinov, B., Leake, D., Turner, R. (eds) Modeling and Using Context. CONTEXT 2005. Lecture Notes in Computer Science(), vol 3554. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11508373_18
Download citation
DOI: https://doi.org/10.1007/11508373_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26924-3
Online ISBN: 978-3-540-31890-3
eBook Packages: Computer ScienceComputer Science (R0)