Skip to main content
SpringerLink
Log in

Modeling Context for Referring in Multimodal Dialogue Systems

  • Conference paper
Modeling and Using Context (CONTEXT 2005)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 3554))

  • 1141 Accesses

Abstract

The way we see the objects around us determines speech and gestures we use to refer to them. The gestures we produce structure our visual perception. The words we use have an influence on the way we see. In this manner, visual perception, language and gesture present multiple interactions between each other. The problem is global and has to be tackled as a whole in order to understand the complexity of reference phenomena and to deduce a formal model. This model may be useful for any kind of man-machine dialogue system that focuses on deep comprehension. We show how a referring act takes place into a contextual subset of objects, called ‘reference domain,’ and we present the ‘multimodal reference domain’ model that can be exploited in a dialogue system when interpreting.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Beun, R.-J., Cremers, A.H.M.: Object Reference in a Shared Domain of Conversation. Pragmatics and Cognition 6(1/2), 121–152 (1998)

    Article  Google Scholar 

  2. Bolt, R.A.: Put-That-There: Voice and Gesture at the Graphics Interface. Computer Graphics 14(3), 262–270 (1980)

    Article  MathSciNet  Google Scholar 

  3. Cosnier, J., Vaysse, J.: Sémiotique des gestes communicatifs. Nouveaux actes sémiotiques (geste, cognition et communication) 52, 7–28 (1997)

    Google Scholar 

  4. Dekker, P.: Speaker’s Reference, Descriptions and Information Structure. Journal of Semantics 15(4), 305–334 (1998)

    Article  Google Scholar 

  5. Edmonds, P.G.: A Computational Model of Collaboration on Reference in Direction-Giving Dialogues. Ms. Thesis, University of Toronto, Canada (1993)

    Google Scholar 

  6. Efron, D.: Gesture, Race and Culture. Mouton, The Hague (1972)

    Google Scholar 

  7. Fitts, M.: The Information Capacity of the Human Motor System in Controlling Amplitude of Movement. Journal of Experimental Psychology 47, 381–391 (1954)

    Article  Google Scholar 

  8. Grosz, B.J., Sidner, C.L.: Attention, Intentions and the Structure of Discourse. Computational Linguistics 12(3), 175–204 (1986)

    Google Scholar 

  9. Itten, J.: The Art of Color. Reinhold Publishing Corp., New York (1961)

    Google Scholar 

  10. Jöhsson, A., Dählback, N.: Talking to a Computer is not like Talking to your Best Friend. In: Proceedings of the Scandinavian Conference on Artificial Intelligence, Tromsø (1988)

    Google Scholar 

  11. Kamp, H., Reyle, U.: From Discourse to Logic. Kluwer, Dordrecht (1993)

    Google Scholar 

  12. Karmiloff-Smith, A.: A Functional Approach to Child Language. Cambridge University Press, Cambridge (1979)

    Google Scholar 

  13. Kehler, A.: Cognitive Status and Form of Reference in Multimodal Human-Computer Interaction. In: Proceedings of the 17th National Conference on Artificial Intelligence, Austin (2000)

    Google Scholar 

  14. Kievit, L., Piwek, P., Beun, R.-J., Bunt, H.: Multimodal Cooperative Resolution of Referential Expressions in the DenK System. In: Bunt, H., Beun, R.-J. (eds.) CMC 1998. LNCS (LNAI), vol. 2155, pp. 197–214. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  15. Kleiber, G.: Anaphores et pronoms. Duculot, Louvain-la-Neuve (1994)

    Google Scholar 

  16. Landragin, F.: Dialogue homme-machine multimodal. Hermes Science Publishing, Paris (2004)

    Google Scholar 

  17. McNeill, D.: Psycholinguistics: A New Approach. Harper and Row, New York (1987)

    Google Scholar 

  18. Moeschler, J., Reboul, A.: Dictionnaire encyclopédique de pragmatique. Seuil, Paris (1994)

    Google Scholar 

  19. Olson, D.R.: Language and Thought: Aspects of a Cognitive Theory of Semantics. Psychological Review 77, 257–273 (1970)

    Article  Google Scholar 

  20. Reboul, A., Moeschler, J.: Pragmatique du discours. Armand Colin, Paris (1998)

    Google Scholar 

  21. Rock, I., Brosgole, L.: Grouping Based on Phenomenal Proximity. Journal of Experimental Psychology 67 (1964)

    Google Scholar 

  22. Salmon-Alt, S.: Reference Resolution within the Framework of Cognitive Grammar. In: Proceedings of the International Colloquium on Cognitive Science, San Sebastian, Spain (2001)

    Google Scholar 

  23. Sperber, D., Wilson, D.: Relevance. In: Communication and Cognition, 2nd edn. Blackwell, Oxford (1995)

    Google Scholar 

  24. Thórisson, K.R.: Simulated Perceptual Grouping: An Application to Human-Computer Interaction. In: Proceedings of the 16th Annual Conference of the Cognitive Science Society, Atlanta, Georgia (1994)

    Google Scholar 

  25. Wertheimer, M.: Untersuchungen zur Lehre von der Gestalt II. Psychologische Forschung 4 (1923)

    Google Scholar 

  26. Wolff, F., De Angeli, A., Romary, L.: Acting on a Visual World: The Role of Perception in Multimodal HCI. In: Proceedings of AAAI 1998 Workshop: Representations for Multi-modal Human-Computer Interaction, Madison, Wisconsin (1998)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Thales Research and Technology, Domaine de Corbeville, F-91404, Orsay Cedex, France

    Frédéric Landragin

Authors
  1. Frédéric Landragin
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Human-Computer Interaction Institute, Carnegie Mellon University, 5000 Forbes Ave, 15213-3891, Pittsburgh, PA, USA

    Anind Dey

  2. Central and East European Center for Cognitive Science, New Bulgarian University, 21 Montevideo Street, 1618, Sofia, Bulgaria

    Boicho Kokinov

  3. Computer Science Department, Indiana University, 47405, Bloomington, IN, U.S.A.

    David Leake

  4. Department of Computer Science, University of Maine, 04469, Orono, Maine, USA

    Roy Turner

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Landragin, F. (2005). Modeling Context for Referring in Multimodal Dialogue Systems. In: Dey, A., Kokinov, B., Leake, D., Turner, R. (eds) Modeling and Using Context. CONTEXT 2005. Lecture Notes in Computer Science(), vol 3554. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11508373_18

Download citation

  • DOI: https://doi.org/10.1007/11508373_18

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-26924-3

  • Online ISBN: 978-3-540-31890-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation