Abstract
Face-to-face interaction is a primordial site for human activity and intersubjectivity. Empirical studies have shown how people reflexively exhibit a face orientation and work to establish a formation in which everyone is facing each other in local participation frameworks. The Face has also been described by, e.g., Levinas as the basis for a first ethical philosophy. Humans have established these Face-formations when interacting since time immemorial, but what happens when one of the participants is present through a telepresence robot? Based on ethnomethodology, Peircean/Goodwinian semiotics, multimodal conversation analysis and video data from a Danish residential rehabilitation center, the article shows the ways in which participants manage to interactively, cooperatively, and moment by moment achieve an F-formation in situ. The article contributes a detailed analysis and discussion of the kind of participant a telepresence robot is, in and through situated interactions: I propose that we term this participant the RoboDoc, given that it is an assemblage of a doctor who controls a robot. By focusing on the affordances of mobility, the article contributes to a renewed understanding of the importance and relevance of establishing Face-orientations in an increasingly technofied telepresence world.
Acknowledgments
Data for this paper is from the research project “Professionals’ use of video meeting” supported by the Velux Foundation, Denmark. Special thanks to Rikke Nielsen for helping with data collection. Earlier versions of the paper were presented at the conferences Nordisco 2018 and Lansi 2018.
References
Arminen, Ilkka, Christian Licoppe & Anna Spagnolli. 2016. Respecifying mediated interaction. Research on Language and Social Interaction 49(4). 290–309. https://doi.org/10.1080/08351813.2016.1234614.Search in Google Scholar
Arundale, Robert B. 2013. Face as a research focus in interpersonal pragmatics: Relational and emic perspectives. Journal of Pragmatics 58. 108–120. https://doi.org/10.1016/j.pragma.2013.05.013.Search in Google Scholar
Bateson, Gregory & Margaret Mead. 1942. Balinese character, a photographic analysis. New York: New York Academy of Sciences.Search in Google Scholar
Birdwhistell, Ray. 1970. Kinesics and context, essays on body motion communication. Philadelphia: University of Pennsylvania Press.10.9783/9780812201284Search in Google Scholar
Broth, Mathias & Leelo Keevallik. 2014. Getting ready to move as a couple accomplishing mobile formations in a dance class. Space and Culture 17(2). 107–121. https://doi.org/10.1177/1206331213508483.Search in Google Scholar
Callon, Michel & John Law. 1997. After the individual in society: Lessons on collectivity from science, technology, and society. Canadian Journal of Sociology/Cahiers Canadiens de Sociologie 22(2). 165–182. https://doi.org/10.2307/3341747.Search in Google Scholar
Clark, Andy. 2004. Natural-born cyborgs: Minds, technologies, and the future of human intelligence. Oxford: Oxford University Press.Search in Google Scholar
Clark, Andy & David Chalmers. 1998. The extended mind. Analysis 58(1). 7–19. https://doi.org/10.1093/analys/58.1.7.Search in Google Scholar
Clynes, Manfred E. & Nathan S. Kline. 1960. Cyborgs and space. Astronautics 5(9). 26–27, 74–76.Search in Google Scholar
Dant, Tim. 2004. The driver-car. Theory, Culture & Society 21(4–5). 61–79. https://doi.org/10.1177/0263276404046061.Search in Google Scholar
De Stefani, Elwys, Mathias Broth & Arnulf Deppermann. 2019. On the road: Communicating traffic. Language & Communication 65. 1–6. https://doi.org/10.1016/j.langcom.2018.04.009.Search in Google Scholar
Deppermann, Arnulf. 2018. Chuck Goodwin’s two ground-breaking contributions to the study of social interaction: Simultaneities in multimodal interaction and professional vision. In Donald Favareau (ed.), Co-operative engagements of intertwined semiosis: Essays in honour of Charles Goodwin, 55–69. Tartu: University of Tartu Press.Search in Google Scholar
Dreyfus, Hubert. 2000. Telepistemology: Descartes’ last stand. In Ken Goldberg (ed.), The robot in the garden: Telerobotics and telepistemology in the age of the Internet, 48–63. Cambridge: MIT Press.Search in Google Scholar
Due, Brain L. 2014. The development of an idea in a context of rejection. Semiotica 202(1–4). 207–239.10.1515/sem-2014-0036Search in Google Scholar
Due, Brain L. 2017. Multimodal interaktionsanalyse og videoetnografisk dataindsamling. Region Hovedstaden: Samfundslitteratur.Search in Google Scholar
Due, Brain L. & Simon Lange. 2018a. Semiotic resources for navigation: A video ethnographic study of blind people’s uses of the white cane and a guide dog for navigating in urban areas. Semiotica 222(1–4). 287–312. https://doi.org/10.1515/sem-2016-0196.Search in Google Scholar
Due, Brain L. & Simon Lange. 2018b. The Moses effect: The spatial hierarchy and joint accomplishment of a blind person navigating. Space and Culture 21(2). 129–144. https://doi.org/10.1177/1206331217734541.Search in Google Scholar
Due, Brain L. & Johan Trærup. 2018. Passing glasses: Accomplishing deontic stance at the optician. Social Interaction 1(2). https://doi.org/10.7146/si.v1i2.110020.Search in Google Scholar
Enfield, Nick J. 2006. Social consequences of common ground. In N. J. Enfield & S. C. Levinson (eds.), Roots of human sociality: Culture, cognition, and interaction, 223–254. Oxford: Berg Press.10.4324/9781003135517-20Search in Google Scholar
Enfield, Nick J. 2009. The anatomy of meaning: Speech, gesture, and composite utterances. Cambridge: Cambridge University Press.10.1017/CBO9780511576737Search in Google Scholar
Enfield, Nick J. 2011. Elements of formulation. In J. Streeck, C. Goodwin & C. LeBaron (eds.), Embodied interaction: Language and body in the material world, 59–67. Cambridge: Cambridge University Press.Search in Google Scholar
Ford, Cecilia E. & Trini Stickle. 2012. Securing recipiency in workplace meetings: Multimodal practices. Discourse Studies 14(1). 11–30. https://doi.org/10.1177/1461445611427213.Search in Google Scholar
Garfinkel, Harold. 1967. Studies in ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall.Search in Google Scholar
Garfinkel, Harold. 1991. Respecification: Evidence for locally produced, naturally accountable phenomena of order, logic, reason, meaning, methods, etc. in and of the essential haecceity of immortal ordinary society (I) – an announcement of studies. In Graham Button (ed.), Ethnomethodology and the human sciences, 10–19. Cambridge: Cambridge University Press.10.1017/CBO9780511611827.003Search in Google Scholar
Garfinkel, Harold & Harvey L. Sacks. 1970. On formal structures of practical actions. In J. C. McKinney & E. A. Tiryakian (eds.), Theoretical sociology, 338–366. New York: Appleton Century Crofts.Search in Google Scholar
Gill, Virginia Teas & Felicia Roberts. 2013. Conversation analysis in medicine. In Jack Sidnell & Tanya Stivers (eds.), The handbook of conversation analysis, 575–592. Chichester: Blackwell.10.1002/9781118325001.ch28Search in Google Scholar
Goffman, Erving. 1964. The neglected situation. American Anthropologist 66(6). 133–136. https://doi.org/10.1525/aa.1964.66.suppl_3.02a00090.Search in Google Scholar
Goffman, Erving. 1967. Interaction ritual: Essays on face-to-face behavior. Garden City, NY: Doubleday.Search in Google Scholar
Goffman, Erving. 1971. Relations in public: Microstudies of the public order. New York: Harper and Row.Search in Google Scholar
Goodwin, Charles. 1979. The interactive construction of a sentence in natural conversation. In G. Psathas (ed.), Everyday language: Studies in ethnomethodology, 97–121. New York: Irvington.Search in Google Scholar
Goodwin, Charles. 1981. Conversational organization: Interaction between speakers and hearers. New York: Academic Press.Search in Google Scholar
Goodwin, Charles. 1995. Co-constructing meaning in conversations with an aphasic man. Research on Language and Social Interaction 28(3). 233–260. https://doi.org/10.1207/s15327973rlsi2803_4.Search in Google Scholar
Goodwin, Charles. 2003. The semiotic body in its environment. In J. Coupland & R. Gwyn (eds.), Discourses of the body, 19–42. New York: Palgrave Connect.10.1057/9781403918543_2Search in Google Scholar
Goodwin, Charles. 2006. Human sociality as mutual orientation in a rich interactive environment: Multimodal utterances and pointing in aphasia. In Nick Enfield and Stephen C. Levinson (eds.), Roots of human sociality, 96–125. Oxford: Berg Press.10.4324/9781003135517-5Search in Google Scholar
Goodwin, Charles. 2017. Co-operative action. Cambridge: Cambridge University Press.10.1017/9781139016735Search in Google Scholar
Goodwin, Marjorie Harness & Charles Goodwin. 2012. Car talk: Integrating texts, bodies, and changing landscapes. Semiotica 191(1/4). 257–286.10.1515/sem-2012-0063Search in Google Scholar
Greimas, Algirdas J. 1987. Actants, actors, and figures. In On meaning: Selected writings in semiotic theory (Theory and history of literature 38), Paul J. Perron & Frank H. Collins (trans.), 106–120. Minneapolis: University of Minnesota Press.Search in Google Scholar
Greimas, Algirdas J. & F. Rastier. 1968. The interaction of semiotic constraints. Yale French Studies 41. 86–105. https://doi.org/10.2307/2929667.Search in Google Scholar
Haddington, Pentti, Tinna Keisanen, Lorenza Mondada & Maurice Nevile. 2014. Multiactivity in social interaction: Beyond multitasking. Amsterdam: John Benjamins.10.1075/z.187Search in Google Scholar
Haddington, Pentti, Lorenza Mondada & Maurice Nevile. 2013. Interaction and mobility, language and the body in motion. Berlin: De Gruyter.10.1515/9783110291278Search in Google Scholar
Haddington, Pentti, Maurice Nevile & Tiina Keisanen. 2012. Meaning in motion: Sharing the car, sharing the drive. Semiotica 191(1–4). 101–116.10.1515/sem-2012-0057Search in Google Scholar
Hall, Edward T. 1966. The hidden dimension. New York: Anchor.Search in Google Scholar
Haraway, Donna. 1990. Simians, cyborgs, and women: The reinvention of nature. New York: Routledge.Search in Google Scholar
Haraway, Donna. 1991. A cyborg manifesto: Science, technology, and socialist-feminism in the late 20th century. In Simians, cyborgs and women: The reinvention of nature, 149–181. London: Routledge.10.1007/978-1-4020-3803-7_4Search in Google Scholar
Haraway, Donna. 2003. The companion species manifesto: Dogs, people, and significant otherness, M. Begelke (ed.). Chicago: Prickly Paradigm Press.Search in Google Scholar
Heath, Christian. 1984. Participation in the medical consultation: The co-ordination of verbal and nonverbal behaviour between the doctor and patient. Sociology of Health & Illness 6(3). 311–388. https://doi.org/10.1111/1467-9566.ep10491964.Search in Google Scholar
Heath, Christain, Jon Hindmarsh & Poul Luff. 2010. Video in qualitative research. London: SAGE.Search in Google Scholar
Heath, Christain & Poul Luff. 1992. Media space and communicative asymmetries: Preliminary observations of video-mediated interaction. Human–Computer Interaction 7(3). 315–346. https://doi.org/10.1207/s15327051hci0703_3.Search in Google Scholar
Heidegger, Martin. 1977. The question concerning technology – and other essays. New York: Garland.Search in Google Scholar
Heritage, John & Douglas W. Maynard. 2006. Communication in medical care. Cambridge: Cambridge University Press.10.1017/CBO9780511607172Search in Google Scholar
Heritage, John & Jeffrey D. Robinson. 2006. Accounting for the visit: Giving reasons for seeking medical care. In John Heritage & Douglas W. Maynard (eds.), Communication in medical care: Interaction between primary care physicians and patients, 48–85. Cambridge: Cambridge University Press.10.1017/CBO9780511607172.005Search in Google Scholar
Hjulstad, John. 2016. Practices of organizing built space in videoconference-mediated interactions. Research on Language and Social Interaction 49(4). 325–341. https://doi.org/10.1080/08351813.2016.1199087.Search in Google Scholar
Jefferson, Gail. 2004. Glossary of transcript symbols with an introduction. In Gene H. Lerner (ed.), Conversation analysis: Studies from the first generation, 13–31. Amsterdam: John Benjamins.10.1075/pbns.125.02jefSearch in Google Scholar
Kendon, Adam. 1976. The F-formation system: The spatial organization of social encounters. Man–Environment Systems 6. 291–296.Search in Google Scholar
Kockelman, Paul. 2007. Agency: The relation between meaning, power, and knowledge. Current Anthropology 48(3). 375–401. https://doi.org/10.1086/512998.Search in Google Scholar
Krummheuer, Antonia Lina. 2016. Who am i? What are you? Identity construction in encounters between a teleoperated robot and people with acquired brain injury. In A. Agah, J. J. Cabibihan, A. Howard, M. Salichs & H. He (eds.), Social Robotics. ICSR 2016 (Lecture notes in computer science 9979), 880–889. Berlin: Springer.10.1007/978-3-319-47437-3_86Search in Google Scholar
Latour, Bruno. 1996a. On actor-network theory: A few clarifications. Soziale Welt 47(4). 369–381.Search in Google Scholar
Latour, Bruno. 1996b. On interobjectivity. Mind, Culture, and Activity 3(4). 228. https://doi.org/10.1207/s15327884mca0304_2.Search in Google Scholar
Latour, Bruno. 2005. Reassembling the social: An introduction to actor-network-theory. Oxford: Oxford University Press.Search in Google Scholar
Law, John. 2009. Actor network theory and material semiotics. In The new Blackwell companion to social theory, 141–158. Chichester: John Wiley.10.1002/9781444304992.ch7Search in Google Scholar
Levinas, Emmanuel. 1979. Totality and infinity: An essay on exteriority. Berlin: Springer.10.1007/978-94-009-9342-6Search in Google Scholar
Lévinas, Emmanuel. 1985. Ethics and infinity, P. Nemo (trans.). Pittsburgh: Duquesne University Press.Search in Google Scholar
Liberman, Kenneth. 2013. More studies in ethnomethodology. Albany, NY: SUNY Press.Search in Google Scholar
Licoppe, Chistian. 2017. Showing objects in Skype video-mediated conversations: From showing gestures to showing sequences. Journal of Pragmatics 110. 63–82. https://doi.org/10.1016/j.pragma.2017.01.007.Search in Google Scholar
Licoppe, Christain & Julin Figeac. 2018. Gaze patterns and the temporal organization of multiple activities in mobile smartphone uses. Human–Computer Interaction 33(5–6). 311–334. https://doi.org/10.1080/07370024.2017.1326008.Search in Google Scholar
Licoppe, Christian, Paul Luff, Christian Heath, Hideaki Kuzuoka, Naomi Yamashita & Sylvane Tuncer. 2017. Showing objects: Holding and manipulating artefacts in video-mediated collaborative settings. In CHI ’17: Proceedings of the 2017 CHI conference on human factors in computing systems, 5295–5306. New York: ACM Press.10.1145/3025453.3025848Search in Google Scholar
Licoppe, Christian & Julien Morel. 2012. Video-in-interaction: ‘Talking heads’ and the multimodal organization of mobile and Skype video calls. Research on Language and Social Interaction 45. 37–41. https://doi.org/10.1080/08351813.2012.724996.Search in Google Scholar
Licoppe, Christian & Julien Morel. 2014. Mundane video directors: Showing one’s environment in Skype and mobile video calls. In M. Broth, E. Laurier & L. Mondada (eds.), Studies of video practices: Video at work, 135–160. London: Routledge.Search in Google Scholar
Licoppe, Christian & Julien Morel. 2018. Visuality, text and talk, and the systematic organization of interaction in Periscope live video streams. Discourse Studies 20(5). 637–665. https://doi.org/10.1177/1461445618760606.Search in Google Scholar
Licoppe, Christian & Clair-Antonie Veyrier. 2017. How to show the interpreter on screen? The normative organization of visual ecologies in multilingual courtrooms with video links. Journal of Pragmatics 107. 147–164. https://doi.org/10.1016/j.pragma.2016.09.012.Search in Google Scholar
Luff, Paul, Christian Heath, Hideaki Kuzuoka, Jon Hindmarsh, Keiichi Yamazaki & Shinya Oyama. 2003. Fractured ecologies: Creating environments for collaboration. Human–Computer Interaction 18(1). 51–84. https://doi.org/10.1207/s15327051hci1812_3.Search in Google Scholar
Luff, Paul, Christian Heath, Naomi Yamashita, Hideaki Kuzuoka & Marine Jirotka. 2016. Embedded reference: Translocating gestures in video-mediated interaction. Research on Language and Social Interaction 49(4). 342–361. https://doi.org/10.1080/08351813.2016.1199088.Search in Google Scholar
Luff, Paul, Marina Jirotka, Naomi Yamashita, Hideaki Kuzuoka, Christian Heath & Grace Eden. 2013. Embedded interaction: The accomplishment of actions in everyday and video-mediated environments. AIS Transactions on Human–Computer Interaction 20(1). 1–22. https://doi.org/10.1145/2442106.2442112.Search in Google Scholar
Marstrand, Ann Katrine & Jan Svennevig. 2018. A preference for non-invasive touch in caregiving contexts. Social Interaction 1(2). https://doi.org/10.7146/si.v1i2.110019.Search in Google Scholar
McGregor, Moira, Barry Brown & Donald McMillan. 2014. 100 days of iPhone use: Mobile recording in the wild. In CHI ’14 extended abstracts on human factors in computing systems, 2335–2340. New York: ACM.10.1145/2559206.2581296Search in Google Scholar
McIlvenny, Paul. 2013. Interacting outside the box: Between social interaction and mobilities. In Pentti Haddington, Lorenza Mondada & Maurice Nevile (eds.), Interaction and mobility: Language and the body in motion, 409–417. Berlin: De Gruyter.10.1515/9783110291278.409Search in Google Scholar
McIlvenny, Paul. 2018. How did the mobility scooter cross the road? Coordinating with co-movers and other movers in traffic. Language & Communication 65. 105–130.10.1016/j.langcom.2018.04.002Search in Google Scholar
McIlvenny, Paul, Mathias Broth & Pentti Haddington. 2009. Communicating place, space, and mobility. Journal of Pragmatics 41(10). 1879–1886. https://doi.org/10.1016/j.pragma.2008.09.014.Search in Google Scholar
Merleau-Ponty, Maurice. 2002. Phenomenology of perception. London: Routledge.10.4324/9780203994610Search in Google Scholar
Meyer, Christian, Jurgen Streeck & J. Scott Jordan. 2017. Intercorporeality: Emerging socialities in interaction. Oxford: Oxford University Press.10.1093/acprof:oso/9780190210465.001.0001Search in Google Scholar
Mikesell, Lisa. 2016. The use of directives to repair embodied (mis)understandings in interactions with individuals diagnosed with frontotemporal dementia. Research on Language and Social Interaction 49(3). 201–219. https://doi.org/10.1080/08351813.2016.1196550.Search in Google Scholar
Mondada, Lorenza. 2009. Emergent focused interactions in public places: A systematic analysis of the multimodal achievement of a common interactional space. Journal of Pragmatics 41(10). 1977–1997. https://doi.org/10.1016/j.pragma.2008.09.019.Search in Google Scholar
Mondada, Lorenza. 2014. The local constitution of multimodal resources for social interaction. Journal of Pragmatics 65. 137–156. https://doi.org/10.1016/j.pragma.2014.04.004.Search in Google Scholar
Mondada, Lorenza. 2018. Multiple temporalities of language and body in interaction: Challenges for transcribing multimodality. Research on Language and Social Interaction 51(1). 85–106. https://doi.org/10.1080/08351813.2018.1413878.Search in Google Scholar
Norman, Donald. 1993. Things that make us smart: Defending human attributes in the age of the machine. New York: Basic.Search in Google Scholar
Peirce, Charles S. 1955. Philosophical writings of Peirce, J. Buchler (ed.). New York: Dover.Search in Google Scholar
Perakyla, Anssi. 1998. Authority and accountability: The delivery of diagnosis in primary health care. Social Psychology Quarterly 61(4). 301–320. https://doi.org/10.2307/2787032.Search in Google Scholar
Philipsen, Johanne S. & Lasse V. Jensen. 2018. “How do signs come to mean?” – Reflections on the Goodwinian interactional approach to empirical investigations of the human semiotic ecology. In D. Favareau (ed.), Co-operative engagements of intertwined semiosis: Essays in honour of Charles Goodwin, 301–305. Tartu: University of Tartu Press.Search in Google Scholar
Ritchie, Graeme. 1999. Developing the incongruity-resolution theory. https://www.era.lib.ed.ac.uk/handle/1842/3397 (accessed 15 September 2020).Search in Google Scholar
Robillard, Albert B. 1999. Meaning of a disability: The lived experience of paralysis. Philadelphia: Temple University Press.Search in Google Scholar
Sacks, Harvey L., Emmanuel A. Schegloff & Gail Jefferson. 1974. A simplest systematics for the organization of turn-taking for conversation. Language 50(4). 696–735. https://doi.org/10.2307/412243.Search in Google Scholar
Scheflen, Albert E. 1968. Human communication: Behavioral programs and their integration in interaction. Behavioral Science 13(1). 44–55. https://doi.org/10.1002/bs.3830130106.Search in Google Scholar
Schegloff, Emmanuel A. 1987. Analyzing single episodes of interaction: An exercise in conversation analysis. Social Psychology Quarterly 50(2). 101–114. https://doi.org/10.2307/2786745.Search in Google Scholar
Schegloff, Emmanuel A. & Harvey L. Sacks. 1973. Opening up closings. Semiotica 8(4). 289–327. https://doi.org/10.1515/semi.1973.8.4.289.Search in Google Scholar
Sørensen, Anders Stengaard & Gitte Rasmussen. 2018. RoBody interaction: A new approach at kinesthetic human robot interaction. In Proceedings of RO-MAN 2018 – IEEE international conference on robot and human interactive communication. Piscataway, NJ: IEEE.10.1109/ROMAN.2018.8525844Search in Google Scholar
Stefani, Elvys De. 2013. The collaborative organisation of next actions in a semiotically rich environment: Shopping as a couple. In P. Haddington, L. Mondada & M. Nevile (eds.), Interaction and mobility language and the body in motion, 123–151. Berlin: De Gruyter.10.1515/9783110291278.123Search in Google Scholar
Stevanovic, Melisa & Anssi Peräkylä. 2012. Deontic authority in interaction: The right to announce, propose, and decide. Research on Language & Social Interaction 45(3). 297–321. https://doi.org/10.1080/08351813.2012.699260.Search in Google Scholar
Stivers, Tanya & Jack Sidnell. 2005. Introduction: Multimodal interaction. Semiotica 156(1/4). 1–20. https://doi.org/10.1515/semi.2005.2005.156.1.Search in Google Scholar
Urry, John. 2007. Mobilities. Cambridge: Polity.Search in Google Scholar
Webb, Helena, Dirk vom Lehn, Christian Heath, Will Gibson & Bruce J. W. Evans. 2013. The problem with “problems”: The case of openings in optometry consultations. Research on Language and Social Interaction 46(1). 65–83. https://doi.org/10.1080/08351813.2012.753724.Search in Google Scholar
© 2020 Walter de Gruyter GmbH, Berlin/Boston
