1 Introduction

You enter the foyer in a gallery and see a large white box sitting next to a shelf and chair. You wouldn’t even have taken note of the box but suddenly it rotates then slides along the side of the shelf. Probably some sort of vacuum cleaner, you think. As you turn away, you catch something unexpected: one of the box’s top corners delicately raises upwards and the whole box seems to gently tilt towards you, before it begins to skitter in your direction. Really? You look around for other witnesses to this curious event, or better, someone controlling the box, half expecting a child with a joystick grinning at you. As you turn back, you find the artefact precariously teetering on one of its edges before forcefully tipping onto a corner, where it slowly sways, as if pondering what to do next. You decide to move closer and it seems to suddenly halt its swivel in mid-air, but you notice an ever so slight tremble. When you bend down to investigate, it rambunctiously bumps onto the ground. Bomp bada bomp.

The scene may be reminiscent of a Disney animation, where shapes and objects ‘come alive’. Abstract animated screen characters, however, are often deliberately anthropomorphized by imbuing the object with human features and a humanlike disposition. Yet while the ‘box’ in the above scene moves in ways that display some form of communicative intention, it does not offer googly eyes or framing strategies that guide our gaze (see Lasseter 2001) to easily pinpoint its expressions. The artefact in question is a robotic prototype, coming out of our Machine Movement Lab (MML) project, which investigates the relational potential of movement and, in particular, movement qualities in human–robot interaction. What the robotic artefact offers, in contrast to screen-based characters, is embodied dynamics and material relations that we can bodily share and kinaesthetically grasp. Even though ‘giving life’ to an artefact is not what we aim for, the effects of a simple artefact moving in dynamic and delicate ways open up an ambiguous zone between subject and object. Such “behavioural objects”, according to Levillain and Zibetti (2017, p. 5), “carry spatial transformations that can be interpreted as actions executed toward a goal, possibly motivated, and possibly intelligent”, without resembling humans or animals.

Human–Robot Interaction (HRI) research frequently sides with traditional animation strategies with regards to portraying artefacts as humanlike subjects, whether seen as technological artefacts that humans interact with as if they were a person or made to look and behave humanlike, thus masking the issue altogether. Considerable ongoing efforts to overcome the ontological gap between people and robots as communicators (see Guzman 2020) rely on blurring the profound differences and “deep asymmetries” (Suchman 2007, p. 11) between them. Yet at its core, human–robot interaction is about interacting and communicating with social entities that dramatically differ from us—ontologically as well as culturally (see Guzman 2020). While dominant HRI research approaches assume that machine ‘otherness’ disrupts successful communication with people (see Sandry 2016), from an artistic research viewpoint, this otherness opens up a productive challenge and ample potential for communication that is about situated meaning-making, rather than information exchange. Human–machine communication (HMC), according to Guzman (2018), is not only about transmitting information, but also about how meaning is created between human and machine participants. This article attempts to develop a counter position to human-centric approaches in HRI by examining how meaning is bodily and dynamically enacted in human-nonhuman encounters.

1.1 Communicating with an ‘other’

Hegel et al. state that “with a functional designed robot it is impossible to express human facial expressions and consequently emotional displays” (2009, p. 173). This limited notion of affective communication starkly contrasts artistic practices exploring the capacity of machinelike artefacts to evoke affective responses or communicate in non-verbal ways. Artists developing kinetic sculptures, robotic artworks or machine performances have studied the potential of movement, sound and other relation-making modes, such as staging, to render artefacts relational or transform their machine identity. The Table: Childhood (1984–2001) by Max Dean, Raffaello D'Andrea and Matt Donovan, for example, produces surprising relational dynamics between audience members and the familiar object of a table (see Drouin-Brisebois 2008). This interactive artwork attempted to reverse the roles of viewer and object by staging a white yet otherwise non-descript table following audience members, apparently selected by the table itself, around the gallery space. The seemingly intentional movement of the table not only renders the object strange but also enables it to open up unexpected, evocative moments of encounter. Simon Penny’s Petit Mal (1989–2005) resembles a strange dicycle, which takes on the role of “an actor in social space” (Penny 2000, p. 400). The work’s unique, non-verbal behavioural performance arises from an eccentric, yet simple mechanism based around a double pendulum, which brings an unpredictable and charming quality to its movements; seemingly struggling to balance, it sways through the gallery space, inviting audiences to bodily interact with it (Penny 2016, p. 57).

From a communication perspective, interactions with such ‘machine performers’ are not governed by familiar turn-taking protocols (see Sandry 2016). Rather, they evoke, or sometimes provoke, flowing interactional coordination, based on spatial or empathic interpretations of movement. Penny argues that artistic practice favours “subtle and evocative modes of communication” (2000, p. 400) over goal-oriented functionality. Communication here is not predicated on the transmission of information but rather focuses on modes of engagement, embodied experience and emergent meaning-making. This often involves a multimodal, embedded approach, as well as an understanding of complex sociocultural contexts. By embracing and promoting multiple interpretations and the emergence of meanings in the interaction, arts-led approaches resonate with conceptions of communication as “an emergent property of systems” (Sandry 2016, p. 188), arising between communicators, rather than being directly produced and transmitted by them.

Our ongoing MML project brings together creative robotics, choreography, performance techniques and machine learning, grounded in an enactive, performative framework (see section 3.2) to promote a performative understanding of communication by exploring the relational potential of human–machine asymmetries. Designing a social artefact, in MML, is about scaffolding a machine’s ability to participate in embodied meaning-making, rather than imbuing the machine with a predesigned sociality, e.g., through humanlike features. To create this intra-corporeal scaffold, our design methodology opens up an intimate link to embodied, performance-based inquiries into the generative potential of movement and its dynamic qualities to enact meaning with abstract robotic artefacts. Starting in 2015, we first set out to search for possible abstract shapes that, when in motion, take on relational, affective qualities without relying on human-like features. To explore the communicative potential of shapes-in-motion, we asked dancers to bodily extend into them and their material affordances by inhabiting and moving with them. A small selection of simple geometric shapes were then rebuilt as wearable costumes of various sizes to serve as embodied mapping interfaces. Standing in for a becoming-robot’s shape, a costume harnesses a dancer’s tactile-kinaesthetic expertise for movement creation to socioculturally situate (see Lindblom 2020; Lindblom and Ziemke 2003) a robot’s learning to move in relational ways (see Fig. 1). We discuss this mapping methodology, called Performative Body Mapping (PBM), in more detail in section 3.

Fig. 1
figure 1

PBM cube costume inhabited by Audrey Rochette (on the left) with Cube Performer #1

Full size image

Based on our PBM dancer-costume entanglements, we have so far realised two robotic prototypes, Cube Performer #1 (see Figs. 1 and 2) and Cube Performer #2. Observing and talking to exhibition audiences and interaction study participants, the blank canvas offered by a plain, regular cube shape when juxtaposed with the rich dynamics of energetic movement qualities seems to open up a large potential for spatial transformations that we can read as meaningful actions (see Levillain and Zibetti 2017).

Fig. 2
figure 2

Machine Movement Lab: Cube Performer #1 at the Games and Performing Arts Festival, UK, 2018

Full size image

Such a relational-performative approach towards HMC is not suitable for all human–robot interaction scenarios. In particular, the bodily, kinaesthetic immersion that is at the heart of our methodology may not be feasible for a particular robot design. Furthermore, our emphasis on the enactment of situated, emergent meanings in our encounters with robotic social entities poses new challenges for interaction scenarios in narrowly defined tasks. However, shifting the focus from representationalism to performativity could advance our understanding of how a robot’s sociality emerges and reveal new pathways for robot design. Machinelike robots whose communicative skills exploit the relational, aesthetic potential of dynamic movement qualities could produce novel, more diverse human–machine relationships that pose fewer ethical risks to potentially vulnerable users than those formed with humanlike robots (see Lee et al 2016; Turkle 2011).

In the following, we begin by outlining a critical perspective on human–robot interaction relying on human likeness (section 2) before taking a closer look at our design methodology. The latter includes a discussion of our methodological approach (section 3.1), participatory studies ( 3.3) and our conceptual framework for performative, relational human–machine interaction (3.2), bringing concepts from posthuman performativity and embodied, affective sense-making into a unique conjunction through performance-based, creative practice.

2 Making machines in our own image

The promotional video for PepperFootnote 1 promotes the humanoid robot as an emotional friend—a robot that can be thought of as “high tech you can high five”. Pepper is one example of a range of commercial humanoid robots marketed as companions, assistants, and carers, heralding a not-too-distant future in which we will live with robots that exhibit traits reminiscent of ourselves—smart, polite and gendered. Robots that are human enough that we can easily empathise with their familiar performances or ‘high five’ them. This marketing sentiment is often sustained by the young fields of Social Robotics and HRI, where great effort is invested in studying how anthropomorphic attributes positively affect people’s acceptance of and interaction with robots (De Graaf and Allouch 2013; Fink 2012; Fong et al 2003). According to popular definitions, a social robot should exhibit design features that permit ‘natural interaction’ (Dautenhahn 2013; Hegel et al 2009), including a distinctive personality and the capacity to express and perceive emotions (Becker 2006; Fong et al 2003). The aim is to render communication between people and machine artefacts ‘more natural’ (Becker 2006; Guzman 2018; Hegel et al 2009), founded on the belief that humanlike features and familiar behaviours can be orchestrated to give a robot a social façade (see Jones 2017; Broadbent 2017; Alač 2015). Despite the dominance of humanoid robots in social robotics, we can also find many studies that question underlying assumptions such as ‘natural interaction’ (Dautenhahn 2013) and the appeal of human likeness (Lee et al 2016; Vlachos et al 2016) and investigate potential social impacts (Šabanović 2010; Turkle 2007).

Successful human–machine communication, seen from this mimetic perspective, is based on what communicators either already have or could have in common (see Sandry 2016). Communication here is framed as a process that has a correct outcome or predefined protocol, where potentially ambiguous meanings or multiple interpretations would be, in Sandry’s words, “an undesirable risk that should be eliminated” (Sandry 2016, p. 179). This narrow view promotes a transmission model of communication, where messages are sent and received or information is passed from one communicator to another (Craig 1999). A constitutive model of communication, in contrast, is based on the production of shared meanings (Craig 1999). The former, focusing on a neatly bounded channelling of pre-scripted meanings, lends itself well to technological functionalist thought and notions of technological control (see Craig 1999). In addition to omitting meanings arising from interaction dynamics, a transmission-focused approach to HMC is also prone to be blind to the sociocultural context that human communicators inevitably bring with them (see Guzman 2018; Craig 1999).

Researchers in Science and Technology Studies (STS) have been calling for more diversified robot designs to allow for human–robot relations that do not rely on beguiling users through imitation (Jones 2018; Castañeda and Suchman 2014; Turkle 2011). Talk of peoples’ or robots’ agency “presupposes a field of discrete, self-standing entities” (Suchman 2007, p. 263), fuelling the assumption that a machine, as a discrete entity, can be rendered social by mimicking human capacities (Alač 2016). Agency and sociality here are seen as intrinsic qualities that can be reverse engineered and programmed into a robot (Jones 2018), irrespective of the wider sociomaterial context. But what if social capacity cannot simply be ‘given’ to a robot and human–robot relationships cannot simply be modelled after human–human relationships (see Jones 2018)? What if, instead, we need to look at human–robot interaction as a relationally enacted, situated meaning-making process, in which a machine’s social agency is brought forth and sustained by the dynamics unfolding in the encounter?

Our arts-led design approach, discussed below, aims to mobilise some of the critical views of dominant assumptions about a machine’s social agency and how they shape the ways we understand successful human–robot communication. MML’s methodological contribution promotes an embodiment-centred, relational model of communication that places movement and its dynamic qualities at the centre of meaning-making.

3 Beyond the mirror and straight through the looking glass: Machine Movement Lab (MML)

Our argument that robots should be conceived of as machinelike, rather than humanlike, communicators has also been raised by researchers in communication studies (Guzman and Lewis 2020; Hoorn 2018; Sandry 2016, 2019). However, the relational dynamics that both robot design and resulting robots are embedded in and how they contribute to a social actor’s capacity to participate in the encounter are often overlooked (see Guzman and Lewis 2020). In fact, we believe that the relational dynamics that both constitute and unfold in the design process play a key role in the relations we can have with a machine. That is to say that the practice of enacting human–machine communication starts at the very beginning of the design process, not once a robot design is apparently complete. The remainder of this article will take a closer look at our MML project and how it seeks to embrace, and aesthetically exploit, the asymmetries between human and machine embodiments to facilitate meaning-making.

3.1 Performative Body Mapping (PBM)

Robot designs that do not rely on mimicking familiar, organic bodies allow for rich encounters, where meaning-making is not predetermined or constrained by the expectations, conceptions, or projections we form in advance of the experience of encounter (Dautenhahn 2002). Guzman observes that people’s perception of the differences between humans and machines co-shape their overall interpretations of the machine communicator, which, in turn, informs their decisions and actions over the process of communicating with the machine (Guzman 2020). To embrace the difference of machines, the challenge is to find a starting point from which to explore the social potential of machinelike agents.

MML takes movement and its “spatio-temporal-energetic” (Sheets-Johnstone 2011, p. 432) qualities as its starting point to investigate the relational potential of abstract, non-humanlike machines and situate these strange artefacts in our social environment (see section 3.2). Movement and, with it, bodily perception and affect are core to embodied meaning-making (Johnson 2007, 2018). Contemporary dance, which purposely and systematically develops movement “for its own sake” (Stevens and McKechnie 2005, p. 243), is a natural ally for investigating embodied, relational meaning-making and bringing to the fore the potential of qualitative movement dynamics (see Sheets-Johnstone 2012). Leach and deLahunta describe the relationality of bodies, affectively reaching through movement as “an extension of feeling, knowing, and sensing into the world with, and of, other bodies” (Leach and deLahunta 2017, p. 464). According to Manning and Massumi, movement “bodies forth” (2014, p. 39), rather than “something the body does” (2014, p. 40). While collaborations between robotics and dance or performance have provided a testbed for evaluating robots’ expressive capacity (Jochum et al 2017), many of these interdisciplinary projects still involve human-like robots or integrate existing robots within a performance event.

MML investigates how the generative capacity of movement can render human–machine differences relational by harnessing choreographic knowledge and dancers’ kinaesthetic expertise to inform the robot design and situate its machine learning. As briefly laid out in the introduction, our experimental, investigatory design process revolves around an embodied mapping interface that combines ideas that underly theatrical costumesFootnote 2 (Suschke 2003) and demonstration learning in HRI (Billard et al 2008). The Performative Body Mapping (PBM) costume serves to bridge between human–machine asymmetries by enabling dancers to: (1) corporeally experience the ‘other’, machinelike morphology and learn to kinaesthetically extend into and move with it, and (2) bypass the correspondence problem, commonly posed by the challenge of mapping between two very different embodiments (Dautenhahn et al 2003). The PBM costume not only serves as a mapping interface but also an instrument for recording the kinetic traces of the dancer’s bodily activation of the costume, manifesting in the resulting movement qualities of the dancer-costume entanglement (see Fig. 3). PBM thus allows (1) delegating much of the difficult morphological mapping to the movement expert without relying on simulating human movement mechanisms, and (2) for the robot prototype to learn from the motion capture data as if it was trained by another robot performer with the same physical shape.Footnote 3

Fig. 3
figure 3

PBM cube costume, inhabited by Audrey Rochette

Full size image

Importantly, we did not begin with a predefined shape for the becoming-robot and, consequently, the costume. Early PBM workshops with choreographer Tess deQuincey and dancers from De Quincey CoFootnote 4 explored a wide range of forms and materials in motion, with the goal to challenge our assumptions and preconceptions with regards to possible machine forms and their movement capacities (see Fig. 4). Later workshops focused on a small selection of simple, geometric forms to investigate the communicative potential of movements resulting from the dancer-costume entanglement, including cuboids, cubes and a tetrahedron (see Fig. 5).Footnote 5 Since we are not interested in examining the robot’s social capacities in terms of how well it performs existing human social tasks, our approach explores how far we can push the relationship between abstract robotic forms and their potential to elicit empathic and affective responses through relational movement qualities.

Fig. 4
figure 4

Early PBM workshop with tube-like costumes, inhabited by performers

Full size image
Fig. 5
figure 5

PBM workshop, showing a dialogue between two PBM costumes inhabited by dancers (Tess de Quincey, on the right)

Full size image

Ultimately, we selected perhaps the most obvious, abstract form, yet not the most apparent in terms of its evocative capacity—a cube—to shape the first robotic prototype.Footnote 6 Starting with experiments, in which dancers inhabited a simple cardboard box, this familiar object demonstrated great potential for being transformed into something more than an ‘object’ when moving in unexpected ways. The cube’s regular, omnidirectional geometry presents a counterpose to organic structures with limbs, two-sided symmetries and the hierarchy of front and back. Furthermore, an intricately moving cube, capable of lifting off the ground, twisting upwards or gently swaying through space, quickly loses its simple, predictable nature. We believe that it may be the apparent schism between the unassuming shape of a cube and dynamic or delicate movement qualities that opens up an aesthetically rich space for transformation. So far, we have iteratively realised two 75 × 75 × 75cm robotic artefacts: Cube Performer #1 and Cube Performer #2. The movement requirements for their mechanical design were derived from an analysis of over ten hours of motion capture recordings to determine the required velocity, acceleration and ranges of vertical, horizontal and rotational movement (Gemeinboeck and Saunders 2018; see Fig. 6).

Fig. 6
figure 6

The mechanical frame of Cube Performer #1

Full size image

Our relational, performative approach proposes that the “spatio-temporal-energetic” (Sheets-Johnstone 2011, p. 432) dimensions of movement can serve to bootstrap the robot’s learning to situate the machine in the “social and cultural scaffolds” (Lindblom 2020, p. 4) that are fundamental to embodied, social meaning-making (Lindblom 2020; see also section 3.2). The PBM costume enables dancers to ‘step into’ the other, machinic shape to explore and put to work its enabling constraints rather than anthropomorphizing or imprinting their human intents onto the becoming-robot. Hence, our goal is to socioculturally situate (see Lindblom 2020; Lindblom and Ziemke 2003) the abstract artefact by entangling human dancers with the artefact and its transformational potential. Motion capture data of this dancer-cube entanglement then serves to bootstrap the learning of the robotic artefact. It comprises granular, discrete movement patterns, derived from short choreographic abstractions (Aviv 2017) that, in the machine learning process, take on the role of aesthetically and socioculturally coded biases and constraints. The latter allows the robotic artefact to learn to compose new movements that are both, grounded in its own unique material embodiment (see Gemeinboeck and Saunders 2018; Saunders and Gemeinboeck 2018) and embedded in our social and cultural scaffolds (see Lindblom 2020). According to Rotman, “[m]otion-capture technology allows the communicational, instrumental, and affective traffic of the body in all its movements, openings, tensings, foldings, and rhythms into the orbit of ‘writing’” (2008, p.47). Intermeshing human and nonhuman affordances, the resulting kinetic alphabet of the PBM entanglement captures a wide range of kinetic dynamics that serve to render the Cube Performer a highly skilled participant in the affective exchanges of the encounter (see Damiano and Dumouchel 2020).

3.2 Performativity and embodied meaning-making in human-machine communication

Underlying our PBM methodology is a conceptual framework, which meshes theoretical work from performative new materialism (Gamble et al 2019; see also Barad 2003, 2007), embodied meaning-making (Johnson 2007; Fuchs 2016; Sheets-Johnstone 2010) and kinaesthetic empathy (Koch 2014; Reynolds and Reason 2012; Behrends et al 2012) and brings them into a unique conjunction through performative, embodied practice. In a nutshell, we argue for and aim to put in practice a relational, performative view of human–machine communication.

3.2.1 Agency cannot be given to a robotic artefact

Both agency and meaning in HRI design are often taken for granted and amenable to technical reappropriation, determined by individually held representations (e.g., external features or pre-programmed gestures). Agency understood performatively, however, is not a property that someone or something can have or be imbued with, rather “agency is a matter of intra-acting … an enactment” (Barad 2007, p. 178). Humans and machines here are no longer regarded as distinct subjects and objects with discreet signals and messages sent in-between. Instead, we are looking at interaction as “a multiplicity of more and less closely aligned, dynamically configured moments of encounter within sociomaterial configurations, objectified as persons and machines” (Suchman 2007, p. 268). More so, we believe that the boundaries between subjects and objects become elastic and renegotiable in this dynamic encounter (see Gemeinboeck 2019, 2021)—a view, which our observations and conversations with audiences and participants seems to support (see section 3.3). Hence, rather than asking how a social agent should look like or what behaviours display ‘its’ agency, we should instead look at human–machine couplings and how they enact agency through the interactional exchange, evolving over time. The robot’s design still plays a constitutive role in this interactional exchange, as it affects how it can act and how its actions are interpreted, both of which are core to meaning making. Yet a distributed, coupled view brings to the fore an artefact’s performative, participatory capacities instead of predefined representational attributes.

Shifting the design focus from representation to performativity can unlock an understanding of how robots’ sociality emerges in the interactional exchange itself, beginning with, for instance, PBM’s dancer-costume entanglement. Our relational, performative stance places meaning making firmly in the ‘here and now’ of the interactional encounter, interrelating human and nonhuman co-agents in a particular situation. Meaning making in HMC then happens as part of a process of embodied, situated material engagement (see Malafouris 2013), rather than based on meanings (pre)ascribed to a certain appearance or behaviour. Both human and machine interactors here are rendered active participants in the meaning-making process, thus empowering them in terms of what they can bring to this negotiation and how they can evolve. A relational understanding of the “ongoing reconfigurings of the world” (Barad 2007, p. 141; see also Haraway 2008; Latour 2005; Law 2004), which we are never outside of, profoundly changes the potential of human–robot communication and opens up new pathways for robot design and studying possible human–robot relationships. It opens up a horizontal ethics of relationality (see section 4) and, from a pragmatic design viewpoint, lays open a potentially rich field of opportunities that may lead to greater freedom and novel, yet unknown ways of communicating with machines (see Sandry 2019). While ‘freeing’ the human communicator to actively negotiate and continuously renegotiate meaning may place a bigger burden on them, it also frees the machine communicator to become its own ‘thing’; a more or less social, unique artefact, depending on both its machinelike abilities to participate in this negotiation and the relational affordances of the unfolding situation.

3.2.2 Meaning making happens in the embodied interactional exchange

Our conceptual framework aligns Barad’s posthuman notion of agency as an intra-active enactment with a notion of meaning-making that is fundamentally “relational, experiential and enactive” (Johnson 2018, p. 244), situated in a particular social, material, cultural and historical context. In contrast to traditional cognitive science, embodied cognitionFootnote 7 places embodiment and interaction dynamics at the centre of the meaning-making process (Lindblom 2020; Di Paolo et al 2010; Johnson 2007; Gallagher 2005; Varela et al 1991). Instead of accessing our world through representations, we bodily participate in the generation of meaning, “often engaging in transformational and not merely informational interactions; [we] enact a world” (Di Paolo et al 2010, p. 39). Such a radical embodied view that emphasizes sociocultural situatedness and environmental embeddedness and favours a mutual link between action and perception over internal representations (Lindblom 2020; see also Fuchs 2018; Gallagher 2005) significantly affects how we can think about and consequently design HMC. Maturana and Varela (1987) characterise communicative behaviours as occurring in processes of social coupling; communication then is the observable effect of behavioural coordination. Hence, from a biological perspective, “there is no ‘transmitted information’ in communication” and, equally, social interaction “cannot be reduced to so-called ‘social information transfer’” (Lindblom 2020, p. 10). Rather, social interaction is always relational (Fuchs 2018; Di Paolo et al 2010; see also Maturana and Varela 1987; Varela et al 1991), where meaning is negotiated in the encounter itself, dynamically arising from intermeshing processes that are corporeal, affective, creative, social, cultural and interactive (see Di Paolo et al 2010; Lindblom 2020; Colombetti 2014).

Current MML research expands its investigations of a robot’s spatial-relational affordances by studying and experimenting with processes of social coupling and the spatial-relational affordances that they produce. Yet social couplings of humans and robots happen across a divide of perceptual worlds; like embodiment, a robot’s perception differs greatly from ours. Hence, while humans and robots can physically share a social space, from a biosemiotic viewpoint, they are each embodied in their own, distinctly different umwelt (Uexküll 1957; see also Ziemke and Sharkey 2001). An entity’s umwelt is the perceptual world in which it exists and acts or, in Colombetti’s words, its “lived or phenomenal environment” (Colombetti 2010, p. 5). Meaning making between humans and robots is thus an intra-bodily enactment across differentiated ecological niches. Designing for embodied HMC then is about developing pathways to negotiate humans’ and machines’ distinct ecological affordances (Fiebich 2014; see also Gibson 1979) through embodied interactional coordination.

To afford dancers an embodied insight into the Cube Performer’s unique machine umwelt, we are extending the costume interface to allow for mapping between human and nonhuman perceptual worlds, the Relational Body Mapping (RBM) costume. Equipped with the same set of sensors that the robot uses, the goal is to enable dancers inhabiting the RBM costume to experience the Cube Performer’s sensorium, made ‘tangible’ to the dancer in the form of a dynamic soundscape. Current workshops with choreographer Marie-Claude PoulinFootnote 8 and dancer Audrey Rochette experiment with the meaning-making potential of dynamically emerging relational spaces in various social couplings and the intra-bodily resonances they produce (see 3.2.3). Relational spaces between agents here are understood as situated enactments that are aesthetically evoked, sculpted and rendered elastic through movement and its dynamic qualities. Being able to tap into the robot’s different ecological affordances permits the dancer-in-costume to creatively work with the asymmetries between the two perceptual worlds and how they affect social coordination. The goal is then to bootstrap the Cube Performer’s learning with the constraints and biases of movement dynamics recorded in relation to specific social couplings; while grounded in its own machine umwelt, the learning is expanded by the dancer’s relational negotiation of two umwelts.

3.2.3 Intra-bodily resonance between bodies-in-motion

At its core, MML seeks to generate possibilities for social coupling (see Maturana and Varela 1987) between humans and machinelike artefacts. In this human-nonhuman coupling, meaning is created where rhythmic coordination (Di Paolo et al 2010) meets embodied, affective sense-making (see Colombetti 2014; Froese and Fuchs 2012). According to Colombetti’s phenomenological approach, embedded in a radical embodied understanding of cognition, our empathic experience of others does not rely on ascribing mental states to them but rather happens ‘directly’, as we perceive bodies and their expressions “as a locus of bodily subjectivity and sensations” (2014, p. 175). Our thoughts, feelings, and behaviours are grounded in our bodily interaction with other bodies and the environment (Meier et al 2012; see also Lindblom 2015; Colombetti 2014). Vice-versa, these thoughts, feelings and behaviours manifest in embodied ways in what Froese and Fuchs have termed “intra-bodily resonance” (Froese and Fuchs 2012, p. 212). As they manifest, they express themselves to others, who interpret them based on their own intra-bodily resonance. Much of our embodied, social meaning-making process thus involves movement and, in particular, movement qualities, allowing us to rhythmically coordinate with others through interaction (Di Paolo et al 2010). Fuchs and Koch thus understand motion and emotion as inherently interconnected: “one is moved by movement (perception; impression; affection) and moved to move (action; expression; e-motion)” (2014, p. 1).

Intra-bodily resonance between bodies-in-motion is referred to by researchers in dance and dance studies as kinaesthetic empathy (Behrends et al 2012). Kinaesthetic empathy is a key interdisciplinary concept concerned with a moving body’s capacity to resonate with us and to literally move and bodily affect us; thus facilitating our understanding of social interaction and embodied communication (Reynolds and Reason 2012; Foster 2008). From a performance perspective, intra-bodily resonance is a bodily processing of forces and tensions expressed in dynamic variations of movement qualities. Importantly, resulting intra-bodily relations serve as scaffolding for non-verbal meaning-making (see Meekums 2012), allowing for social interaction to be initiated and sustained without relying on stereotypical and limited modelling of artificial emotions (see Damiano and Dumouchel 2020). Hence, rather than programming social abilities into robots’ cognitive interior and relying on their anthropomorphic exterior to express them (see Damiano and Dumouchel 2020), we look to movement qualities and how they afford robots active participation in socially meaningful encounters. Our relational, performative approach thus mobilises the intra-affective capacity of the movement to actualise Damiano and Dumouchel’s “affective loop” (2020, p. 190), which locates sociality in the interactional dynamics, coenacted by the robot’s ability to engage human interactors in affective encounters.

3.3 First encounters

The relational, embodied communication, which MML promotes, is already familiar to us from the relationships we form with our animal companions. Embodied communication is, in Haraway’s words, “more like a dance than a word: the flow of entangled, meaningful bodies in time—whether jerky and nervous or flaming and flowing, whether both partners move in harmony or are painfully out of synch or something else altogether—is communication about relationship, the relationship itself, and the means of reshaping relationship and so its enacters” (Haraway 2008, p. 26). Watching participants, whether audience members or study participants, encounter our Cube Performer for the first time, we are often reminded of Haraway’s sometimes jerky, sometimes flowing dance that is embodied communication. Dynamics unfold in unpredictable configurations and participants find themselves, alternating, in moments of harmony or ‘painfully out of synch’ with the cube. In one of our studies, five out of ten participants compared their responses to the Cube Performer with the kinds of responses they have towards animals. “I was surprised how intimate it was. I responded to it like another species and increasingly so”, said one participant. Another commented, “[i]t comes across as playful with an honest curiosity, like a wild animal”. We believe that this view of relational inter-species communication presents promising pathways for human–robot communication, but without the need to render machines animal-like. The difference between the Cube Performer and, for instance, a robot vacuum cleaner is that the machine performer, situated by dancers, is better equipped to spatially and temporally coordinate with other bodies. This, in turn, mobilizes us to correlate our bodies in response to produce “moments of moving complicity” (Suchman 2007, p. 265) by forming new constellations with the machine that are likely to produce alternate openings for entanglement and so on.

We have so far studied social encounters with the Cube Performer (at various prototyping stages) as part of four public exhibitions/eventsFootnote 9 and two dedicated participatory studies.Footnote 10 The main aim of sharing even early prototypes in exhibitions and studies was to gain some insights into their capacity to generate intra-bodily resonance in unscripted, first-encounter scenarios. The robot’s mechanical structure has been conceived to allow changing its outer ‘shell’ to allow the Cube Performer to integrate itself in various (performance) contexts. For the first public exhibition, we decided to stage Cube Performer #1 as a gallery plinth, disguised amongst a group of other, immobile plinths (see Fig. 7). In the open-studio scenario, without an exhibition context, the robot took on the utilitarian identity of a simple wooden box (see Fig. 8). These ‘humble’ stagings suited our prototype stages and the contexts of encounter; at the opening of Repair (2017), for instance, two audience members jumped when the apparent plinth, which they had placed their glasses on, began to twist toward them.

Fig. 7
figure 7

Cube Performer #1, shown at RePair, The Big Anxiety Festival Sydney, 2017

Full size image
Fig. 8
figure 8

Study participants engaging with Cube Performer #2, 2019

Full size image

At an early prototyping stage, we were interested in getting feedback on the robot’s expressive and affective qualities and whether they would, in the participants’ eyes, render the robot more humanlike. On average, participants reported that they perceived the robot as evocative and affective, although not humanlike. Participants also reported perceiving the robot as spontaneous and responsive, which was surprising given its limited adaptive capabilities at the time of the study. Audiences often used affective terms, e.g., ‘curious’, ‘shy’, ‘cheeky’ or ‘playful’ to describe the ways in which they perceived the Cube Performer. Granted, it seems fair to say that we tend to attribute more agency and intent to machines that behave with a certain degree of complexity than is technically warranted (see also Levillain and Zibetti 2017), particularly in first-encounter scenarios. However, having observed people’s surprise, affect and curiosity in the course of the exhibitions and participatory studies, we believe that the rich movement qualities performed by the Cube Performer and the dynamic, affective relations they produce are a contributing factor to rendering encounters with this very plain artefact evocative and meaningful. In general, audiences either preferred to observe the robot from a distance, often circulating around it, or engaged with it directly for more than 5 min and sometimes significantly longer. These latter encounters can be characterised as engagements, where people (1) are occupied with bodily probing how the robot ‘works’ and/or how they are being sensed, (2) meet the Cube Performer ‘on its own terms’, often based on an interplay of following the robot’s movements, tilting with it, crouching or trying to keep up with it on their hands and knees and moving in sudden, unexpected ways to elicit responses from it, or (3) behave in a combination of the two, at first being inquisitive regarding its workings and increasingly developing a ‘dance’ with the robot.

In contrast to our first study, which involved members of the public, the follow-up study aimed at evaluating our PBM methodology with ten professional experts in interaction design and dance improvisation.Footnote 11 To do this, we developed a three-minute-long sequence with the PBM costume to study how the cube’s plain, omnidirectional geometry could be transformed in different spatial-affective ways when moving in different dynamics of tension, amplitude and projection (see Sheets-Johnstone 2012). We then trained the robot to perform similar movement trajectories and grouped the different spatio-temporal-energetic dynamics in three categories, described by the choreographer and dancer as ‘light–airy’, ‘boisterous–chunky’ and ‘playful–unpredictable’. Engaging our participants with this specific set of movement qualities in multiple configurations allowed us to compare their perception of these qualities and the different relational-affective affordances they produce to the dance experts’ interpretation (see Table 1). The study involved a later iteration of the robot prototype (Cube Performer #2, see Fig. 8) which could perform more movement qualities with higher sensorimotor fidelity, and participants engaged between ten and fifteen minutes with the robotic artefact. Although the robot prototype lacked improvisational skills at the time, participants tended to relate its movements to their own and reported affective responses akin to intra-bodily resonances. Table 1 shows that all participants described the artefact’s qualitative dynamics in terms that closely align with the choreographer’s and dancer’s descriptions. One participant commented, “it’s obvious that it does what it does because I’m here”; others expressed surprise about connecting with a ‘wooden box’: “I felt quite tender towards it” or “I like its non-humanness … there is a companionability to it. Wow”. The results thus indicate that the dynamics of the PBM entanglements when reinterpreted by the robot can, in the participants’ “kinetically-sensitive eyes” (Sheets-Johnston 2010, p. 124), produce kinaesthetic empathic responses. We previously referred to this form of HMC as human–robot kinesthetics (Gemeinboeck and Saunders 2018).

Table 1 Participants’ descriptors of different movement qualities (Gemeinboeck and Saunders 2019)
Full size table

We alsodeployed PBM in design workshops with primary school children, aged 8–10,Footnote 12 to engage them in creative, embodied explorations of possible relationships with machine-like robots. Children were invited to get together in pairs to design a simple robot using cardboard boxes and to develop a playful human–robot scenario with it, where one of them wears the cardboard robot costume (see Fig. 9). While some children opted for cutting holes in the box to be able to use their arms, many of the children were inspired by our demonstrations of the Cube Performer and developed intricate movement patterns with the box, sometimes coupled with moving tentacles or flaps, to give their robot a distinct affective identity.

Fig. 9
figure 9

PBM design workshop with school children, demonstrating Cube Performer #1 at AMATA, Falmouth University (on the left), and building cardboard robot costumes at The Exchange Gallery, UK, 2018

Full size image

4 Embodied meaning-making in HMC—some concluding reflections

In this article, we introduced a movement-centric design approach to HMC that counters HRI approaches whose communication potential relies on rendering the machine and its behaviours as humanlike as possible. Building on a conceptual framework that meshes concepts from performativity and embodied, affective meaning-making, HMC in this arts-led approach is fundamentally relational, embodied and performative. Its central premise is that social capacity is not a property of the machine but rather is enacted in the encounter or evolving relationship, thus shifting the design focus from the representation of social agency to how it is performatively enacted.

A core insight from our performative, relational approach is that agential enactment in meaning-making between human and nonhuman agents not only plays a key role in the interactional process but is equally core to the design process. Hence, human–machine encounters begin with the design process, not only once the robot is apparently “ready for relationships” (Turkle 2005, p. 288). Talking about effective encounters at the human machine interface, Suchman points to “those moments of moving complicity between persons and things achieved through particular, dynamic materialities and extended socialities” (Suchman 2007, p. 245). Moments of complicity brought about by dynamic configurations of specific materialities and distributed socialities not only shape the interactional encounter but also constitute significant junctures in the design process that shape the social potential of our human–machine relationships. Conceived to socioculturally situate the learning of an abstract robotic artefact, the PBM costume also serves to explore and experiment with the first instances of human–machine encounter. It permits dancers to corporeally extend and kinaesthetically probe into a particular set of material, performative possibilities to reimagine and bodily coenact the artefact’s spatial-affective affordances. It is along the embodied interface and the dancer’s kinaesthetic-material probings that the framing of possible human–machine encounters and the potential for relational meaning-making is beginning to take shape, quite literally.

Arguing for an extended notion of HMC that considers machinelike robots as social communicators, Sandry (2019) distinguishes between evocative or relational artifacts, depending on their ‘personality’. An evocative artefact, according to Sandry (2019), reveals itself in the process of interaction based on its behaviour, without suggesting a humanlike capacity for reciprocal communication, while a relational artefact directly engages the human interactor. This focus on direct engagement, she argues, is more likely not only to be considered humanlike, but also to be compared against a human assistant or companion. But opposing the evocative and relational seems unnecessarily limiting, particularly as it assumes the relational to be confined to and compared against human qualities. Both from a performance or an embodied cognition perspective embodied meaning-making is always relational and contextual, without being confined to the human domain. Indeed, it could be argued that only relational forms of HMC situate machines (whether machinelike or humanlike) in a wider sociomaterial ecology. Beyond a human-centric perspective, relational artefacts, such as the Cube Performer, open up HMC to the enactive, relational nature of meaning-making that we can observe in embodied animal communication (see Smuts 2008; Haraway 2008), as well the performative, agential role that artefacts play in our embodied, material engagements (see Malafouris 2013).

A relational, performative notion of HMC could open up a horizontal ethics of human–machine relations, one that releases the robot from Čapek’s original vision of humanlike non-humans built to serve mankind (see Madigan 2009), which arguably still weighs on current social robot design. Jones (2017) detected the beginnings of a relational turn in HRI, arguing that researchers begin to consider people’s experience and social judgement as part of the making of a robot’s sociality. Hegel et al (2009), for instance, speak of robots with a social interface, where the latter is a metaphor for a machine’s properties that, in the observer’s eyes, render it social. However, shifting the attribution of sociality from an internal property to peoples’ perception still understands social agency as something given to the artefact by humans, albeit through psychological attribution (see Levillain and Zibetti 2017). Aligned with our argument here, Damiano and Dumouchel (2020) argue that a genuine relational turn warrants a further step, which requires a radically different understanding of sociality and how it emerges. The relational only opens up once we position ourselves on a horizontal plane and in the middle of the encounter.

Observing audiences and participants encountering the Cube Performer, it is the in-between where transformations are not only triggered but literally take on shape: without an interactor, the Cube Performer may twist, tilt and sway but it is just an object-in-motion. But as soon as a person (or another artefact for that matter) approaches the cube and, for instance, crouches facing one of its gently twisting corners, the same movement turns into a gesture toward them. Hence, the same spatial–temporal dynamics take on a meaning when in relation to someone/thing. This transformation, we claim, does not only happen in the interactor’s or the observer’s eyes as it requires two bodies to intra-act to co-construct this relational space. Movement becomes a relational gesture in the dance that is embodied communication (see Haraway 2008), a form of empathic ‘being toward’ that renders the previously unseen or obscure meaningful (see Dimitrova 2017). A horizontal, relational ethics thus unhinges and breaks open traditional visions of what a robot is, requiring and at the same time opening up “a more differentiated set of starting points for the robot” (Castañeda and Suchman 2014, p. 340).

Our Cube Performer, in many ways, can be considered a material, situated research proposition, and we recognise, as earlier noted, that our relational, performative approach to HRI design is not suitable for all robot designs. Yet to advance our understanding of possible human–robot relationships, it is critical that we question our assumptions and widen our perspective to include more differentiated notions of human-nonhuman communication and its social potential. Robots, such as the Cube Performer, that are able to skillfully utilise motion dynamics as building blocks for affective, social coordination could not only open up novel, participatory forms of human-nonhuman meaning-making but also bring to the fore alternative ethical dimensions. In contrast to human–robot relationships that remodel human relationships, such an alternative, horizontal human-nonhuman ethics is more reflective of our embeddedness in the “ongoing reconfigurings of the world” (Barad 2007. p. 141). Intra-acting with a plain cube, meaning- and relation-making cannot follow familiar social protocols but can only unfold and evolve in the intra-actional process. In fact, we selected this simple yet commonly expressionless shape precisely because it cannot rely on already known social attributes. Instead, we found that a cube equipped to participate in the encounter through the relational dynamics of movement can engender a surprisingly broad potential for social meaning to be enacted in-between, far beyond the mirror image.