1 Introduction

Sensemaking as an active construction of meaning has recently attracted attention in several scientific fields connecting to technology development, meaningful understanding of technology, and integration of technology in societal and cultural environments. According to Henkel (2023) it affects essential qualities of human life when recognizing the cultural environment of humans (see Fig. 1). They become particularly important, as recent technological advancements towards everyday scenarios of mixed realities, such as the metaverse merging VR and XR technologies, may lead to mindless communication (Laut-Berger 2023), and thus, change cultural habits significantly.

Fig. 1
figure 1

Sensemaking as an essential part of a culturally framed understanding of sustainability (according to Henkel 2023)

Full size image

“Sensemaking involves not only finding information but also requires learning about new domains, solving ill-structured problems, acquiring situation awareness, and participating in social exchanges of knowledge” (Pirolli et al. 2011: 1). As such it has the character of a process of interpretation, i.e., assignment of meaning to experiences of individuals or groups (Klein et al. 2006; Maitlis and Sonenshein 2010), as well as of “freezing new understandings or meanings of experiences” (Kramer 2017) in a specific situation, it has driven recent research in a variety of fields, connecting to both, the field of digital environments, new technologies and media, and the field of change and transformation processes in organizations which can be applied to digital transformation, but are also relevant to change and transformation in general:

  • Cyber-resilience practice as grounded in sensemaking (Dupont et al. 2023)

  • Digital support of team creativity contextualized through sensemaking (Talat eta l., 2023)

  • Sensemaking when reflecting on chatbot behavior (Schuler et al., 2023)

  • Reflecting domain-specific (transformation) processes, such as Revenue Management (Yeoman 2023)

  • The cognitive and emotional perspective on change processes and their understanding (Dahmen 2023)

  • Vulnerability of organizations when handling situations of crises (Kročil et al. 2023)

  • Innovation processes from a resilient sensemaking perspective (Ciasullo et al. 2023)

Sensemaking has a long tradition in the context of considering it relevant when digital technologies are introduced to reflect and guide transformation processes triggered by emerging technologies, e.g., Pirolli et al. (2011), Chua et al. (2019), Techatassanasoontorn et al., (2023), only a few studies refer to applying digital technologies for sensemaking, such as Steffik et al. (2009), introducing the term digital sensemaking as “sensemaking mediated by a digital information infrastructure” (p. 38), and the performative character of digital culture (Beyes et al. 2017; Penny 2017), or calling for approaches that trigger sensemaking to allow for meaningful interaction with digital technologies (Mekler and Hornbaek 2019).

When looking at the nature of sensemaking as “the ongoing retrospective development of plausible images that rationalize what people are doing” (Weick et al. 2005: 409), organizational members make sense of events through a process of action, selection, and interpretation (Weick et al. 1993). Specifically within the process of digital transformation where individuals are often thrown back onto their own previous experience, UX design conventions, and are increasingly removed from, e.g., business processes or industrial production that they oversee through digital media or in a digitized form, it is important to ask how sensemaking can take place in this digitized environment, in cyber-physical systems, and with digital technologies (Brill et al. 2022a, b). It is important to understand this process though, as decision-makers and actors in situations of change and crisis are increasingly confronted with pressure, lacking and uncertain information, and digitally mediated conversations and perspectives on situations they face to overcome—and of which they need to make sense (Comes 2016). Thus, it is important to understand the potential of human sensemaking in digital contexts and inquire about ways of enriching the sensemaking process in digital environments. In this vein, it is important to connect to sensemaking to approach the problem of alienation in a digitalized world (Haga 2022). Boumeester (2014) even warns from un-reflected media consumption by stating: “To stack meaning upon meaning, sign upon sign, semiotic on semiotic and convention on convention beyond the point at which the system collapses, to the point where we simply can’t make any sense out of it.” (p.74).

Reflection in corresponding research processes should include the body better in, as for example Penny (2017) argues. Overcoming a body-mind duality connects to meaning making in the first instance of a sensemaking process. Recent findings, e.g., by Sandberg & Tsoukas (2020) reveal the strong contribution of the body, and the process of embodying knowledge, experiencing with senses connecting to the body and cognition during this process, which is—as a process—happening over time. Performance art as a relational practice enables focusing on the body, building relations by “being-in-the-world” with other bodies, objects, spaces, etc., over time (temporality) (Greil 2021).

Sensemaking scholars identified four core constituents that have been prevalent in the discourse and development of sensemaking theory: (1) “being-in-the-world” or sense-action nexus, (2) temporality, (3) language, (4) embodiment (Sandberg and Tsoukas 2020). Especially the core constituent embodiment has been understood as undertheorized (Maitlis and Christianson 2014) and brought to the foreground as an important part of sensemaking in the past few years (Rond et al. 2017). At the same time, researchers started to explore the sense-action nexus in the light of sensory experience, shedding light on the aesthetic dimension of sensemaking (Brill et al. 2022b; Baldessarelli et al. 2022). We address aesthetics and bodily experiences when looking beyond cognitive formats and capturing both, sensemaking moments and sensemaking activities. They refer to a situation being sensible to the world, with, as Strati (2007) pointed out, ‘empathic openness to the dimension of the Other ‘ (p.63). Such a situation ‘renders perceptive experience paramount in knowledge because it is ‘co-born’ (ibid.)

Performance art and employing the performative process and performers’ ability to reflect on embodiment has been increasingly discussed in terms of enriching the grounds of meaning and space for articulation to enrich research that tackles the dimensions of embodiment of processes and aesthetic dimensions in organizations, work settings, or interaction with digital technologies (e.g., Biehl 2017; Brill et al. 2022b; Cuan et al. 2019). Specifically, designing situations in which the researchers can tap into the performers’ kinesthetic awareness are of interest in the design of research settings (Gemeinboeck and Saunders 2016, 2022). In the research presented in this paper, we investigate sensemaking in digital environments and the generation of individually meaningful knowledge (‘knowing’) in two different settings involving performance art. With these experimental settings, we aim at grasping bodily experience, perception and embodied knowledge, and aesthetics (as in sensed knowledge (Strati 2007)) in the sensemaking process. Aesthetic knowledge serves as a carrier of sensed and thus embodied meaning and puts humans in relation with themselves and the world (Gherardi et al. 2007). We aim not only to codify the moments of constructing meaning but also to immediately and share these experiences for further use in digital ecosystems. Thereby, we aim to contribute to the development of digital technologies that are fit for embodied sensemaking (Hummels et al. 2015).

To generate such guiding principles, we need to understand the fundamental mechanism of how humans generate meaning in a specific situation, especially in the context of encountering digital environments, cyber-physical systems, or digital technologies. As Weick (2022) explored, meaning is continuously formed in sensemaking processes and then attributed to individual encounters. Following the conceptual foundation of constructing meaning in Sect. 2, we report on 2 subsequent performative interactions as part of our empirical research process. In Sect. 3 we provide the context for both. In Sect. 4, we discuss the first empirical setting with performance artists in which they are getting familiar with digital media beyond cognitive experiences. The second one, as discussed in Sect. 5, reports on a collaborative digital setting featuring meaning-making through active participation and direct digital feedback loops. In Sect. 6 we wrap up our insights in light of sensemaking support and sketch follow-up research.

2 Meaning making—sense comes into being

“Sense is a mode of perceiving the world. It designates the form of order of human experience and action. So meaning is a fundamental structure that underlies the operations of all psychological and social systems unconditionally and without exception.” (Laut-Berger 2023, 30f) When sensemaking involves freezing of ‘meanings of experiences’ (Kramer 2017) we should develop an understanding of how meaning is constructed, i.e. how information emerges enabling humans to act (leading to know-how) (North, 2021).

The emergence of information is related to observing and recognizing objects as information carriers from the perspective of a viewer, i.e. in a certain way directly from an observed object. As soon as the process of cognition recedes into the background, the decision about meaning is in a sense placed 'in the object under consideration'. The emergence of information is thus shaped by object relevance. The meaning of objects themselves can be derived from the primary purpose of creation and/or be the result of construction processes.

These processes happen through the object creators and/or viewers, who can access the information contained in the object and also establish it socially. They also come from the 'social life' of an object, i.e. specific cultural and historical milieus (Lüthi et al. 2018), and can only be perceived according to the existing personal experiences and expertise of the individual (Vester 1975). These represent the basis for recognizing and thus grasping meaning—it is therefore a process of relating something that has already been internalized to something that has been recognized in the environment. The recorded meaning helps people to orientate themselves in the world (cf. Dilthey 1984).

What is essential in the development of meaning is the set of instruments that accompanies this process, i.e. everything that is available to visitors at the moment of encountering the object for their constitution of meaning: everything they perceive at the moment of reception, read or based on what they see and can activate what has been read in contexts of understanding. This has far-reaching consequences for the analysis. The relevant analysis question is then namely, which communicative manifestations are responsible for the fact that a certain object is understood as something natural or something man-made (an artefact) […]' as Kesselheim summarizes the findings of studies in the context of exhibitions (Kesselheim 2021: 46).

The emergence of information as a construct for gaining meaning, therefore, has a real, socially institutionalized aspect for determining knowledge in certain situations (such as in a work organization), as well as a technical reference that brings us back into the vicinity of objects, i.e. leads to object relevance. Both aspects 'legitimize' the emergence and recognition of meaning. Consequently, the question arises as to how the emergence of information can be observed or how the process of recognizing meaning can be recorded as human practice and subsequently described, especially since the process itself is not visible as a thought process (Hampe 2006).

The recognition of meaning and thus the emergence of information can be understood as self-organization, which is constituted as practice in observing and self-observing, in thinking and in communication. Thinking is constituted as an open-ended practice—a practice that is not fixed on knowledge in the sense that the participants claim to want to know beforehand what they will know afterwards (cf. Hoppe, 1999). Rather, a ‘shaping and ordering of life’ takes place in recognition (according to Lessing in Böhm, 1986: 81). Seeing reason has cognitive ‘power’ with regard to the dimension of intelligibility (ibid.).

It is, however, structurally determined: In the interaction between the living being and the environment within this structural congruence [between living being and environment], the perturbations of the environment do not determine what happens to the living being; rather, it is the structure of the living being that determines what change occurs in it as a result of perturbation. (Maturana et al., 1984, p. 106) In order for us to recognize meaning of an active process—living beings do not adapt their environment, but shape their environment according to their own structure.

According to the concept of self-organization, the ability to act (know-how or ability) requires 'physical (sensorimotor) structures', which represent the 'substance of experience' (Varela, 1994: 23), not only for motor skills but also for recognizing and dealing with meaning. Meaning thus represents an embodiment: Recognition therefore does not consist of representations, but of embodied action. Accordingly, we must hold that the world we know is not simply given to us, but is negotiated through the history of the structural connections we have made, and that the temporal turning points at which such performances of action begin are rooted in a multitude of alternative micro-worlds that can be activated in any situation. These alternatives are the source of both common sense and the creativity of our cognition. (Varela, 1994: 25).

The cognition process as embodiment is anchored in a particular situation: 'We always operate in the immediacy of a given situation' (Varela, 1994: 16). It represents a 'micro-world' that is experienced through an act of recognizing: 'The world is not simply given to us, but it is something that we get involved in, in which we move in it, touch it, breathe it in and eat. For this reason, I like to speak of cognition as an act" (Varela, 1994: 15). Thus, acting in a situation also represents a process of cognition that is 'concrete' in the same sense as the experienced world itself.

Instead of the reconstruction of properties of the world from the perceptual ‘data’ (sic!), which is typical for information processing, in the implementation orientation the control of the respective actions in concrete situations by the perceiving person occurs—'Every doing is knowing, and every knowing is doing' (Maturana et al., 1984: 32). Meaning can also be inferred through the execution of language: 'The meaning of a word is its use in language.' (Wittgenstein, 1984, PU § 43) The individual understanding of meaning not only leads to a variety of perceived meanings but also provides the basis for thinking: 'How am I supposed to know what I'm thinking before I perceive what I'm saying'. (Weick 1995: 195).

'Since the concrete situation is constantly changing due to the actions of the perceiver, there is no predetermined world independent of the perceiver, but only the sensorimotor structures of the cognitive actor […] It is this structure—the physical condition of the perceiver—and not such as a given world that determines how the perceiver acts and is influenced by events in his or her environment. The primary concern of the performance-oriented approach, then, is not to figure out how to reconstruct a perceptually independent world, but to examine the common principles or lawful connections between the sensory and motor systems that explain how perceptually guided action occurs in a perceptually dependent world is possible' (Varela 1994: 20).

'So there is a progressive relating between the ever deeper and more distant zones with the ever more intimate operations of one's own activity. Intelligence thus begins neither with the knowledge of the ego nor with that of things as such, but with the knowledge of their interaction, and it organizes the world and itself by attending simultaneously to the two poles of this interaction' (Piaget 1974: 339) The tight coupling of constructing meaning with action has also been recognized in Malhotra’s (2004) framework of knowledge development—Attention / Motivation / Commitment in relation to Performance Outcomes indicates that, and at the same time, questions the need of language: ‘When meaning corresponds to use, there is no point in speaking of it. The grasping of the meaning happens in one fell swoop and is certainly something different than the temporally extended use.' (translated from Wittgenstein 1984, PU §138).

(Normative) specifications for defining meaning do not generate the variety of possibilities of self-organizing perceivers (cf. Axelrod, 1991). Rather, the art is to deal with ambiguity and ambiguity with an undetermined meaning. The play of meanings can lead to conventions that entail a uniform (but not prescribed) usage that becomes observable as meaning. "Conventions are now understood as coordination logics that actors can use in situations to 1) interpret practices, objects and processes in a meaningful way, Fig. 2) to make appropriate forms of coordination succeed, to legitimize or to criticize, as well as to 3) act as a framework for the evaluation and attribution of "value" and "quality". Conventions are not to be equated with routines, with simple standards of behavior or with customs. Rather, as logics for interpretation, coordination and evaluation, they are realities that can be interpreted in terms of institutional theory. In addition, actors draw on conventions to pragmatically decide for themselves the meaning (meaning) and the situational handling of rules and formal specifications.' (Diaz-Bone 2021: 475).

Fig. 2
figure 2

Non-prestructured routines and processes of finding meaning (according to Malhotra, 2004)

Full size image

Weick (1985) aptly calls the acquisition of meaning in this way 'production of ambiguity' (referred to in self-organization as perturbation) which requires specific mechanisms, as shown in Fig. 3, to cooperate as a collective. With the help of selection, the ambiguity can be reduced and incorporated into the collective structure through storage, taking into account beliefs, feelings and thought processes (retention). The other self-organizing processes build on this structure.

Fig. 3
figure 3

Variety of goals and means at the center of collective voting processes according to Weick, (1985: 134)

Full size image

A group as an organization learns about the cyclic formation of resonance about goals and means. It communicates on the basis of perceived meaning to act together. This creates the resonance that makes it possible to set common goals on the basis of individual self-reflection. To form knowledge, communication is part of the formation of understanding, since the formation of collective knowledge requires that each individual relate to the common practice of meaning (Hoppe, 1999).

Weick (1985) understand collective meaning construction as cyclic and mutual coordination process (see Fig. 3), which not only allows the great openness of meaning caused by individual perception to be processed but also ensures the ability to work: 'So, in other words, meaning is a representation of world complexity that is capable of being up to date Moment. However, the discrepancy between the complexity of the real world and the capacity of consciousness or communication can only be bridged by keeping the space of the current intention small and making everything else potential, i.e. reducing it to the status of a mere possibility.' (translated from Luhmann 1990: 44) It should be added at this point that Luhmann understands self-organized systems as being able to change their structure (Luhmann 1997). It gives way for considering ‘affective resources, emotional guidelines and sensuous trajectories for the formation of subjectivity and for sensemaking efforts.’ (Alacovska et al., 2023: 965f).

3 Context of the empirical studies

To empirically explore this theoretical argument in which we understand sensemaking and meaning-making grounded in aesthetics and embodiment in the sensemaking process, we present two empirical studies. Both empirical studies were framed as empirical experimentation with performance artists to understand the role of the body in digital sensemaking processes (i.e., sensemaking processes with digital technologies, cyber-physical systems, and digitized environments). In each of the two empirical studies, the team of researchers (composed of artscience scholars, organization studies researchers, and business informatics researchers) collaborated with performance artists to investigate cyber-physical systems and digital technologies as provided by the research team. We worked with performance artists to tap into their potential to reflect on their bodily process (‘kinesthetic awareness’, Gemeinboeck and Saunders 2022) and to create performances through which we could tap into the embodiment of digitized (work) processes (Biehl 2017; Brill et al. 2022b). In the second empirical study, we worked with a performance artist to even go beyond this focus on the performers’ competencies by opening up the performance to lay audiences to explore the interaction with digital technologies and to bodily explore digitized processes. Thereby, we worked with the performance artist as a specialist in embodiment to develop a choreography for the audience who becomes active performers in the interactive performative installation.

By juxtaposing these two approaches, we can shed light on the digital sensemaking process through the expert perspective of performers’ processes and the engagement of non-experts in a digital sensemaking process. How do they attribute meaning to this process? What are the roles of the body, aesthetic elements, and guidelines in the sensemaking process? How are instructions interpreted and what is the role of aesthetics and the body in the process of using these instructions to make sense of digital technology?

4 Approaching the construction of meaning with the body in digitally mediated environments

In the first empirical study with performance artists on digital sensemaking, two performance artists were invited to engage with a set of technologies stemming from ongoing research at the institute of business informatics. The process started by introducing the two performance artists to these technologies with which they were not familiar at all (i.e., they did not have a previous background in coding nor in media art). Departing from this introduction, they went through a process of understanding, handling, and applying the technologies to create performances. This period lasted three months. They explored and worked with two technologies:the agile robot system Spot© and the (IoT) components M5Stack. The artists proposed unexpected approaches to explore the technology based on their background as artists. Additionally, the research team asked them to also focus their exploration and self-reflective processes on their bodily experience with the elements, the sensations, emotions, and the learning processes which entailed aesthetic elements, both as sensible knowledge (Strati 2007) as well as in the form of feedback or responses from the technologies. The research team was interested in doing so, specifically to address the two performers’ kinesthetic knowledge, as proposed in previous research (Gemeinboeck and Saunders 2022).

This phase was intended as a structural ground on which to build the next two phases of the project, getting an initial insight into the processes of performance artists making sense of digitalization technologies. We invited the artists to document several moments along the process, creating micro-performances (i.e., ephemeral and unannounced actions, happenings, and moments that are a direct manipulation of matter, movement, language, among other elements for a limited period of time) as either a way to document the process or a way to visualize the result of the process. The results of this phase mainly pointed to the aesthetic elements in the process, both referring to the “sensed” element of working with these technologies and the visual, haptic and auditory elements as feedback during the sensemaking process. The main result of the phase was the central aspect of aesthetic feedback in the process of understanding digital technologies through the body, for example through movement, touch, and visual analysis. In addition, the two performers decided to deliver one performance with the robot, which they called “Mimicry”. The analysis of this performance provided the research team with additional insights.

Throughout the three months of the process information and data were gathered following this mixed-method design, comprising experimental studies, participatory observation, and interviews. The focus in the data collection was on aesthetic perception and bodily interaction, as they are considered essential constituents of sensemaking in organizational contexts but are still under-researched (Vitry et al. 2020; Stigliani and Ravasi 2012), combined with methods from research into human–computer-interaction and robotics that employ dance (LaViers et al. 2018; Cuan et al. 2019; Barnard & de LaHunta, 2017; Prpa et al. 2020). In total, visual data was gathered in the form of videos (a total of 120 videos between 20 s and 47 min long), as well as written data (notes, research diaries by the artists), participatory observation from the project team, and a series of qualitative interviews with the artists.

All videos and all interview data were additionally coded in the qualitative evaluation software MAXQDA, a software to support the analysis of qualitative research data. Thick descriptions with links to the videos, stills from the videos, photos, video still sequences, and sequences focusing on the movements (e.g., close-ups of the hand movements) were put together. In the gathered data, we were able to identify several elements of the sensemaking process and conclude that the body, embodied knowledge and the senses are a crucial part of it.

The recorded data was transcribed using MAXQDA, (the conversations were transcribed, but also the visual data was carefully described) and some specific moments of the videos that zoomed into the moments of the sensemaking process were coded. We recognized and analyzed separately:

  • The sensemaking process of the artists during the interaction and learning processes with the provided technologies

  • The role of the body, the senses, and the aesthetic elements of the process

  • Moments of meaning making or knowledge production through conversations, theoretical input and hands-on activity

A total of 514 moments, segments, and interactions were coded using MAXQDA, classifying the processes and the steps and the elements for the recognition of the sensemaking processes. The moments of meaning making and knowledge production mostly happened through conversations, theoretical input and hands-on activity. Codes referring to meaning-making were “proposal of a solution” (59 moments), “thinking out loud / speculations about function or meaning” (40 moments), “aha-moments” (22 moments), “retrospective explanation of past situations step by step” (30 moments). In the course of these coded moments of the interactions, there were different ways in which the artists gave meaning to either the tools, the functions of the tools—starting from speculation to retrospective explanation -, or even a previous situation that had a meaning for the current moment. Overall, meaning was attributed to the elements of the situation, either verbally or bodily, and aesthetic elements such as the visual, sound and haptic properties and feedback of the technologies or performative actions were crucial in this process.

4.1 Data analysis: give meaning to technological tools using the body

In our data analysis, we found three main insights that focus on the body making sense of the technologies. We present them in the following three concrete examples:

4.1.1 Give meaning to a technological tool by experiencing its physical properties with the body

Not being familiar with the technological tools presented, and having the task to make sense of them using their bodies, the two performance artists proposed unexpected and unusual explorations of the technologies physical bodies and properties. While learning about the functions, use instructions and possible implementations of the robot, the artists suggested different ways of perceiving the robot's physical properties with and on their bodies. This experiment was done in several steps, which included looking at the robot, walking around it, feeling the robot's materiality by touch, and then, to experience further properties, like its weight, lying beneath it (see Fig. 4). This way, abstract and cognitive information, for example, of the weight of the robot, can be better understood by feeling the weight of the robot on their bodies. The information needed to understand the technology, in this case, the weight of it, is gained corporeally. Thus, the performers got thoroughly acquainted with the robot and its capabilities: what does it mean to work with a robot with a weight of 30 kg? What are its maneuvers, physical potentials and limitations? How to translate instructions and really understand safety regulations? Meaning is given to it through the body.

Fig. 4
figure 4

The artist laying down beneath the robot to feel its weight on her body

Full size image

4.1.2 Give meaning to the instructions through the body—translate languages into actions

The first interaction of the artists with the IoT Elements (M5-Stack sensors) was organized outside of the university at a performance space, with no support of the research team. It was recorded by video to analyze the process later. The first video, which is 47 min long, records both artists sitting at a desk, unpacking, looking at, wondering about and trying to use the IoT elements.

In almost every case of attempting to understand a tool or a new technology, the artist's first action was to look for the written instructions, or for videos containing the information to use it. In the case of the M5-Stack sensors, the initial instructions were given to the research team by the institute of information engineering. These instructions were put together by a team of students from the university, partially in German and partially in English. Since the research is taking place in English, their first way to go was to translate the instructions (which they did with online translation interfaces). In several parts of the steps described in the instructions, the artists had to look for other and additional information (e.g., online, in social media), since the specific academic/engineering language used was difficult to follow (see Fig. 5). Since the sensors are produced in China, most of the online instructions were written in Chinese. The goal became then to first translate the instructions from different languages into English, and understand the terms of the specific language of engineering used in the instructions since they were written to be used by the students of the institute and not by people coming from different backgrounds. This goal eventually proved not to have the desired outcomes, leading to moments of frustration. The first day of the experiment goes by, and the artists cannot translate, use or understand the sensors.

Fig. 5
figure 5

The two performance artists trying to figure out introductions to use the M5 Stacks, by translating languages (both Chinese, English and German, and visual languages such as visual representations)

Full size image

After this initial interaction of the artists with the sensors, the research team conducted a written interview with them to better understand the process. To the question “describe the process of the previous day”, one of the artists answers: “There were several steps before understanding the equipment. First we had sensors in our hands, looking at them piece by piece and trying to think what they do and how they could be assembled. Secondly, going through instructions with online information, we were going through all possible options, what we could find to see how to put them into work. From the information we got as examples, but also using other Google resources, downloading applications and drivers. For me, it was a matter of selective approach to understanding what is not working was actually very beneficial.”

As seen in her answer, the process started by looking, and having the sensors in their hands, then they proceeded to look for instructions, and then they selected what worked and what did not work. In the end, it was a translation of the instructions (from other languages and discipline-specific terms) into actions, movements, and sensed experiences that brought them closer to understanding the sensors. They gave meaning to the different elements by translating written and cognitive information into bodily action and sensed decision-making processes. The technology was understood because they mixed the cognitive information with their sensed processes, giving meaning to it through their body.

4.1.3 Give meaning, translating a movement into another movement

With the task of creating several micro-performances for visualizing the sensemaking process with the different technologies, the artists proposed approaches from their own artistic background. One of the artists works with movement, and with giving meaning to different movements in his performances. To study the movements, properties, and abilities of the robot, both artists decided to follow the robot, mimic its movements, and to propose a translation of the robot's movements into human movements. To propose micro-performances or artistic presentations, there is a process of abstraction of the information and languages gathered during the initial sensemaking process. In this process, the movements of the robot are carefully studied in several steps. Starting with 1. Looking at the robot moving, 2. Making the robot move, being the person able to control it using the controller, 3. Mimicking the movements: the artists mimicking the robot and vice versa, the robot mimicking the artists by directly translating the movements made by the robot into human movements, by relating the different body parts of the robot to their own body parts (arms, knees, upper body, etc.).

After having experienced the movements from these three perspectives, the artist was able to suggest a new layer of translation, when confronted with the moment in which the body of the robot is not directly translatable into the body of the human and vice versa. When attempting a translation from the human movements into movements by the robot (the robot mimicking the human), there were several movements in which the person controlling the robot did not know how to proceed. A new language was needed: it was decided that when the person did a movement that the robot could not make, it would move differently: “We decided that it needed to move differently because it has different capacities and limbs than me. When I moved my arms to the side, making a movement that the robot was unable to follow, the robot would translate this movement into moving in circles, for example”. Meaning was given to the movements by translating them into movements that the properties of the artist's body / of the robot's body were able to follow (see Fig. 6).

Fig. 6
figure 6

The artist moves using specific parts of his body that the robot cannot directly relate to. Other movements need to be used in order to mimic the artist's movements

Full size image

With their goal of developing the long performance “Mimicry”, they needed to explore this translation of robot movements to human movements and vice versa into a more elaborate body language that was meaningful to them and became visible and experienceable to the audience. Thus, their process of attributing meaning intensified in this later stage.

5 Digital co-construction: drawing with the body collaboratively

In the second empirical study, the research team collaborated with an artist with a background in performance, choreography, media art and computer science. The artist created a performative installation, where the audience (two at a time) became the performers in the installation. The installation, called “Drawing Exercises,” is an interactive performative installation in which a series of drawings are performed by two participants together, who have to find out the movements, velocities, and strategies to carry them out. The experienced process in the performative installation takes the participants through a sensemaking process in which they will develop different strategies and methods to draw, with their bodies, the figure they imagine with the given task. Sets of sensors on the bodies of the participants digitize the information produced by their bodies: their movements, their orientation, and at a certain point their heart rate and stress levels, to introduce moments of sensebreaking and sensegiving (Gioia and Chittipeddi 1991; Vitry et al. 2020) to the process. This combined digital information produces a digital twin of their combined actions—a line to draw their envisioned outcomes. The process was conducted by the participating audience, allowing a closer look into moments of generation of meaning with the body.

The process was recorded with video and sound, and written questionnaires and verbal conversations were conducted after each round with the participating persons. In the data generation process and data analysis, the research team used methodologies in the same vein as in the first empirical study, referring to studies in sensemaking, but also dance and performance in researching interactions with robotics and digital technologies (i.e., Stigliani and Ravasi 2012; LaViers et al. 2018; Cuan et al. 2019; Barnard and de LaHunta 2017).

While analyzing the gathered visual and written data (questionnaire and qualitative interviews with each participant), consisting of 13 videos between 11 and 27 min long (one video for each participating group), several moments of the process can be pointed out to study how the participants attributed meaning to different aspects and elements throughout the process with their body.

5.1 Drawing exercises: technological setup and functionalities

As shown in Fig. 7. the setup of the performative installation is the following: in a closed space, two persons face a screen or a large projection. They are both wearing the technology developed for the phase, each one has a belt or harness that has a small box with a movement sensor and electronics, and cables with sensors going to their fingers (measuring their stress level sensor through the Galvanic Skin Response) and to their earlobes (heart rate sensor). Both participants don’t previously know how the installation works. On the screen tasks are projected, such as “draw a horizontal line”, “draw a vertical line” or “draw a landscape”. Once they start moving, they realize that their movements create lines, but they have to figure out what kind of movement creates what kind of line. The installation is programmed to draw a vertical line when one of the participants moves to the front and the back, and to draw a horizontal line when the other participant moves to the sides, making it necessary for the participants to find out who draws which line. Both movements need to happen with the upper body, since the sensors perceive the orientation of the body, and not the position in space, making it necessary for both participants to move their upper bodies together to be able to draw, for example, a circle. Sensebreaking is introduced when the line changes due to the input of the participants' stress level or heart rate. After the initial moment of confusion, they need to figure out the reason and new functionality—and to readjust.

Fig. 7
figure 7

On the left: Positioning of the two participants in front of the projection with the sensor sets on their chest (Video 12, Min:5:33) On the right: Close-up of the sensors as fixed on straps to mount them on the performers

Full size image

Both participants go through the sensemaking process together, finding out, in steps, how they have to move, and finding different strategies to achieve the tasks. In the process, the participants learn, cognitively but mostly with their bodies, how the installation works, how to achieve a drawing, and attribute meaning to different elements of the experiments. We expose three elements and processes of meaning attribution next.

5.2 Data analysis: collaboratively exploring the interface and experiencing through the body

Based on our data analysis, we will discuss the following three meaning-making moments:

5.2.1 Meaning making moment 1—How to collaboratively understand the tasks? WHEN is tangible meaning making with the body necessary?

In the course of the installation, the tasks given to the participants gradually became more complex. The first two tasks, “draw a horizontal line”, and “draw a vertical line” led the participants to find out the movements each one needed to make to complete the drawings. “draw a circle”, “draw a landscape” and “draw a house” were only achievable if the participants were in sync and had found, either cognitively or sensibly, a movement strategy. For the two last tasks, “draw a landscape” and “draw a house”, the participants gave meaning to different elements to proceed, and often talked, before starting the movement, to reach a common meaning of the objects “house” and “landscape”.

The process of drawing the landscape usually started cognitively, i.e., coming up with a meaning of the word “landscape” in a conversation. Out of the 26 participants, 10 verbally asked what a “landscape” means, and discussed what they imagine drawing as a landscape, for example “a landscape means that the line cannot cross” (Video 1, Min: 18:43) or “landscape means we have to draw a map”(Video 12, Min:7:45) or “it means we draw a line from right to left” (Video 13, Min: 7:15). 10 participants verbally agreed on one object in the representation of the landscape, such as “let’s draw a tree” (Video 9, Min:13:25), or on an abstract representation of a landscape, such as a horizontal line to draw the horizon (see Fig. 8, which can be interpreted as a single bell flower or a dying flower on a flat landscape). These objects or “representations” often correlated to their personal experiences and memories of a specific existing landscape, like a specific mountain in the area where they live “let’s draw the Pöstlingberg” (Video 10, Min: 6:48)(see Fig. 9). 8 participants immediately proposed a movement to start drawing the landscape “let me move to the right, and you move up and down” (Video 6, Min: 9:38) or “I would say we make a horizontal line” (Video 5, Min:6:17), without previously talking about how a landscape looks.

Fig. 8
figure 8

(Video 3 Min: 12:27) One of the minimalist landscapes that shows a minimally jittering line reflecting calm participants

Full size image
Fig. 9
figure 9

(Video 10, Min:7:20) Two of the participants attempting to draw the Pöstlingberg in Linz, Austria

Full size image

To agree on the meaning of the word “house”, the participants frequently discussed how they imagined a house to be and what elements constitute it. The basic representation of a house, as seen in Fig. 10 on the right, was a starting point for their intention to draw a house for all 13 pairs. This house was seldom achieved. 2 pairs ended up with a house that they interpreted from abstract lines they produced (as seen in Fig. 10 on the left). Out of the 26 participants, 18 participants (= 9 pairs) immediately started moving to make the square as the base of the house, continuing with the triangular roof. These participants did not discuss previously the specific lines they wanted to draw. The other 8 participants (= 4 pairs) first discussed the plan to draw the house to later make the movements. They talk to each other to plan out the lines, “let’s start at the bottom left and go up” (Video 11, Min:7:54), or “let’s make the tip of the roof” (Video 3, Min.14:51). 3 participants (in 3 different pairs) proposed a specific house before starting, “do you know the house of Nikolaus?” (Video 4, Min: 6:00, Video 11, Min:7:40 and video 8, Min: 15:47). A third participant commented on the resulting house to represent “the house of Nikolaus” (Video 6, Min:13:30). Two participants go up to the wall and draw the house they imagine on the wall with their fingers, to decide on the movements they need to make and on the form of the house. In this case, the participants needed to make their goal tangible before they started. They walked to the wall to be able to explain the meaning of the word “house” by touching the wall and moving their hands as if they were drawing their imagined image of a house on the wall. Meaning was given to the abstract words either through conversations or with movement, or bodily actions (see Fig. 11).

Fig. 10
figure 10

Two results of a house. On the left two participants are discussing how to interpret the lines as a modern house (Video 9, Min:21:30), on the right you see an approximation to a basic shape of a house as represented in children’s drawings and games (Video 11, Min:11:52)

Full size image
Fig. 11
figure 11

(Video 1 Min: 20:48) Two of the participants mapping out the house with their hands on the wall before starting their joint drawing process

Full size image

In general, this process of drawing a house is strongly differentiated from the process of giving a meaning to the word “landscape” because the participants don’t agree beforehand on what kind of house they want to draw. The meaning or form of “house” was discovered while or after the lines were drawn, “this is a very abstract house” (Video 9, Min:21:37), or “we are doing a church” (Video 4, Min: 6:26).

As a general observation of video material, we see that there are three main processes of giving meaning to the task: 1. To talk about what the word means (how a landscape or a house might look like), 2. To start moving and planning at the same time, with no previous cognitive process of giving meaning to the word, and 3. Finding ways to communicate to the other participant the meaning of the word beyond language, for example by going up to the wall to draw or asking the research team for advice.

5.2.2 Meaning making moment 2—give meaning to the other persons' movement through movement

In the course of the installation, the participants came up with several strategies to draw together. Most commonly, one of the participants mimicked the second one, or watched their movements, following the output of the projection, as to know what the result of the particular movement was. At the beginning, one of the participants needed to see and understand the meaning of the movement the other participant made, to speculate on possible movements he/she could make to make the desired line. The movement of the other person was given a meaning depending on the visual output on the projection, and this meaning allowed this person to move accordingly.

5.2.3 Meaning making moment 3—give meaning to the properties of the line through the body and reconsider the meaning after sensebreaking

The installation was conceived in a way, that on every step or task, the participants needed to either find a new way to move or a strategy to produce the desired output, the participants had the opportunity to make decisions on their velocities, their rhythms, among others. It is relevant to point out that, the interface and its functionalities were not known to the participants previously, every experiment was conducted with closed doors, so the participants were immersed in a completely new situation. In the beginning, there were two tasks, in which the participants did not have a direct decision on the properties of the projected line. Two sensebreaking moments were introduced, one in which the line was affected by the participants' stress level and another one in which the line only appeared when the heartbeats of the participants were synchronized for a brief moment of time. In these moments, the participants needed to give a meaning to the new, different and seemingly incoherent line (for example, as it “jittered” with their level of stress), to be able to continue with the process. Once the line appeared different from before, either trembling or only existing in specific moments, the participants had to find out why it was different to try to control it, to be able to draw. Until then, the process had been consecutive steps towards making sense of the connection of the bodily movements and the resulting lines. Although all participants were informed which sensors they wore on the body at the beginning of each performance, the performers barely could make the connection between the “jittering” as a representation of the stress level and the beating hearts as a representation of their heartbeats and that they should find how to deal with this new information. In 12 out of 13 pairs, the artist provided information about the jittering line at the beginning of the experiment, allowing the participants to know why the line had changed, but introducing them to the question of how to consciously control the appearance of the line, they had to find a way to calm down their stress levels to make the line less jittering. The participants found different strategies to do so, for example breathing at the same time, or even holding hands. This additional explanation of the sensebreaking moment was not anticipated by none of the pairs. Even with additional information, one pair was extremely disturbed by this sensebreaking moment. The second sensebreaking moment that introduces the heart rate is not immediately revealed to the participants. Thus, all participants start drawing later, especially as all participants first want to understand the new mechanism and the sensebreaking moment before giving meaning to the house they want to draw.

To sum up, first, the movements to create independent lines (horizontal and vertical) were learned, and then collaborative movement strategies were developed for the more complex drawings. When the line starts to appear differently, the participant’s focus is lo longer only on the common movements, rhythms, and velocities, but there’s a necessity to understand the impact of elements that seem uncontrollable, such as the heartbeat. Strategies change from a focus on the movement of the body, to for example, needing to ask the research team about this step. After understanding the new line, often with the support of the research team, the participants looked for strategies to control these elements, by breathing slower or taking each other by the hands to synchronize their biofeedback. The process of “Drawing Exercises” was both a learning process about collaborative strategies on movement and a process to step by step understand and give meaning to the interfaces and interactions programmed as part of the installation.

6 Conclusion

Current cyber-physical technologies for aesthetic and embodied control allow new forms of expression, interaction, and communication for humans in an increasingly digitized world. In this work, we showed that digital sensemaking utilizing these technologies enables individual and participatory meaning identification processes. In the sense of Weick’s (2022) recently explored dynamics of knowledge generation through sensemaking, when meaningful items are recognized and propagate to formal structures—the reflected performative actions of the initial performative study presented in this paper lay ground to all micro phases of sensemaking:

  • perceptual recognition of so far undifferentiated situations, e.g., when a robot enters a workspace by demonstration and needs to be explored aesthetically

  • conceptualizing through crude differentiation, e.g., detecting the transport function of a robot through body experience

  • forming entities that can be communicated, e.g., describing the digital transport system in terms of functional work items

  • developing patterns, e.g., identifying functional movements as part of a task that could be automated or supported by a robot system

  • formalizing patterns, e.g., arranging preparation, control, and monitoring tasks in a specific sequence

  • incorporating patterns for implementation in the workplace, e.g., leading to allocations like “This transport task IS a robot process”.

The second performative study also bridged the gap between aesthetic and cognitive articulation featuring the co-construction of meaning through interaction. It revealed how bodily experiences differ from exclusive cognitive explorations of actions when tasks need to be accomplished in cooperation.

Both studies show the potential of digital sensemaking support at the method and tool level to enrich the human reflection of digitalization and related transformation processes. Future research will not only address implication for HCI but also capture drawing by hand versus body movement and localizing multi-part body control when providing visual feedback. An already ongoing effort includes audio information that is accompanying movement and visual feedback. Finally, talking could influence the accuracy of the investigated articulation patterns when being included to share thoughts and intentions.