1 Introduction

Machine learning (ML) is a technology used to make predictions about likely outcomes based on the processing of a large amount of collected data, thereby producing “probabilistic results” (Elish & Boyd, 2018, 62). Machine learning is a core technology in Artificial Intelligence (AI) and it has offered many promises of improving work tasks across an array of different fields. The technology has produced impressive results in recent years, in object recognition, translation, and speech recognition, for example. It is associated with promises to “present insights that can transcend human limitations” (Elish and Boyd 2018, 58). Some argue that its potential is exaggerated. Elish and Boyd write that “[p]art of what makes the phenomena of Big Data and AI so compelling is the hyped imagination of what is possible, not what is realistic” (Elish and Boyd 2018, 58). In this context, Elish and Boyd use the word “magic” to highlight what they understand to be unrealistic promises of the technology and underscore that technology is different from magic in this regard. Elish and Boyd criticize a magical understanding of ML, pointing out that in “glossing over the limitations of technological systems, they run the risk of undermining the power and potential of the systems they are building” (2018, 74). Several scholars have discussed the relation between ML technology and magic in a similar way (e.g., Elish and Boyd 2018; Jatzlau et al. 2019; Mathis et al. 2018; Rosenberg 2021; Zhou 2016).

Comparisons between technology and magic are not unique to ML technology. The relation between technology and magic (and sometimes religion or other phenomena that can be described as supernatural) have been discussed by several scholars (e.g., Baladrón et al. 2019; Boholm 2014; de Waal Malefyt 2017; Durkheim 1976 [1912]; Elish nd Boyd 2018; Francisco 2015; Gell 1988; 1992; Hickman 1987; Hunter et al. 2001; Kravets 2017; Lévi-Strauss 1974; Noble 1999; Rosenberg 2021; Tambiah 1990; Zhou 2016). In recent scholarly work, a reference to magic is often used simply to signify that technology has great potential (Elish and Boyd 2018; Francisco 2015) or to criticize the view that technology can function as an effortless realization of human intentions (Elish and Boyd 2018; Rosenberg 2021). These scholarly articles rarely fail to cite and criticize the sci-fi author Clark’s statement that “Any sufficiently advanced technology is indistinguishable from magic” (1962, 73), signifying a technology that works perfectly to achieve the intention of the user. The relationship between magic and technology is not only a matter for scholarly debate, it is also common in popular discourse. A study by Stahl (1995) demonstrated that 36 percent of all computer-related articles in Time Magazine used magical or religious language to describe the functionality of the technology.

Although contemporary scholarship often reveals a simplistic understanding of magic, the relationship between technology and magic/religion has also been discussed in more depth. Research in historical anthropology has shown that throughout much of history magic has not been separated from science to the same degree it is today. In his book Magic, Science, Religion, and the Scope of Rationality, social anthropologist Tambiah shows that the opposition of magic to science is quite recent in many regards. He highlights examples of influential empirical scientists such as Isaac Newton and Galileo Galilei who practiced astrology and alchemy (Tambiah 1990). Historical anthropologist Boholm has studied similarities between the discourses of alchemy and nanotechnology, taking a longue durée (i.e., long-term historical structure) perspective on technological imagination (Boholm 2014). Noble (1999) links modern technology to religion by arguing that the technological developments and discoveries of the late Middle Ages were driven by a certain interpretation of Christianity, namely, that technological development was a way of restoring the world to the state it was before the fall of man in the garden of Eden (the prelapsarian state). Noble thus makes the argument that technological development in the European context is underpinned by rationales not directly connected to everyday needs or explained merely by material factors.

In his conservative critique of technology, Stivers (2001) talks about technology as magic in the sense that contemporary society has unrealistic expectations of technology for achieving “happiness,” “success” and creating a “utopian future.” From this perspective, technology is magic because it is “irrational” and a “superstition.” Magic is thus understood as the opposite of something that functions perfectly. Stivers’ definition of magic (as well as that of technology) is, however, very broad, vague and contradictory as he repeatedly conflates the two concepts. He seems to be somewhat aware of this as he writes that in his understanding “magic is the most adaptive of enterprises and does not operate according to a single principle or logic” (2001, 28), a definition most certainly not suitable for scholarly scrutiny. This is not the place for an in-depth critique of Stivers’ book but “unrealistic expectations” of technology hardly qualify for describing it as magic. To have unrealistic expectations should rather be understood as a condition of humans being in the world. Although conservative in nature Stivers’ critique of technology is not far from a (post) Marxist understanding of the fetishization of technology in contemporary capitalist society (e.g., Harvey 2003; Hornborg 2014). For example, Hornborg writes: “[the] Marxian concept of fetishism can be extended from our understanding of money and commodities to explain how we tend to be deluded by modern technologies” (Hornborg 2014, 121). Departing from this point of view, Hornborg uses the concept of magic to describe technology as an irrational or exaggerated belief (2014).

Departing from social anthropological understandings of magic and oracles, this paper aims to compare ML technology and magic to identify similarities that are of interest in understanding ML technology. In describing ML technology, the paper will rely on previous scholarship but also use empirical examples from a project we have studied that aims to develop an ML-powered system for controlling the propulsion of cargo ships to improve their energy efficiency. In the project, historical ship and weather data were used to create models to predict how the ship would react under certain conditions to optimize fuel consumption given a certain departure time and estimated arrival time. In the project, “magic” was frequently used to talk about some aspect of ML technology or their customer’s view of technology. While the concept of magic was used to discuss what the system could or could not do, these emic uses of “magic” are not the main inquiry of this paper.

Therefore, what do we mean by magic? The paper will rely on anthropological perspectives on magic and oracles, where oracles are understood as a specific category of magic aimed at producing knowledge rather than manipulating events or matter. Magic and religion have been prominent topics in anthropological scholarly debate from the very onset, originally in the study of non-western small-scale societies. Frazer, the nineteenth century armchair anthropologist, distinguished magic from religion, claiming that magic was a primitive form of religion, lower on the scale of social evolution (Frazer 1990 [1890]). Subsequent studies have shown that the institution of magic coexisted with religion in many instances and that they can be very hard to distinguish in actual empirical settings (Cunningham 1999). However, from the perspective of this paper a distinction between magic and religion is useful. While religion offers an explicit worldview including deities and the relationship between humans and the divine, magic does not necessarily include such elements. In his pioneering work on magic, Durkheim’s nephew Mauss defines magic as “any rite that is not part of an organized cult” (Mauss 2001 [1902]). Magic can sometimes be linked to ideas of divine reciprocity or to deities or ancestors, but for practitioners it need not be connected to an elaborate worldview. Magic’s detachment from comprehensive belief systems is substantiated by the fact that the same magical elements (e.g., rituals and spells) can be transferred between people of different origins and different beliefs (Evans-Pritchard 1929).

Furthermore, magic—as opposed to religion as commonly understood—is primarily concerned with achieving things in the natural world, such as health and wealth. “Counter-magic,” that is, a measure taken against (often imaginedFootnote 1) attacks of other people’s magic, is another common use of spells and magical rituals (Evans-Pritchard 1976), but this kind of magic can also be said to have as its objective the protection of wealth and health. Magic is a method aimed at achieving a certain goal through some kind of action (e.g., a ritual, a spell, or a fetish). This goal can be a materialization of something (e.g., someone falling in love with you) or, in the case of oracles, gaining knowledge on a subject matter (e.g., who is guilty of a crime or what is going to happen in the future). The latter is referred to as divination and is a way of using oracles, either persons or objects, to materialize predictions or statements about the world under certain conditions (a well-known example is reading tea leaves). The understanding of magic presented so far stands in sharp contrast to magic as the realization of intentions or magic as unrealistic expectations (or false consciousness in Marxist terminology).

Because of the utilitarian aspect of magic, it has been described as a technology (Naess 1987). However, defining magic as technology in a paper wanting to compare the two concepts will not do. In this paper, technology is reserved for human artifacts designed to perform intentional tasks and whose predicted effects can be given a functional explanation that is coherently aligned with theoretical knowledge from natural sciences as they stand today. Defining technology as an artifact excludes techniques such as psychotherapy, organizational structures or ways of using your body, all of which are sometimes included in the definition of technology (e.g., Stivers 2001). Uttering a spell to achieve a certain goal cannot be a technology as it does not include any external object (and also because its function cannot be explained by modern science).

Defined as such, technology, as opposed to magic, should be explainable and understandable in principle to anyone competent in the relevant area of scientific knowledge. However, there might be instances in which the predicted effects of an artifact, such as a medical drug, for example, are not fully understood by science; it is a well-established fact that two people can respond very differently to the same treatment, an effect due to the extremely complex constitution of human bodies and the interaction of psychological and bodily functions where all interacting factors have still not been identified. Although a full scientific explanation might be lacking, such an artifact must still be considered a technology because its use is based on empirical observations and medical science continues to search for possible casual explanations. Though magic is also a way of dealing with the world around us, its purported functional explanation is not compatible with science, at least not as it stands today. It might be assumed that we will reach a time when all instances of magic and science can be clearly distinguished from one another once and for all. However, some natural phenomena still remain outside the reach of human comprehension, and will do so almost certainly forever. Dieticians, for example, advice you to eat a varied diet (as opposed to just taking food supplements such as vitamins and minerals) because it is likely that foods found in nature contain nutrients not yet discovered (Mertz 1984). We should remain humble that some things will never be discovered and might have fundamentally different explanations than are thought of today (the paradigmatic shift from the Newtonian understanding of the universe to the Einsteinian understanding stands as a reminder).

This article will elaborate on aspects of our understanding of magic in relation to ML technology, each subsection presenting previous scholarly discussions on magic and discussing the relationship to ML technology. The analysis will rely on anthropological and ethnographical examples from the research literature as well as empirical examples from our own research on the implementation of ML technology in shipping (described above). The themes are not formulated to provide a comprehensive understanding or theory of magic but to highlight aspects that allow us to discuss the relationship between magic and ML. Having established the difference between magic and technology here in this introduction, the thematical sections that follow will focus on identifying similarities.

2 The cost and effort put into magic

The authors of this paper began to discuss magic in relation to ML technology while we were conducting an ethnographic study of the development and implementation of ML technology aimed at increasing energy efficiency in the operation of cargo ships. As social scientists participating in the project, we had the opportunity to observe the technical development of ML algorithms as well as the productization and implementation of the technological solution. One of the developers repeatedly referred to the ML solution as “magic” (or to be more precise “automagic”) when describing it to potential customers. The same developer also referred to the ML product as a “crystal ball,” enabling users to “see into the future.” As highlighted by Elish and Boyd, claiming that a product “works like magic” is common in marketing new technologies, especially those involving AI (Elish and Boyd 2018, 62). In contrast to the developer, one data science manager did not think that “magic” was a good way to describe the product. He was concerned that the customers did not understand the technology but saw it as some kind of “hocus pocus” and recognized this as a problem because it made customers overestimate the capability of the technology. Magic was thus used both to characterize ML technology and to distinguish it from blind faith, but with both perspectives sharing an implicit view that “magic” signifies that something works perfectly.

In the examples above, “magic” is used to signify something that functions perfectly well and without any effort. It does seem quite strange that academics who would most likely dismiss magic as superstition use the concept to describe technology that functions without any problems. In their article “Situating methods in the magic of Big Data,” Elish and Boyd (2018) share a similar understanding of magic, seeing it as representing an easy quick fix. They write the following:

In a brief essay on the correspondences between magic and technology, anthropologist Gell (1988) proposed that a defining feature of magic, as an orientating framework of actions and consequences in the world, is that it is “‘costless’ in terms of the kind of drudgery, hazards, and investments that actual technical activity inevitably requires. Production ‘by magic’ is production minus the disadvantageous side-effects, such as struggle, effort, etc.” (Gell 1988, p. 9). To evoke magic is not only to provide an alternative regime of causal relations, but also to minimize the attention to the methods and resources required to carry out a particular effect.

However, examples from throughout the world show that magic is rarely costless and cannot be described as an instant manifestation of intentions. Some magic rituals can be said to be quite effortless, such as spitting three times over your shoulder when a black cat crosses your road to prevent a potential large accident, a common superstition throughout the western world. However, while some magic performance might be understood as an attempt to take short cuts, in most cases it is not effortless but quite ‘costly’. An example is the practice of using albino body parts as fetishes in contemporary East Africa (Bryceson et al. 2010), a practice that is both expensive and risky in terms of the possibility of criminal charges. The underlying message in Levi-Strauss’ essay on the North American shaman Quesalid is that the function of magic is related to the effort that is put into the spell or the spectacle of the ritual—drama, blood, and fire is required to perform successful healing (Lévi-Strauss 2016). A possible psychological explanation is that the idea of magic is embedded in a view of a reciprocal universe, where a great reward can only be achieved through a great sacrifice. Be that as it may, magic—especially potent magic—is rarely costless or effortless.

With an understanding of magic as ‘costly’, the distinction between magic and ML technology becomes less obvious in this regard. Indeed, ML technology, like magic, does not provide an easy solution or quick fix; rather it is hard work. In the case of the project that we have been studying, it includes finding the best data variables to use, communicating with the shipping companies and the individual ships on what data are available and how these can be accessed, scaling these data, cleaning the data by removing faulty values, and evaluating the results that are produced in the ML libraries. Furthermore, the results from the training models must be made accessible through a comprehensible interface for end users. In sum, in the case of the shipping project, more effort is put into creating the ML system, designing and evaluating the ML training models, than the fuel savings of a single journey. The idea is that it will provide a scalable solution that can be used for many commercial cargo ships and produce large fuel savings over time.

Elish and Boyle also recognize the work put into ML technology: “When the glitz of AI hype is brushed aside, a great deal of mundane work underlies the practices of doing machine learning. This work includes collecting, cleaning, and curating data, managing training dataset, choosing or designing algorithms, and altering code based on outputs. In addition, as with any development process, engineers must grapple with the practical tasks of debugging and optimization, not to mention making sense of poorly documented code written by others.” (2018, 69) Elish and Boyle use this example to underline the difference between ML and magic. However, the large effort put into the rituals of ML fits quite well with an understanding of magic as costly. The comparison between ML and magic can be used to highlight the precise problem Elish and Boyd want to address by removing magic from the discourse of ML.

Therefore, if magic is not an easy quick fix why is it used? A common explanation among the early evolutionist anthropologists was that magic is a “failed science,” a faulty understanding of how things could be achieved linked to a “primitive mindset” that is replaced with more elaborated thought systems in the established religions, and eventually perfected in a scientific mindset. The understanding was that this “primitive” mindset developed throughout human history towards more complex thought systems represented in religion and then ultimately in science. Without a doubt, these evolutionist theorists could observe magic practices in their contemporary time.Footnote 2 Tylor (1958 [1871]) explained these phenomena as “survivals,” practices that continued in society even though they did not have a function (like the appendix or the tailbone in the human body).

Subsequent understanding has instead emphasized that magic is used differently from technology or science. In this spirit, anthropologist Rosaldo (1975) writes that “[m]agic fills in the gap where technology is absent.” This view is aligned with Malinowski’s understanding of magic as a way to handle the psychological stress of situations in which one’s own abilities do not suffice (Malinowski 1982 [1925]). However, technology or other methods are often used to achieve a goal that was attempted through magic. We will return to this discussion in the next section; however, we wish to point out that technology and other methods being used simultaneously somewhat disqualifies the theory that magic is only applied where science has failed.

3 The discursive dissonance and need of interpretation

Although Elish and Boyd have a poor understanding of magic, there is one good observation included in the quote above, namely that magic “provides an alternative regime of causal relations” (2018, 63). Indeed, magic takes place in an alternate epistemic regime in which an attempt is made to achieve something through actions that are not physically or causally related to the goal. The relation between the desired outcome y and what is done to achieve it can be written as the formula a:b:: x:y, where there is an attempt to achieve y through the influence of a on b and not through a natural causal way, represented by x. Perhaps the best example lies in the relation between a doll (b) and a needle (a) in a commonly depicted voodoo ritual (a:b) and their relationship with a danger/harm (x) and the person that is going to be harmed (y). However, something equivalent is true for most kinds of magic rituals; that is, something is performed to influence the relations in a different space. Some magic also includes a closeness or metonymic relation between (a:b) and (x:y) (what Levi-Strauss refers to as the relation between relations). For example, an injury might be treated by performing some ritual at the wound (Levi-Strauss, 1974) and some magic might require a metonymically related item (such as hair or nails of the person to be affected by the magic). An example from the fieldwork in Tanzania of the first author of this paper is a love potion that was made by a woman who cooked a stew with meat that had been inserted into her vagina for consumption by her partner of choice. This magic includes several metaphorical and metonymical relationships, such as the relationship between genitals and sexual desire. However, these aspects of closeness do not disprove the point that there is a gap between the magic ritual that is intended to perform something that might have another parallel causal explanation. Magic is a performance in which there is an attempt to manipulate the natural world through words or the rearrangement of objects in a different discursive or symbolic universe.

This structural relation between (a:b) and (x:y) establishes a disjunction between the act of magic and the effect it is supposed to have. To put it simply, one thing is done to achieve another thing. In the case of the voodoo ritual described above, for example, the objective is not to put needles in a doll, but to achieve something else through this action. In most cases, there are other ways to achieve what is attempted through magic. If magic is used to become wealthy, which is common in contemporary Africa (Comaroff and Comaroff 1999), the mundane way to achieve this is to start some kind of business venture, a church or a funeral home (as the saying goes). A common magic spell throughout the world is a love potion to make someone fall in love with you. Even the New York Times had a recent article on how to produce love potions (Wollan 2020). However, there are also known ways to try to make someone fall in love with you that are closer in terms of space and causality than preparing potions or casting spells (although truly making someone fall in love with you is not an exact science). A practitioner of magic typically knows this, and the performance of magic generally does not exclude trying to achieve an objective through more mundane means. As is evident in both Malinowski’s (1982 [1925]) and Evans-Pritchard’s (1976) empirical examples, the use of magic does not generally exclude working towards the desired goal or taking necessary safety precautions when magic is used to try to avoid accidents. In the case discussed by Malinowski (1982 [1925]) in which magic is used to try to ensure safety during long sea voyages, the magic act does not exclude doing the hard work of operating the boat and taking the necessary safety precautions to maximize the probability of a successful journey. The “magic” explanation co-exists with a mundane understanding of causality, as most famously illustrated in Evans-Pritchard Witchcraft, Oracles and Magic Among the Azande in which the collapse of a granary killing several children is explained both as malevolent magic and as the result of termites destroying the supporting beams (Evans-Pritchard 1976).

In short, magic can be described as a way of trying to achieve one thing by doing another thing, which can also be said to be true for ML technology. In the case of the ML algorithms for shipping propulsion, the goal of operating a ship as efficiently as possible in terms of energy consumption is attempted using methods that differ very much from those used to operate a ship as efficiently as possible in everyday navigation. The method of running various variables through ML libraries is indeed very different from operating a ship and is the concern of data science, an entirely different field of scientific inquiry. The data experts are concerned with the relations between numbers and have little interest in domain-specific knowledge (with the exception of some basic knowledge of which variables can be used to predict ship behavior). In this regard, ML differs from earlier generations of expert systems which still depended on domain knowledge and domain expertise (c.f., Huysman and de Wit 2004). Contemporary AI systems such as the ML systems described here are developed almost or completely independently from domain experts in the field in which the application is intended to be used (c.f., Michalski 1986). To take another example from a common ML application, the facial recognition algorithm does not know what a face is or what it looks like. It mathematically interprets the relation between the pixels in an image and can provide an estimated value of whether or not a picture analyzed is a human face, based on the images that have been used for training. The data model is another regime of knowledge or semantic system that requires different methods of interpretation.

Because of this discursive or symbolic disjunction in both magic and ML technology, interpretation between domains is necessary. Such interpretation is especially necessary in the case of divination (the brand of magic that is used to obtain knowledge through oracles). An oracle can be a person (like the famous Pythia of Delphi) or objects, such as bones or poison, that are used in a certain fashion to materialize some truth about the world. The message obtained through divination is seldom straightforward and often require some interpretation. Such was the case for the messages of the Pythia of Delphi whose predictions were notoriously cryptic. It is also true for bone oracles and poison oracles and various other forms of divination. In many cases of divination, the ritual is performed and interpreted by someone with ritual expertise, such as a nganga or a shaman. From what is known of the oracle of Delphi the oracle transmitted a message that had to be interpreted by the receiver of the message. Other examples include the oracle bones of the Chinese Shang dynasty which were read by diviners; later kings were the ones capable of reading the messages in the divination process. There is also the Bakongo nganga (ritual expert) who was required for mediating the power of the nkisi (an object providing protection from, among other things, spirit possession).

An important part of interpreting magic is explaining why the magic does not work in some instances (because counter to what Elish and Boyd (2018) and Rosenberg (2021) seem to suggest, magic does not work every time). The practitioner of magic is well aware that magic will not always work. The failure of magic to work can be given various explanations by the practitioner or ritual expert—that the ritual has been performed incorrectly, for instance, or circumstances in the natural or cosmic world are wrong, or in the case of oracles that the result has been interpreted the wrong way. Another common explanation for the failure of magic is that counter magic has been used, an explanation that is credible since rituals of counter magic are also common among many practitioners of magic (e.g., Evans-Pritchard 1976).

Like magic, ML technology builds upon the relation between two different discursive or symbolic systems and requires interpretation. The methods of ML generally cannot be performed by domain experts within the field in which they are to be used, and the results from ML libraries require interpretation. When constructing a model, the output data from the ML algorithms have to be interpreted and verified so that the output actually represents values that are meaningful and useful to end users. Furthermore, the results need to be communicated in a way that can be understood by end users. The interpretation of the system output made by domain experts is yet another layer of interpretation. Evaluating ML output is generally different from the way other input information is used and evaluated. In the case of the shipping project, interviews show that the estimations made by the system often contradicted the skill and embodied knowledge of the crew based on their previous experience of ship behavior and weather. One of the problems with the interpretation of ML outcomes is that they provide estimations that in the best-case scenario are correct within a certain margin of error, and it is inherently difficult to relate to and make judgment based on statistics.

In interpreting and assisting further interpretation of ML output after the system is built and implemented, the data experts become sort of ritual experts in this regard, translating knowledge from one domain into another. Therefore, when Zhou (2016) writes that “[m]achine learning is still a kind of magic: Even with sufficient training data, most end users, except machine learning experts, can hardly produce strong models,” he quite accurately describes magic and the similarity between ML and magic. Similarly, Mathis et al. writes:

As with all methodologic approaches, machine learning is not without drawbacks. The most hotly contested is the difficulty of understanding mechanisms driving the prediction models presented. Herein lies the “black magic” of machine learning […] the question of how to interpret and act upon the information generated remains wholly unanswered. (Mathis et al. 2018)

Mathis et al. accurately identify an important aspect of the “magic of machine learning,” namely the difficulties related to the interpretation of the outputs. The interpretation of ML relates to fundamental questions regarding the epistemology or nature of ML technology. Burrell’s (2016) discussion on the opacity of AI algorithms notes that only a limited professional class with highly specialized skill is able to read or understand algorithms. It also refers to the fundamental “mismatch between mathematical procedures of machine learning algorithms and human styles of semantic interpretation” (Burrell, 2016, p. 2).

As with oracles, interpreting between domains entails explaining inaccuracies and faulty results produced by an ML system. No matter how good the model is there will always be the chance of erroneous results with potentially significant consequences. Recall, for example, Google’s infamous labeling of photos of African Americans as gorillas (Zhang 2015).

In the shipping project, there are many sources of potential errors. If a ship encounters weather conditions that were not included in the training data, the results can deviate very much from the predictions. In addition, as mentioned above, no matter how good the predictions are they will always deviate within the margins of error since they are statistical predictions. Another similarity is that faulty results do not disprove the methods used for making predictions. In the shipping project, there were many situations in which the predictions did not meet the expectations of the user. Sometimes such dissonance was explained in terms of problems with the design of the model, but other reasons were also frequently given (for example that the tool had not been used correctly). With results that are produced and communicated digitally, there is also the possibility that the data have been intentionally corrupted through a cyberattack of some sort. Elish and Boyd write: “Models that made sense in one instance are incorrect in another, or undermined by malicious or unwitting actors” (Elish and Boyd 2018, 72). In the shipping project, there was also a concern with spoofing (disguising a digital communication from an unknown source as being from a known, trusted source) that could potentially corrupt the output of the system. Once again, there is a striking similarity to the idea of counter magic corrupting the results of a system.

Thus, failure to produce accurate results does not necessarily disprove the general methods in ML. An important difference between magic and technology, however, is that many of the potential errors of an ML system can be explained in a scientific manner. There are also ways of systematically evaluating the results produced by ML predictions and making the results more reliable, a quality it does not share with magic. Evaluation includes monitoring the input signals and output signals to determine if they are outside a plausible range to administer warning signals to users. In many instances, however, it will not be possible to detect incorrect results in the predictions.

The features ML shares with magic, namely the need for interpretation and contingency of results, relates to a critique of knowledge claims of ML (e.g., Crawford and Calo 2016; Elish and Boyd 2018; Selbst et al. 2019). As formulated by Elish and Boyd: “Not only does the construction of models hold significant epistemological implications, but also the ways in which models may be interpreted generates epistemological fault lines in the kinds of truth and knowledge that AI system produces.” (Elish and Boyd 2018, 71). Based on the opacity of ML system designs and on the difficulty of reproducing results, previous scholars have argued that ML is not a science in the common understanding of science as producing statements that are falsifiable (Popper 2005 [1934]). Elish and Boyd explicitly write that “machine learning is not a science, at least not in the traditional sense. Unlike disciplines that leverage the scientific method as a tool for interrogating phenomena, machine learning techniques do not require formulating hypotheses rooted in earlier theories to test for validity” (Elish and Boyd 2018, 72).

4 The function of magic in dealing with contingency

This section is concerned less with the beliefs and praxis around magic and ML; it takes as its point of departure scholarly interpretation of the function of magic. While anthropologists have documented beliefs and practices of magic, their main concern has been to discuss and try to explain the function that magic has in a society. The intellectualist approach emphasizes that magic provided intellectual explanations of the world. This was the approach of the early evolutionist armchair anthropologist and has been a focus for other anthropologists, for example in the work of Lévi-Strauss in emphasizing the importance of meaning and sense-making in understanding witchcraft beliefs (2016). Other anthropologists have stressed that magic serves individual human needs by addressing psychological needs (Malinowski 1982 [1925]) or by contributing to social cohesion (Durkheim 1976 [1912]; Mauss 2001 [1902]; Radcliffe-Brown 2013 [1922]). The correctness of these various intellectualist and functional approaches to understanding magic is not at issue here. However, if magic is said to contribute to social cohesion some disclaimers are in place. Social cohesion is not equated with a good or just society. Some magic, such as identifying witches through oracles and killing them, might maintain a normative order by making people align with established norms, also getting rid of those who are socially marginal or outsiders and other odd people. The fact that magic and witchcraft do not always contribute to social cohesion and stability in a society is well documented in African societies in decline (for example, as an effect of colonial violence) where the decline has created a prominent role for ngangas to execute witchcraft processes (Comaroff and Comaroff 1999; Ekholm-Friedman 1991; Larsson 2016), sometimes with socially disturbing effects.

Although magic is interpreted differently by different scholars within anthropology, we argue that the function of magic can best be described as a method to deal with and ultimately reduce contingency. That is, magic helps people address the fundamentally indeterminate condition of human existence, including the fact that tomorrow can be radically different from today and that we as humans have limited knowledge and limited capabilities. In this regard, we are using “contingency” in its original theological meaning as a fundamental condition of the postlapsarian world, where the created is differentiated from the creator. This condition of the world is expressed quite well in one of Paul’s letters to the Corinthians: “For now we see in a mirror, dimly, but then face to face. Now I know in part, but then I shall know just as I also am known” (New King James Version, 1 Corinthians 13:12). As such, contingency represents the lack of knowledge of the forces and mechanisms guiding the events of the world as well as a lack of ability to control and manipulate these events. Wanting to reduce contingency by controlling events and increasing knowledge is a fundamental human desire; it has been argued that a reduction of contingency is a fundamental aspect of human organization and human institutions (Luhmann 1989). In such a tradition, magic can be understood as an institution providing explanations and methods for trying to control events (Boholm 2015), to prevent or deal with existing illness and misfortunes, to accumulate riches or become more powerful. A view of magic as a way of dealing with contingency encompasses the intellectualist approach and the various functional approaches to understanding and describing magic (providing explanation, responding to physical and psychological needs, and to preserve social cohesion).

While the fundamental uncertainty in ML is important to take into consideration when trying to understand ML, ML is ultimately a method to reduce or deal with contingency. For example, in translation and speech recognition, ML-assisted digital tools can provide translations that make them more accessible for people who otherwise would not be able to read these texts. That ML is a way to reduce contingency is certainly true in the case of the shipping project. By making predictions based on historical data and weather prognoses, the ML system provides knowledge that may change the operation of a ship. The project actually tries to make a prediction about the future, trying to explain and predict events that have not yet happened by providing information on estimated arrival time given the condition of the ship and the weather. This knowledge is not obtainable in any other way, and it does not hide or displace any existing data, which might be the case if a well-established and well-functioning practice were to be replaced with an ML-powered system.

That the ML system is intended to reduce contingency is further underscored by technical developers who were interviewed stressing that the ML system would be particularly useful when uncertainty was high. In calm weather conditions, constant speed will be equivalent to constant power consumption, and there will thus be little need for the ML system to make predictions and offer advice on how to operate the ship. But in rough weather conditions or weather conditions that vary throughout the journey, that is, in situations of high uncertainty, the system will be useful. This fact about contingency in the ML system aligns with Malinowski’s statement that “deep magic” among the Trobriands is utilized in sea voyages in deep waters where the uncertainty and stakes are high (Malinowski 1982 [1925]).

By describing magic and technology as a way of dealing with contingency, we believe that we touch upon the single most important aspect of the technology–magic analogy. Technological development is driven by a need to control the world and to safeguard provision of food, safety, and other human necessities and desires. However, science and technological development are also driven by curiosity in understanding and explaining events and mechanisms of the natural world. The desire to find connections between things that do not appear to be connected seems to us to be more important than immediate human needs as a driving force of science and technology. Magic and science alike are aimed at making connections and trying to understand a world with a surplus of symbols and a deficiency of meaning. What Durkheim wrote about religion is also true for magic, namely that it “sets itself to translate […] realities into an intelligible language which does not differ from that employed by science; the attempt is made by both to connect things with each other, to establish internal relations between them, to classify them and to systematize them” (Durkheim, 1976 [1912], 429). Much of the inquiry of modern science has been driven by a firm belief that it is possible to find explanations and connections even though they are not known to exist at the time. By encompassing both magic and science in this human desire, we can also understand the close connection between scientific inquiry and magic among some of the most prominent scientists throughout history, such as Isaac Newton, Galileo Galilei, and Tycho Brahe. What is more, conceptualizations do not necessarily have to be correct to be productive in terms of innovation and development. As Lévi-Strauss convincingly argues in La Pensée sauvage (1966), much of the development during the Neolithic revolution was driven by conceptualizations beyond empirical observations that were not scientific. Lévi-Strauss understands that a systematization of the world provides humans with a great advantage even if it is faulty in its basic understanding of the world.

5 Concluding discussion

As shown in the introduction, magic is frequently used as an analogy for AI and ML technology, in popular discourse as well as in scholarly debate. Most scholars that have commented on the relationship between magic and ML, however, have a rudimentary understanding of magic, namely that it functions perfectly and that, in this regard, it differs from science, or that magic is irrational and that, in this regard, it is different from technology or similar to technology depending on the argument they want to make about technology. The first finding of this paper is that magic is poorly understood and defined even in scholarly work, and generally poorly connected to anthropological or religious studies, thereby failing to reflect the existing knowledge and debates on the praxis and function of magic.

The argument made throughout this paper is that technology in general, and ML technology in particular, have similarities to magic; at the very least it is not “the furthest thing from magical” (Rosenberg 2021, 1299), as has been argued in previous research. The point of this article is not to say that ML is just like magic, or that the results produced from ML algorithms are just as arbitrary as bone oracles or other methods for divination, but to highlight certain similarities that are interesting for discussing some aspects of ML technology. Recognizing these similarities between magic and ML can provide insight into how to view and deal with ML technology in real life such as in a factory or other organizational settings.

Magic and ML technology alike require a translation between domains. In the case of magic, interpretation of oracles might require someone with ritual expertise (such as a nganga in the Bantu context of Sub-Saharan Africa or a shaman in Europe or North America) to translate real-life problems into the logic of the magic or spiritual world and, in the case of divination, to interpret information from the discourse of magic in terms of knowledge that is relevant to everyday problems. In the case of ML technology, data specialists and data scientists are required to translate domain-specific knowledge into the symbolic language of data science and mathematics and then back to domain-specific knowledge. The way results of ML must be interpreted and trusted by users has similarities with the interpretation of divination, including the structural similarity between the technical elite that are required to interpret the results of the ML technology and the ritual experts that are commonly required to interpret and contextualize the message resulting from divination. The act of translation and interpretation is important for considering how ML can be used and how the results should be communicated to the end user.

As noted by several scholars, ML is not a science in the sense that the results can be reproduced in any easy way or the hypotheses embedded in the methods can be falsified; in this, it is like magic. In contrast with magic, however, the methods and results of ML can be systematically evaluated using logical and scientific explanations. But failure to produce accurate results cannot immediately disprove the method used, and the reasons for the failure need to be communicated to the end user. In using AI for predicting, for instance, the likelihood of relapse in setting bail conditions (Fry, 2018), you will not get a fixed unambiguous answer but a likelihood ratio that needs to be made useful for the actual work of the individual judge. How this information should be used and how it should be brought to bear on the institutionalized ways of doing things is not straightforward. This complication creates concerns over the conditions under which AI results can be used, particularly if the prediction feeds directly into an automated system, raising questions as to whether or not other important values such as safety, existing regulation, fairness in decision making, and transparency are compromised. Mathis et al. (2018) argues that “in cases where mechanisms are critical, and penalties for error are high—as is often the case in health care, and particularly in anesthesiology—a machine-learning approach falling anywhere short of nearly perfect remains unviable” (Mathis et al., 2018).

In this paper, we have also argued that there is a similarity between magic and ML (and also technology in general) in the sense that both are deployed to deal with contingency. Magic and technology are both related to the general human desire to reduce the contingency of the world, which answers to biological, psychological and societal needs. Some literature on AI and ML stresses the indeterminate outputs of the technology (also discussed in the previous paragraph). Certainly, such technologies can increase indeterminacy if ML learning applications replace established and well-functioning operations performed by humans. But the goal with such technologies is to gain information that in some regard deals with or tries to reduce the contingency of the world by producing knowledge that is otherwise difficult to obtain. When used within the judicial system the aim is to make verdicts and other court decisions fairer and to reduce human bias (e.g., Chen and Eagel, 2017; Sweeney and Fry 2018). In the case of the shipping project, we have been studying the goal with the ML application was to produce knowledge on the estimated arrival time of a ship given a predetermined fixed consumption to enable fuel savings. This knowledge is very hard or impossible to obtain in any other way. In addition, although the results will have a margin of error, if done correctly ML will provide useful information for saving fuel and arriving on time. This aspect is important to recognize since much of the social science debate on ML tends to focus on the way ML obscures knowledge and increases indeterminacy, with Mathis et al. arguing that ML is unacceptable in many cases if the results are falling anywhere short of nearly perfect. It is, however, important to remember that humans and other technologies also do not produce perfect outcomes and an ML system that outperforms humans might still be a good option. Although magic and ML are both methods to deal with contingency, ML is the one that is better equipped to actually successfully reduce it.