Sensorimotor debilities in digital cultures

Abstract

This paper reflects on the qualities of living and learning in digital cultures, the design of digital technologies and the philosophical history that has informed that design. It takes as its critical perspective the field of embodied cognition as it has developed over the last three decades, in concert with emerging neurophysiology and neurocognitive research. From this perspective the paper considers cognitive, neurological and physiological effects that are increasingly becoming noticed in user populations, especially young populations. I call this class of conditions ‘sensorimotor debility’, to distinguish it from other psychological, social, cultural and political symptoms associated with computer, internet and social media use.

1 Introduction

This paper reflects on the qualities of living and learning in digital cultures, the design of digital technologies and the philosophical and commercial history that has informed that design.¹ It takes as its critical perspective the field of embodied cognition as it has developed over the last three decades, in concert with emerging neurophysiology and neurocognitive research.² From this perspective, the paper considers cognitive, neurological and physiological effects that are (mostly) associated with younger user populations, increasingly noticeable among Millennial and GenX ‘digital natives’ (the first born-digital generations), as they age.³ I draw upon Jasibir Puar’s articulation of debility to develop a theorization of sensorimotor debilities in digital cultures (Puar 2017). This discussion is framed by a more general historical and philosophical critique of AI and cognitive science, including a critique of congnitivism/mentalism/internalism—glossed below.⁴ I am provoked to provide this historically and philosophically informed critique because, in my experience, far too few computing professionals are familiar with the philosophical commitments of their discipline and how these conditions them to take a dualist approach to their technology.

In this paper, I focus on sensorimotor debilities, as opposed to social, psychological and political implications. What I call sensorimotor debility has been rendered largely invisible by digital cultural rhetorics, both philosophical and commercial. I propose that these debilities and the systems, and structures they arise in might be illuminated by applying perspectives from post-cognitivist discourses. Pronouncements, utopian or dystopian, regarding the internet, social media, gaming, computer use or screen-time tend to be highly polarizing. Concerns regarding child development tend to be even more polarizing.

2 Cognition and computing

The philosophical basis upon which computer technologies have formed is inherently rationalist. Computing as a discipline is axiomatic of the Cartesian mind/body dualism on which it is founded. This is evidenced by the unquestioned ‘belief’ in the immateriality of information, reified in the fundamental distinction between hardware and software that is taken as axiomatic. This encourages certain ways of thinking and precludes others.⁵ The discipline’s core formal ideas: Turing’s universal machine, George Boole’s mathematical logic; von Neumann’s serial processing architecture; Newell and Simon’s Physical Symbol System hypothesis; Putnam’s functionalism: are all consistent with rationalist–humanist commitment to the idea of abstract, immaterial information. Through the second half of the C20th, conventional philosophy of mind (a philosophical tradition that axiomatically embraced a Cartesian separation of mind and body) was reinvigorated by notions of thinking-as-reasoning that took as their models the operations of digital computers: symbols, code, programs and algorithms. Computationalist theories of human cognition became more hegemonic as the new discipline of cognitive science accrued authority and power. This occurred largely due to the mutually justifying nature of arguments in cognitive science and in artificial intelligence (AI). AI itself was propelled due to the rapid development of digital computing. Putnam’s functionalism (later recanted) provided a philosophical argument that cognition was a matter of manipulation of symbolic tokens in an abstract reasoning space. By these lights, intelligence was intelligence, whether implemented in silicon or neural tissue.

As digital computing became our paradigmatic technology (Bolter 1984), this disembodied conception of information and cognition came (rather insidiously) to be explanatory of our biological systems, from DNA to brain and all the rest.⁶ While social media is replete with spiritualisms, alien seduction stories and conspiracy theories, popular digital cultures are undergirded by an armature of technologically instrumental procedures with specific affordances and constraints,⁷ that, when metaphorised, contribute to a rhetorical environment, in a way that I have previously likened to a Trojan Horse.⁸ In the late 1980s, the so-called common-sense problem hit the AI community with a force that could no longer be evaded.⁹ AI systems made absurd gaffs. Symbolic logical procedures were unable to embrace the complexities of the real world, and in attempting to cope with weight of contingency upon contingency, systems broke, in a condition designated brittleness. Hubert Dreyfus argued that logical operations on symbols cannot provide the kind of practical knowledge required by creatures, and robots, to operate successfully in the world (Dreyfus 1972) In conceiving our ability to go-about-the-world as being amenable to computational rationalization, the AI community not only asserted that mathematical logical procedures were not only ‘necessary and sufficient’ to fulfill animal cognition but this assertion, as a case of Heideggerian enframing, rendered cognition itself as a standing reserve.

These issues came under scrutiny by thinkers and researchers such as James Jerome Gibson, Hubert Dreyfus, Lucy Suchman, Rodney Brooks, Francisco Varela, David Kirsh, Edwin Hutchins, Andy Clark, Philip Agre, Andy Pickering, Maxine Sheets Johnstone, Mark Johnson, Evan Thompson and others. They showed that such internalist explanations of cognition were by no means universally explanatory, in some case not explanatory, and in some cases entirely obfuscating, particularly in the case of practices with a substantial embodied component. Thus emerged new paradigms of cognition that provided for embodied, social and spatial dimensions of cognition, called Situated, Embodied, Distributed, Enactive and Extended. The more recent, related notion of Material Engagement (Malafouris 2013) complements this list.¹⁰ Taken together, these paradigms provide the basis for the development of a new vocabulary with which to discuss the cognitive dimensions of intelligent practices in the world that characterize human cultures. While humanism places substantial emphasis on the individual cogito, post-cognitivist approaches destabilize the autonomy of the individual from both directions. They help us understand ourselves as creatures for whom cognition is a phenomenon of the entire organism (influenced by internal symbionts, our microbiome) and for whom cultural artifacts and systems—from language to architecture to databases to dancing—are constitutive of socially, spatially and temporally distributed modes of supra-individual cognition.

3 Technosocial change

Students of the history of technology, especially of the industrial revolution, will be familiar with the idea that the development of regulatory social and legal mechanisms necessarily tend to lag decades behind new technologies.¹¹ A technology has to become well established and entrenched before troubles are noticed. An historical process unfolds: a technology is developed, it is then made commercially viable and marketed. It then, slowly, embeds itself into society, in ways that are often surprising—as William Gibson aptly noted “the street finds its own uses for things” (Gibson 1982). It is only then that the way social mechanisms are being distorted can be observed, and occupational health issues can be correlated (and usually dismissed as spurious by vested interests). Studies have to be funded, tests and measurements have to be made, and their results contested. It takes a groundswell of public opinion, usually driven by activism and lobbying by the most egregiously effected, before laws are passed and regulations are enforced. History shows us that this process takes 25 or 30 years, about a generation.¹² So it is that, in 2020, roughly 35 years since the introduction of the personal computer, 25 years since the emergence of the world wide web, and 15 years since the emergence of mobile computing on handheld devices, we are now coming to understand some subtle and insidious aspects of emerging technosocial formations that span political, economic, social, and psychological and physiological dimensions. The ecstasy of communication (Baudrillard 1988) has become the agony of communication.

Corporations are always keen to show us the advantages that their products offer, and no doubt, they have wonderful qualities: we enjoy access to, and sharing of, information of all kinds, opportunities to create communities of niche interests, location of obscure artifacts and commodities, and the rest. Along with these qualities of digital technologies that we take pleasure and profit in, there is emerging an increasingly long litany of social, physiological, and cognitive deficits. These can seem trivial (or are represented that way by the same vested interests), but taken together, they indicate a disquieting trend.

It would be foolish to imagine that the computer products and tools we have, exist to serve the common good. The products we have are the products that sell, that people buy (often with manipulation of public opinion via advertising). They are designed to be maximally profitable. They have emerged as products of an alliance between an academic research community that, as Philip Agre would observe, has been disinclined to social or philosophical reflection, and a corporate community who recognized the opportunity for the accumulation of unimaginable wealth, and is (still) distancing itself from social or moral culpability (Agre 1997).¹³ Like any powerful and wealthy group, the new technopolitical establishment will be slow to recognize or admit culpability that may negatively impact their reputation or bottom line. These industries hire highly trained specialists who can render ‘dangerous’ results dubious and buy off potentially critical institutions with gifts.¹⁴ Once invested (and thus complicit), states, corporations and educational institutions will be slow in acknowledging unexpected costs and problems. Given all this, any attentive student of technopolitical history will be unsurprised when:

• 1. Negative effects of new technologies show up.

• 2. Some people, communities and environments are negatively impacted.

• 3. These impacts are suppressed, denied or otherwise made dubious by vested interests.

I believe we are seeing all three aspects of this scenario, especially among the (now various) born-digital generation(s). We are also seeing autonomous recognition and compensation, evident in the enthusiasm among Silicon Valley professionals and other computerati for highly physical sports like rock climbing and mountain biking, the rise of maker culture and the reported popularity of tech-free Montessori-style schooling for their children.¹⁵

4 Skill and intelligence

The computationalist ethos asserts a clear separation between information and matter, just as Cartesianism asserts a separation of mind and body. According to this logic, the distinction between intelligence and skill is obvious. But the idea of a separation between mind and body is ideological, counterfactual and without any scientific basis.¹⁶ Postcognitivists put a counter argument: cognition is an activity of the whole person. Skilled making involves learned bodily practices that may be difficult to linguistically articulate. From paleolithic flint knapping to blacksmithing to shaping a pot on the wheel, to watchmaking—for the entirety of human history, human culture has been defined and made glorious by skilled activities that involve the combination of refined perception and judgment with precisely applied muscular force.

In the industrial period, the harnessing of fossil fuels and the development of fast precise machines, led to a reconfiguration of the role of human cognition and muscle power. But it still took muscular precision to control the steam engines, locomotives, and the heavy and light machinery of mines and mills that functioned, effectively, as force amplifiers (a term taken up by the US military mid C20th). Through the C20th, as electrical infrastructure proliferated (in more wealthy, industrialized countries) the idea of labor-saving devices animated industrial design and production from vacuum cleaners and washing machines to power steering. This trend has proceeded with little in the way of checks and balances, the reduction of physical effort by the application of external power sources being seen as an unquestionable good—a scenario that has clearly been to the benefit of power and petroleum companies.¹⁷ In ‘postindustrial’ nations, physical labor has been reduced to the bare minimum—the swiping of touchscreens and tapping of keyboards, a kind of work in which both sensorimotor precision and physical effort have been actively reduced,¹⁸ as has cognitive effort, such as using mental arithmetic to calculate the amount of change required in a purchase.¹⁹

It is easy to forget that until the 1980s, for the entire prior history of the human race, screen-based interaction did not exist.²⁰ Many of us now spend a substantial part of our waking lives looking at a computer screen or a phone screen, and poking little buttons or images of buttons.²¹ Occupational conditions will arise when a person devotes a significant portion of their working life to a specific kind of activity. Cowboys get bow-legs, people who stir vats of whisky get ‘monkey-shoulder’, and people who sit at desks all day get backaches. It is to be expected that extended periods of computer use will result in new occupational conditions. Without adequate care, one might expect a decline in physical fitness, agility and stamina, weight increase and possibly eye problems. All this is common-sense and is known to be the case (RSI being an obvious example). What is less straightforward is whether such activity leads to cognitive changes or deficits—sensorimotor debilities.

One way to think of sensorimotor debility is to understand the computer as a cognitive prosthetic. Cognitive activities, from remembering phone numbers, correcting grammar and spelling, doing mental arithmetic, to visually aligning objects, identifying faces or navigating cities, are now ‘off-loaded’ onto machines. The result is the atrophying of the capacity to perform such activities without computer assistance. Such adaptations—and acculturation to them—necessarily change the qualities of cognition and thus human capabilities. Such adaptations have always occurred as new technologies are embraced. Many non-literate peoples have extraordinary capacity for memory that literate people have largely lost. The advent of the camera led to a general decline in draftsmanship. Digital texts offer sensorimotor affordances and constraints that differ from those of books. Widespread use of typing has led to a decline in handwriting ability (mine has become atrocious).

A complementary approach would be to acknowledge the embodied nature of cognition, and to recognize that, in a technology that axiomatically endorses the idea of the separability of information and matter, this quality of human cognition is obscured. The embodied nature of cognition is necesssarily meaningless if internalist characterisations of cognition are found adequate: the body is little but a meat marionette whose strings are pulled by the self in the skull. On the other hand, if we are to take the ideas of the various post-cognitivist paradigms seriously, then cognition is an activity of the whole creature, and a dogmatic distinction between mind and body is untenable.²² This then brings into question the conception of the nature of cognition that is reified in computational technology. What kind of a creature is implied by the machine? Long ago, some wit remarked that if in the future, an alien archeologist were to discover a computer (but no human remains) they would infer that humans were a one-eyed, one-limbed creature with 19 digits.²³

Postcognitivist approaches bring into question the validity of the humanist conception of cognition. What if cogito ergo sum is simply wrong and sum res cogito (I am a thinking thing) is a better conception of the human condition? What if it takes a full human-like embodiment to cognize like a human? This, essentially, was Hubert Dreyfus’s phenomenological refutation of AI: to think like a human, you have to have a human-like embodiment (Dreyfus 1972). A humanoid robot will not do, you need a life-history of embodiment, (and probably an evolutionary history) because the structure of the brain and the way a person thinks are a product of their development, of interactions between a developing brain, a growing body and a physical world. (A more radical position would be to hold that thinking itself is an activity of the whole person, not just the brain). Jakob von Uexküll would frame this idea from an ethological perspective in terms of the umwelt of a creature—the world it is given by its particular sensorimotor embodiment (Uexküll 1957). Being embodied in particular way, we know those qualities of the world we can know through our human suite of senses. We experience the physics of the world and develop kinesthetic and proprioceptive capabilities isomorphic with the impact of that physics on our particular embodiment. I cannot know what it is like to be a bat (nor can I know what it is like to be an immaterial intelligence, if indeed such a thing were possible.²⁴

5 In(tro)ducing sensorimotor debility

The infiltration of digital tools into human culture has been slow enough that younger generations have naturalized to them as aspects of the pervasive sociotechnical background, as water is to a fish. In Neil Postman’s terms, these technologies have become part of a technosocial ecology: we look at a friend or colleague through our phone, through the internet and through Facebook (Postman 1992). The technologies recede (when they’re ready to hand), so drawing attention to the specificities of these dispersed technologies can be difficult.²⁵ Jasibir Puar’s formulation of debility is relevant to my current subject. In The Right to Maim, she explains: “Debility addresses injury and bodily exclusion that are endemic rather than epidemic or exceptional, and reflects a need for rethinking overarching structures of working, schooling, and living rather than relying on rights frames to provide accommodationist solutions.” [p2]. She asserts “Capacity and debility are, on the one hand, seeming opposite generated by increasingly demanding neoliberal formulations of health, agency, and choice—what I call a liberal eugenics of lifestyle programming—that produce, along with biotechnologies and bioinformatics, population aggregates.” (Puar 2017). Puar observes: “Technological platforms—new media, prosthetic technologies, biomedical enhancements—mediate bodily comportments, affects, and what is recognized as bodily capacity and bodily debility. Technology acts both as a machine of debility and capacity and as portals of affective openings and closures.” [ibid p3] She later asks “Which bodies are made to pay for “progress”?” [ibid p13], appropriately putting ‘progress’ in quotes. It is entirely pertinent for us to consider which bodies pay for that ‘progress’, the ways those bodies pay, and who profits. These bodies include factory workers in Shenzen, Amazon warehouse staff, as well as call center workers and students in schools.²⁶

While occupational hazards of adult life are demonstrably real, a more subtle and pernicious threat lies in the impact of computational technologies in infant and child physiological, cognitive and psychological development. Clinical and anecdotal evidence point to rapid changes (mostly declines) in a variety of key markers of bodily competence—from visual acuity to manual dexterity—among young adults over the last 15–20 years. This time period corresponds with the emergence of the born-digital generation who have been naturalized to digital touch screens in infancy, as previous generations were naturalized to books. The substantial reconfiguration of cognitive and sensorimotor capabilities is occurring in communities where screen-based technologies are ubiquitous—especially, for obvious developmental reasons, where infants and children are overexposed; this justifies further research.

In a 2018 article in The Guardian entitled “Medical students raised on screens lack skills for surgery”, Roger Kneebone, a professor of surgical education at Imperial College London, lamented that his students often do not have a basic understanding of the physical world: “We are talking about the ability to do things with your hands, with tools, cutting things out and putting things together … which is really important in order to do the right thing either with operations, or with experiments. You need to understand how hard you can pull things before you do damage to them or how quickly you can do things with them before they change in some way.”²⁷ Here Kneebone gestures toward a lack of general background knowledge about the world and how to proceed in it (that children once normally accumulated), that Polanyi called ‘tacit knowledge’ (Polanyi 1966).²⁸ Hands-on making and play that were central in childhood experience have been substantially replaced by screen-based activities. An assessment of the general impact of this is overdue. Anyone with a grounding in developmental psychology, or anyone who has a child, understands the unrelenting hard work an infant does to integrate their senses and their physiology and to understand the world of space and mass and light and gravity. As the famous ‘kittens in baskets’ experiment of Hein and Held showed, an infant remains functionally blind unless—at a critical time in neural development—its visual system is calibrated through bodily movement.²⁹ Proprioception and kinesthesia are the source for knowing ourselves in the world, and these senses bring vision and hearing ‘into focus’.³⁰ Human cognitive systems have evolved to auspiciously match our specific environmental contexts, as Jakob von Uexküll noted in his conception of the umwelt, and as Abraham Maslow pointed to, in his essay ‘Isomorphic interrelationships between knower and known’ (Maslow1966).

A recent study shows that childhood and adolescent myopia rates have risen alarmingly in the last 15–20 years. “Another remarkable change shown by our survey was that the proportion of high myopia (7.9–16.6%), especially very high myopia (0.08% to 0.92%) significantly increased during a 15-year period.” The authors continue: “The etiology of myopia still remains unclear. However, genetic and environmental factors are widely believed to play an important role. Near work is one of the important environmental factors.”³¹ (For ‘near work’ read ‘screen-time’). If neuro-optical and visuomotor capabilities are developed through active practice in childhood, and if a child’s visual focus is largely on a flat, perpendicular, smoothly illuminated surface 40 cm from their face, then clearly, visual capacities that involve focusing on the horizon, rapidly changing focal distance, rapidly changing location of visual attention across a wide visual field, attending to events in peripheral vision, or in low or high light conditions—simply will not develop. The impact of these changes, and the specificity of the historical period, is not lost on public health personnel. A 2019 study “Associations Between Screen-Based Media Use and Brain White Matter Integrity in Preschool-Aged Children” conducted at the Cincinnati Children's Hospital Medical Center, states “In this study of 47 preschool-aged children, increased use of screen-based media in the context of the AAP guidelines was associated with lower microstructural integrity of brain white matter tracts that support language, executive functions, and emergent literacy skills, controlling for child age and household income. Screen use was also associated with lower scores on corresponding behavioral measures, controlling for age.”³² Although a small sample size, these results should give pause. In considering the implications of these studies, a vast territory for research presents itself.

6 Screens and screenal interaction

The general concern over ‘screen-time’ is so familiar as to be tedious, and skeuemorphic in the sense that it refers to a television-era concern, ported into digital times. What is missed in such ‘porting’ is that digital interaction is experientially, ontologically different from passive screen-viewing.³³ Or at least it can be: clicking buttons to do the equivalent of turning pages seems trivial, but making interactive choices and actions that change narratives, promote reaction from characters or even cause their demise, ethically implicates users in a way that passive observers are never implicated. The user is cognitively and neurologically engaged in sensorimotor ways entirely absent in passive screen-viewing—different neural pathways are activated. Users take actions and those actions have consequences (on screen at least)—interactive engagement breaks the ‘4^th wall’ of the subject/object dual.³⁴ Driving a ‘car’ in a car race game is neurocognitively entirely different from watching a car race on TV.

This discussion raises profound questions about the ‘knowledge’, skills or capabilities developed in digital environments. Computer interaction of this sort is a thoroughly enactive neurocognitive mode involving iterative sensorimotor feedback loops. Learning in these contexts is sensorimotorically integrated—actions have consequences. Not only does this make learning really ‘stick’ (otherwise simulators of all kinds would be worthless) but the veracity of the representation of real-world physics becomes critical to what is learned. The false physics of a Coyote and Roadrunner is obvious cartoon slapstick, Wile E. Coyote constantly being blown up, falling off cliffs and being flattened by anvils. But when a child is piloting a car in a game where objects collide with other objects but incur no damage, and falling objects do not accelerate due to gravity, one has to wonder if the child is gaining false understandings or expectations of the way things behave in the real world of materiality and fundamental physics.³⁵

The question of the neurocognitive significance of screen-based digital interactions and the transferability of skills learned in simulation to the real world are vexing. How do we measure or characterize the ‘tacit’ sensorimotor ‘knowledge’ gained in driving virtual car? What does the user bring to digital environments from their experience of the real physical world that makes those environments intelligible? Does “real-world” experience and cognitive learning contribute to success in digital environments? Do skills gained in digital environments transfer to the real world, and if so, what qualities make them transferable? I suppose it depends on whether the qualities of digital environments have relation to, or are modeled upon, real-world environments and real-world physics. Are these questions even relevant for life lived in immersive digital workspaces? If we are neurocognitively wired for terrestrial physics as our evolutionary umwelt, only certain kinds of interactive cause- and-effect will make sense.

Professional e-sports gamers exhibit extraordinarily fast hand–eye coordination combined with intimate familiarity with the architectures of game worlds and their interfaces—highly attuned skills for a highly specialized environment. Is the question of transferability irrelevant because these are not simulations but actual work environments? Gaming is not that different from other kinds of contexts in which skilled practices occur in technologically encapsulated environments, such as race-car driving or piloting a drone—a radio-controlled toy or a military drone on the other side of the world. These are new cognitive ecologies (Hutchins 2010) and offer distinct challenges to (embodied) cognitive analysis because of their prosthetic, symbolic and remote aspects.

It is well understood by more senior engineers and architects that CAD tools are double edged. In an architecture or engineering design package, an environment of perfect Euclidean geometry is simply given: planes are perfectly flat, infinitely thin and precisely perpendicular. Dimensions and angles are automatically available. Objects automatically align. It is not necessary to know how to make measurements.³⁶ Taking the map for the territory is a real danger here. Any software simulation is a model, and a model is, by definition, a simplification and a rule-based abstraction. Users become adept at manipulating these abstracted environments, but there remain questions regarding the transferability of these skills of abstract manipulation to the real world.

The effectiveness of simulators in creating knowledge which is applicable to real physical tasks is not straightforward, it depends of many factors. If a child had a screen-based game in which they could learn to ‘put’ images ‘into’ an image, (ie images of lego bricks into an image of a cup), would this be cognitively equivalent to the physical activity? The belief in the effectiveness of this kind of learning is a premise of various kinds of educational software. This is a special case of the problem of the transferability of simulator-based learning, demanding a semiotic analysis of interactive simulated procedures.³⁷

7 Skill, again

Physiologically, if the gamut of manual activities is reduced to slapping a screen or poking a button, a diverse range of sensorimotor acuities and capabilities will have no opportunity to develop. Much has been made of Martin Heidegger’s notion of ‘ready to hand’. What is usually absent from that discussion is a recognition that ‘ready to hand’ implies the integration of an intact tool with a well-practiced bodily knowledge. ‘Ready to hand’ is not so much a quality of the tool but a quality of the user’s skill. A tool cannot be ‘ready to hand’ without a complementary corpus of proprioceptive, kinesthetic ‘knowledge’ specific to that kind of tool, the context such a tool is used in, and the behavior of the materials worked with it.

Consistent with the observation of Roger Kneebone and other specialists in diverse fields, in my experience of teaching sensorimotor skill-based practices for 30 years, I observe that student’s general familiarity with manipulating matter and sensitive handling of hand-tools has declined precipitously. Students are often demonstrably unfamiliar with activities in which precise movement and application of carefully judged force occur simultaneously—such as holding a screwdriver forcefully against the head of a screw while applying a rotating force to the tool and thus the screw. This kind of dexterity is ubiquitous in making, yet the majority of these students appear unfamiliar with this general capacity for effective action in the world.³⁸ As a corollary, I have noted that students who have some native sensitivity to tasks involving sensorimotor judgment and force often have training in traditional musicianship (playing guitar, violin, or piano) where ability to modulate manual force is critical.

Over the last 30 years or so, in elementary, secondary and tertiary education, hands-on practices have largely been replaced by text and screen-based modes. This is not in question. The question is whether valuable and generalisable cognitive lessons have thus been missed. If one approaches the question of education with a dualistic mindset, then abstract knowledge is what is important, and manual practices are just a means to the end. If, on the other hand, we embrace a ‘post-cognitivist” conception of cognition, where valid kinds of knowledge cannot all be classed as propositional knowledge, then the streamlining of technologically facilitated modes of pedagogy becomes a far more dubious enterprise. This was the argument Gilbert Ryle was making in his distinction between know-how and know-that (Ryle 1949).

8 User-friendly interfaces

Interface design has been directed by the principle of least effort, for clear (often market-driven) reasons—designing a product that was intentionally difficult to use would be commercial suicide. Ease-of-use has been a selling point for domestic commodities, from blenders to handguns. To make applications instantaneously ‘accessible’ they must have qualities we call user friendly and intuitive. These terms mean that minimal learning is required. In these hyper-designed tools, manual dexterity and sensorimotor sensitivity have been designed out. Digital tools are designed with the intention of de-skilling and dumbing-down.³⁹ Perhaps the appropriate guiding adage is “as simple as possible but not simpler”. No one expects to sit down at a piano and immediately be adept—we recognize the necessity of arduous training and practice in the attainment of skill or ‘chops’. Effective use of tools generally requires the development of skills and sensibilities specific to that tool—hence the traditions of apprenticeships and trade schools and ateliers and conservatoires.

In first-generation digital consumer technologies, existing pre-digital skills were leveraged: the QWERTY keyboard, complete with anachronistically named ‘shift’ and ‘return’ keys. Given the provenance of computers, these skills almost exclusively referred to the already rationalized office environment. Our interfaces are still populated with ‘desktops’ ‘folders’ and ‘wastebaskets’.⁴⁰ One can argue for the usefulness of analogies as ways of making new contexts comprehensible, but analogies are useful until they aren’t. As more time is spent among these interfaces, a Baudrillarian precession of simulacra occurs, in which each generation of interface abstracts elements of the previous generation. If user friendly and intuitive interfaces leverage preexisting skills, they induce conservatism in design because the market is (users are) ‘lazy’. Viewed from the perspective of this paper, this trend presents the specter of a cognitive downward spiral epitomized by the interface at the intake desk at St God’s hospital in the movie Idiocracy that has cartoon icons for injuries and medical conditions.⁴¹ A shortage of experience in embodied practices (broadly conceived), combined with overexposure to digital tools that are designed to preclude the necessity for fine neuromuscular judgment (in the interests of being ‘user friendly’ and ‘intuitive’) is generating sensorimotor debility.

There was no room in the interfaces of the original desktop computers for skills like slicing tomatoes or shuffling cards or riding bicycles. This is not a big deal if one adheres to an internalist explanation of cognition: they’re ‘just’ skills. But in any more expansive conception of cognition, it is not so easy to sort skill from intelligence, like wheat from chaff. This sensorimotor apartheid is rooted in a humanist assumption—as fallacious as it is fundamental—that there is a distinction between mind work and body work. But what if the mind–body binary, and related others like hardware–software, and skill-intelligence are simply wrong? If embodied, enactive (etc.) paradigms have credibility, then it behooves us to reassess the qualities of cognition and computer use in these terms. One would then be forced to undertake a philosophical excavation of technical practices of the kind recommended so long ago by Philip Agre (1997): a critical technical practice.⁴²

Embodied experience informs thinking—it provides the basis for a common-sense understanding of materials and terrestrial physics that informs design decision making (for instance, in the construction of a bridge). Clearly, as we adapt to new technologies, our capabilities and skills change. Some may argue that skills of map reading, like mental arithmetic, are just redundant in our technological context. Does embracing particular technologies and abandoning others cause more generalizable cognitive deficits? Is it possible to distinguish between technology-specific skills (like using an abacus or a slide rule or reading a clock face) and ‘abstract’ cognitive understandings which undergird not simply bodily skills, but our ability to form intuitions and utilize concepts and metaphors? Where do concepts come from? A traditional internalist explanation usually relies on some mysterious Platonism. According to George Lakoff and Mark Johnson, embodied experience provides a key source for metaphors and concepts applied in more abstract thinking (Lakoff and Johnson 1980). That is, the meaning of a statement like “the set of digits contains the number 3” is premised upon a concept of container and the possibility for things to be thereby ‘contained’—the kind of activity infants occupy themselves with.⁴³

9 Post-corporeal pedagogy

The general phenomenon of the digital, and in particular the overexposure of children, from a young age, to touch screens, graphical interfaces and the false physics of animation and games has created a context in which lived experience of material engagement has been reduced in a crucial period of cognitive development. Today these abilities are measurably diminished, but curriculum designers appear not to have noticed.⁴⁴ Sensorimotor competence has traditionally been taken for granted in pedagogical planning as part of the formation of students. This emerging debility has not been accommodated in pedagogical programs, which still assume this background of sensorimotor competence. On the contrary, the neoliberal rationalization of schooling, focusing on STEM learning, the increasingly academic and abstract tone of education in general has resulted in the general elimination of hands-on arts, crafts and vocational classes and facilities in the academy. In the academy, numerical and text-based scholarship continues to be the focus. These assume embodied competence and leverage concepts and intuitions that, traditionally, have developed in childhood.

As discussed above, the usefulness and convenience of apps are self-evident. We appreciate the convenience of navigating with interactive maps, sourcing obscure information almost instantly with a few carefully chosen keywords, or shopping online. By the logic of the market, this is why they survive and are profitable, while apps with clunky interfaces rapidly go extinct. These applications are popular because they make complex tasks simpler, and that’s why we like them. These packages deliver a success experience by making automatic fixes to user errors, hidden behind the GUI. Our word processors, email clients and text apps spellcheck, our cameras deploy sophisticated algorithms not simply to stabilize hand-shake and fix lighting problems, but more subtly, to identify the subject of the image (usually a face) and enhance it. Sensorimotor skills like focusing and managing depth of field all used to be part of photography. Now, the user generally has no idea the computer is doing so much behind the scenes to make the image ‘nice’.

Pedagogical software—being largely marketed by companies operating in similar commercial contexts to other consumer software development—is usually designed according to similar logic. There appears to have been little consideration whether such rationalizations are appropriate in pedagogical contexts. Software is conceived around the premise that the ‘customer’—in this case students—expects a success experience. To remain commercially profitable, pedagogical software must continually deliver a success experience. This cannot be good for students, for educational institutions or for society and economy at large.

The exigencies of neoliberal education often call for constraint of context to deliver the ‘lesson’ defined by the syllabus with surgical precision, unencumbered by tedious setup procedures. As such, fundamental aspects of acting in the world are abstracted away. It is a commonplace in conversations about technical educational software that there is so much to learn, it makes sense to strip away the tedium of laboratory setup and specification of components, to focus learning on the ‘more important’ abstract verities. But is it wise to elide those hands-on materially engaged practices, again reenacting Cartesian assumptions?⁴⁵ This is more than simply de-skilling, it gives users a falsely inflated sense of their own ability.

Education is by definition, and should be, inherently challenging. It is necessary to fail. Recognizing and analyzing failure makes the goal state explicit and articulates subtleties of the task that the student was previously unaware of, and provides a reflexive understanding of the student’s process. Successive attempts refine these abilities resulting in success experiences that have deeper value.

10 Conclusion

In Western cultures, the notion that there is a distinction between mind work and body work is deeply entrenched, philosophically rooted in the Cartesian mind/body dualism. The skill/intelligence distinction is a corollary and is similarly axiomatic and ideological. The (false) distinction between skill and intelligence has directed the development of technologies (and specifically technologies that are deemed ‘cognitive’), along paths that seek to minimize bodily engagement, dexterity, and physical effort. The rise of ‘information technologies’—themselves rooted in dualistic notions—has compounded the problem: the rapid development of computers and network infrastructure and the highly profitable nature of software, internet and social media ‘industries’ and the reconfiguration of entire economies around ‘data’ and datamining. The long-standing Enlightenment-humanist privileging of reason and of abstraction, combined with the emergence of a technology of abstract symbol manipulation, and neoliberal educational agendas that slash ‘soft’ or ‘applied’ aspects of learning (because they are expensive)—under the smokescreen of valorizing STEM when they are actually valorizing abstract symbol manipulation have created a perfect storm for sensorimotor competence. Leveraging post-cognitivist, embodied, enactive and distributed approaches to cognition to analyze human computer interaction can provides new insights into growing social and public health concerns around emerging computer-use issues. They provide this leverage because, in reasserting the holism of the cognizing organism, they destabilize axiomatic assumptions about the separability of mind and body, and thus of intelligence and skill.

Social and public health concerns have become tangible as a result of the slow and ongoing integration of these technologies into diverse aspects of life, and their longer term use. Here I have restricted myself to considering sensorimotor/cognitive issues, noting that in an internalist critique, such concerns are rendered obscure or non-existent. Emerging debilities of diverse kinds prompt a new kind of inquiry that demands transdisciplinary assessment. More substantial and more diverse studies of ‘digital native’ infant and child development are called for (this would imply establishing pre-digital ‘yardsticks’).

There are pressing physiological, psychophysiological, neurocognitive questions. There is a broad set of questions about the design of apps in popular usage and a similar set of questions to ask about pedagogical and professional software tools. The current period of covid quarantine and the concomitant rapid expansion of remote working and learning and (and socializing) only exacerbates and highlights these issues.