There is a disturbing discrepancy between the AI ethics frameworks that highlight the technology’s ability to promote the social good and the relationship between AI and humans as it has played out in one of the places where its deployment has been the most elaborate so far, Amazon’s fulfillment centers.

That AI should promote human beneficence is the first of five principles advanced in the Ethical Framework for a “Good AI Society” (Floridi et al. 2018). This framework declares that AI offers opportunities for the promotion of human dignity and flourishing in four fundamental ways: by the advancement of autonomous self-realization, “who we can become”, by promoting human agency, “what we can do”, by enhancing individual and societal capabilities, “what we can achieve”, and by contributing to societal cohesion, “how we can interact with each other and the world”.

Warehouse robots have been described with similar optimism. Former CEO of Amazon Worldwide Consumer, Jeff Wilke, has tweeted that such robots make warehouse jobs “better and safer”, and he has said that they allow “people to take advantage of their innate human creativity instead of doing rote things over and over again” (Evans 2020). Dave Clark, then Amazon’s Operations Chief, stated: “It’s better for everybody” when Amazon’s warehouse in California was equipped with new robots in 2014 (Evans 2020). These robots use cloud-based AI-software for sensing and navigation around the warehouse.

The reality is more complicated than that. AI-powered technologies have two features that are particularly important in this context: they have considerable limitations when it comes to dexterity, and considerable capacities when it comes to analyzing data about human behavior. Amazon’s director of robotics fulfillment, Scott Anderson, has stated that a fully automated warehouse operation is yet a decade away (Statt 2019). Putting merchandise on shelves and packing them into boxes will be particularly hard to replace. Hence, robots generally cannot match the quick and flexible fingers and eyes of humans when they handle objects as diverse as balls, pens and sweaters, but cameras and algorithms can cheaply evaluate and control human behavior by surveillance and continuous feedback.

These features are mirrored in the working conditions at Amazon’s fulfillment centers. After the introduction of robots in 2014, workers no longer had to walk to find customer orders. Instead, they remained at one workstation where they repeatedly picked and stowed merchandise onto shelving pods, which  robots delivered to them. However, this implied more repetitive tasks than before. Isolation at a workplace meant that workers interacted less with each other. Software tracked how many items they were scanning every hour, encouraging them to work faster. Gamification was a method to keep them make rates. Faster and more accurate picking was rewarded, and if their rate was too low, workers could be disciplined. Production quotas increased after robots were introduced. One ex-employee has stated that rates went from 120 items when they started, to 280 three years later, and that error rates allowed decreased from one per 1000 to one per 2200 items.

This seems to have come with a surge in workplace injuries. Weekly data from more than 150 Amazon warehouses have shown that the number of serious injuries, which is defined as injuries that require days away from work or a job restriction, increased by 33% between 2016 and 2019 to 7.7 per 100 workers (Evans 2020). For each of the investigated years, the number of injuries were higher in the roboticized warehouses. Injuries spiked around Prime Day, an annual deal event. In 2020, the rate of serious injuries was almost 80% higher in Amazon’s warehouses than in non-Amazon warehouses, according to a report from 2021 by the Strategic Organizing Center, a coalition of four labor unions in the US.

A majority of workers’ compensation injury claims in Southern California are related to ergonomic risk factors, such as lifting and repetitive motion (Delp et al. 2021). The underlying cause is primarily the production quotas and the relentless work pace, according to the workers’ own accounts. A New York Times article has reported that the turnover among Amazon’s hourly associates amounted to 150% a year, even before the pandemic, which is nearly twice as high as that of the retail and logistics industries in general. These high rates suggest that that the work pace might not be sustainable.

The robots have generated huge benefits for a company in which fast shipping is key. Amazon can now hold more stock, retrieve it faster, and reduce the cost of fulfillment. Increased automation is essential to this development. In 2012, Amazon acquired Kiva Systems, a warehouse robotics company, which it swiftly renamed Amazon Robotics. Recently, it bought Canvas Technologies, a company that specializes in autonomous robotic carts in warehouses. Currently, the company has more than 200,000 robots (Tech Vision 2020).

With the vision of human flourishing as a backdrop, the challenges arising from AI’s deployment at one of the leading AI-companies stands out in stark contrast. They suggest that what humans may sometimes become with AI is monitored and injured, what they can do is repetitive tasks, what they are capable of is fatigue, and that societal cohesion is hampered by a dearth of opportunities to talk to other humans. They suggest that the relationship between agile humans and rigid robots may sometimes result in the withering, rather than the flourishing of human capabilities.

Machines may well be able to replace the worst parts of human labor, tasks that have been denoted as 3d: dirty, dangerous and demeaning, but whether this will happen will depend on the strengths of different actors in the labor market. The adoption of AI in an industry that is characterized by a just-in-time model with flexible labor may not be consistent with the conditions that a good AI society presupposes. It follows that the positive social impact of AI at work is primarily a political rather than a technological issue.

Recently, the executive chairman of Amazon, Jeff Bezos, recognized in his letter to shareholders that the company has to improve conditions for its employees. This is promising, and a reminder that human flourishing in the age of the AI machine will not come about automatically. It demands due attention from those in control of the technology. In this era, the essential tension is not the one between humans and technology, but between the humans who make decisions about it and those who need to wrap themselves around it to stay afloat.