Political Machines: Ethical Governance in the Age of AI

2.1 Ego depletion

Experiments have shown conclusively that all variants of voluntary effort – cognitive, emotional, or physical – draw at least in part on a shared pool of mental energy (Kahneman 2011, p. 41). Typically, after exerting ourselves in one task, ego depletion means that we do not feel like making an effort in the next one (although this can be overridden with a strong incentive).

One well-known report (Danziger et al. 2011, pp. 6889ff.) demonstrated worrying depletion affects amongst a group of eight Israeli judges reviewing applications for parole. Despite the default decision for such an application being denial – only 35 % are generally approved – experimenters found that after meal breaks the number of requests granted leapt to 65 %. They also dropped steadily during the two hours before the judges’ next meal. The implication being that tired and hungry judges may fall back on default positions.

2.2 The halo effect

The sequence in which we observe people is determined by chance, and yet the ‘halo effect’ (Thorndike 1920, pp. 25ff.) weights first impressions so heavily that later information can be completely wasted. Asch (1946, p. 272) asks what the reader thinks of Alan and Ben?

Alan: intelligent – industrious – impulsive – critical – stubborn – envious

Ben: envious – stubborn – critical – impulsive – industrious – intelligent

Most view Alan more favorably, though they both have identical traits. The stubbornness of an intelligent person may be seen as justified and educe respect, but intelligence in an envious and stubborn person makes him dangerous (Kahneman 2011, p. 82). We are naturally attracted to coherence and convincing stories, and thus interpret each later trait in the context of the earlier ones.

Researchers have also found that the same halo effect can compel us to find physically attractive people more likeable (William Lucker, Beane, and Helmreich 1981, pp. 69ff.), as well are more politically knowledgeable and persuasive (Palmer and Peterson 2016, pp. 353ff.) because of the increased weight of first impressions.

2.3 The affect heuristic

Humans can sometimes give primacy to conclusions when our emotions are involved. Psychologist Paul Slovic termed this the affect heuristic, in which our existing likes and preferences (e. g. political allegiance) determine our beliefs about everything else. Moreover, if we dislike something passionately, then we are likely to believe that its risks are higher, and its benefits are negligible (Finucane 2000).

Slovic ran an experiment (1999, pp. 689ff.) which surveyed opinions on a number of technologies, asking participants to list the pros and cons of each. He found that when someone approved of a technology they perceived minimal risks and large benefits. When they disliked it, they struggled to find any advantages.

However, it should be noted that when respondents were shown positive passages about the technologies they initially disliked – either describing ample benefits or low risks – the emotional appeal also changed. Those who learned about the benefits also changed their minds about the risks, despite receiving no further information about the latter. Similarly, those who learned that the risks of a technology were mild, also developed a more favorable view of its benefits, simply as a byproduct.

2.4 Belief overkill

Baron (2009)^[2] demonstrated a similar cognitive phenomenon with online opinion polling data about hypothetical political candidates.

Upon making the initial observation that many US Republican Party voters were originally attracted to the Party for its stance on social issues (e. g. abortion) but came to espouse the party’s economic positions too, Baron set out to prove what he termed ‘belief overkill’ – the idea that when some decision is salient, once a voter moves towards an option they bring all of their other beliefs into line so that none of their beliefs oppose the favored option.

In his study, Baron presented subjects with two separate political positions which were attributed to same political candidate and asked for an overall judgment of that candidate as well as a judgment about how each position affected the candidate’s attractiveness. Confirming Baron’s hypothesis, the subject’s appraisal of one issue predicted their evaluation of the other and of the candidate. He concluded that our political beliefs seem to come in ‘pre-packaged’ bundles.

2.5 The law of small numbers

We are prone to exaggerate the consistency and coherence of what we see, even on limited evidence. This means we tend to concentrate on the content of any given message, rather than its reliability (Kahneman 2011, p. 118). Such a flaw is particularly problematic when it comes to small samples of information, as they often give more extreme and misleading results than larger samples.

Wainer and Zwerling (2006, pp. 300ff.) give the example of a large investment made by the Gates Foundation on the basis of research into the most successful schools. The study, which looked at 1,662 schools in Pennsylvania, found that the best schools were usually small in size. This prompted the foundation to plunge $1.7 billion into the creation of more small schools (even splitting some existing schools in the process). They were joined in their enthusiasm by at least half a dozen other agencies and a US government department.

Though this move seems reasonable – small schools should be able to give children more time, attention and encouragement – the causal analysis doesn’t quite work. Had researchers also looked at the characteristics of poor performing schools, they would’ve found they also tended to be small. Small schools are not better, they are just more variable, but cognitively we wish to simplify the findings and make the world more intelligible than the data justify.

2.6 Availability cascade

An availability cascade is the mechanism by which policies become infused with biases and the allocation of resources can be distorted (Kuran and Sunstein 1999, p. 683). It refers to a chain of events, often beginning from media reports of a relatively minor event, which escalate into public panic and government action. In these cases, availability provides a heuristic, which leads to judgments based on fluency and/or emotional charge when ideas come to mind.

In the late 1980s the chemical Alar was sprayed on apples to regulate their growth and improve their appearance. Press stories then emerged about tumors occurring in rodents that had consumed huge amounts of the chemical. A frightened public reacted, which in turn fueled more media. Apples became objects of fear, Meryl Streep gave testimony before Congress, the manufacturer withdrew the product and the FDA banned it (Kahneman 2011, p. 143). Though subsequent research confirmed the substance could pose a very small risk as a mild carcinogen, this was still an overreaction to a minor problem. The phenomenon has been called ‘probability neglect’ (Sunstein 2002, p.63), an over-attendance to a minor risk due to the ease and frequency with which it comes to mind.

4.1 Arguments for AI

Writing for Wired magazine, Joshua Davis (2017) made a (tongue-in-cheek) case for an ‘AI president’. As his main support, he cites the superiority of computational judgments over human ones, which – as we’ve seen – can be problematic. Of late, he argues, advanced AI has demonstrated that it can cut through complexity even more easily than some of the most sophisticated human minds:

Over the past 12 months, an AI built by Google has won 60 games in a row against the world’s best Go players. To do this, it had to master a game that is far more complex than chess. (There are more possible Go games than there are atoms in the known universe). The AI faced a huge array of choices and had to think dozens of steps ahead. It needed to make difficult decisions, fashion a strategy that involved risk and operate with incomplete information. It did all this, and it also innovated. “It won by doing things we hadn’t seen before,” says Myungwan Kim, a professional Go player. “We thought it would take 50 years for software to beat the top players in the world but, over five months, this program became the best player in the world.”

Davis pondered further:

Imagine an AI presidency in 2003. The software would have analyzed decades’ worth of reports on Saddam Hussein, absorbed the intelligence about WMDs, and concluded that an invasion of Iraq was obviously a dumb idea and unlikely to spread democracy. Ditto Vietnam. ^[3]

As Davis acknowledges, the complexities of the world cannot fairly be compared to a game of Go. Nevertheless, clearly there are good reasons to favor AI over human judgment. I believe that these reasons become even stronger when we consider the potential ramifications of political judgments; this is what leads me to insist that AI systems should be an integrated part of governmental decision-making.

The first argument for AI has already been made. Unlike human judgment and cognition, decision-making systems are not at risk of the halo effect, ego depletion, powerful emotion, or the other challenges outlined above. AI systems do not get tired, suffer from low blood sugar, or let subjective personal beliefs warp their perspectives on important issues. In this sense at least, they seem to be objective.

The second reason to prefer AI, which perhaps flows naturally from the first, is because artificial intelligence is often extremely accurate in its predictions and judgments. Where it outperforms humans, it is usually by some considerable margin. This accuracy is not even unique to AI or machine learning. It is simply the case that statistical predictions tend to triumph over those of human experts. This has long been known. The famous comparison between clinical predictions and statistical predictions, first noted by Meehl, demonstrates well the preeminence of the latter (1954). Kahneman writes that even after 50 years, and roughly 200 studies, there are still no convincingly documented examples of human intuitions besting algorithmic forecasts in this area (2011, p. 223).

Tetlock’s experiments with political experts also found that it was impossible to find any domain in which humans clearly outperformed even crude extrapolation algorithms, less still sophisticated statistical ones (2017, p. 54). He asserted that – unlike algorithms – humans experts dig themselves into intellectual holes, basing the probability of something happening on nothing more than their worldview. Interestingly, Tetlock describes a human dilemma which presupposes an advantage for data-driven AI, explaining with regard to human judgment: ‘if we only accept evidence that conforms to our worldview, we will become prisoners of our preconceptions, but if we subject all evidence, agreeable or disagreeable, to the same scrutiny, we will be overwhelmed’ (2017, p. 19). Of course, our superfast processors now can examine all available evidence and – just like the type of pundits he finds to have better political judgment (‘foxes’) – this means their survey of the world could be more balanced. Furthermore, just like foxes, AI is a ‘belief updater’ by its very nature.

Many years on from Meehl’s early discovery about statistics, AI now deploys automated algorithms to supplant human judgment in a growing number of domains. Recently, Stanford University created a machine learning algorithm which can identify lung tissue slides exhibiting a specific type of cancer with a far greater accuracy than human pathologists, two of whom will only agree 60 % of the time (Yu et al. 2016, p. 2). Meanwhile, Northwestern University developed a model that performs as well or better than 75 % of Americans on a standard intelligence test (Lovett and Forbus 2017, p. 77). Even the CIA has begun to employ ‘anticipatory intelligence’, which in some cases allows it to detect social unrest from open datasets as far as three to five days out (Russan 2016). ^[4]

As I’ve already touched upon, another key advantage of artificial intelligence is the extraordinary speed at which it is capable of processing material. A video-game playing ‘bot’ recently completed 19,000 actions per minute (Kim and Lee 2017). ^[5] This speed is critical, given that AI systems can now draw upon information that is infinitely broader in range and more complex in detail than a human could ever reasonably make sense of (thanks largely to the increasing ‘datafication’ of our world). The combination of speedy technology and the avalanche of so-called ‘big data’ means that computational, algorithmic systems can usually outpace human analysis of the available information cues. It is precisely this speed and almost unlimited scope that allows AI machine learning systems to become so accurate.

4.2 A caveat

Having noted these ‘pros’, I do want to nuance the point I am stating. I do not wish to imply that AI systems are always preferable to human decision-makers. It is important (and only fair) to note that there are scenarios where human instincts, based on experience, can be very reliable indeed. In their joint effort, ‘Conditions for Intuitive Expertise: A Failure to Disagree’, Kahneman and Klein (2009, pp. 515ff.), though rivals on the topic, agreed the conditions under which human judgment is very reliable:

1. When an environment is sufficiently regular to be predictable; and

2. When there has been an opportunity to learn these regularities through prolonged practice.

If these are satisfied, intuition is likely to be skilled. Into this category of decision-maker might fall physicians, athletes, nurses, and firefighters, all of whom face what Kahneman calls ‘complex but orderly situations’ (2011, p. 240), also known as ‘high validity environments’ where we can find stable relationships between objectively identifiable cues and subsequent events, or between cues and the outcomes of possible actions. In contrast, in ‘zero validity environments’ outcomes are unpredictable. As an example, Kahneman and Klein offer that ‘predictions of the future value of individual stocks and long-term forecasts of political events are made in a zero-validity environment’ (2009, p. 524).

There are two observations here which are decisive for my argument. First, that Kahneman and Klein do not append politicians and policymakers to this list of skilled decision-makers. The unpredictability the future scenarios they face (e. g. the fluctuation of economies, or the movements of foreign governments) are not accessible via intuition. Second, the agreement of these rival scholars that people perform significantly more poorly than algorithms in low-validity environments (2009, p. 523). Corroborating this is an analysis of 136 studies comparing the accuracy of clinical and mechanical judgments, mostly in noisy and highly complex environments (Grove et al. 2000, pp. 19ff.). Though the algorithms were often still wrong, in about half of the studies they prevailed. In the other half there was no difference.

Add to this evidence that algorithmic systems also perform extremely well in high-validity environments, and it seems fair to conclude they are as good an option as humans – and frequently better – when it comes to making judgments, including political ones.

5.1 Aversion

Many of those who do not want automated decisions to become an integral part of governance do so because of a common, visceral aversion which is usually furnished with a causal story about fearing the demotion or displacement of humans as a species. A kind of unqualified moral repugnance (Kass 1997, pp. 17ff.).

This is, in itself, a bad intuition which leads many to vastly overweight the perceived threat of AI. The idea of becoming subject to rule-by-system, or even rule-by-robot (as per recent media hype) could lead us to reject perfectly reasonable technological advancements. It is a kind of ad hominem argument. It is certainly a slippery slope argument.

A study by Dietvorst et al. (2015, p. 114) demonstrated the phenomenon of what has been termed algorithmic aversion. Their work found that subjects mistrusted algorithmic predictions even when they witnessed them outperforming humans. Similarly, subjects lost confidence in algorithmic predictions much more quickly than they did in human forecasters.

Meehl points out that this baseless mistrust is not just a misguided position, but also an unethical one. He writes in defense of algorithms:

If I try to forecast something important about a college student, or criminal, or a depressed patient by inefficient rather than efficient means, meanwhile charging this person or the tax 10 times as much money as I would need to achieve greater predictive accuracy, that is not sound ethical practice. That it feels better, warmer and cuddlier to me as a predictor is a shabby excuse indeed. (1986, p. 374)

A preference for what is natural and human, is just that: a preference, and not an argument. If we have developed better methods (as is happening all the time), we should employ them. It is a moral obligation in the case of governance. Even if this means AI must encroach on our notions of human essentialism as well as our preciously guarded status professions.

More extreme long-term fears about unruly or despotic AI are fairly easily defeated. It is worth pointing out that even the most prominent voices warning against the existential threat of what might be termed true AI (e. g. scientist Stephen Hawking, Microsoft founder Bill Gates, and celebrity tech-speculator-in-chief Elon Musk), do not use this concern as a basis to resist the deployment of all intelligent systems.

In response to the type of AI aversion typical of those who fear future dystopias or the much-talked-of ‘Singularity’, I defer to Floridi (2016) who remarks:

How some nasty ultraintelligent AI will ever evolve autonomously from the computational skills required to park in a tight spot remains unclear. Climbing on top of a tree is not a small step towards the Moon; it is the end of the journey. ^[6]

5.2 Concern

Much worthier of close consideration is the concern that, in practice, data-driven AI decisions could be responsible for actively encoding and propagating pernicious biases while continuing to appear objective. If unfair machines were at the center of a political system, it follows that this could result in distinctly unethical action. Any potential skew could wrongfully target or marginalize citizens and communities at scale and in direct contradiction of Brennan’s competence principle.

Our current AI systems issue biased decisions for a number of reasons; sometimes they are trained on datasets that already contain bias, other times datasets are simply incomplete or unrepresentative of the categories they purport to represent. Moreover, these unthinking goal-oriented systems do not ‘care’ what harms they cause in achieving their end objectives which are, of course, determined and programmed by the necessarily biased humans. Data scientist and social activist Cathy O’Neill gives the full and disturbing detail of how data-driven systems can ruin lives in her epoch-defining book, Weapons of Math Destruction (2016).

This is a legitimate fear, but it is critical that current imperfections do not become an obstacle to all or any serious consideration of future uses. My ‘moral obligation’ argument is conditional and only obtains if we, as a society, are successful in developing ethically sensitive AI. I choose to look ahead to the idea of ‘political machines’ because the outlook for this condition is improving. AI ethicists have described a burgeoning focus on mitigating troubling effects (like bias) as ‘promising’, adding that there is growing movement seeking to develop AI that is attuned to underlying issues of fairness and equality (Campolo et al. 2017).

As noted, biased AI judgments often arise from data problems rather than the system itself, and at the time of writing measures are being taken to try to rectify this. For example, the practice of supplementing narrow datasets with synthetic data to ensure full representation of all ‘types’ in a spectrum (Zemel et al. 2013), and the creation of new de-biasing algorithms that can remove embedded gender stereotypes from collections of language data (Bolukbasi et al. 2016, pp. 4349ff.)). While such methods are still largely untested, practitioners are also trying to find new ways of soliciting error feedback through algorithmic audits and even citizen engagement where systems are used in the public domain.

Technical researchers are also in the early stages of thinking about how we might build human morality into AI (Conitzer et al. 2017). This might mean – for example – that a system used to determine whether a new rule will save money will refrain if it can also predict a bad social outcome for a minority group. This involves coding for our moral values right at the development stage of AI.

Within this scope, recent research by Vamplew et al. (2018, pp. 27ff.) considered how ethical values (both utilitarian and deontological) might sit among the competing objectives within a human-aligned AI system. Their discussion advocates for a MOMEU (multi-objective maximum expected utility) approach which explicitly allows a system designer to specify desired trade-offs between components of utility. They claim that this approach could also help avoid the risks of unconstrained maximization and the exploitation of goal-oriented systems, either by separating out AI behaviors into separate utility functions or by developing separate independent utility functions each with the same aim. This research builds on the multi-objective approach developed by Keeney (1973, 1988, 1996). As far back as 1971, Keeney attempted to ‘systematically attack’ a public project – an airport – in Mexico City, and used his (non-automated) technique to balance competing factors of geography, political context, finances, and issues like traffic, displacement and noise (1973, pp. 101ff.).

Beyond mere technical mitigation, there are also calls for greater diversity with regards to precisely who creates AI models, given that those who design, develop and maintain AI systems will shape them in-line with their own understanding of the world (Campolo et al. 2017, p. 37f.). Campaigners and experts are also pushing for a much better understanding of the provenance of datasets – including all embedded socio-political salience – through interdisciplinary collaboration. The same cohort argue that public agencies should not be allowed to use ‘black box’ AI or algorithmic systems, and that any system used should be subject to ongoing tests (including public auditing) and constant monitoring across different contexts and communities (2017, pp. 2ff.).

In sum, many minds – scientific, ethical, sociological, philosophical – are now engaged in developing more moral, fair, diverse, and transparent AI. If they are successful in producing a decision-making system which is at once accurate and ethical, then this concerned resistance will presumably give way to vigilant operational scrutiny.

7.1 Determining ‘the best’ outcome

In his assessment of political judgment, Tetlock acknowledges that if it were easy to set standards for judging judgment that would be honored across the opinion spectrum someone would have patented the process long ago (2017, p. 1). Political judgments are inarguably among the most controversial of them all, and yet that does not mean we cannot assess their validity. Like Tetlock, I wish invoke Lewis (1973) discussion of counterfactuals and their use in a method that helps remove doubt and establish that a political decision resulted in ‘the best’ of all possible outcomes. Tetlock argues that we have a warrant to think of a policy as great when we can only think of ways that it would’ve worked out for the worse, and there is already a substantial corpus of literature dedicated to exploring these kinds of proofs (Tetlock and Belkin 1996, p. 4). I venture that AI developers could use similar technology to evaluate decisions ex post.

Typical counterfactual arguments look for a causal link between an antecedent and consequent by constructing alternative causal paths and imagining ‘what could have been’. Lars-Erik Cederman considers how this could be applied in hindsight to moments in history and made the subject of computer simulations able to randomize or manipulate many variables to replicate real world conditions in a series of complex counterfactuals (1996, pp. 247ff.). From there users could winnow out which combination of influencing factors give rise to ‘the best’ possible outcome by all measures. As Cedarman argues, it is tempting but fallacious to regard actual history as the inevitable causal path. Counterfactuals could help both decision-makers and algorithms learn and amend.

This type of evaluation is undoubtedly useful, but in the end, it still requires that we collectively agree which of these possible other worlds is objectively better. In many cases, the advantage will be obvious. War or peace. Improving schools or badly performing schools. Higher or lower employment figures. However, others will be less convincing and the counterfactuals themselves will cause contention. For example, imagine an instruction to forge closer ties with a previously hostile foreign government; this could be viewed a) as a positive move towards peace and good for trade, or b) a negative and dangerous allegiance that could alienate other, closer international allies. Such conundrums make the problem of political contention seem intractable.

In response to this I still reject any suggestion that a counterfactual evaluation system would require a further layer of evaluation. Our requirement was only ever that AI took better decisions than those proposed by small groups of humans, not perfect decisions. If it is not easy to discern whether the system chose the best of all alternative options, then as a corollary it cannot be easily discernable that it did any meaningful damage.