BSG policy memo Page 1 Killer Robots: Regulate, don't ban Executive summary Lethal Autonomous Weapon Systems ('LAWS') are here. Technological development will see them become widespread in the near future. This is in a matter of years rather than decades. When the UN Convention on Certain Conventional Weapons meets on 10-14th November 2014, well-considered guidance for a decision on the general policy direction for LAWS is clearly needed. While there is widespread opposition to LAWS-or 'killer robots', as they are popularly called-and a growing campaign advocates banning them outright, we argue the opposite. LAWS may very well reduce suffering and death in war. Rather than banning them, they should be regulated, to ensure both compliance with international humanitarian law, and that this positive outcome occurs. This policy memo sets out the basic structure and content of the regulation required. The technology is in use and developing fast The characteristic forerunners of this technology are drones armed with missiles. At present, a pilot flies the system remotely; intelligence analysts assess the target information provided; and military commanders supported by legal advice make the decision to attack. Current R&D programmes aim to automate these processes for systems deployable on land, at sea, underwater and in the air. In time, a system's movement, target acquisition, and ultimately decision to kill, could be made autonomously. (For overview and research plans until 2038, see US BSG Policy Memo November 2014 Policy recommendations z Expand remit of war crimes legal regimes-both international and national-to include illegitimate distribution and use of LAWS z Establish national and international technical standards and licensing bodies for LAWS z Enshrine compliance with International Humanitarian Law's principles of distinction and proportionality, in the technical standards and licensing bodies This Policy Memo represents the views and recommendations of the author(s). They are not necessarily held or endorsed by the Blavatnik School of Government. Taranis Drone Copyright © 2014 BAE Systems. All rights reserved. http://www.sophia.de http://orcid.org/0000-0002-4144-4957 BSG policy memo Page 2 Department of Defense 2013; also Singer 2009 and Birmingham Policy Commission 2014.) Some LAWS are already in use. The 'Phalanx' Close-In Weapons System used to defend ships can autonomously identify and attack oncoming missiles, rockets, artillery fire, aircraft and surface vessels according to criteria set by the human operator. Similar systems exist for tanks. These systems could not perform their defensive function if they needed human authorisation to engage each target. Fundamental principles to clear the muddy moral waters The basic case in favour of LAWS is simple: We can spend treasure, to save blood. Fewer lives lost at war is a valuable thing. So is better digital documentation of war and thus prosecution of war crimes. Just as it is culpable negligence to send in the infantry without body armour and helmets, so-other things equal-it is also negligent to fail to develop LAWS. The most significant moral question arises about the attribution of responsibility for LAWS' killings. When things go wrong and civilians are killed, who is to blame? We propose that responsibility for the effects of LAWS should be attributed in exactly the same way as any other technological system. Consider medicines. These have generally predictable results but with a risk of negative side-effects. So drugs are tested during development and only then licensed for prescription. When prescribed in accordance with the guidelines, neither the doctors nor drug companies are responsible for side-effects (though reliability for defective manufacturing, defective design or failing to warn still holds). Instead, the body that licenses the medicine, such as the FDA or NICE, is responsible for ensuring overall beneficial outcomes. Apply this to LAWS. The same division of responsibility occurs between engineers, military users, and the government. These implications follow: z There is pressing need for a regulatory regime to govern LAWS, both legally and in terms of technical standards. This is a moral responsibility of governments and international organisations. z The deployment of LAWS is acceptable only if that system reduces risk to both combatants and civilians. Risk reduction is the overall beneficial outcome that justifies their development. It is not permissible to reduce risk to soldiers by increasing that to civilians. z LAWS are weapons of war, and so they must comply with the regulations of International Humanitarian Law (IHL) in the Geneva Conventions. In particular, they must comply with the principle of distinction, viz. have the ability to discriminate combatants from non-combatants. They must also comply with the principle of proportionality, viz. that damage to civilians is proportional to the military aim. Current legal regimes can provide the appropriate regulatory mechanisms Regulating LAWS will be challenging in practice. This is because their ability to distinguish legitimate from illegitimate targets, and attack in a proportionate manner, is not in the direct self-interest of a particular nation at war. For a particular nation, deployment of LAWS reduces the risk to their own soldiers, and there is a temptation to do so by 'off-loading' risk onto foreign civilian populations. So long as legitimate targets are attacked, tactical military effect is still achieved regardless of how many non-combatants LAWS also attack. 'Shoot everything that moves' is the logical extreme, but less crazy attack parameters still off-load risks onto civilians. As such, LAWS which reduce risk to civilians are an international public good: everyone has reason to prefer that everyone uses only technologies which comply with this requirement, but nations have no individual interest in ensuring their own compliance. We propose that the same legal regime which enforces accountability for individual war criminals be applied also to LAWS. War crimes have the same international public good incentive structure. Moreover, victimisation of noncombatants for the sake of reducing risk to one's own soldiers is a war crime. Internationally agreed standards on conduct in war, notably the Geneva Conventions, are incorporated into domestic law. Domestic judicial institutions are then primarily responsible for the enforcement of these standards. Where states are unable or unwilling to enforce these standards, an international body (the International Criminal Court) has 'back-up' jurisdiction. "There is a strong moral imperative for ensuring that regulation precedes tragedy, rather than coming only as a response." BSG policy memo Page 3 Legal and technical regulation International National Legal International Humanitarian Law Criminal Law Technical Technical standards for performance Control regimes for techical standards; national-specific standards Five policy recommendations for regulating LAWS 1. Establish an international technical standards agency for LAWS LAWS's reliability in meeting principles of distinction and proportionality is in part a moral and political judgment. It is then expressed in technical specifications. An international standards agency is required to establish and maintain those specifications as the technology evolves. In particular, it must ensure a high degree of success for LAWS in distinguishing legitimate from illegitimate targets, and attacking the former only when it is proportional and necessary to do so. 2. Establish national technical standards and licensing bodies for LAWS National or supranational regulatory bodies are required to then incorporate, and/or establish domestic standards of technical specifications, issue licenses and control manufacturers. The institutional burden here is less than it seems. This is likely best fulfilled as an additional function of those bodies that will have to be established for civil autonomous robotics anyway-e.g. autonomous cars, search & rescue robots, autonomous mini-copter postal services, etc. (see Palmerini et al. 2014 for the case of the EU). The same point applies to the wider need for an international standards agency on robotic technology. 3. Extend war crimes legal instruments to the illegitimate use of LAWS Alongside national and international standards bodies, a legal regime is required to enforce compliance. We propose that the remit of existing legal instruments for war crimes be extended to include the illegitimate use of LAWS. Illegitimate use includes: the distribution of non-standards-compliant LAWS; the use of nonstandards-compliant LAWS; and the intentionally malign or culpably negligent use of LAWS. 4. Permit the distribution of LAWS only when there is better-than-human performance Civilians must be exposed to less risk when LAWS are pursuing military targets than when humans are doing the same. Unless this is so, it is an unfair redistribution of risk, exploiting civilians' vulnerability. So LAWS must achieve rates of compliance with the IHL principles of distinction and proportionality that are better than humans' in order to be deployed. Contrary to popular belief, it is wholly plausible that some LAWS will reach this standard in the near future. For example, tanks are useful solely for military purposes, and they give distinctive and recognisable heat signatures that can be used for targeting. LAWS are different from other weapons, in that war crimes can not only be committed by users, but also by manufacturers. A manufacturer who distributes a system that does not meet the standards of distinction and proportionality is also guilty of a war crime. Therefore, technical standards of performance must be established and enforced through public licenses. In this regard, the instructive parallels are the national and international bodies that govern other important technologies, such as the International Civil Aviation Organisation. US Navy: A common unmanned surface vehicle patrols for intruders during Trident Warrior 2011 BSG policy memo Page 4 Timeframe for implementation There is general agreement, on both sides of the debate, that the 10-14th November 2014 meeting of the UN Convention on Certain Conventional Weapons is propitious for a decision on the general policy direction for LAWS. For the 'ban now' camp, it is better to stop up the bottle than try and put the genie back in once escaped. For the 'regulate' camp-where we sit-there is a strong moral imperative for ensuring that regulation precedes tragedy, rather than coming only as a response (cf. Heyns 2013). The period till 2020 constitutes a realistic and necessary timeframe to put the basic regulatory structures in place. The arguments stated here are developed in detail in Autonomous Killer Robots Are Probably Good News, Müller and Simpson (http://bit.ly/1zzTrra), and Just War and Robots' Killings, Simpson and Müller (http://bit.ly/111vV8v). References Birmingham Policy Commission (2014), 'The security impact of drones: Challenges and opportunities for the UK', (October 2014). http://www.birmingham.ac.uk/research/impact/policy-commissions/remote-warfare/index.aspx. Heyns, Christof (2013), 'Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions', UN General Assembly, Human Rights Council, 23 (3), A/HRC/23/47. Müller, Vincent C. and Simpson, Thomas W (forthcoming), 'Autonomous Killer Robots Are Probably Good News', in Ezio Di Nucci and Filippo Santoni de Sio (eds.), Drones and responsibility: Legal, philosophical and socio-technical perspectives on the use of remotely controlled weapons (London: Ashgate). Palmerini, Erica, et al. (2014), Guidelines on Regulating Robotics, 22/09/2014 (http://www.robolaw.eu/) 215pp. Simpson, Thomas W and Müller, Vincent C. (Draft 2014), 'Just war and robots' killings'. Singer, Peter (2009), Wired for war: The robotics revolution and conflict in the 21st Century (New York: Penguin). US Department of Defense (2013), 'Unmanned Systems Integrated Road Map FY2013-2038'. http://www.defense.gov/pubs/ DOD-USRM-2013.pdf. Authors (in alphabetical order) Vincent C Müller is Professor of Philosophy at ACT/Anatolia College in Thessaloniki. He is a former conscientious objector (to German National Service). His website is at www.sophia.de and he can be contacted at vmueller@act.edu Thomas W Simpson is the Associate Professor of Philosophy and Public Policy at the Blavatnik School of Government, University of Oxford, and a Senior Research Fellow at Wadham College. He formerly served as an officer with the UK Royal Marines Commandos, deploying to Iraq, Afghanistan and Northern Ireland. He can be contacted at thomas.simpson@bsg.ox.ac.uk. The Blavatnik School of Government at the University of Oxford exists to inspire and support better public policy and government around the world. www.bsg.ox.ac.uk | enquiries@bsg.ox.ac.uk Images: BAE Systems Taranis drone (Copyright © 2014 BAE Systems. All rights reserved) US Navy photo by Mass Communication Specialist Seaman Scott Youngblood 5. Permit the use of LAWS for killing only when there is compelling military reason Remotely-operated weapons systems achieve the same reduction of risk to a military's soldiers as automated systems do. By definition, remotelyoperated systems comply with IHL to the same degree as humans. So the uncertainty about non-compliance with IHL, that goes with LAWS, should be permitted only when there is compelling military reason to automate. There are three such reasons: 1) When the speed of cognition achieved by a computer exceeds that of a human, and the combat situation requires it, automated systems enjoy a tactical advantage over mannedor remotely-operated adversaries; 2) When the number of automated systems is so high that human operators cannot feasibly control them, as in 'swarm' concepts; and 3) When the environment is such that communication links between system and operator cannot be guaranteed, automation makes military operations possible.