Predictions about autonomous weapon systems are typically thought to channel fears that drove all the myths about intelligence embodied in matter. One of these is the idea that the technology can get out of control and ultimately lead to horrifi c consequences, as is the case in Mary Shelley’s classic Frankenstein. Given this, predictions about AWS are sometimes dismissed as science-fiction fear-mongering. This paper considers several analogies between AWS and other weapon systems and ultimately offers an argument (...) that nuclear weapons and their effect on the development of modern asymmetrical warfare are the best analogy to the introduction of AWS. The fi nal section focuses on this analogy and offers speculations about the likely consequences of AWS being hacked. These speculations tacitly draw on myths and tropes about technology and AI from popular fi ction, such as Frankenstein, to project a convincing model of the risks and benefi ts of AWS deployment. (shrink)

In this article, we focus on the scholarly and policy debate on autonomous weapon systems and particularly on the objections to the use of these weapons which rest on jus ad bellum principles of proportionality and last resort. Both objections rest on the idea that AWS may increase the incidence of war by reducing the costs for going to war or by providing a propagandistic value. We argue that whilst these objections offer pressing concerns in their own right, (...) they suffer from important limitations: they overlook the difficulties of calculating ad bellum proportionality; confuse the concept of proportionality of effects with the precision of weapon systems; disregard the ever-changing nature of war and of its ethical implications; mistake the moral obligation imposed by the principle of last resort with the impact that AWS may have on political decision to resort to war. Our analysis does not entail that AWS are acceptable or justifiable, but it shows that ad bellum principles are not the best set of ethical principles for tackling the ethical problems raised by AWS; and that developing adequate understanding of the transformations that the use of AWS poses to the nature of war itself is a necessary, preliminary requirement to any ethical analysis of the use of these weapons. (shrink)

In October 2011, the “2nd European Network for Cognitive Systems, Robotics and Interaction”, EUCogII, held its meeting in Groningen on “Autonomous activity in real-world environments”, organized by Tjeerd Andringa and myself. This is a brief personal report on why we thought autonomy in real-world environments is central for cognitive systems research and what I think I learned about it. --- The theses that crystallized are that a) autonomy is a relative property and a matter of degree, b) (...) increasing autonomy of an artificial system from its makers and users is a necessary feature of increasingly intelligent systems that can deal with the real-world and c) more such autonomy means less control but at the same time improved interaction with the system. (shrink)

Despite resistance from various societal actors, the development and deployment of lethal autonomous weaponry to warzones is perhaps likely, considering the perceived operational and ethical advantage such weapons are purported to bring. In this paper, it is argued that the deployment of truly autonomous weaponry presents an ethical danger by calling into question the ability of such weapons to abide by the Laws of War. This is done by noting the resonances between battlefield target identification and the process (...) of ontic-ontological investigation detailed in Martin Heidegger’s Being and Time, before arguing that the nature of lethal autonomous weaponry precludes them from being able to engage in such investigations—a key requisite for abiding by the relevant legislation that governs battlefield conduct. (shrink)

The development of complex autonomous systems that use artificial intelligence (AI) is changing the nature of conflict. In practice, autonomous systems will be extensively tested before being operationally deployed to ensure system behavior is reliable in expected contexts. However, the complexity of autonomous systems means that they will demonstrate emergent behavior in the open context of real-world conflict environments. This article examines the novel implications of emergent behavior of autonomous AI systems designed (...) for conflict through two case studies. These case studies include (1) a swarm system designed for maritime intelligence, surveillance, and reconnaissance operations, and (2) a next-generation humanitarian notification system. Both case studies represent current or near-future technology in which emergent behavior is possible, demonstrating that such behavior can be both unpredictable and more reliable depending on the level at which the system is considered. This counterintuitive relationship between less predictability and more reliability results in unique challenges for system certification and adherence to the growing body of principles for responsible AI in defense, which must be considered for the real-world operationalization of AI designed for conflict environments. (shrink)

In this article we focus on the jus in bello principle of necessity for guiding the use of autonomous weapons systems (AWS). We begin our analysis with an account of the principle of necessity as entailing the requirement of minimal force found in Just War Theory, before highlighting the absence of this principle in existing work on AWS. Overlooking this principle means discounting the obligations that combatants have towards one another in times of war. We argue that the (...) requirement of minimal force is an important requirement for considering ethical uses of force. In particular, we distinguish between lethal and non-lethal purposes of use of force and introduce the prospect of non-lethal AWS before reviewing a number of challenges which AWS pose with respect to their non-lethal use. The challenges arise where AWS generate unpredictable outcomes impinging upon the situational awareness required of combatants to ensure that their actions meet the requirement of minimal force. We conclude with a call for further research on the ethical implications of non-lethal uses of AWS as a necessary contribution for assessing the moral permissibility of AWS. (shrink)

In this article we focus on the jus in bello principle of necessity for guiding the use of autonomous weapons systems (AWS). We begin our analysis with an account of the principle of necessity as entailing the requirement of minimal force found in Just War Theory, before highlighting the absence of this principle in existing work on AWS. Overlooking this principle means discounting the obligations that combatants have towards one another in times of war. We argue that the (...) requirement of minimal force is an important requirement for considering ethical uses of force. In particular, we distinguish between lethal and non-lethal purposes of use of force and introduce the prospect of non-lethal AWS before reviewing a number of challenges which AWS pose with respect to their non-lethal use. The challenges arise where AWS generate unpredictable outcomes impinging upon the situational awareness required of combatants to ensure that their actions meet the requirement of minimal force. We conclude with a call for further research on the ethical implications of non-lethal uses of AWS as a necessary contribution for assessing the moral permissibility of AWS. (shrink)

The ethics of warfare and military leadership must pay attention to the rapidly increasing use of artificial intelligence and machines. Who is responsible for the decisions made by a machine? Do machines make decisions? May they make them? These issues are of particular interest in the context of Lethal Autonomous Weapon Systems. Are they autonomous or just automated? Do they violate the international humanitarian law which requires that humans must always be responsible for the use of lethal (...) force and for the assessment that civilian casualties are proportionate to the military goals? The article analyses relevant documents, opinions, government positions, and commentaries using the methods of applied ethics. The main conceptual finding is that the definition of autonomy depends on what the one presenting it seeks to support. Those who want to use lethal autonomous weapon systems call them by another name, say, automated instead of autonomous. They impose standards on autonomy that machines do not meet, such as moral agency. Those who wish to ban the use of lethal autonomous weapon systems define them much less broadly and do not require them to do much more than to be a self-standing part of the causal chain.The article’s argument is that the question of responsibility is most naturally perceived by abandoning the most controversial philosophical considerations and simply stating that an individual or a group of people is always responsible for the creation of the equipment they produce and use. This does not mean that those who press the button, or their immediate superiors, are to blame. They are doing their jobs in a system. The ones responsible can probably be found in higher military leadership, in political decision-makers who dictate their goals, and, at least in democracies, in the citizens who have chosen their political decision-makers. (shrink)

Autonomous Weapon Systems (AWS) are artificial intelligence systems that can make and act on decisions concerning the termination of enemy soldiers and installations without direct intervention from a human being. In this article, I provide the positive moral case for the development and use of supervised and fully autonomous weapons that can reliably adhere to the laws of war. Two strong, prima facie obligations make up the positive case. First, we have a strong moral reason to (...) deploy AWS (in an otherwise just war) because such systems decrease the psychological and moral risk of soldiers and would-be soldiers. Drones protect against lethal risk, AWS protect against psychological and moral risk in addition to lethal risk. Second, we have a prima facie obligation to develop such technologies because, once developed, we could employ forms of non-lethal warfare that would substantially reduce the risk of suffering and death for enemy combatants and civilians alike. These two arguments, covering both sides of a conflict, represent the normative hill that those in favor of a ban on autonomous weapons must overcome. Finally, I demonstrate that two recent objections to AWS fail because they misconstrue the way in which technology is used and conceptualized in modern warfare. (shrink)

We propose a multi-step evaluation schema designed to help procurement agencies and others to examine the ethical dimensions of autonomous systems to be applied in the security sector, including autonomous weapons systems.

In this paper, the current AI view that emergent functionalities apply only to the study of subcognitive agents is questioned; a hypercognitive view of autonomous agents as proposed in some AI subareas is also rejected. As an alternative view, a unified theory of social interaction is proposed which allows for the consideration of both cognitive and extracognitive social relations. A notion of functional effect is proposed, and the application of a formal model of cooperation is illustrated. Functional cooperation shows (...) the role of extracognitive phenomena in the interaction of intelligent agents, thus representing a typical example of emergent functionality. (shrink)

The two-volume set LNCS 10350 and 10351 constitutes the thoroughly refereed proceedings of the 30th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2017, held in Arras, France, in June 2017. The 70 revised full papers presented together with 45 short papers and 3 invited talks were carefully reviewed and selected from 180 submissions. They are organized in topical sections: constraints, planning, and optimization; data mining and machine learning; sensors, signal processing, and data (...) fusion; recommender systems; decision support systems; knowledge representation and reasoning; navigation, control, and autonome agents; sentiment analysis and social media; games, computer vision; and animation; uncertainty management; graphical models: from theory to applications; anomaly detection; agronomy and artificial intelligence; applications of argumentation; intelligent systems in healthcare and mhealth for health outcomes; and innovative applications of textual analysis based on AI. (shrink)

The purpose of this article is to provide a multi-perspective examination of one of the most important contemporary security issues: weaponized, and especially lethal, artificial intelligence. This technology is increasingly associated with the approaching dramatic change in the nature of warfare. What becomes particularly important and evermore intensely contested is how it becomes embedded with and concurrently impacts two social structures: ethics and law. While there has not been a global regime banning this technology, regulatory attempts at establishing a ban (...) have intensified along with acts of resistance and blocking coalitions. This article aims to reflect on the prospects and limitations, as well as the ethical and legal intensity, of the emerging regulatory framework. To allow for such an investigation, a power-analytical approach to studying international security regimes is utilized. (shrink)

Autonomous lifelong development and learning are fundamental capabilities of humans, differentiating them from current deep learning systems. However, other branches of artificial intelligence have designed crucial ingredients towards autonomous learning: curiosity and intrinsic motivation, social learning and natural interaction with peers, and embodiment. These mechanisms guide exploration and autonomous choice of goals, and integrating them with deep learning opens stimulating perspectives.

When a computer system causes harm, who is responsible? This question has renewed significance given the proliferation of autonomous systems enabled by modern artificial intelligence techniques. At the root of this problem is a philosophical difficulty known in the literature as the responsibility gap. That is to say, because of the causal distance between the designers of autonomous systems and the eventual outcomes of those systems, the dilution of agency within the large and complex teams (...) that design autonomous systems, and the impossibility of fully predicting how autonomous systems will behave once deployed, determining who is morally responsible for harms caused by autonomous systems is unclear at a conceptual level. I review past work on this topic, criticizing prior works for suggesting workarounds rather than philosophical answers to the conceptual problem presented by the responsibility gap. The view I develop, drawing on my earlier work on vicarious moral responsibility, explains why computing professionals are ethically required to take responsibility for the systems they design, despite not being blameworthy for the harms these systems may cause. (shrink)

Cognitive science's basic premises are under attack. In particular, its focus on internal cognitive processes is a target. Intelligence is increasingly interpreted, not as a matter of reclusive thought, but as successful agent-environment interaction. The critics claim that a major reorientation of the field is necessary. However, this will only occur when there is a distinct alternative conceptual framework to replace the old one. Whether or not a serious alternative is provided is not clear. Among the critics there is some (...) consensus, however, that this role could be fulfilled by the concept of a 'behavioral system'. This integrates agent and environment into one encompassing general system. We will discuss two contexts in which the behavioral systems idea is being developed. Autonomous Agents Research is the enterprise of building behavior-based robots. Dynamical Systems Theory provides a mathematical framework well suited for describing the interactions between complex systems. We will conclude that both enterprises provide important contributions to the behavioral systems idea. But neither turns it into a full conceptual alternative which will initiate a major paradigm switch in cognitive science. The concept will need a lot of fleshing out before it can assume that role. (shrink)

Ensuring the trustworthiness of autonomous systems and artificial intelligence is an important interdisciplinary endeavour. In this position paper, we argue that this endeavour will benefit from technical advancements in capturing various forms of responsibility, and we present a comprehensive research agenda to achieve this. In particular, we argue that ensuring the reliability of autonomous system can take advantage of technical approaches for quantifying degrees of responsibility and for coordinating tasks based on that. Moreover, we deem that, in (...) certifying the legality of an AI system, formal and computationally implementable notions of _responsibility_, _blame_, _accountability_, and _liability_ are applicable for addressing potential responsibility gaps (i.e. situations in which a group is responsible, but individuals’ responsibility may be unclear). This is a call to enable AI systems themselves, as well as those involved in the design, monitoring, and governance of AI systems, to represent and reason about who can be seen as responsible in prospect (e.g. for completing a task in future) and who can be seen as responsible retrospectively (e.g. for a failure that has already occurred). To that end, in this work, we show that across all stages of the design, development, and deployment of trustworthy autonomous systems (TAS), responsibility reasoning should play a key role. This position paper is the first step towards establishing a road map and research agenda on how the notion of responsibility can provide novel solution concepts for ensuring the _reliability_ and _legality_ of TAS and, as a result, enables an effective embedding of AI technologies into society. (shrink)

Focused on city-scale automation, and using self-driving cars (SDCs) as a case study, this article reflects on the role of AI—and in particular, computer vision systems used for mapping and navigation—as a catalyst for urban transformation. Urban research commonly presents AI and cities as having a one-way cause-and-effect relationship, giving undue weight to AI’s impact on cities and overlooking the role of cities in shaping AI. Working at the intersection of data science and social research, this paper aims to (...) counter this trend by exploring the reverse perspective: how do cities affect the development, and expose the present limits, of SDCs? The contribution of this paper is threefold. First, by comparing urban and nonurban environments and thoroughly examining the relationship between computer vision and city-specific sociality and form, it defines machine autonomy/automation as a function of the sociotechnical milieu in which an AI system operates. Second, and related, the paper problematizes the notion of SDCs as autonomous technologies and the role it plays in envisioning contending policy arrangements and technical solutions for achieving full driving automation. Finally, the article offers insight into a materialist and spatialized understanding of AI—namely, not as an abstract quality susceptible to replication within discrete machines, but rather as a distributed property emerging through embodied interactions among a multiplicity of agents (human, non-human, and technological) within/with their environments. (shrink)

The design of collaborative robotics, such as driver-assisted operations, engineer a potential automation of decision-making predicated on unobtrusive data gathering of human users. This form of ‘somatic surveillance’ increasingly relies on behavioural biometrics and sensory algorithms to verify the physiology of bodies in cabin interiors. Such processes secure cyber-physical space, but also register user capabilities for control that yield data as insured risk. In this technical re-formation of human–machine interactions for control and communication ‘a dissonance of attribution’ :7684, 2019. https://doi.org/10.1073/pnas.1805770115) (...) is created between perceptions of phenomena, materials and decision-making. This reconfigures relations not only between humans and machines, objects and subjects, but possibly disrupts attributive functions in the social system of Law. What it requires is shifting a legal accountability for action from a sovereignty of the human to a new materialist account based on a ‘cognitive assemblage’ between physiological data, computation and algorithmic sensing. This paper investigates the function of law as a guidance system to acknowledge this account of sensory and algorithmic computation as autonomous ‘sensing agents’ that may be accountable in situations of risk. This assemblage of robotic computation and sensory determination requires a clearer legal differentiation across the current static terminologies of person, property, liability and rights that maintain strict separations of object from subject. To neglect this, we argue, law will solely impute attributions of error to humans despite evidence of operation via mutual control. (shrink)

This article conducts a literature review of current and future challenges in the use of artificial intelligence in cyber physical systems. The literature review is focused on identifying a conceptual framework for increasing resilience with AI through automation supporting both, a technical and human level. The methodology applied resembled a literature review and taxonomic analysis of complex internet of things interconnected and coupled cyber physical systems. There is an increased attention on propositions on models, infrastructures and frameworks of (...) IoT in both academic and technical papers. These reports and publications frequently represent a juxtaposition of other related systems and technologies. We review academic and industry papers published between 2010 and 2020. The results determine a new hierarchical cascading conceptual framework for analysing the evolution of AI decision-making in cyber physical systems. We argue that such evolution is inevitable and autonomous because of the increased integration of connected devices in cyber physical systems. To support this argument, taxonomic methodology is adapted and applied for transparency and justifications of concepts selection decisions through building summary maps that are applied for designing the hierarchical cascading conceptual framework. (shrink)

This paper takes stock of all the various factors that cause the design-time opacity of autonomous systems behaviour. The factors include embodiment effects, design-time knowledge gap, human factors, emergent behaviour and tacit knowledge. This situation is contrasted with the usual representation of moral dilemmas that assume perfect information. Since perfect information is not achievable, the traditional moral dilemma representations are not valid and the whole problem of ethical autonomous systems design proves to be way more empirical (...) than previously understood. (shrink)

This paper is focused on the Sapient and Sentient Intelligence Value Argument or SSIVA and the ethics of how that applies to autonomous systems and how such systems might be governed by the extension of current regulation, as well as providing a computable model of ethics for AGI research. SSIVA is based on some static core definitions of "Intelligence" as defined by the measured ability to understand, use, and generate knowledge or information independently, all of which are (...) a function of sapience and sentience. The SSIVA logic places the value of any individual human and their potential for Intelligence, and the value of other systems to the degree that they are self-aware or "intelligent", as a priority. Further, the paper lays out the case for how the current legal framework could be extended to address issues with autonomous systems to varying degrees depending on the SSIVA threshold as applied to autonomous systems. Further, from a research standpoint it is important to have a descreet model to measure against without ambiguity, which the SSIVA theory provides. (shrink)

Contemporary medical diagnostics has a dynamic moral landscape, which includes a variety of agents, factors, and components. A significant part of this landscape is composed of information technologies that play a vital role in doctors’ decision-making. This paper focuses on the so-called Intelligent Decision-Support System that is widely implemented in the domain of contemporary medical diagnosis. The purpose of this article is twofold. First, I will show that the IDSS may be considered a moral agent in the practice of (...) medicine today. To develop this idea I will introduce the approach to artificial agency provided by Luciano Floridi. Simultaneously, I will situate this approach in the context of contemporary discussions regarding the nature of artificial agency. It is argued here that the IDSS possesses a specific sort of agency, includes several agent features (e.g. autonomy, interactivity, adaptability), and hence, performs an autonomous behavior, which may have a substantial moral impact on the patient’s well-being. It follows that, through the technology of artificial neural networks combined with ‘deep learning’ mechanisms, the IDSS tool achieves a specific sort of independence (autonomy) and may possess a certain type of moral agency. Second, I will provide a conceptual framework for the ethical evaluation of the moral impact that the IDSS may have on the doctor’s decision-making and, consequently, on the patient’s wellbeing. This framework is the Object-Oriented Model of Moral Action developed by Luciano Floridi. Although this model appears in many contemporary discussions in the field of information and computer ethics, it has not yet been applied to the medical domain. This paper addresses this gap and seeks to reveal the hidden potentialities of the OOP model for the field of medical diagnosis. (shrink)

With the rapid development of big data, artificial intelligence teaching systems have gradually been developed extensively. The powerful artificial intelligence teaching systems have become a tool for teachers and students to learn independently in various universities. The characteristic of artificial intelligence teaching system is to get rid of the constraints of traditional teaching time and space and build a brand-new learning environment, which is the mainstream trend of future learning. As the carrier of students’ autonomous learning, the (...) artificial intelligence teaching system provides a wealth of learning resources and learning tools on the one hand, and on the other hand, it gradually accumulates more and more learning behaviors, learning status, and other large amounts of data, which is an in-depth study of online learning and provides valuable and generative dynamic resources. Based on relevant researches on domestic and foreign related learning analysis and common big data analysis methods, combined with actual learning evaluation goals, this paper proposes an artificial intelligence teaching system using big data analysis methods and a modeling process framework for online learning evaluation and uses student data to carry out predictive evaluation modeling to evaluate student learning outcomes. The evaluation results can enable teachers to predict whether students can successfully complete the course of learning after a period of teaching. Through the final evaluation, students’ learning problems can be discovered in time based on the evaluation results, and targeted interventions can be made for students who are at risk. The scientific and objective learning evaluation obtained in this study through data analysis can not only provide teachers with relevant information and provide personalized guidance to students, but also improve the adaptive and personalized service functions of the learning platform of the artificial intelligence teaching system, greatly reducing teachers teaching burden. Artificial intelligence teaching evaluation can help educators understand the problems in teaching, adjust teaching strategies in time, and improve teaching results. (shrink)

Abstract: Many believe that, in addition to cognitive capacities, autonomous robots need something similar to affect. As in humans, affect, including specific emotions, would filter robot experience based on a set of goals, values, and interests. This narrows behavioral options and avoids combinatorial explosion or regress problems that challenge purely cognitive assessments in a continuously changing experiential field. Adding human-like affect to robots is not straightforward, however. Affect in organisms is an aspect of evolved biological systems, from the (...) taxes of single-cell organisms to the instincts, drives, feelings, moods, and emotions that focus human behavior through the mediation of hormones, pheromones, neurotransmitters, the autonomic nervous system, and key brain structures. We argue that human intelligence is intimately linked to biological affective systems and to the unique repertoire of potential behaviors, sometimes conflicting, they facilitate. Artificial affect is affect in name only and without genes and biological bodies, autonomous robots will lack the goals, interests, and value systems associated with human intelligence. We will take advantage of their general intelligence and expertise, but robots will not enter our intellectual world or apply for legal status in the community. -/- . (shrink)

World War II research into cryptography and computing produced methods, instruments and research communities that informed early research into artificial intelligence and semi-autonomous computing. Alan Turing and Claude Shannon in particular adapted this research into early theories and demonstrations of AI based on computers’ abilities to track, predict and compete with opponents. This formed a loosely bound collection of techniques, paradigms, and practices I call crypto-intelligence. Subsequent researchers such as Joseph Weizenbaum adapted crypto-intelligence but also reproduced aspects of its (...) antagonistic precepts. This was particularly true in the design and testing of chat bots. Here the ability to trick, fool, and deceive human and machine opponents was a premium, and practices of agent abuse were admired and rewarded. Recognizing the historical genesis of this particular variety of abuse can help researchers develop less antagonistic methodologies. (shrink)

ChatGPT launched in November 2022, triggering a global debate on the use of artificial intelligence (AI). A debate on AI-enabled lethal autonomous weapon systems (LAWS) has been underway far longer. Two sides have emerged: one in favor and one opposed to an international law ban on LAWS. This essay explains the position of advocates of a ban without attempting to persuade opponents. Supporters of a ban believe LAWS are already unlawful and immoral to use without the need of (...) a new treaty or protocol. They nevertheless seek an express prohibition to educate and publicize the threats these weapons pose. Foremost among their concerns is the “black box” problem. Programmers cannot know what a computer operating a weapons system empowered with AI will “learn” from the algorithm they use. They cannot know at the time of deployment if the system will comply with the prohibition on the use of force or the human right to life that applies in both war and peace. Even if they could, mechanized killing affronts human dignity. Ban supporters have long known that “AI models are not safe and no one knows how to reliably make them safe” or morally acceptable in taking human life. (shrink)

The aim of this exploratory paper is to review an under-appreciated parallel between group agency and artificial intelligence. As both phenomena involve non-human goal-directed agents that can make a difference to the social world, they raise some similar moral and regulatory challenges, which require us to rethink some of our anthropocentric moral assumptions. Are humans always responsible for those entities’ actions, or could the entities bear responsibility themselves? Could the entities engage in normative reasoning? Could they even have rights and (...) a moral status? I will tentatively defend the (increasingly widely held) view that, under certain conditions, artificial intelligent systems, like corporate entities, might qualify as responsible moral agents and as holders of limited rights and legal personhood. I will further suggest that regulators should permit the use of autonomous artificial systems in high-stakes settings only if they are engineered to function as moral (not just intentional) agents and/or there is some liability-transfer arrangement in place. I will finally raise the possibility that if artificial systems ever became phenomenally conscious, there might be a case for extending a stronger moral status to them, but argue that, as of now, this remains very hypothetical. (shrink)

Agents and agent-based systems are becoming essential in the development of various fields, such as artificial intelligence, ubiquitous computing, ambient intelligence, autonomous computing, and intelligent robotics. The concept of autonomous agents, inspired by the observed agency in living systems, is also central to current theories on the origin, development, and evolution of life. Therefore, it is crucial to develop an accurate understanding of agents and the concept of agency. This paper begins by discussing the role (...) of agency in natural systems as an inspiration and motivation for agential technologies and then introduces the idea of artificial agents. A systematic approach is presented for the classification of artificial agents. This classification aids in understanding the existing state of the artificial agents and projects their potential future roles in addressing specific types of problems with dedicated agent types. (shrink)

Technological progress has brought about the emergence of machines that have the capacity to take human lives without human control. These represent an unprecedented threat to humankind. This paper starts from the example of chemical weapons, now banned worldwide by the Geneva protocol, to illustrate how technological development initially aimed at the benefit of humankind has, ultimately, produced what is now called the “Weaponization of Artificial Intelligence (AI)”. Autonomous Weapon Systems (AWS) fail the so-called discrimination principle, yet, the (...) wider public is largely unaware of this problem. Given that ongoing scientific research on AWS, performed in the military sector, is generally not made available to the public domain, many of the viewpoints on this subject, expressed across different media, invoke common sense rather than scientific evidence. Yet, the implications of a potential weaponization of our work as scientists, especially in the field of AI, are reaching further than some may think. The potential consequences of a deployment of AWS for citizen stakeholders are incommensurable, and it is time to raise awareness in the public domain of the kind of potential threats identified, and to encourage legal policies ensuring that these threats will not materialize. (shrink)

The proposed article is devoted to the description of an intelligent urban passenger transport system based on unmanned electric vehicles, sequentially moving along a separate line. This system is a passenger transport system of a new urban mobility, formed under the influence of social conditions generated by high population density in cities, that suppose the development of pedestrian zones and ecological modes of transport, "transport as a service", etc. In this historical context, public transport systems acquire special relevance. (...) The described transport system belongs to intelligent systems, since it is capable of functioning in autonomous mode without human intervention, adaptively responding to changes in the dynamics of the flow of passengers during the day. Passengers are transported by electric cars, which can be combined into cassettes according to the principle of road trains based on the transportation plan drawn up by the intelligent center of the transport system according to the matrix of correspondences, filled in taking into account the incoming requests for service from passengers. When drawing up a transportation plan, the algorithms of the transport system give preference to transportation according to the "source-destination" principle, that is, when the passenger goes to the destination with a minimum number of intermediate stops, and ideally without them. The paper formulates also a criterion of arising of a conflict in the movement of vehicles, which allows to identify situations when an electric vehicle driving in front can detain vehicles following after. The work has relevance because the criterion will allow to make adjustments in the schedule of movement of vehicles and exclude the loss of time and energy that carries the transport system during braking and acceleration of electric cars, as well as to reduce waiting time and travel of passengers. (shrink)

Technological developments involving robotics and artificial intelligence devices are being employed evermore in elderly care and the healthcare sector more generally, raising ethical issues and practical questions warranting closer considerations of what we mean by “care” and, subsequently, how to design such software coherently with the chosen definition. This paper starts by critically examining the existing approaches to the ethical design of care robots provided by Aimee van Wynsberghe, who relies on the work on the ethics of care by Joan (...) Tronto. In doing so, it suggests an alternative to their non-principled approach, an alternative suited to tackling some of the issues raised by Tronto and van Wynsberghe, while allowing for the inclusion of two orientative principles. Our proposal centres on the principles of autonomy and vulnerability, whose joint adoption we deem able to constitute an original revision of a bottom-up approach in care ethics. Conclusively, the ethical framework introduced here integrates more traditional approaches in care ethics in view of enhancing the debate regarding the ethical design of care robots under a new lens. (shrink)

The question of whether new rules or regulations are required to govern, restrict, or even prohibit the use of autonomous weapon systems has been the subject of debate for the better part of a decade. Despite the claims of advocacy groups, the way ahead remains unclear since the international community has yet to agree on a specific definition of Lethal Autonomous Weapon Systems and the great powers have largely refused to support an effective ban. In this (...) vacuum, the public has been presented with a heavily one-sided view of Killer Robots. This interdisciplinary volume presents a more nuanced approach to autonomous weapon systems that recognizes the need to progress beyond a discourse framed by the Terminator and HAL 9000. Re-shaping the discussion around this emerging military innovation requires a new line of thought and a willingness to challenge the orthodoxy. Lethal Autonomous Weapons focuses on exploring the moral and legal issues associated with the design, development and deployment of lethal autonomous weapons. The volume brings together some of the most prominent academics and academic-practitioners in the lethal autonomous weapons space and seeks to return some balance to the debate. As part of this effort, it recognizes that society needs to invest in hard conversations that tackle the ethics, morality, and law of these new digital technologies and understand the human role in their creation and operation. (shrink)

Choice architecture describes the environment in which choices are presented to decision-makers. In recent years, public and private actors have looked at choice architecture with great interest as they seek to influence human behaviour. These actors are typically called choice architects. Increasingly, however, this role of architecting choice is not performed by a human choice architect, but an algorithm or artificial intelligence, powered by a stream of Big Data and infused with an objective it has been programmed to maximise. We (...) call this entity the autonomous choice architect. In this paper, we present an account of why artificial intelligence can fulfil the role of a choice architect and why this creates problems of transparency, responsibility and accountability for nudges. We argue that choice architects, be them autonomous computational systems or human-beings, at a most basic level select, from a range of designs, the design which is most likely to maximise a pre-determined objective. We then proceed to argue that, given the growing demand for targeted, personalised choice architecture and for faster, dynamic reconfigurations of choice architecture, as well as the ever-expanding pool of data from which feedback can be drawn, the role of the human choice architect is increasingly obscured behind algorithmic, artificially intelligent systems. We provide a discussion of the implications of autonomous choice architects, focusing on the importance of the humans who programme these systems, ultimately arguing that despite technological advances, the responsibility of choice architecture and influence remains firmly one human beings must bear. (shrink)

In this book, the author examines the ethical implications of Artificial Intelligence systems as they integrate and replace traditional social structures in new sociocognitive-technological environments. She discusses issues related to the integrity of researchers, technologists, and manufacturers as they design, construct, use, and manage artificially intelligent systems; formalisms for reasoning about moral decisions as part of the behavior of artificial autonomous systems such as agents and robots; and design methodologies for social agents based on societal, (...) moral, and legal values. Throughout the book the author discusses related work, conscious of both classical, philosophical treatments of ethical issues and the implications in modern, algorithmic systems, and she combines regular references and footnotes with suggestions for further reading. This short overview is suitable for undergraduate students, in both technical and non-technical courses, and for interested and concerned researchers, practitioners, and citizens. (shrink)

Recent decades have witnessed tremendous progress in artificial intelligence and in the development of autonomous systems that rely on artificial intelligence. Critics, however, have pointed to the difficulty of allocating responsibility for the actions of an autonomous system, especially when the autonomous system causes harm or damage. The highly autonomous behavior of such systems, for which neither the programmer, the manufacturer, nor the operator seems to be responsible, has been suspected to generate responsibility gaps. (...) This has been the cause of much concern. In this article, I propose a more optimistic view on artificial intelligence, raising two challenges for responsibility gap pessimists. First, proponents of responsibility gaps must say more about when responsibility gaps occur. Once we accept a difficult-to-reject plausibility constraint on the emergence of such gaps, it becomes apparent that the situations in which responsibility gaps occur are unclear. Second, assuming that responsibility gaps occur, more must be said about why we should be concerned about such gaps in the first place. I proceed by defusing what I take to be the two most important concerns about responsibility gaps, one relating to the consequences of responsibility gaps and the other relating to violations of jus in bello. (shrink)

Proposals by the San Francisco Police Department (SFPD) to use bomb disposal robots for deadly force against humans have met with widespread condemnation. Media coverage of the furore has tended, incorrectly, to conflate these robots with autonomous weapon systems (AWS), the AI-based weapons used in armed conflict. These two types of systems should be treated as distinct since they have different sets of social, ethical, and legal implications. However, the conflation does raise a pressing question: what _if_ (...) the SFPD had proposed using AWS for law enforcement purposes? This article argues that current debate on AWS takes place within a ‘killing paradigm’ that leaves us ill-placed to understand the implications of using autonomous force outside of armed conflict. This is because ‘lethality’ is taken as the paradigmatic form of harm, meaning that other harms are understood in a way that is derivative of lethal harm. The article calls for more research on how other sorts of goods, such as freedom from domination, are imperilled by the use of autonomous force. The article also suggests that bringing research on AWS into dialogue with the sorts of concerns that have typically motivated political theory—e.g. power, coercion, the state—provides a fruitful starting point for addressing these issues. (shrink)

As artificial intelligence systems are becoming increasingly autonomous and will soon be able to make decisions on their own about what to do, AI researchers have started to talk about the need to align AI with human values. The AI ‘value alignment problem’ faces two kinds of challenges—a technical and a normative one—which are interrelated. The technical challenge deals with the question of how to encode human values in artificial intelligence. The normative challenge is associated with two questions: (...) “Which values or whose values should artificial intelligence align with?” My concern is that AI developers underestimate the difficulty of answering the normative question. They hope that we can easily identify the purposes we really desire and that they can focus on the design of those objectives. But how are we to decide which objectives or values to induce in AI, given that there is a plurality of values and moral principles and that our everyday life is full of moral disagreements? In my paper I will show that although it is not realistic to reach an agreement on what we, humans, really want as people value different things and seek different ends, it may be possible to agree on what we do not want to happen, considering the possibility that intelligence, equal to our own, or even exceeding it, can be created. I will argue for pluralism which is compatible with objectivism. In spite of the fact that there is no uniquely best solution to every moral problem, it is still possible to identify which answers are wrong. And this is where we should begin the value alignment of AI. (shrink)

Lethal Autonomous Weapons (LAWs) are robotic weapons systems, primarily of value to the military, that could engage in offensive or defensive actions without human intervention. This paper assesses and engages the current arguments for and against the use of LAWs through the lens of achieving more ethical warfare. Specific interest is given particularly to ethical LAWs, which are artificially intelligent weapons systems that make decisions within the bounds of their ethics-based code. To ensure that a wide, (...) but not exhaustive, survey of the implications of employing such ethical devices to replace humans in warfare is taken into account, this paper will engage on matters related to current scholarship on the rejection or acceptance of LAWs—including contemporary technological shortcomings of LAWs to differentiate between targets and the behavioral and psychological volatility of humans—and current and proposed regulatory infrastructures for developing and using such devices. After careful consideration of these factors, this paper will conclude that only ethical LAWs should be used to replace human involvement in war, and, by extension of their consistent abilities, should remove humans from war until a more formidable discovery is made in conducting ethical warfare. (shrink)

Social machines are systems formed by technical and human elements interacting in a structured manner. The use of digital platforms as mediators allows large numbers of human participants to join such mechanisms, creating systems where interconnected digital and human components operate as a single machine capable of highly sophisticated behaviour. Under certain conditions, such systems can be described as autonomous and goal-driven agents. Many examples of modern Artificial Intelligence (AI) can be regarded as instances of this (...) class of mechanisms. We argue that this type of autonomous social machines has provided a new paradigm for the design of intelligent systems marking a new phase in the field of AI. The consequences of this observation range from methodological, philosophical to ethical. On the one side, it emphasises the role of Human-Computer Interaction in the design of intelligent systems, while on the other side it draws attention to both the risks for a human being and those for a society relying on mechanisms that are not necessarily controllable. The difficulty by companies in regulating the spread of misinformation, as well as those by authorities to protect task-workers managed by a software infrastructure, could be just some of the effects of this technological paradigm. (shrink)

The development of the systems of artificial intelligence (AI) and digital transformation in general lead to the formation of multitude of autonomous agents of artificial and mixed genealogy, as well as to complex structures in the information and regulatory environment with many opportunities and pathologies and a growing level of uncertainty in making managerial decisions. The situation is complicated by the continuing plurality of understanding of the essence of AI systems. The modern expanded understanding of AI goes (...) back to ideas formulated more than 100 years ago. In official national policy documents on the development of AI, working definitions of AI are preferred. The current stage of AI systems life cycle can be assessed as the completion of the initial period in the development of systems associated with weak AI. The ability of artificial systems to realize themselves as a separate person becomes one of the serious scientific and practical challenges. Attention to the issues of the ethics of AIS indicates the expansion of the diversity of its forms and the beginning of the work in the field of goal-setting. New moral and ethical problems also arise in connection with the possibility of the creation of genuine conscious subjects in the foreseeable future. There is an increasing phenomenon of degradation of natural intelligence. It is required to take into account the issue of the heterogeneity of data generated by humans, electronic sensors and network devices in the dynamic complex environments of the digital economy, the issue of the complexity of the process of co-evolution of AI systems, collective and individual natural consciousness. A special area of opportunities and risks is the development of neurotechnologies. The object of control is digital twins, through which there can be manipulation of real attitudes, preferences, and behavior of individuals. As a result, there are the development of technological capabilities that provoke destructive phenomena as well as the formation of a new class of mass addictions. (shrink)

Artificial Neural Networks have reached “grandmaster” and even “super-human” performance across a variety of games, from those involving perfect information, such as Go, to those involving imperfect information, such as “Starcraft”. Such technological developments from artificial intelligence (AI) labs have ushered concomitant applications across the world of business, where an “AI” brand-tag is quickly becoming ubiquitous. A corollary of such widespread commercial deployment is that when AI gets things wrong—an autonomous vehicle crashes, a chatbot exhibits “racist” behavior, automated credit-scoring (...) processes “discriminate” on gender, etc.—there are often significant financial, legal, and brand consequences, and the incident becomes major news. As Judea Pearl sees it, the underlying reason for such mistakes is that “... all the impressive achievements of deep learning amount to just curve fitting.” The key, as Pearl suggests, is to replace “reasoning by association” with “causal reasoning” —the ability to infer causes from observed phenomena. It is a point that was echoed by Gary Marcus and Ernest Davis in a recent piece for theNew York Times: “we need to stop building computer systems that merely get better and better at detecting statistical patterns in data sets—often using an approach known as ‘Deep Learning’—and start building computer systems that from the moment of their assembly innately grasp three basic concepts: time, space, and causality.” In this paper, foregrounding what in 1949 Gilbert Ryle termed “a category mistake”, I will offer an alternative explanation for AI errors; it is not so much that AI machinery cannot “grasp” causality, but that AI machinery (qua computation) cannot understand anything at all. (shrink)

The use of autonomous weaponry in warfare has increased substantially over the last twenty years and shows no sign of slowing. Our chapter raises a novel objection to the implementation of autonomous weapons, namely, that they eliminate moral cost-sharing. To grasp the basics of our argument, consider the case of uninhabited aerial vehicles that act autonomously (i.e., LAWS). Imagine that a LAWS terminates a military target and that five civilians die as a side effect of the LAWS bombing. (...) Because LAWS lacks moral agency, and in particular the capacity for moral emotions, moral cost-sharing is limited to dead civilians and their loved ones. We argue that's unjust insofar as those responsible for unjust harm to others ought to share in those costs. Our worry expands to other strategic uses of AI, e.g., cyber warfare. This presents a dilemma: Either we design autonomous weaponry capable of moral emotions, or we limit the use of autonomous weaponry. The former undermines the risk-mitigation purpose of creating autonomous weaponry and expands the number of sentient individuals whose welfare is risked in war. The latter risks worsening combatant casualties and achieving strategic aims. (shrink)