v. 1. Editorial introduction -- v. 2. The English and Latin texts (i) -- v. 3. The English and Latin texts (ii).

All humans have an equal basic moral status. They possess the same fundamental rights, and the comparable interests of each person should count the same in calculations that determine social policy. Neither supposed racial differences, nor skin color, sex, sexual orientation, ethnicity, intelligence, nor any other differences among humans negate their fundamental equal worth and dignity. These platitudes are virtually universally affirmed. A white supremacist racist or an admirer of Adolf Hitler who denies them is rightly regarded as beyond the (...) pale of civilized dialogue. However, a very simple line of argument developed by Peter Singer challenges our understanding of these platitudes and forces us to rethink the basis and nature of the moral equality of all humans. What I call the “Singer Problem” arises if one accepts that the morally significant cognitive capacities that are relevant to the determination of the fundamental moral status of a being vary from individual to individual by degree. The problem is to specify a moral principle determining fundamental moral status that assigns a superior status to humans compared to other animals on the basis of the superior cognitive capacities of humans but also assigns all human persons an equal fundamental moral status regardless of their differing cognitive capacities. More broadly posed, the problem is to specify moral principles that yield intuitively satisfactory implications for the treatment of human individuals and other individual animals given that cognitive capacities differ across species and individuals. A search for a resolution of the Singer problem leads to disappointing results. (shrink)

What is morality? Where does it come from? And why do most of us heed its call most of the time? In Braintrust, neurophilosophy pioneer Patricia Churchland argues that morality originates in the biology of the brain. She describes the "neurobiological platform of bonding" that, modified by evolutionary pressures and cultural values, has led to human styles of moral behavior. The result is a provocative genealogy of morals that asks us to reevaluate the priority given to religion, absolute rules, and (...) pure reason in accounting for the basis of morality. Moral values, Churchland argues, are rooted in a behavior common to all mammals--the caring for offspring. The evolved structure, processes, and chemistry of the brain incline humans to strive not only for self-preservation but for the well-being of allied selves--first offspring, then mates, kin, and so on, in wider and wider "caring" circles. Separation and exclusion cause pain, and the company of loved ones causes pleasure; responding to feelings of social pain and pleasure, brains adjust their circuitry to local customs. In this way, caring is apportioned, conscience molded, and moral intuitions instilled. A key part of the story is oxytocin, an ancient body-and-brain molecule that, by decreasing the stress response, allows humans to develop the trust in one another necessary for the development of close-knit ties, social institutions, and morality. A major new account of what really makes us moral, Braintrust challenges us to reconsider the origins of some of our most cherished values. (shrink)

A controversial scientific vision predicts a time in which humans and machines will merge and create a new form of non-biological intelligence, explaining how the occurrence will solve such issues as pollution, hunger, and aging.

The era of the Cyborg is now upon us. This has enormous implications on ethical values for both humans and cyborgs. In this paper the state of play is discussed. Routes to cyborgisation are introduced and different types of Cyborg are considered. The author's own self-experimentation projects are described as central to the theme taken. The presentation involves ethical aspects of cyborgisation both as it stands now and those which need to be investigated in the near future as the effects (...) of increased technological power have a more dramatic influence. An important feature is the potential for cyborgs to act against, rather than for, the interests of humanity. (shrink)

That the successful development of fully autonomous artificial moral agents (AMAs) is imminent is becoming the received view within artificial intelligence research and robotics. The discipline of Machines Ethics, whose mandate is to create such ethical robots, is consequently gaining momentum. Although it is often asked whether a given moral framework can be implemented into machines, it is never asked whether it should be. This paper articulates a pressing challenge for Machine Ethics: To identify an ethical framework that is both (...) implementable into machines and whose tenets permit the creation of such AMAs in the first place. Without consistency between ethics and engineering, the resulting AMAs would not be genuine ethical robots, and hence the discipline of Machine Ethics would be a failure in this regard. Here this challenge is articulated through a critical analysis of the development of Kantian AMAs, as one of the leading contenders for being the ethic that can be implemented into machines. In the end, however, the development of Kantian artificial moral machines is found to be anti-Kantian. The upshot of all this is that machine ethicists need to look elsewhere for an ethic to implement into their machines. (shrink)

The United States Army’s Future Combat Systems Project, which aims to manufacture a “robot army” to be ready for deployment by 2012, is only the latest and most dramatic example of military interest in the use of artificially intelligent systems in modern warfare. This paper considers the ethics of a decision to send artificially intelligent robots into war, by asking who we should hold responsible when an autonomous weapon system is involved in an atrocity of the sort that would normally (...) be described as a war crime. A number of possible loci of responsibility for robot war crimes are canvassed; the persons who designed or programmed the system, the commanding officer who ordered its use, the machine itself. I argue that in fact none of these are ultimately satisfactory. Yet it is a necessary condition for fighting a just war, under the principle of jus in bellum, that someone can be justly held responsible for deaths that occur in the course of the war. As this condition cannot be met in relation to deaths caused by an autonomous weapon system it would therefore be unethical to deploy such systems in warfare. (shrink)

I survey four categories of factors that might give a digital mind, such as an upload or an artificial general intelligence, an advantage over humans. Hardware advantages include greater serial speeds and greater parallel speeds. Self-improvement advantages include improvement of algorithms, design of new mental modules, and modification of motivational system. Co-operative advantages include copyability, perfect co-operation, improved communication, and transfer of skills. Human handicaps include computational limitations and faulty heuristics, human-centric biases, and socially motivated cognition. The shape of hardware (...) growth curves, as well as the ease of modifying minds, are found to have a major impact on how quickly a digital mind may take advantage of these factors. (shrink)

Ethical decisions related to computer technology and computer use are subject to three primary influences: (1) the individual's own personal code (2) any informal code of ethical behavior that exists in the work place, and (3) exposure to formal codes of ethics. The relative importance of these codes, as well as factors influencing these codes, was explored in a nationwide survey of information system (IS) professionals. The implications of the findings are important to educators and employers in the development of (...) acceptable ethical standards. (shrink)

After 50 years, the fields of artificial intelligence and robotics capture the imagination of the general public while, at the same time, engendering a great deal of fear and skepticism. Isaac Asimov recognized this deep-seated misconception of technology and created the Three Laws of Robotics. The first part of this paper examines the underlying fear of intelligent robots, revisits Asimov’s response, and reports on some current opinions on the use of the Three Laws by practitioners. Finally, an argument against robotic (...) rebellion is made along with a call for personal responsibility and suggestions for implementing safety constraints in intelligent robots. (shrink)

As with other emerging technologies, advanced robotics brings with it new ethical and policy challenges. This paper will describe the flourishing role of robots in society—from security to sex—and survey the numerous ethical and social issues, which we locate in three broad categories: safety & errors, law & ethics, and social impact. We discuss many of these issues in greater detail in our forthcoming edited volume on robot ethics from MIT Press.

A fundamental problem in artificial intelligence is that nobody really knows what intelligence is. The problem is especially acute when we need to consider artificial systems which are significantly different to humans. In this paper we approach this problem in the following way: we take a number of well known informal definitions of human intelligence that have been given by experts, and extract their essential features. These are then mathematically formalised to produce a general measure of intelligence for arbitrary machines. (...) We believe that this equation formally captures the concept of machine intelligence in the broadest reasonable sense. We then show how this formal definition is related to the theory of universal optimal learning agents. Finally, we survey the many other tests and definitions of intelligence that have been proposed for machines. (shrink)

Self-improvement was one of the aspects of AI proposed for study in the 1956 Dartmouth conference. Turing proposed a “child machine” which could be taught in the human manner to attain adult human-level intelligence. In latter days, the contention that an AI system could be built to learn and improve itself indefinitely has acquired the label of the bootstrap fallacy. Attempts in AI to implement such a system have met with consistent failure for half a century. Technological optimists, however, have (...) maintained that a such system is possible, producing, if implemented, a feedback loop that would lead to a rapid exponential increase in intelligence. We examine the arguments for both positions and draw some conclusions. (shrink)

Is morality rational? In this book Gauthier argues that moral principles are principles of rational choice. He proposes a principle whereby choice is made on an agreed basis of cooperation, rather than according to what would give an individual the greatest expectation of value. He shows that such a principle not only ensures mutual benefit and fairness, thus satisfying the standards of morality, but also that each person may actually expect greater utility by adhering to morality, even though the choice (...) did not have that end primarily in view. In resolving what may appear to be a paradox, the author establishes morals on the firm foundation of reason. Gauthier's argument includes an account of value, linking it to preference and utility; a discussion of the curcumstances in which morality is unnecessary; and an application of morals by agreement to relations between peoples at different levels of development and different generations. Finally, he reflects on the assumptions about individuality and community made by his account of rationality and morality. (shrink)

The most cursory examination of the history of artificial intelligence highlights numerous egregious claims of its researchers, especially in relation to a populist form of ‘strong’ computationalism which holds that any suitably programmed computer instantiates genuine conscious mental states purely in virtue of carrying out a specific series of computations. The argument presented herein is a simple development of that originally presented in Putnam’s (Representation & Reality, Bradford Books, Cambridge in 1988) monograph, “Representation & Reality”, which if correct, has important (...) implications for turing machine functionalism and the prospect of ‘conscious’ machines. In the paper, instead of seeking to develop Putnam’s claim that, “everything implements every finite state automata”, I will try to establish the weaker result that, “everything implements the specific machine Q on a particular input set (x)”. Then, equating Q (x) to any putative AI program, I will show that conceding the ‘strong AI’ thesis for Q (crediting it with mental states and consciousness) opens the door to a vicious form of panpsychism whereby all open systems, (e.g. grass, rocks etc.), must instantiate conscious experience and hence that disembodied minds lurk everywhere. (shrink)

The most cursory examination of the history of artificial intelligence highlights numerous egregious claims of its researchers, especially in relation to a populist form of ‘strong’ computationalism which holds that any suitably programmed computer instantiates genuine conscious mental states purely in virtue of carrying out a specific series of computations. The argument presented herein is a simple development of that originally presented in Putnam’s monograph, “Representation & Reality”, which if correct, has important implications for turing machine functionalism and the prospect (...) of ‘conscious’ machines. In the paper, instead of seeking to develop Putnam’s claim that, “everything implements every finite state automata”, I will try to establish the weaker result that, “everything implements the specific machine Q on a particular input set ”. Then, equating Q to any putative AI program, I will show that conceding the ‘strong AI’ thesis for Q opens the door to a vicious form of panpsychism whereby all open systems,, must instantiate conscious experience and hence that disembodied minds lurk everywhere. (shrink)

A principal goal of the discipline of artificial morality is to design artificial agents to act as if they are moral agents. Intermediate goals of artificial morality are directed at building into AI systems sensitivity to the values, ethics, and legality of activities. The development of an effective foundation for the field of artificial morality involves exploring the technological and philosophical issues involved in making computers into explicit moral reasoners. The goal of this paper is to discuss strategies for implementing (...) artificial morality and the differing criteria for success that are appropriate to different strategies. (shrink)

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the propects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from (...) a human to a "posthuman" society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could case our extinction or destroy the potential of Earth - originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (shrink)

In Good and Real, Gary Drescher examines a series of provocative paradoxes about consciousness, choice, ethics, quantum mechanics, and other topics, in an effort to reconcile a purely mechanical view of the universe with key aspects of our ...

Focusing on the breakthrough field of molecular engineering--a new technology enabling scientists to build tiny machines atom by atom--the author offers projections on how this technological revolution will affect the future of computer science, space travel, medicine, and manufacturing.

Computers are already approving financial transactions, controlling electrical supplies, and driving trains. Soon, service robots will be taking care of the elderly in their homes, and military robots will have their own targeting and firing protocols. Colin Allen and Wendell Wallach argue that as robots take on more and more responsibility, they must be programmed with moral decision-making abilities, for our own safety. Taking a fast paced tour through the latest thinking about philosophical ethics and artificial intelligence, the authors argue (...) that even if full moral agency for machines is a long way off, it is already necessary to start building a kind of functional morality, in which artificial moral agents have some basic ethical sensitivity. But the standard ethical theories don't seem adequate, and more socially engaged and engaging robots will be needed. As the authors show, the quest to build machines that are capable of telling right from wrong has begun. -/- Moral Machines is the first book to examine the challenge of building artificial moral agents, probing deeply into the nature of human decision making and ethics. (shrink)

Second, this collection of cognitive programs evolved in the Pleistocene to solve the adaptive problems regularly faced by our hunter-gatherer ancestors-...

Thomas Hobbes took a new look at the ways in which society should function, and he ended up formulating the concept of political science. His crowning achievement, Leviathan, remains among the greatest works in the history of ideas. Written during a moment in English history when the political and social structures as well as methods of science were in flux and open to interpretation, Leviathan played an essential role in the development of the modern world. This edition of Hobbes' landmark (...) work is based on the original text. It incorporates the author's own corrections and retains the period spelling and punctuation, offering both flavorful authenticity and the utmost clarity of expression. (shrink)

The writer commences:—“There was a time, when the earth was to all appearance utterly destitute both of animal and vegetable life, and when according to the opinion of our best philosophers it was simply a hot round ball with a crust gradually cooling. Now if a human being had existed while the earth was in this state and had been allowed to see it as though it were some other world with which he had no concern, and if at the (...) same time he were entirely ignorant of all physical science, would he not have pronounced it impossible that creatures possessed of anything like consciousness should be evolved from the seeming cinder which he was beholding? Would he not have denied that it contained any potentiality of consciousness? Yet in the course of time consciousness came. Is it not possible then that there may be even yet new channels dug out for consciousness, though we can detect no signs of them at present? (shrink)

What happens when machines become more intelligent than humans? One view is that this event will be followed by an explosion to ever-greater levels of intelligence, as each generation of machines creates more intelligent machines in turn. This intelligence explosion is now often known as the “singularity”. The basic argument here was set out by the statistician I.J. Good in his 1965 article “Speculations Concerning the First Ultraintelligent Machine”: Let an ultraintelligent machine be defined as a machine that can far (...) surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion”, and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make. The key idea is that a machine that is more intelligent than humans will be better than humans at designing machines. So it will be capable of designing a machine more intelligent than the most intelligent machine that humans can design. So if it is itself designed by humans, it will be capable of designing a machine more intelligent than itself. By similar reasoning, this next machine will also be capable of designing a machine more intelligent than itself. If every machine in turn does what it is capable of, we should expect a sequence of ever more intelligent machines. This intelligence explosion is sometimes combined with another idea, which we might call the “speed explosion”. The argument for a speed explosion starts from the familiar observation that computer processing speed doubles at regular intervals. Suppose that speed doubles every two years and will do so indefinitely. Now suppose that we have human-level artificial intelligence 1 designing new processors. Then faster processing will lead to faster designers and an ever-faster design cycle, leading to a limit point soon afterwards. The argument for a speed explosion was set out by the artificial intelligence researcher Ray Solomonoff in his 1985 article “The Time Scale of Artificial Intelligence”.1 Eliezer Yudkowsky gives a succinct version of the argument in his 1996 article “Staring at the Singularity”: “Computing speed doubles every two subjective years of work.. (shrink)

Utilizing the film I, Robot as a springboard, I here consider the feasibility of robot utilitarians, the moral responsibilities that come with the creation of ethical robots, and the possibility of distinct ethics for robot-robot interaction as opposed to robot-human interaction. (This is a revised and expanded version of an essay that originally appeared in IEEE: Intelligent Systems.).

As arti® cial intelligence moves ever closer to the goal of producing fully autonomous agents, the question of how to design and implement an arti® cial moral agent (AMA) becomes increasingly pressing. Robots possessing autonomous capacities to do things that are useful to humans will also have the capacity to do things that are harmful to humans and other sentient beings. Theoretical challenges to developing arti® cial moral agents result both from controversies among ethicists about moral theory itself, and from (...) computational limits to the implementation of such theories. In this paper the ethical disputes are surveyed, the possibility of a `moral Turing Test ’ is considered and the computational di culties accompanying the diŒerent types of approach are assessed. Human-like performance, which is prone to include immoral actions, may not be acceptable in machines, but moral perfection may be computationally unattainable. The risks posed by autonomous machines ignorantly or deliberately harming people and other sentient beings are great. The development of machines with enough intelligence to assess the eŒects of their actions on sentient beings and act accordingly may ultimately be the most important task faced by the designers of arti® cially intelligent automata. (shrink)

This paper is reprinted in the book Machine Ethics, eds. M. Anderson and S. Anderson, Cambridge University Press, 2011.

_This paper outlines the case for believing that we will have superhuman artificial intelligence_ _within the first third of the next century. It looks at different estimates of the processing power of_ _the human brain; how long it will take until computer hardware achieve a similar performance;_ _ways of creating the software through bottom-up approaches like the one used by biological_ _brains; how difficult it will be for neuroscience figure out enough about how brains work to_ _make this approach work; (...) and how fast we can expect superintelligence to be developed once_ _there is human-level artificial intelligence._. (shrink)