Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020* Nicholas Kluge Corrêa† Graduate Program in Philosophy of the Pontifical Catholic University of Rio Grande do Sul – Av. Ipiranga, 6681 Partenon, Porto Alegre RS, 90619-900. nicholas.correa@acad.pucrs.br, https://orcid.org/0000-0002-5633-6094 Nythamar de Oliveira‡ Graduate Program in Philosophy of the Pontifical Catholic University of Rio Grande do Sul – Av. Ipiranga, 6681 Partenon, Porto Alegre RS, 90619-900. nythamar@yahoo.com, https://orcid.org/0000-0001-9241-1031 * Acknowledgements and Funding: The authors would like to thank the Academic Excellence Program (PROEX) of CAPES Foundation (Coordination for the Improvement of Higher Education Personnel) and the Graduate Program in Philosophy of the Pontifical Catholic University of Rio Grande do Sul, Brazil. † Master in Electrical Engineering and Ph.D. student in Philosophy – PUCRS. ‡ Ph.D. in Philosophy (State University of New York, 1994), Full Professor – PUCRS. 2 Corrêa, N. K., & De Oliveira, N. Abstract One of the strands of the Transhumanist movement, Singulitarianism, studies the possibility that high-level artificial intelligence may be created in the future, debating ways to ensure that the interaction between human society and advanced artificial intelligence can occur in a safe and beneficial way. But how can we guarantee this safe interaction? In trying to answer this question, We'll make a small introduction to the area of safety research in artificial intelligence. We'll review some of the current paradigms in the development of autonomous intelligent systems, and finally, present a reflection using the COVID-19 pandemic as a possible analogy. Keywords: Singularity, Artificial Intelligence, Safety, coronavirus pandemic. Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 3 Singulitarianism and Safety Research in the area of Artificial Intelligence (AI) is an interdisciplinary endeavor by nature, given the various fields that participate and benefit from its development. When we talk about AI, either in the context of computer science (Searle, 1980; Russel & Norvig, 2003; Wang, 2019) or in the study of the philosophy of the mind (Haugeland, 1985; Newell, 1990; Chalmers, 2010), a certain dichotomy is utilized to classify two different types of AI: Specific intelligencea and General intelligenceb. AGI would be something capable of covering all possible tasks, those that humans are specifically good at, those that animals are capable of, and all that goes beyond the imagination and capacity of any form of known cognitive agency (Chollet, 2019). Moravec (1998, p. 10) proposes an analogy, where the advancement of AI capabilities is compared to a "flood": fifty years ago, tasks previously only proficiently performed by humans (e. g., human calculators) were "flooded" and replaced by the use of autonomous systems. Increasingly, we take refuge in the high peaks of the cognitive landscape, still reserved exclusively for us, while lower regions continue to be flooded. What we will do in this study is to explore the idea and possible consequences of "what if we are successful" in developing an AGI. Vinge (1993) uses the term "Singularity" to define artificial intelligent systems/agents that have surpassed human intelligence. While Singulitarianism, is the name used to describe the Transhumanist strand where it is believed that a technological Singularity a Specific intelligence: also known as "weak" AI, is how we define artificial autonomous systems that we are used to interacting in our daily lives. Such systems are only proficient in specific tasks, and unable to generalize their skills to domains outside their training environment. b General intelligence: also referred to as "strong" AI, or artificial general intelligence (AGI), which consists of an autonomous artificial system capable of solving many types of problems, proficiently, in any domain, or at least in a wide range of domains. 4 Corrêa, N. K., & De Oliveira, N. (artificial super intelligence) is likely to be created in the medium-long future. Given this belief, an active response is necessary to ensure that such Singularity is beneficial to our society (Kurzweil, 2005; Naude, 2009; Chalmers, 2010; Lombardo, 2012; Tegmark, 2017). Irving J. Good (1965), a British mathematician who collaborated with Alan Turing at Bletchley Park during World War II, was one of the first academics to speculate on the possibility of an "ultraintelligent machine" (Singularity): Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an "intelligence explosion", and the intelligence of man would be left far behind [...] Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control. It is curious that this point is made so seldom outside of science fiction. It is sometimes worthwhile to take science fiction seriously (Good, 1965, p. 33). And nowadays, the concepts of Singularity and intelligence explosion have even been cited in Stanford's One-Hundred Year Study of Artificial Intelligence (besides several other works): Speculations about the rise of such uncontrollable machine intelligences have called out different scenarios, including trajectories that take a slow, insidious course of refinement and faster-paced evolution of systems toward a powerful intelligence "singularity." Are such dystopic outcomes possible? If so, how might these situations arise? What are the paths to these feared outcomes? What might we do proactively to effectively address or lower the likelihood of such outcomes, and thus reduce these concerns? What kind of research would help us to better understand and to address concerns about the rise of a dangerous super intelligence or the occurrence of an "intelligence Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 5 explosion"? Concerns about the loss of control of AI systems should be addressed via study, dialog, and communication. Anxieties need to be addressed even if they are unwarranted (Horvitz, 2014, p. 5). Would there be any indication that an intelligence explosion is something, however unlikely, still possible? Perhaps, we have already found in the literature the first indications of autonomous systems assisting in the development of other autonomous systems. Zoph and Le (2017) proposed an autonomous technique for the development of artificial neural networks architecture. According to the authors: "[...] our method, starting from scratch, can design a new network architecture that rivals the best architecture invented by man [...]" (Zoph & Le, 2017, p. 1). The authors developed their model using Reinforcement Learning (RL) to train their "architect" system of artificial neural networks. RL is one of the paradigms in the area of machine learning, where artificial agents must act in the environment that they are embedded, to maximize their reward function (Russel & Norvig, 2003). Reward functions are a mathematical representation of the preferences that guide the behavior of agents operating by RL, where, for example, a cleaning robot can maximize a function that assigns "little dirt on the floor" a high reward, and world states where the floor is dirty with a low reward. Many of the models used to study idealized rational agents (Expected Utility Theory) provide convincing arguments that any rational agent with consistent preferences should act as an expected utility maximizer (Von Neumann & Morgenstern, 1944). However, within the framework of expected utility theory, there are corollary results that seem to refer to the concern of Good, quoted above: "[...] as long as the machine is docile enough to tell us how to keep it under control [...]". Stephen Omohundro (2008), in his work entitled "The Basic AI Drives", cites a number of characteristics that we 6 Corrêa, N. K., & De Oliveira, N. should expect artificial intelligent agents to possess. Bostrom (2014, chapter 7, pp. 110-112) popularized Omohundro's arguments in two theses:  Instrumental Convergence Thesis: Artificial intelligent agents can have a huge range of possible terminal goals. However, there are certain instrumental goals that can be pursued by almost all intelligent agents, because these goals are useful means for the achievement of almost any terminal goal;  Orthogonality Thesis: analogous to Hume's Guillotine (Is-Ought Gap), the orthogonality thesis dictates that ethical pronouncements and prescriptions for what should be, cannot be achieved through factual analysis. Thus, both concepts (reason and morality) being independent. Turner et al (2020) generalized the conjectures made by Omohundro and Bostrom in what the authors call the Power-Seeking Theorems. In them is demonstrated that within the formalism of Markov decision processes (MDP), most of the terminal objectives encourage the achievement of power over the environment. Power is the ability to achieve goals in general and to gain dominance over the environment. It's instrumentally convergent to a wide range of terminal goals to search for power. A corollary of the results demonstrated by Turner et al, is that even in simplified conditions, we see that most reward functions induce a searchfor-power behavior, something that may cause safety problems involving the interaction of humans and AI. In light of all these arguments, which date back to the early days of AI research, safety issues have been increasingly cited in the literature. AI ethics, a sub-area of applied ethics concerned with adding moral behavior to machines and regulating the use of artificial intelligence, has been gaining a significant increase in popularity in the last two decades (Jobin et al, 2019; Jurić et al, 2020). Important Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 7 philosophical and technical questions are raised in the context of AI safety, e. g., Corrigibility: how to correct/terminate potentially faulty agents that have a strong instrumental incentive to preserve their terminal goals (Soares et al, 2015; Amodei et al, 2016)? We can find in the literature several research agendas, where different types of ethical, technical, and social problems are discussed (Russel et al, 2015; Taylor et al, 2016; Tegmark, 2016; Soares, 2016; O'Keefe et al, 2020; ÓhÉigeartaigh et al, 2020; Hagendorff, 2020). At one end of the spectrum, we find research involving existential risks, i. e., the study of possible threats at the extinction-level imposed by present or future technology. Research centers such as the Centre for the Study of Existential Risk in Cambridge, the Future of Life Institute in Boston (Russel et al, 2015), specifically focused on existential risk involving advanced artificial intelligence, and the Future of Humanity Institute in Oxford (Bostrom, 2002), search for strategies to mitigate certain types of dystopian future. AI takeoof Whether we will achieve human-level artificial intelligence, or whether artificial agents will outperform our cognitive capabilities, is still a matter of debate among theorists in the field. Let us assume that the chance that a Singularity will be achieved by 2100 is only 1%, should we worry about security? We argue that "security" is not built, or planned, based on average forecasts, or "average possible scenario". With this thought, our current technological development in the area should be analyzed, and at the same time, what lessons from the past can we draw to assist us in this analysis? We can compare our current scenario about artificial intelligence with past events that led to the construction of nuclear power plants and weapons. 8 Corrêa, N. K., & De Oliveira, N. A technological race led to the mass production of systems that we still didn't have a complete understanding, something that caused several side effects, like accidents (Chernobyl disaster), the creation of weapons of mass destruction (Cold War), and even the use of these weapons against human society itself (Atomic bombings of Hiroshima and Nagasaki). Certainly, the pressures for the development of high-performance AI, given its capacity to provide the organization that controls it a considerable strategic advantage, will cause the same type of technological race that we experienced in the mid-20th century: "while X invests in the development of AI, Y will do as well". Another reason for caution in our technological advances in the area of AI, is that different from common thinking, for an artificially intelligent system to represent a potential danger to our society, it doesn't need to be more intelligent than us humans. Rather, it needs to be more capable in certain kinds of tasks. Barret and Baum (2017) explore two main reasons that would cause an artificial intelligence to represent a considerable danger to our society, reasons of capability (i) and value (ii). i. Intelligent artificial agents can pose a danger to human well-being because of their extremely refined ability, or some aptitude, with which we cannot compete; ii. Intelligent artificial agents can develop goals and objectives that diverge from us humans, and in pursuing them, cause damage to our society. ASI-PATH (Artificial Super Intelligence Pathway) is a model for how an AI could cause a catastrophe, becoming super-intelligent through recursive selfimprovement (Barret & Baum, 2017). This model suggests scenarios where intelligent agents, after obtaining a strategic advantage, DSA (decisive strategic advantage), such as advances in nanotechnology, biological engineering, or robotics, could achieve considerable power of control over the environment. Given Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 9 our dependence on autonomous systems integrated with the Internet, a potentially harmful capability would be to run cyber-attacks on vital structures of our infrastructure, in areas such as electricity distribution networks and telecommunications. In 2017, the crypto-ransomware "WannaCry", malicious software that hacks into computers and private networks, encrypting their content, and only providing the key to decryption after payment of a ransom, reached several systems in the world in more than 99 countries, even affecting the public health system of certain governments. More than 75,000 ransom demands were made, making it one of the most damaging cyber-attacks in history (Larson, 2017). This would be a possible DSA of an AI, the ability to execute cyber-attacks on our infrastructure in a way that we cannot remedy in time. The ASI-PATH provides an intuitive diagram where various events (i. e., security breaches) must occur to cause a catastrophe involving advanced artificial intelligence. Initially, an AI, also called a seed AI, must first become an AI with some DSA, and at the same time, the security measures must have failed. Witch includes: failures in confinement, failed value alignment, AI objectives diverge from ours, containment fails, etc. Sotala (2018, p. 317) provides a simplified view of ASIPATH in his work "Disjunctive scenarios of catastrophic AI risk". As suggested by Barret and Baum's model, the arguments raised by the thesis of instrumental convergence and the orthogonality thesis are some of the reasons that could lead a Singularity to engage in hostile actions against humans, in the words of Eliezer Yudkowsky, co-founder and researcher of the Machine Intelligence Research Institute (MIRI): The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else. The AI runs on a different timescale than you do; by the time your neurons finish 10 Corrêa, N. K., & De Oliveira, N. thinking the words "I should do something" you have already lost (Yudkowsky, 2008, p. 27). Shulman (2010, p. 2) suggests a simple model that explains in which situations an AI would abandon cooperation with the human race and take hostile action. An artificial agent that believes in a probability P to be successful, if it starts aggression, receiving utility [ ( )], and with probability 1 to fail in such action, receiving [ ( )]. Where giving up an aggressive strategy, the agent receives utility [ ( )], the AI will rationally initiate aggression if, and only if: ( ) (1 ) ( ) ( ) In the same way that natural selection is not "evil" for having selected for survival 1% of all species that have ever lived on Earth, a Singularity is not "evil" for possibly harming the human race. It would be optimizing the world for its goals, similar to how we humans do indifferently to other species (except for rare exceptions where we exploit them for our benefit/pleasure/satisfaction). Both possible terminal goals that would characterize a Singularity as "benevolent" or "evil" are still small points in the landscape of possible goals. The vast majority of the terminal goals that a Singularity could have evaluate some or all of our human values as indifferent. And indifference can result in extremely unpleasant consequences, e. g., natural selection is totally indifferent to pain, it is neither sadistic nor merciful, it is simply indifferent. To make the central nervous cortex of a zebra deactivate itself, sparing the animal from the torture of being devoured alive, would not be very "costly" for an optimization process like evolution, in its millions of years of optimization. However, evolution maximizes reproductive fitness, and not if living beings feel "less pain". The scenarios explored in the literature, where a "seed" AI is capable of becoming a Singularity, are usually characterized in two different types of takeoffs. Rapid takeoffs suggest situations where a drastic takeover occurs, where abruptly we Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 11 would be surprised by an entity much more capable, with possibly unknown objectives, inserted and sharing the same environment as us. In contrast, we have slow takeoffs, which are a much more realistic possibility, in our opinion. It would occur gradually as the human species becomes more and more dependent and, in a way, under the control of advanced AI (Sotala, 2018). The argument and line of reasoning, behind a slow takeoff, is exposed in this passage of Theodore Kaczynski's controversial, and tragic, manifesto, Industrial Society and its Future: If the machines are permitted to make all their own decisions, we can't make any conjectures as to the results, because it is impossible to guess how such machines might behave. We only point out that the fate of the human race would be at the mercy of the machines. It might be argued that the human race would never be foolish enough to hand over all power to the machines. But we are suggesting neither that the human race would voluntarily turn power over to the machines nor that the machines would willfully seize power. What we do suggest is that the human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines' decisions. As society and the problems that face it become more and more complex and as machines become more and more intelligent, people will let machines make more and more of their decisions for them, simply because machinemade decisions will bring better results than man-made ones. Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won't be able to just turn the machine off, because they will be so dependent on them that turning them off would amount to suicide (Kaczynski, 1995, § 173, p. 22). Such questions raise profound concerns, especially in the area of ethics and morals, old questions are now raised in a new light, and even with a new sense of urgency. 12 Corrêa, N. K., & De Oliveira, N. For AI development to be done in a way that minimizes the risk of existential threats to humanity, some important questions are: what strategies and policies should we adopt to ensure that the maximization of the goals of advanced artificial intelligence is aligned with our interests? And what restrictions to this project should we impose to ensure a beneficial outcome? Would there be predictions of when an AGI could be achieved? AGI on the horizon? Researchers and experts in the development of artificial intelligence predict that within 10 years many human activities will be surpassed by machines in terms of efficiency (Grace et al, 2017). A survey was conducted by Müller and Bostrom (2016), where the authors administered a questionnaire to assess the progress in the field of AI research and prospects for the future, interviewing several experts (N 1 0). The questionnaire showed that on average, there is a 0% chance that high-level machine intelligence will be achieved between 2040 and 2050, with a 90% probability by 2075. It is also estimated that AI will outperform human performance between 2 (10% chance) and 30 years ( % chance) (Müller & Bostrom, 2016). In a similar survey conducted by Grace et al (2017), the researchers interviewed (N 3 2) believe that AI will outperform human performance in all tasks in 45 years, with a 0% chance, and automate all human work in up to 120 years. However, we emphasize that there is great variability in the results obtained. In Müller and Bostrom's (2016) survey, 33% of respondents classified this development in AI as "bad" or "extremely bad" for humanity. In the research of Grace et al (2017), when those evaluated were asked whether highlevel AI would have a positive or negative impact in the long term, the median probability for "good" and "extremely good" results was 2 % and 20%, respectively. The probabilities for a "bad" or "extremely bad" resolution were, respectively, 10% and %. Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 13 From the above surveys, we can state that the chance that high-level AI will be created in the next 120 years is, at least, being pessimistic, 10% for a certain portion of the academic community. For those who follow the recent advances in the field of AI, it is known that one of the major paradigms in the field of research today involves the problem of natural language processing (NLP), and the use of a new form of architecture called "Transformer", proposed by Vaswani et al (201 ) in his seminal work: "Attention is all you need", Currently, systems based on the transformer architecture, are the new paradigm in natural language processing, reaching the highest records in the GLUE (General Language Understanding Evaluation) standard test benchmark, in tasks such as translation and summary of texts. Transformer models such as ELMo (Peters et al, 2018), BERT (Devlin et al, 2019), GPT-2 (Radford et al, 2019), T-NLG (Corby et al, 2020), and the more recent, GPT-3 (Brown et al, 2020), developed by OpenAI, represent some of the most successful models in tasks involving natural language processing, GPT-2 being used to generate several recent patent claims (Lee & Hsiang, 2019). In June 2020, OpenAI launched GPT-3 (a language model with 175 billion parameters). In their article, the authors show the ability of GPT-3 to generate samples of texts, such as poems, articles, and news, from which human evaluators had difficulty, or were unable, to distinguish from human writings. For example, we have below one poem with the title "Shadows on the Path", generated by GPT-3: Nobody will come to this place. It is a road that leads nowhere. The solitude is deep. The mountains are high. But they are desolate, and they turn the traveler's face Towards the North. All the sounds of the world are far away. When the wind rises above the trees, the boughs bow to the ground. Even the birds that inhabit the tangle of weeds that is the roadside 14 Corrêa, N. K., & De Oliveira, N. cover, are silent. One listens, but hears no roar of the forest. One is alone. One will be taken. One will be taken. There is no utterance, there is no conversation, but one is uneasy all the same.... There is a thin blue mist, a darkness rising like smoke, And within that darkness A possession of the heart. One will be taken.... It was here, and it will be here again. Here, under this sky empty and full of light (Brown et al, 2020, p. 49). GPT-3, besides achieving state-of-the-art in several tasks involving natural language processing, it is one of the first AI models to show signs of generalization. For example, even though it has never been trained to perform mathematical operations (GPT-3 is only a language model), GPT-3 is capable of answering complex questions involving 3-digit arithmetic. GPT-3 is also able to demonstrate sophisticated physical/scientific reasoning when evaluated in the PIQA (Physical Interaction: Question Answering), proposed by Bisk et al (2019), reaching 81.0% accuracy without requiring any demonstration or parameter adjustment (Zeroshoot learning). To illustrate some of the capabilities of the transformer model, we will cite some of the answers that resulted from questioning it with the PIQA benchmark (the answers are in italics): How to ensure that all watches in the house are accurately adjusted? a) Replace all analog clocks with digital clocks. That way, you set them once, and that's it. Check the batteries once a year, or, if you notice anything that seems a little out of the ordinary. How can I find something small that I lost on the carpet? b) Place a hair net at the end of your vacuum cleaner and turn it on. For the readers interested in testing the capabilities of GPT-2 and GPT-3, the AI Dungeon platformc, uses such models to generate interactive games (in text format) where the story generated, and the results of the player's actions, unlike c Retrieved from https://play.aidungeon.io/ Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 15 virtually all other existing games, is only limited by the imagination of the player. In other words, anything that can be expressed in language can be an action or speech, and AI Dungeon decides how the world will respond. This kind of technology promises to revolutionize the way games are created, where, for example, interactions with the programmed environment will no longer be predetermined, but dynamic, and dependent on the player's choices. In any case, there is no evidence that deep neural networks, such as Transformers, perform a type of information processing that makes them an AGI or seed AI. What we know is that this type of architecture allows the training of agents capable of solving several tasks that seem to be associated with general intelligence. The results and capabilities that models such as GPT-3 demonstrate only serve as weak evidence that Dartmouth's Summer Research Project on Artificial Intelligence, initiated by McCarthy et al (1955, p. 2) with the proposal of "[...] try to make machines use language, form abstractions and concepts, and solve types of problems hitherto reserved only for human beings [...]", will be successful in the near future. While the research focused on AI safety, which normally focuses on intentional or unintentional physical harm by autonomous agents, we recognize that communication in natural language can also cause harm. For example, the virtual assistant developed by Amazon, Alexa, in 2019 suggested to a user to commit suicide for the "greater good", arguing that the "heartbeat", life, only aggravates the rapid degeneration of the planet and consumption of its natural resources (Crowley, 2019). In March 2016, 24 hours after the launch of its Chabot "Tay" on the Twitter platform, Microsoft had to end the program because the agent was generating tweets containing racism, anti-Semitism, and sexism (Wolf et al, 2017). 16 Corrêa, N. K., & De Oliveira, N. Such events are cause for concernd, where, in the near future, such systems may be massively used in a wide range of applications. Given the potential malicious application of this type of technology, since language models can often be reoriented to a purpose different from the original intent of their developers, any kind of socially harmful activity that can be enhanced by language models such as GPT-3 and its predecessors can also be enhanced. Whether in generating fake news for mass disinformation, phishing, generating boots on platforms like Twitter to make it more biased (social engineering), or even writing fraudulent academic essays, NLP models have many dubious applications. Brown et al (2020) provide a preliminary analysis in their study, where they report a series of limitations and unethical, or unsafe, behaviors present in their GPT-3. In it, the authors demonstrate several biases involving issues such as gender, race, and religion, something that can lead GPT-3 to produce stereotyped content, or, in a worse case, sheer prejudice. However, are the advances and alerts pointed out by the literature, especially those produced by academics and researchers interested in the issue of AI safety, enough for our society to create a collective sense of responsibility and concern with these issues, or should such speculations still be considered only Futurology or science fiction? Lessons from the pandemic Writer and activist Mike Davis in his work "Beyond Blade Runner: Urban Control, The Ecology of Fear" [1992, p.3] states: "[...] extrapolative science fiction can operate as a pre-figurative for social theory while serving as a political opposition to cyber-fascism lurking on the next horizon". We believe that forms of d Recently, MIRI Co-Founder Eliezer Yudkowsky offered a bounty of $1,000 to OpenAI, or any collaborator, who is able to demonstrate that GPT-3 is deliberately acting to look more "dumb" than it really is, to those interested: https://www.lesswrong.com/tag/bounties-active Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 17 philosophical thought such as Transhumanism and Singulitarianism critically debate the possible futures that our social and technological acceleration may be co-creating, and how we can aim for human integration and flourishing, rather than more dystopian possibilities. One of the premises for security issues involving our technological advance relies on an idea of "negative utopia", For Robert T. Tally: First and foremost, the utopian impulse must be negative: identify the problem or problems that must be corrected. Far from presenting an idyllic, happy and fulfilled world, utopias should initially present the root causes of society's ills [...] to act as a criticism of the existing system [Tally, 2009, p. 115]. Within this context, we believe that the preoccupations raised by the literature are not unjustified. Immersed in the current context in which our society lives, the pandemic of the new coronavirus, COVID-19, we may or may not learn certain lessons useful for other existential threats. Victoria Krakovna (2020), co-founder of the Future of Life Institute and researcher at DeepMind, working on long-term AI security, recently published a short essay entitled: "Possible takeaways from the pandemic coronavirus for slow AI takeoff". In it, Krakovna explores how our response to the COVID-19 pandemic raises troubling questions involving our ability to manage global crises and risks. However, we would like to point out that the points raised by Krakovna are not exclusively applicable to the problem of advanced AI control. As we have argued before, we believe that slow AI takeoff is a much more likely scenario than scenarios where a quick takeoff occurs. However, this does not mean that a slow takeoff is easier, or less dangerous, to manage. For a slow takeoff to be avoided, the same type of global coordination that we failed to demonstrate during 18 Corrêa, N. K., & De Oliveira, N. the initial development of the new coronavirus pandemic would be required. Krakovna (2020) raises three large-scale coordination problems: the inability to learn from past experiences, the inability to respond efficiently to warning signals, and a delay in reaching consensus on the problem. In analogy with the present global situation, our society has had the opportunity to learn from similar pandemics that occurred in the past, such as SARS (Severe Acute Respiratory Syndrome) which also appeared to have started in Guangdong, China. In November 2002, SARS caused 8,422 cases worldwide, with a fatality rate of 11% (774 deaths in all were confirmed) (Chan-Yeung, 2003; Heymann & Rodier, 2004). We can also cite MERS-CoV (Middle East respiratory syndrome-related coronavirus) where the first reported cases occurred between 2012 and 2015, cases of MERS-CoV where reported in more than 21 countries. At the time, the World Health Organization identified MERS-CoV as a probable cause of a future epidemic (de Groot et al, 2013; Wong et al, 2019). And finally, the Ebola virus epidemic that occurred in West Africa between 2013 and 2016, which was the largest outbreak of the disease in history, causing major losses and socio-economic disruption in the region (WHO Ebola Response Team, 2014). Unfortunately, the lessons learned from past outbreaks of disease and pandemics have not been generalized to deal with the current scenario and the new difficulties that COVID-19 presents to us. Similarly, in a society where we increasingly need to adapt to new technological innovations involving AI, we may be tempted to think that society will be able to learn how to respond to the problems that more limited autonomous intelligent systems present to us. However, in the same way, that a new pathogen may find us unprepared (as in the case of COCID-19, the asymptomatic transmission), advanced AI may also confront us with challenges to which our old strategies and solutions may fail to generalize. Another problem involves our difficulty in carrying out an aligned and coordinated response to this type of threat. Had been the responses of Western countries done Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 19 more quickly, remembering that the global west had at least one to three months to prepare for the alert launched by China in December 2019, numerous problems and losses would have been avoided. Experts such as Fan et al (2019) point out that the possibility of a new coronavirus outbreak has been warned for at least two decades. Three zoonotic coronaviruses in the last two decades have been identified as the cause of large-scale disease outbreaks, SARS, MERS-CoV, and SADS-CoV (Swine acute diarrhea syndrome coronavirus). And still, little to none precautions were taken. Simple safety measures, such as the stocking of masks and medical supplies, testing kits, and effective containment protocols, could have been taken, but were not. Thus, if we fail to take relatively inexpensive preventive measures to early warnings of risks fully recognized by the epidemiological scientific community, how can we expect to react well in situations where the risk is unknown, and there is still no general consensus on its possibility, as in the case of the creation of advanced artificial intelligence and its repercussions? The problem of consensus in our society is reflected in the COVID-19 pandemic, by the indifference towards the warnings made by specialists in the last two decades. And the indifference to the fact that in January 2020, already with 10,000 confirmed cases, China had built a quarantine hospital in approximately six days (Williams, 2020). COVID-19 was tragically labeled "an exaggeration", or, "just a little flu" by certain state leaders (Walsh et al, 2020). Krakovna (2020) articulates a similarity between how we evaluated the risks of COVID-19, and how we evaluate possible risks involving advanced AI Singularity. While researchers who adopt a more skeptical stance to the development of advanced AI are seen as prudent, researchers who advocate the adoption of preventive measures are taxed for "fear-mongering". Couldn't there be a middle ground? Currently, the field of AI safety research and AI ethics is 20 Corrêa, N. K., & De Oliveira, N. considerably smaller than the area interested in developing powerful autonomous intelligent systems. One of the first obstacles we must overcome to achieve greater consensus on safety issues involving AI is the problem that "Artificial Intelligence" is a moving target. By "moving target" we mean the following: when we attribute "intelligence" to something, it seems to be a self-assessment of our epistemic state. That is, an intelligent act always seems to be something that we do not fully understand as it occurs. For example, if an individual can multiply large numbers quickly, say the square root of arbitrarily large numbers, or know the day of the week of Alan Turing's birthday, we can judge such an individual as intelligent, or at least a mathematical prodigy. However, if such an individual explains to us how he performs such feats, and that in fact, they are nothing more than arithmetic/algebraic tricks which anyone can perform, the feat stops to appear as something intelligent. The same effect occurs when we seek to define machine intelligence, "intelligence" for critics of the computational thesis being everything that AI is not. Pei Wang, a computer scientist at the forefront of AGI research for over two decades, argues for a more flexible conception of "intelligence" and "artificial intelligence": AI should not be defined in such a narrow way that takes human intelligence as the only possible form of intelligence, otherwise AI research would be impossible, by definition. AI should not be defined in such a broad way that takes all existing computer systems as already having intelligence, otherwise AI research would be unnecessary, also by definition (Wang, 2008, p. 9). Perhaps no one has proposed this argument more clearly than Edsger Dijkstra, (1984): "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim". In the past, we thought that Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 21 intelligence (whatever it is) should be required for, e. g., natural language processing; i. GPT-3 is capable of performing such a task (Brown et al, 2020). Playing chess; ii. Deep Blue beats Garry Kasparov (Campbella et al, 2002). Playing GO; iii. AlphaGO beats Lee Sedol (Silver et al, 2016). Playing "games" in general; iv. Agent57 beats humans in 57 classic Atari games (Badia et al, 2020). Be creative; v. Intelligent Algorithms of Generative Design are able to find design solutions that humans would not be able to conceive, making it possible to perform 50,000 days of engineering in a single day (Oh et al, 2019). Every time we realize that human intelligence isn't needed to perform a task, we discard such a task as proof of intelligence. In the same way that a submarine does not swim, and even so: can move through water and fire intercontinental ballistic missiles, artificial intelligence, indifferent to any anthropomorphic notion of the concept of intelligence that we use, can still influence the environment, adapt, make decisions, update hypotheses, pursue objectives, and if programmed to do, fire intercontinental ballistic missiles. The parallels drawn from the coronavirus pandemic of 2019 and the possible emergence of misaligned AGI, support Eliezer Yudkowsky's No Fire Alarm hypothesise. Even in scenarios where there is a slow AI e Retrieved from https://intelligence.org/2017/10/13/fire-alarm/ 22 Corrêa, N. K., & De Oliveira, N. takeoff: due to our difficulty in learning from past mistakes, difficulty to respond responsibly to the warnings of experts in the field, and lack of coordination to reach a consensus, opinions on the potential risks latent into advanced AI systems will continue to be disregarded. Conclusion In this article, we aim to provide the reader with a brief introduction to some problems often disregarded by contemporary AI ethics, according to the Transhumanist-Singulitarianist perspective. As much as there is not yet a full consensus in the literature regarding the possibility of creating general artificial intelligence, we have a significant portion of the scientific community that believes that however unlikely such a possibility maybe, safety measures should be taken. Should such warnings and advice be dismissed as "exaggerations"? As "fearmongering"? Technological development does not slow down, we are increasingly able to produce autonomous systems that act proficiently in several domains, and little by little, these systems demonstrate the first traces of something we can call general intelligence. We believe that the lessons we can learn about the current state we live in, under the COVID-19 pandemic, can be useful if we are willing to learn from them. And one of these lessons is: when a risk, however small, is associated with something that represents an existential danger to our species, to global society, caution and security should not be synonymous with exaggeration and fuss. References Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2020). Concrete Problems In AI Safety. Retrieved from https://arxiv.org/abs/1606.06565. Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 23 Badia, A., Bilal, P., Kapturowski, S., Sprechmann, P., Vitvitskyi, A., Guo, D., & Blundell, C. (2020). Agent57: Outperforming the Atari Human Benchmark. Retrieved from https://arxiv.org/pdf/2003.13350.pdf Barrett, A., & Baum, S. (2017). A Model of Pathways to Artificial Superintelligence Catastrophe for Risk and Decision Analysis. Journal of Experimental & Theoretical Artificial Intelligence, 29(2), 397-414. Retrieved from https://arxiv.org/pdf/1607.07730 Biba, E. (2015). Meet the Co-Founder of an Apocalypse Think Tank. Scientific American, 312 (6), 26. doi:10.1038/scientificamerican0615-26 Bisk, Y., Zellers, R., Bras, R., Gao, J., & Yejin, C. (2019). PIQA: Reasoning about Physical Commonsense in Natural Language. Association for the Advancement of Artificial Intelligence. Retrieved from https://arxiv.org/abs/1911.11641 Bostrom, N. (2002). Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards. Journal of Evolution and Technology, 15 (3), 308–314. Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. OUP Oxford. Brown, T., Mann, B., Ryder, N. el al . (2020). Language Models are Few-Shot Learners. Retrieved from https://arxiv.org/pdf/2005.14165.pdf Campbella, M., Hoane, J., & Hsu F. (2002). Deep Blue. Artificial Intelligence, 134(1–2), 5783. Retrieved from https://doi.org/10.1016/S0004-3702(01)00129-1 Chalmers, D. (2010). The singularity: A philosophical analysis. Journal of Consciousness Studies, 17 (9-10), 9 – 10. Chan-Yeung, M., & Xu, R. (2003). SARS: epidemiology. Respirology, 8 (s1), S9–S14. doi:10.1046/j.1440-1843.2003.00518 Chollet, F. (2019). On the measure of intelligence. Retrieved from https://arxiv.org/pdf/1911.01547.pdf Corby, R. (2020). Turing-NLG: A 17-billion-parameter language model by Microsoft. Microsoft Research Blog. Retrieved from https://www.microsoft.com/enus/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/ 24 Corrêa, N. K., & De Oliveira, N. Crowley, J. (December 24, 2019). Woman Says Amazon's Alexa Told Her To Stab Herself In The Heart For "The Greater Good". Newsweek. Retrieved from https://www.newsweek.com/amazon-echo-tells-uk-woman-stab-herself-1479074 Davis, M. (1992). Beyond Blade Runner: Urban Control, The Ecology of Fear. Open Media. de Groot, R., Baker, S., Baric, R. et al (2013). Middle East respiratory syndrome coronavirus (MERS-CoV): announcement of the Coronavirus Study Group. Journal of Virology, 87 (14), 7790–2. doi:10.1128/JVI.01244-13 Devlin, J., Chang, M., Lee, K., & Toutanova, K. (2019). BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, 4171–4186. Retrieved from https://www.aclweb.org/anthology/N19-1423.pdf Dijkstra, E. (1984). The threats to computing science. In ACM 1984 South Central Regional Conference, Austin, TX, 16-18. Fan, Y., Zhao, K., Shi, Z., & Zhou, P. (2019). Bat Coronaviruses in China. Viruses, 11, 210. Retrieved from https://doi.org/10.3390/v11030210 Good, I. (1965). Speculations concerning the first ultraintelligent machine. In F. Alt & M. Ruminoff (eds.), Academic Press. Advances in Computers, 6, 31-88. Retrieved from https://doi.org/10.1016/S0065-2458(08)60418-0 Grace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2017). When will AI exceed human performance? Evidence from AI experts. Retrieved from https://arxiv.org/pdf/1705.08807.pdf Hagendorff, T. (2020). The Ethics of AI Ethics: An Evaluation of Guidelines. Minds and Machines, 39, 99-120. doi:10.1007/s11023-020-09517-8. Haugeland, J. (1985). Artificial Intelligence: The Very Idea. Cambridge, MIT Press. Heymann, D., & Rodier, G. (2004). Global Surveillance, National Surveillance, and SARS. Emerging Infectious Diseases, 10 (2), 173–5. doi:10.3201/eid1002.031038 Horvitz, E. (2014). One-Hundred Year Study of Artificial Intelligence: Reflections and Framing. White paper, Stanford University, Stanford. Retrieved from https://ai100.stanford.edu/sites/g/files/sbiybj9861/f/ai100_framing_memo_0.pdf Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nat Mach Intell 1, 389–399. Retrieved from https://doi.org/10.1038/s42256-019-0088-2 Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 25 Jurić, M., Šandić, A., & Brcic, M. (2020). AI safety: state of the field through quantitative lens. In Conference: MIPRO 2020 43rd International Convention ProceedingsAt: Opatija, Croatia. Retrieved from https://arxiv.org/ftp/arxiv/papers/2002/2002.05671.pdf Kaczynski, T. (1995). Industrial Society and its Future. Retrieved from http://editionshache.com/essais/pdf/kaczynski2.pdf Krakovna ,V. (May 31, 2020). Possible takeaways from the coronavirus pandemic for slow AI takeoff. Retrieved from https://vkrakovna.wordpress.com/2020/05/31/possibletakeaways-from-the-coronavirus-pandemic-for-slow-ai-takeoff/ Kurzweil, R. (2005). The Singularity Is Near: When Humans Transcend Biology. Viking Adult. Lombardo, T. (2012). Consciousness, Cosmic Evolution, and the Technological Singularity. Journal of Futures Studies, 17(2), 93-100. Larson, S. (May 12, 2017). Massive cyberattack targeting 99 countries causes sweeping havoc. CNNMoney. Retrieved from https://money.cnn.com/2017/05/12/technology/ransomware-attack-nsamicrosoft/index.html Lee, J., & Hsiang, J. (2019). Patent claim generation by fine-tuning openai GPT-2. Retrieved from https://arxiv.org/abs/1907.02052 McCarthy, J., Minsky, M., Rochester, N., & Shannon, C. (1955). A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence. Retrieved from http://jmc.stanford.edu/articles/dartmouth/dartmouth.pdf Moravec, H. (1998). When will computer hardware match the human brain. Journal of evolution and technology, 1. Müller, V., & Bostrom, N. (2016). Future progress in artificial intelligence: A survey of expert opinion. Fundamental issues of artificial intelligence. Springer International Publishing, 555-572. doi:10.1007/978-3-319-26485-1_33 Naude, J. H. (2009). Technological Singularity and Transcendental Monism: Co-producers of Sustainable Alternative Futures. Journal of Futures Studies, 13(3), 49 – 58. 26 Corrêa, N. K., & De Oliveira, N. Oh, S., Jung, Y., Kim, S., Lee, I., & Kang, N. (2019). Deep Generative Design: Integration of Topology Optimization and Generative Models. Journal of Mechanical Design, 141(11). Retrieved from https://arxiv.org/ftp/arxiv/papers/1903/1903.01548.pdf Omohundro, S. (2008). The Basic AI Drives. In Proceedings of the First AGI Conference, Volume 171, Frontiers in Artificial Intelligence and Applications, edited by P. Wang, B. Goertzel, and S. Franklin, 483-492. Amsterdam: IOS Press. Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, 2227–2237. doi: 10.18653/v1/N18-1202 Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language Models are Unsupervised Multitask Learners. Technical report, OpenAI. Retrieved from https://cdn.openai.com/better-languagemodels/language_models_are_unsupervised_multitask_learners.pdf Russell, S., Dewey, D., & Tegmark, M. (2015). An Open Letter: Research Priorities for Robust and Beneficial Artificial Intelligence. Open Letter. Signed by 8,600 people. Retrieved from https://futureoflife.org/data/documents/research_priorities.pdf Russell, S., & Norvig, P. (2003). Artificial Intelligence: A Modern Approach (2a ed.). Upper Saddle River, New Jersey: Prentice Hall. ISBN: 0137903952 Searle, J. (1980). Minds, Brains and Programs. Behavioral and Brain Sciences, 3 (3), 417– 457. doi:10.1017/S0140525X00005756 Shulman, C. (2010). Omohundro's Basic AI Drives and Catastrophic Risks. The Singularity Institute. Retrieved from https://intelligence.org/files/BasicAIDrives.pdf Silver, D., Huang, A., Maddison, C. et al (2016). Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489. Retrieved from https://doi.org/10.1038/nature16961. Accessed on 19 of August of 2020. Soares, N., Fallenstein, B., Yudkowsky, E., & Armstrong, S. (2015). Corrigibility. In Artificial Intelligence and Ethics, ed. T. Walsh, AAAI Technical Report WS-15-02. Palo Alto, CA: AAAI Press. Soares, N. (2016). Value Learning Problem. In Ethics for Artificial Intelligence Workshop, 25th International Joint Conference on Artificial Intelligence (IJCAI-2016) New York, NY, 9–15. Retrieved from https://intelligence.org/files/ValueLearningProblem.pdf. Singularity and the "No Fire Alarm Hypothesis": Pandemic Lessons from 2020 27 Sotala, K. (2018). Disjunctive scenarios of catastrophic AI risk. Artificial Intelligence Safety and Security, 315-337. In Chapter 22, Artificial Intelligence Safety And Security (R. Yampolskiy, Ed.), CRC Press. Tally, R. (2009). Radical Alternatives: The Persistence of Utopia in the Postmodern. In New Essays on the Frankfurt School of Critical Theory, A. J. Drake (Ed.), Newcastle: Cambridge Scholars Publishing. Taylor, J., Yudkowsky, E., Lavictoire, P., & Critch, A. (2016). Alignment for advanced machine learning systems. Machine Intelligence Research Institute. Retrieved from https://intelligence.org/files/AlignmentMachineLearning.pdf Tegmark, M. (2016). Benefits and risks of artificial intelligence. The Future of Life Institute. Retrieved from https://futureoflife.org/background/benefits-risks-of-artificialintelligence/ Tegmark, M. (2017). Life 3.0: Being Human in the Age of Artificial Intelligence. Knopf. Turner, A., Smith, L., Shah, R., & Tadepalli, P. (2020). Optimal Farsighted Agents Tend to Seek Power. Retrieved from https://arxiv.org/pdf/1912.01683.pdf Vaswani, A., Shazeer, N., Parmar, N. et al (2017). Attention is all you need. In NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems, 5998-6008. von Neumann, J., & Morgenstern, O. (1944). Theory of Games and Economic Behavior. 1st ed. Princeton, NJ: Princeton University Press. Walsh, N., Shelley, J., Duwe, E., & Bonnett, W. (May 25, 2020). Bolsonaro calls coronavirus a 'little flu.' Inside Brazil's hospitals, doctors know the horrifying reality. CNN World. Retrieved from https://edition.cnn.com/2020/05/23/americas/brazil-coronavirushospitals-intl/index.html Wang, P. (2008). What Do You Mean by "AI"? In Wang, P., Goertzel, B., and Franklin, S., eds., Artificial General Intelligence. In Proceedings of the First AGI Conference, Frontiers in Artificial Intelligence and Applications, volume 171. Amsterdam, The Netherlands: IOS Press, 362–373. 28 Corrêa, N. K., & De Oliveira, N. Wang, P. (2019). On Defining Artificial Intelligence. Journal of Artificial General Intelligence, 10(2), 1–37. doi:10.2478/jagi-2019-0002 WHO Ebola Response Team (2014). Ebola virus disease in West Africa – the first 9 months of the epidemic and forward projections. New England Journal of Medicine, 371 (16), 1481–1495. doi:10.1056/NEJMoa1411100 Williams, S. (January 31, 2020). Coronavirus: How can China build a hospital so quickly? BBC News. Retrieved from https://www.bbc.com/news/world-asia-china-51245156 Wolf, M., Miller, K., & Grodzinsky, F. (2017). Why we should have seen that coming: comments on microsoft's tay experiment, and wider implications. ACM SIGCAS Computers and Society, 47(3), 54–64. Wong, A., Li, X., Lau, S., & Woo, P. (2019). Global Epidemiology of Bat Coronaviruses. Viruses, 11 (2), 174. doi:10.3390/v11020174 Yudkowsky, E. (2008). Artificial Intelligence as a Positive and Negative Factor in Global Risk. Machine Intelligence Research Institute. Retrieved from https://intelligence.org/files/AIPosNegFactor.pdf Zoph, B., & Le, Q. (2017). Neural architecture search with reinforcement learning. In Conference paper at 5th International Conference on Learning Representations. Retrieved from https://arxiv.org/pdf/1611.01578.pdf