Correcting Errors in the Bostrom/Kulczycki Simulation Arguments draft, extended with proofs R. Dustin Wehr www.logicforprogress.org February 26, 2017 Abstract Both patched versions of the Bostrom/Kulczycki simulation argument contain serious objective errors, discovered while attempting to formalize them in predicate logic. The English glosses of both versions involve badly misleading meanings of vague magnitude terms, which their impressiveness benefits from. We fix the errors, prove optimal versions of the arguments, and argue that both are much less impressive than they originally appeared. Finally, we provide a guide for readers to evaluate the simulation argument for themselves, using well-justified settings of the argument parameters that have simple, accurate statements in English, which are easier to understand and critique than the statements in the original paper. TL;DR go straight to page 22 1 Contents 1 First Patch and Error 1 7 1.1 Effect on the argument . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 Second Patch and Error 2 11 2.1 Effect on the argument . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3 Improved formalization, Error 3, and Guide to Evaluating the Simulation Argument for Yourself 15 3.1 The special case of Theorem 3 that you should pay attention to, and Error 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.1.1 Example 4-disjunct 1-Parameter Simplified English Version . . 21 3.1.2 Example Implication form, 1-Parameter Simplified English Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.1.3 Example Implication form, 0-Parameter Simplified English Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.1.4 Why I suspect Prop 2 is true (making the implication trivial) . 23 4 Other pernicious vagueness 24 5 Conclusion 25 6 Proofs (supplemental) 27 6.1 Proof of Theorem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 6.2 Proof of Corollary 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 6.3 Proofs that Theorem 1 and Corollary 1 are optimal . . . . . . . . . . 33 6.3.1 Proof of Theorem 2 . . . . . . . . . . . . . . . . . . . . . . . . 33 6.3.2 Proof of Corollary 2 . . . . . . . . . . . . . . . . . . . . . . . 34 2 This note concerns the widely-known Simulation Arguments, first published by Bostrom in [Bos03], and "patched" in two different ways years later in [BK10] to correct an error. Here, we correct three further serious errors, one in the Patch 1 version and two in the Patch 2 version. We also improve the formalization, give complete proofs, and demonstrate the significance of the errors. For the corrected proofs, we show that the impressiveness of the informal English statements is largely due to an accidental equivocation pattern, which [Wal08] calls variability of strictness of standards, with the problematic terms used in [BK10] including "significant number of", "very likely", "extremely small", "almost certainly", "astronomically large", etc (see Sections 1.1, 2.1, and 3 in particular). Variability of strictness of standards occurs when the intended interpretation of a (or several) vague predicate(s) with a natural 1-dimensional notion of magnitude (the "strictness") is left too vague, and the persuasiveness of the argument benefits from using a strong/large-magnitude interpretation for some assumptions, and a weak/small-magnitude interpretation for others.i In Section 3 we give new concise statements of the corrected simulation arguments, which should be used in preference to those in [BK10] or [Bos03]. This paper assumes familiarity with [BK10]. Throughout, PH abbreviates "posthuman". Note 1. This paper is narrowly focused, and does not attempt to provide a summary of other kinds of criticisms of the simulation argument [Wea03] [Bru08] [Lew13] [Bir13], which might be more important than this quite technical one. Also, throughout the paper, whenever possible we use via quotation the informal English glosses of [Bos03] and [BK10] for axioms, definitions and theorems, rather than introducing i[Wal08] demonstrates the pattern by giving a proof of "Nobody should ever give married", which exploits the vagueness of two predicates: roughly, person p can safely predict proposition A, and person p and person q are compatible. For a formalization of the argument in predicate logic, and recommendation on how to methodically criticize it, see page 17 of [Weh15]. 3 our own. In general, we take a completely uncritical stance on the nuanced epistemological issues involved, focusing, essentially, only on the math and logic. Note 2. The following Definition 1 is given at a level above the formality of predicate logic. If given in a fully formal manner, it would define a set of first-order L-structures for a particular language L, which includes not only the five symbols tC,CPH, pop,#sims, N, countEu that are particular to the Simulation Argument (and a couple more for the Patch 2 version), but also many mathematical symbols that have standard meanings, which would be fixed in the definition of the set of L-structures. We will save the reader from excessive jargon and pedantry by counting on our shared understanding of the standard meanings of symbols for numbers and finite sets. Definition 1. A Simulation Argument 1-modelii, or just 1-model, is given by: • A finite nonempty set C, for the "human-level technological civilizations" [BK10]. We use the clearer term advanced human-like civilizations, where advanced means as advanced as the current state of the human race. • A possibly-empty subset CPH of C, for the civilizations that eventually reach a PH stage. • For each c P C, a natural number poppcq, for the cumulative pre-PH population size of c. • For each c P CPH, a natural number #simspcq, for the number of pre-PH-phase ancestral simulations that c does in its PH phase. • A positive integer N , for the number of ancestor simulations that a civilization must eventually run in order to be considered to have run "a significant number of ancestor simulations"[BK10].iii iiIn contrast to 2-model, defined in the Section 2. iiiIntuitively it is an argument parameter, like the parameters q1, q2, q3, d introduced later, but for the purpose of Theorem 1 we make it part of the model. 4 Whenever a model is fixed, we also have the following abbreviations: • CPH – C{CPH, the civilizations that never reach a PH stage. • CăN – tc P C | #simspcq ă Nu and CěN – tc P C | #simspcq ě Nu, the civilizations that run fewer than N and at least N ancestor simulations, respectively. • #U " ř cPC poppcq is the total number of unsimulated observers. iv • #S – ř cPCPH poppcq #simspcq is the total number of simulated observers. The formula conveys [BK10]'s intended interpretation of "ancestor simulation", in which each of a civilization c's simulations contains exactly as many simulated observers as there were unsimulated observers in the pre-PH phase of c.v • avgpopăN –   ř cPCăN poppcq   {|CăN | and avgpopěN –   ř cPCěN poppcq   {|CěN | are the average number of unsimulated observers in the pre-PH phase of civilizations in CăN and CěN , respectively. Next, we introduce some symbols (with different notation) and definitions from [BK10]. Note that all quotations in the following three definitions are from that paper. Definition 2 (fPH, fPH, q1,Prop 1). fPH is informally defined as "The fraction of human-level civilizations that reached a posthuman stage." It and fPH are uncontroversially defined formally by: fPH – |CPH| |CPH| ` |CPH| fPH – 1  fPH Prop 1 is intended to express "The human species is very likely to go extinct before ivNote the implicit assumption that all civilizations are non-overlapping. vNote the implicit assumption that all ancestor simulations are non-overlapping. 5 reaching a PH stage." fPH ă q1 where q1 is a r0, 1s parameter of the argument. In the appendix of [BK10], an example proof with q1 " .01 is demonstrated. Definition 3 (fěN , q2,Prop 2). fěN is informally defined as "The fraction of posthuman civilizations that are interested in running a significant number of ancestor simulations". Here, "significant number" means N . It is uncontroversially defined by fěN – |CěN | |CPH| Prop 2 is intended to express that fěN is "extremely small." Formally: fěN ă q2 where q2 is another r0, 1s argument parameter, set to .01 in [BK10]'s example proof. Definition 4 (fsim, q3,Prop 3). fsim is informally defined as "...the fraction of all observers in the universe with human-type experiences that are living in computer simulations." It is uncontroversially defined by fsim – #S #S `#U Prop 3 is intended to express "We are almost certainly living in a computer simulation." Formally: fsim ą q3 where q3 is another r0, 1s parameter, set to .99 in [BK10]'s example proof. 6 1 First Patch and Error 1 Patch 1 is described in [BK10] as: ...a very weak assumption to the effect that the typical duration (or more precisely, the typical cumulative population) of the pre-posthuman phase does not differ by an astronomically large factor between civilizations that never run a significant number of ancestor simulations and those that eventually do. For example, in an appendix we show how by assuming that the difference is no greater than a factor of one million we can derive the key tripartite disjunction. Formally: Definition 5 (Patch 1). avgpopăN avgpopěN ď d vi where d is a natural number parameter of the argument. The previous quoted passage tells us that the appendix uses the parameter settingvii: avgpopăN avgpopěN ď 1 million (1) Unfortunately, rather than inequality (1) above, the appendix of [BK10] erroneously uses the much strongerviii assumption: avgpopăN avgpopěN ď N 1 million (2) viSome models have avgpopěN " 0, in which case the left side is undefined and so the inequality is false, and then all the results in this paper that depend on Patch 1 hold trivially. viiTechnically there is exactly one other a priori reasonable interpretation of "the difference is no greater than a factor of one million", in which the numerator and denominator of the left hand side of the inequality are flipped. However, that alternative can be ruled out from other statements in the paper, and in any case could not have been intended since it does not help to prove the trilemma. viiiTechnically, (2) is only stronger than (1) when N ď 1012. As we note in Section 3, such a large value of N makes Prop 2 very likely, but not necessarily for an interesting reason. 7 Then, under the additional, quite reasonable assumption N ě 9900,ix they proved that Patch 1Ñ Prop 1_ Prop 2_ Prop 3 However, a massively larger lower bound on N , close to one trillion (see Corollary on page 9), is needed when the erroneous (2) is replaced with (1). This is Error 1. It turns out to be easier to state and understand the dependency of the argument on its parameters N, d, q1, q2, q3 if we do a change of variables. We restrict our attention to settings of the parameters where q1, q2, q3 are all in p0, 1q, x and replace them with R` parameters: k1 " 1  q1 q1 k2 " 1  q2 q2 k3 " q3 1  q3 and then the reader can check: Prop 1 " |CPH| ą k1|C PH | Prop 2 " |CPHXCăN | ą k2|CěN | Prop 3 " #S ą k3#U For example, the parameter setting q1 " .01, q2 " .01, q3 " .99 used in [BK10] corresponds to k1 " k2 " k3 " 99. Let ~k abbreviate k1, k2, k3. Define the expression (LB for lower bound): LBpd,~kq– dk3pk1 ` k2 ` k1k2q ` k3 Now we can state the main results for Patch 1. Let (1 denote entailment for 1-models (Definition 1).xi ixReasonable with respect to the given intended interpretation that a "significant number of" ancestor simulations means ě N ancestor simulations. xi.e. none are 1 or 0. This does not affect the criticism. xiThat is, if A is a sentence and Γ a set of sentences, then Γ (1 A means every 1-model that satisfies each sentence in Γ must also satisfy A. 8 Theorem 1. For all settings of the parameters d P N, ~k P pR`q3: N ą LBpd,~kq,Patch 1 (1 Prop 1_ Prop 2_ Prop 3 The following companion theorem shows that the lowerbound on N in Theorem 1 cannot be weakened, and so in that sense Theorem 1 is as strong as possible given the other assumptions. Theorem 2. xii For all settings of the parameters d P N, ~k P pN`q3: N ě LBpd,~kq,Patch 1 *1 Prop 1_ Prop 2_ Prop 3 For example, if we fix the parameters q1, q2, q3, d as they are (or should be, in the case of d) in the appendix of [BK10], then we get:xiii Corollary. If d " 1 million, q1 " .01, q2 " .01, q3 " .99, xiv then N ą 0.99 trillion,Patch 1 (1 Prop 1_ Prop 2_ Prop 3 N ě 0.989 trillion,Patch 1 *1 Prop 1_ Prop 2_ Prop 3 1.1 Effect on the argument Recall the prose definitions of Prop 2 and Patch 1 from [BK10]: Prop 2: The fraction of posthuman civilizations that are interested in running a significant number of ancestor simulations is extremely small. xiiIt is not a mistake that the domain of ~k is different here. The proof of Theorem 2 (6.3.1), as it is now, uses the fact that k is an integer, whereas the proof of Theorem 1 does not. Thanks to an anonymous refereefor pointing out that this deserves explanation, since it seems like it could be an error. xiiiActually what one gets from substitution is 989, 901, 000, 099; we round in the sound direction for both statements of the Corollary. xivi.e. k1 " k2 " k3 " 99 9 Patch 1: ...the typical cumulative population... of the pre-posthuman phase does not differ by an astronomically large factor between civilizations that never run a significant number of ancestor simulations and those that eventually do. Recall that "astronomically large factor" means d, and "significant number" means N , which we now know must be larger than dk3pk1 ` k2 ` k1k2q ` k3 where the magnitudes of k1, k2, and k3 should be chosen to reflect the severity of the terms "very likely", "extremely small", and "almost certainly", respectively. To see the significance of the error, we consider the effect on the one fully-fleshed out proof given in [BK10]. There the partial parameter setting k1 " k2 " k3 " 99 (equivalently q1 " .01, q2 " .01, q3 " .99) is used. Although we believe that the use of vague magnitude terms should in general be avoided when giving interpretations of proofs, their use for these parameters is harmless enough: 1  q1 = probability 0.99 interpreted as "very likely" q2 = probability 0.01 interpreted as "extremely small" q3 = probability 0.99 interpreted as "almost certainly" But when we consider the meaning of the lower bound on N now, we see a serious problem. Substituting in the settings ~k into N ą dk3pk1 ` k2 ` k1k2q ` k3, we get: N ą d 989, 901 And then, substituting in the the intended interpretations of N and d, we get: "significant number" ą "astronomically large factor" 989, 901 Thus, the persuasiveness of [BK10] benefits from what is clearly a wildly misleading definition of "significant number". 10 2 Second Patch and Error 2 The Patch 2 argument introduces the idea of E-observers, which are the observers (unsimulated and simulated) that satisfy a chosen fixed predicate E. You, by appropriate choice of the predicate E, are an E-observer. The bland indifference principle[Bos03] of the Patch 1 argument, from which the authors of [BK10] justify the jump fsim fraction of observers are simulated Ñ you should believe with credence fsim that you are simulated is made dependent on E: since you cannot tell whether or not you are a simulated E-observer, you should believe, with credence equal to the fraction of simulated Eobservers over all E-observers, that you are simulated. As with the Patch 1 Simulation Agument from the previous section, we will accept the qualitative assumptions of the Patch 2 argument uncritically, focusing only on the mathematical aspects. The Patch 2 argument was not fleshed out in [BK10]. We do that here, and it turns out that the mathematics of the Patch 1 and Patch 2 arguments are practically the same. The 6.2 follows easily from Theorem 1. The (unfixed and fundamentalxv) language of the Patch 2 argument is the language of the Patch 1 argument tC,CPH, pop,#sims, Nu plus the new symbol countE. Definition 6. A Simulation Argument 2-model, or just 2-model, is a 1-model together with a function countE that counts the number of E-observers in any given civilization. When a 2-model is fixed, we also use the following abbreviations: • avgpopEěN and avgpopEăN , the average number of E-observers in civilizations from CěN and CăN , respectively. That is, avgpop E ěN " " ř cPCěN countEpcq ı {|CěN |. xve.g. the symbol { for set difference is fixed, and the symbol CăN is defined in terms of fixed and fundamental symbols, and thus is not itself fundamental. See Note 2 (pg 4). 11 • CEě1ăN and CEě1ěN , the civilizations in CăN (resp. CěN) that contain at least one E-observer. • #UE – ř cPC count Epcq, the total number of unsimulated E-observers. • #SE – ř cPCPH count Epcq #simspcq, the total number of simulated observers. Definition 7 (fEsim, q3, k3,Prop 3 1). The role of fsim in the previous argument is played by fEsim – #SE #SE `#UE The role of Prop 3 in the previous argument is played by Prop 31, defined by fEsim ą q3 Or #SE ą k3#U E using the alternate parameterization introduced in the previous section, which we use in this section as well. The quantitative partxvi of the informal definition of Patch 2 is provided by the following quote from [BK10] (page 4): (i) In a substantial fraction of those pre-posthuman histories that end up running (significant numbers of) ancestor simulations, there is some Eobserver. (ii) Let HspEq be the average number of E-observers among those preposthuman histories that contain some E-observer and that end up running (significant numbers of) ancestor simulations. Let HnpEq be the average number of E-observers among those pre-posthuman histories that contain some E-observer and that do not end up running (significant numbers of) ancestor simulations. It is not the case that HnpEq is vastly greater than HspEq. xviSee footnote xxiii on page 19 for a third item (iii) about not having "other information that enables us to tell that we are not in a simulation". Error 3 relates to that. 12 Unfortunately, that does not quite suffice; this is Error 2 of [BK10], which when fixed results in the absurdity explained in Section 2.1. We actually need the fraction mentioned in (i), which is |CEě1ěN |{|CěN |, to be "substantial" relative to the corresponding fraction for civilizations that run fewer than N ancestor simulations, where the meaning of "substantial" is an unnamed argument parameter. There is also the unnamed parameter that defines (ii)'s "vastly greater than". Fortunately, it is only the product of those parameters that matters for stating the following theorems, so our version of Patch 2, Definition 8, collapses them. Temporarily adopting the notation from the previous quote, we are using these facts:   |CEě1ěN | |CěN |    HspEq " avgpop E ěN and  |CEě1ăN | |CăN |    HnpEq " avgpop E ăN Definition 8 (Patch 2). avgpopEăN avgpopEěN ď d An important special case of the Patch 2 argument, used in [BK10], is clarified by the following: Fact 1. If E is such that no civilization has more than one E-observer, then Patch 2 is equivalent to |CEě1ăN |{|CăN | |CEě1ěN |{|CěN | ď d That is, the fraction of CăN civilizations with an E-observer is at most d times larger than the fraction of CěN civilizations with an E-observer. The reader should convince themself intuitively at this point that whether we are counting all observers, or only E-observers, is inconsequential, since the E-restriction is applied to all civilizations. One way of formalizing this involves proving that the 2-models of the Patch 2 argument can be mapped (in a suitable truth-preserving way) to the 1-models of the Patch 1 argument, where the E-observers of the former become 13 the observers of the latter.xvii That is the approach we take in the proof of Corollary 1 (6.2). Let (2 denote entailment with respect to Definition 6. Corollary 1. For all settings of the parameters d P N, ~k P pR`q3: N ą LBpd,~kq,Patch 2 (2 Prop 1_ Prop 2_ Prop 3 1 Corollary 2. For all settings of the parameters d P N, ~k P pN`q3: N ě LBpd,~kq,Patch 2 *2 Prop 1_ Prop 2_ Prop 3 1 Note 3. The Patch 1 argument is a special case of the Patch 2 argument; just take E to be "I am a member of my civilization". We have gone to some trouble to formalize the Patch 2 argument results in such a way that Corollary 1 and Corollary 2 are almost identical to Theorem 1 and Theorem 2, as it makes the proofs of Corollary 1 (6.2) and Corollary 2 6.3.2) easier. 2.1 Effect on the argument The English glosses of [BK10]'s Patch 2 result suffers from a problem similar to that of their statement of their Patch 1 result, though it takes more effort to show this since the authors provide only a sketch of the Patch 2 argument. Recall that in Section xviiIn the case of [BK10]'s example where E is "My computer age birth rank is 1 billion", where every civilization in a 2-model has 0 or 1 E-observers (as in Fact 3), the corresponding 1-model civilizations have cumulative population size 0 or 1. Of course, a civilization with no observers is probably not compatible with what the authors of [BK10] had in mind by "human-level technological civilizations"[BK10]. However, models containing such trivial civilizations are acceptable according to the assumptions needed in order to prove the mathematical statement Theorem 1. Even if the definitions were tightened to exclude civilizations with no observers, we would still be able to use Theorem 1 to prove Corollary 1. In fact, in the proofs in the appendix, we make use of the permissibility of mathematical models of civilizations with zero cumulative pre-PH population size that nonetheless do pre-PH ancestor simulations. We give further explanation of why such reasoning is beyond reproach in the appendix. 14 1.1 we showed that their assumptions, in relation to their assigned informal English interpretations, imply "significant number" ą "astronomically large factor" 989, 901 Recall from our restatement on page 12 of [BK10]'s gloss of their version of Patch 2: "It is not the case that HnpEq is vastly greater than HspEq." It turns out that, by their word usage and suggested parameter settings: "X is vastly greater than Y " means X Y ą "significant number" 989,901 Thus, just to get the meaning of "X is vastly greater than Y " to be the trivial X ą Y , we need the meaning of "significant number" to be nearly 1 million. We leave deriving the previous absurdity as an exercise for the reader, with this tip: Examine Definition 8 and the two equations that precede it – importantly, the objective error in [BK10]'s Patch 2argument sketch must be fixed before deriving the absurdity. A second serious error in the Patch 2 argument is presented in Section 3. 3 Improved formalization, Error 3, and Guide to Evaluating the Simulation Argument for Yourself The reader may wonder why we have gone through so much trouble to give optimized versions of the Simulation Arguments in terms of the function LB. It is for two reasons. First, we wanted to be sure that we were treating not just the published form of Bostrom/Kulczycki's arguments fairly, but also the ideas behind that form. 15 Thus, we did not settle for merely demonstrating absurd consequences of the errors in the arguments, as we did in Sections 1.1 and 2.1, since such a thing could in principle be fixed; we also proved theorems that attempt to characterize the limitations of the ideas used in the arguments (Theorem 2 and Corollary 2). The second reason is that we use, in this section, settings of the parameters that require knowing the exact form of LB. Observation: Whether or not you have extra concerns about the assumptions and interpretation of the Patch 2 argument that you don't have about the Patch 1 argument, without loss of generality we may restrict attention to the Patch 2 argument. The reason is, first of all, as explained in Note 3, the corrected Patch 1 argument is mathematically a special case of the corrected Patch 2 argument. Second, the next subsection will delay asking the reader to limit their choices for E until Step 3 (and Error 3), so a reader who prefers the Patch 1 argument can stop just before then, and use E = "I am a member of my civilization" to reduce to the Patch 1 argument, instead of the E we recommend (which is a strengthening of [BK10]'s example E). With that observation in mind, let us first take what we have learned about the corrected Simulation Argument to give an equivalent, concise statement that is free from problematic vague magnitude terms such as "significant number of" and "astronomically large". After that, we will try to persuade the reader to fix a couple of the parameters, to get a simpler form. Theorem 3. Let d,N be natural numbers and ~k a triple of positive real numbers. Define four propositions: Prop 1: There are ą k1 times more advanced-human-like civilizations that never reach the PH stage than there are that eventually reach the PH stage. Prop 2: Among the advanced human-like civilizations that eventually reach the PH stage, the number that run fewer than N ancestor simulations is ą k2 times greater 16 than the number that run at least N ancestor simulations. Prop 3: There are ą k3 times more simulated E-observers than non-simulated Eobservers. Prop 4xviii: The average number of E-observers in advanced human-like civilizations that run fewer than N ancestor simulations is ą d times larger than the average number of E-observers in human-like civilizations that run at least N simulations. Let (2 denote entailment with respect to Definition 6. Then N ą dk3pk1 ` k2 ` k1k2q ` k3 (2 Prop 1_ Prop 2_ Prop 3_ Prop 4 and when the ~k are integersxix, the bound is tight: N ě dk3pk1 ` k2 ` k1k2q ` k3 *2 Prop 1_ Prop 2_ Prop 3_ Prop 4 We will now be able to see more clearly the fundamental difficulty of evaluating the simulation argument: When k1k2k3 xx is large, as suggested in [BK10], the truth (or probable truth) of the less-interesting Prop 2 and Prop 4 are difficult to assess, and moreover: • Making d large makes Prop 4 less likely and Prop 2 more likely. • Making d small makes Prop 2 less likely and Prop 4 more likely. But we need both Prop 4 and Prop 2 to be unlikely in order to conclude that the more exciting proposition, Prop 1 _ Prop 3, is likely. This is the sense in which the Simulation Arguments exploits variability of strictness of standards, as mentioned in the introduction. xviiiFormerly Patch 2. xixProbably unnecessary, but it's a current limitation of the proof. xxNote that dk1k2k3 is the dominating term in LBpd,~kq 17 3.1 The special case of Theorem 3 that you should pay attention to, and Error 3 Let A-Civilizations abbreviate "advanced human-like civilizations". Step 1 Set k3 " 1, so that Prop 3 becomes "There are more simulated E-observers than nonsimulated E-observers." Then, if you accept [BK10]'s interpretation of the proposition using the "bland indifference principle", you get Prop 3: You are probably simulated (more likely than not). Convince yourself that this simplification does not undercut the strength of the Simulation Argument. In particular, convince yourself that we should make Prop 2 and Prop 4 as unlikely as possible (since they're less interesting than Prop 1 and Prop 3) subject to the constraint that Prop 3 remains both profound and easy to internalize. Minimizing k3 makes Prop 2 and Prop 4 as unlikely as possible. Step 2 Set k1 just large enough to strongly overestimate your best-guess subjective probability that the human race is destroyed before reaching the PH stagexxi. [BK10]'s suggestion of k1 " 99 should suffice for all but the most pessimistic among us, but a smaller value may suffice as well. Keep in mind that a larger value of k1 (or k2, k3, or d) weakens the Simulation Argument by forcing a larger value of N . For k2 " 99, if you accept [BK10]'s reasoning that we should, roughly, treat our civilization as a random sample from the A-Civilizations, then you get xxiRecall that "the PH stage" does not merely mean super-intelligence. It demands all the advances necessary to allow for an ancestor simulation that is as convincing as the world we live in now. 18 Prop 1: There's at least a 99% chance that the human race is destroyed before it reaches the posthuman stage. Step 3 (and Error 3) Take E to be an elaboration of a proposition that, first of all, like [BK10]'s "My computer age birth rank is 1 billion", singles you out within the human race, and singles out at most 1 entity in each of the A-Civilizations (so that the average number of E-observers in any set of A-Civilizations is at most 1). Second, your E should contain any knowledge about our world that is pertinent to whether we might be living in a simulation, or pertinent to whether we are living in a civilization that dies out before the PH stage or does fewer than N ancestor simulations. The bolded point is because that question is not independent, in the Bayesian probability sense, of the question of whether we are living in a simulation. To see this, simply note that the main novelty of Bostrom's Simulation Argument is the use of the assumption P pp is simulated | p's originalxxiicivilization does ě N ancestor simulationsq ě N N ` 1 which is must closer to 1 than any reasonable prior P pp is simulatedq. The authors of [BK10] miss that point with their example Exxiii(this is Error 3). Thus, "My computer age birth rank is X" is not sufficient. You must at least strengthen it to: xxiiUnsimulated xxiiiThey are, however, aware that care must be taken in defining E, as evidenced by item (iii) (page 4) of their sketch of Patch 2, which we reprint here: There is no defeater, i.e. we have no other information that enables us to tell that we are not in a simulation. (A defeater could be some more specific centered proposition E1 such that we know that we are E1-observers and such that we have empirical grounds for thinking that most E1-observers are not in simulations.) As we attempt to convey with our suggested (start at a definition of) E above, any grounds for thinking that we are in a civilization with a significant chance of destroying itself before reaching the posthuman state, is grounds for thinking that E-observers are less likely to be in simulations. 19 My computer age birth rank is X, I live in a single-planet civilization, with several nations that have thermonuclear arsenals capable of destroying the world, the militaries of the two nations with the largest nuclear arsenals are commanded by thugs, 〈something about climate change〉, etc... Moving on, regardless of the specific definition of E, restricting E to definitions that single out at most 1 entity in every A-Civilization simplifies Prop 4 to: Prop 4 version (a): The fraction of E-observer-having civilizations among A-Civilizations that run fewer than N ancestor simulations is more than d times greater than the fraction of E-observer-having civilizations among A-Civilizations that run at least N ancestor simulations. Optional Step 4 Set d " 1, as suggested by one of the authors of [BK10] in private communication. Then Prop 4 further simplifies to: Prop 4 version (b): The frequency of E-observer-having civilizations among our-technology-or-better civilizations that run fewer than N ancestor simulations is greater than the frequency of E-observer-having civilizations among our-technology-or-better civilizations that run at least N ancestor simulations. [BK10] argues that the previous simplified version of Prop 4 is false, which would take us back to a tripartite disjunction, but that is too generous in light of Error 3 (subsection Step 3 (and Error 3) on page 19). Moving on, we now have an argument with a single parameter, k2. The final result of these simplifications is given on the next page. 20 I believe the following is essentially the most compelling single parameter version of [BK10]'s repaired Simulation Argument possible. If you wish to consider two parameter versions, which I believe benefits the Simulation Argument, I recommend stopping the simplifications before Optional Step 4. We give the simplified English statement of the theorem for k2 " 1 in the next section. 3.1.1 Example 4-disjunct 1-Parameter Simplified English Version Let k2 be any positive real number. Recall A-Civilizations abbreviates "advanced human-like civilizations". See Step 3 (and Error 3) for the meaning of "E-observer". Then at least one of the following is true: • Prop 1: There's at least a 99% chance that the human race is destroyed before it reaches the posthuman stage. • Prop 2: There are some A-Civilizations that reach the posthuman stage, but among those there are at least k2 times as many that run fewer than N " 100pk2 ` 1q ancestor simulations than there are that run at least N ancestor simulations. • Prop 3: You are probably simulated (more likely than not). • Prop 4: The fraction of E-observer-having civilizations among A-Civilizations that run fewer than N ancestor simulations ă than the fraction of E-observerhaving civilizations among A-Civilizations that run at least N ancestor simulations. We find the result is easier to understand as an implication, putting the negations of the more-technical Prop 2 and Prop 4 on the left hand side and the clearly-profound Prop 1 _ Prop 3 on the right hand side, as follows. 21 3.1.2 Example Implication form, 1-Parameter Simplified English Version Let k2 be any positive real number. Recall A-Civilizations abbreviates "advanced human-like civilizations". See Step 3 (and Error 3) for the meaning of "E-observer". Assume that at least one of the A-Civilizations reach the posthuman stage, since otherwise Prop 1 is trivially true. If Prop 2: Among the A-Civilizations that eventually reach the posthuman stage, there are less than k2 times as many that run fewer than N " 100pk2`1q ancestor simulations than there are that run at least N ancestor simulations, and Prop 4: The fraction of E-observer-having civilizations among A-Civilizations that run fewer than N ancestor simulations ě the fraction of E-observerhaving civilizations among A-Civilizations that run at least N ancestor simulations. then At least one of Prop 1 or Prop 3 are true ("There's at least a 99% chance that the human race is destroyed before it reaches the posthuman stage", or "You are probably simulated (more likely than not)"). Observe that the dependence of N on k2 makes for a very difficult task of setting k2 to maximize the acceptability of Prop 2 in the antecedent. Making k2 large (say, 99 as in [BK10]) seems at first like a good strategy of making the antecedent true, but then you notice that doing so raises N , and so shifts more of the eventually-posthuman civilizations into the category of running fewer than N ancestor simulations. 22 3.1.3 Example Implication form, 0-Parameter Simplified English Version Here we do a final simplification, fixing the parameter k2 to 1. Recall A-Civilizations abbreviates "advanced human-like civilizations". See Step 3 (and Error 3) for the meaning of "E-observer". Assume that at least one of the A-Civilizations reach the posthuman stage, since otherwise Prop 1 is trivially true. If Prop 2: Among the A-Civilizations that eventually reach the posthuman stage, there are fewer that run ă 200 ancestor simulations than there are that run ě 200 ancestor simulations, and Prop 4: The fraction of E-observer-having civilizations among A-Civilizations that run ă 200 ancestor simulations is greater than the fraction of Eobserver-having civilizations among A-Civilizations that run ě 200 ancestor simulations. then At least one of Prop 1 or Prop 3 are true ("There's at least a 99% chance that the human race is destroyed before it reaches the posthuman stage", or "You are probably simulated (more likely than not)"). 3.1.4 Why I suspect Prop 2 is true (making the implication trivial) I will make this quick. First, in such posthuman civilizations that are so profoundly advanced that they can build a simulation as realistic as what we perceive as real life, wealthy individuals would simply have too little to gain to put in the expense of fully programming something so complex, and then running it N times. We must 23 be very careful not to succumb to the issue of variability of strictness of standards (mentioned in the introduction) when it comes to evaluating the plausibility of this. In particular, the more feasible it is for an institution in an A-Civilization to run an ultra-realistic ancestor simulation (and some argue it's not possible at all), the less interesting it will be to do so. Second, I believe that in a civilization capable of surviving long enough to run such simulations, most people will believe that running such realistic ancestor simulations is profoundly unethical, since it involves, for example, prolonged torturing of individuals, who they would consider sentient, in ways that would astonish any of us who has not done a brief review of the ghastly history of torture. I, for example, would ardently advocate for and participate in pre-emptive military action against any institution who was planning such a thing. If that view was popular, ancestor simulations would need to be carried out in secret. Moreover, for Prop 2 to be false, it would need to be more common than not that in such posthuman civilizations, the psychopaths responsible for secretly perfecting such ancestor simulationsxxiv run, on average, at least 200 such simulations before getting bored (for k2 " 1). 4 Other pernicious vagueness Proponents of the simulation argument, when faced with the criticism of the plausibility of a civilization running stupendously expensive ancestor simulations, will respond with the possibility of software tricks to reduce the energy cost, e.g. • classical approximations of quantum physics whenever possible (since there is no good reason to have confidence that quantum computing will ever be a reality). • editing of the simulation when bugs arise. xxivI say perfecting since, in order to count in the argument, an ancestor simulation needs to reach our point in technological advancement before crashing or veering off course. 24 • starting the simulation from a "recent" but complex state, e.g. the beginning of life on Earth rather than the big bang. Those are not complete responses, since those tricks impose a much greater cost of software development and during-simulation maintenance. For example, the first idea presumes that our descendants will be able to specify when quantum-classical differences affect the simulation in a way that humans would notice, and when they don't. Even if our descendants have superintelligence-powered program generators, there is no shortcut for software specification. The second requires manual modification of the state, and a software model that supports that (and we have no good reason to believe that neural nets will ever support that). The third requires specifying the initial state; again, there is no shortcut for writing such a complex specification. In general, do not let the question of the cost of running N ultra realistic ancestor simulations, which is dominated by energy, distract you from the cost of running one ultra realistic ancestor simulation, which is dominated by the effort of intelligent agents, and constrained by the nature of reality. 5 Conclusion We corrected three remaining serious errors in two "patched" versions of the Simulation Argument, and analyzed their significance carefully in the language of mathematical logic. We found that, although the corrected arguments are sound, their meaning is highly dependent on the settings of their (interdependent) parameters, and we do not believe the parameters can be set in a way that makes the Simulation Argument nearly as impressive as it appeared in [BK10], where the example parameter settings turned out to rely on inconsistent interpretation of vague magnitude terms. 25 References [Bir13] Jonathan Birch. On the "simulation argument" and selective scepticism. Erkenntnis, 78(1):95–107, 2013. [BK10] Nick Bostrom and Marcin Kulczycki. A patch for the simulation argument. Analysis, pages 54–61, 2010. [Bos03] Nick Bostrom. Are we living in a computer simulation? The Philosophical Quarterly, 53(211):243–255, 2003. [Bru08] Anthony Brueckner. The simulation argument again. Analysis, 68(3):224– 226, 2008. [Lew13] Peter J Lewis. The doomsday argument and the simulation argument. Synthese, 190(18):4009–4022, 2013. [Wal08] D.N. Walton. Informal Logic: A Pragmatic Approach. Cambridge University Press, 2008. [Wea03] Brian Weatherson. Are you a sim? The Philosophical Quarterly, 53(212):425–431, 2003. [Weh15] Dustin Wehr. Rigorous deductive argumentation for socially relevant issues. CoRR, abs/1502.02272, 2015. 26 6 Proofs (supplemental) Notation: To cut down on some of the clutter, we drop the cardinality function symbol when it is easily inferred from the context. Specifically, whenever a finite set valued term S appears where a number is expected, it is shorthand for |S|. 6.1 Proof of Theorem 1 Recall the statement: Let (1 denote entailment with respect to Definition 1 (models for the Patch 1 Simulation Argument). For all settings of the parameters d P N, ~k P pR`q3: N ą LBpd,~kq,Patch 1 (1 Prop 1_ Prop 2_ Prop 3 Proof. This proof builds on the proof in the appendix of [BK10]. Let d,~k be arbitrary, and assume all of N ą LBpd,~kq, Patch 1, Prop 3, Prop 2 The remainder of the proof is to derive Prop 1. Define R – N k3d   1 d (3) From Prop 3, Prop 2 and Patch 1, we will derive: CPH ě CěN pR   k2q (4) 27 Starting from Prop 3: q3 ě fsim " ř cPCăN poppcq#simspcq ` ř cPCěN poppcq#simspcq ř cPCăN poppcq#simspcq ` ř cPCěN poppcq#simspcq ` avgpopěNCěN ` avgpopăNCăN Since the fraction is in [0,1], we drop a positive term above and below: ě ř cPCěN poppcq#simspcq ř cPCěN poppcq#simspcq ` avgpopěNCěN ` avgpopăNCăN Again since the fraction is in [0,1], we may soundly substitute in the lower bound ř cPCěN poppcq#simspcq ě NCěNavgpopěN above and below: ě NCěNavgpopěN NCěNavgpopěN ` avgpopěNCěN ` avgpopăNCăN " 1 1` 1 N   1` CăNavgpopăN CěNavgpopěN   ě 1 1` 1 N   1` CăN CěN d   by Patch 1 28 And so q3 ě 1 1` 1 N   1` CăN CěN d   Ø 1 q3 ď 1` 1 N  1` CăN CěN d   Ø 1` k3 k3 ď 1` 1 N  1` CăN CěN d   defn of k3 Ø  1` k3 k3   1     1 N ď CăNd CěNN Ø 1 k3   1 N ď CăNd CěNN Ø N k3   1 ď CăNd CěN Ø CăN ě CěN d  N k3   1   Ø CPHXCăN ` C PH ě CěN d  N k3   1   since CPHXCăN ,C PH partitions CăN By Prop 2 " CPHXCăN ď k2CěN and the previous inequality we have k2CěN ` C PH ě CěN d  N k3   1   Ø CPH ě CěN „ 1 d  N k3   1     k2  " CěN „ N k3d   1 d   k2  " CěN pR   k2q So finally, Inequality (4) is proved. From the definition of fPH, Inequality (4), and Prop 2 again, we'll derive fPH ě R   k2 R ` 1 (5) 29 fPH " CPH CPH ` CPH " CPH CěN ` C PH XCăN ` C PH since CěN ,C PH X CăN partitions C PH ě CPH CěN ` k2CěN ` C PH by Prop 2 ě CěNpR   k2q CěN ` k2CěN ` CěNpR   k2q by Inequality (4) " R   k2 1` k2 `R   k2 CěN cancels " R   k2 R ` 1 Equation (3) If we can prove Goal: R   k2 R ` 1 ą 1  q1 then we're done, since then fPH ě 1   q1, which is equivalent to fPH ď q1, which is Prop 1. We finally use the N lower bound assumption: N ą LBpd,~kq " dk3pk1 ` k2 ` k1k2q ` k3 N ą dk3pk1 ` k2 ` k1k2q ` k3 is equivalent to k1 ` k2 ` k1k2 ă N dk3   1 d " R Solving for k1 in the inequality just derived, obtain: k1 ă R   k2 1` k2 Since by definition k1 " 1 q1 q1 : 1  q1 q1 ă R   k2 1` k2 30 Solving for q1, obtain: q1 ą 1 R k2 1`k2 ` 1 Thus 1  q1 ă 1  1 R k2 1`k2 ` 1 " 1  1 R`1 1`k2 " R`1 1`k2   1 R`1 1`k2 " R k2 1`k2 R`1 1`k2 " R   k2 R ` 1 That completes the proof. 6.2 Proof of Corollary 1 Recall the statement: Let (2 denote entailment with respect to Definition 6. For all settings of the parameters d P N, ~k P pR`q3: N ą LBpd,~kq,Patch 2 (2 Prop 1_ Prop 2_ Prop 3 1 Proof. Fix a setting of the parameters. Let M be any 2-model that satisfies N ą LBpd,~kq and Patch 2. We construct a 1-model N that satisfies N ą LBpd,~kq and Patch 1, and so by Theorem 1 we get that N satisfies Prop 1 _ Prop 2 _ Prop 3. Lastly, we observe that this implies M satisfies Prop 1_ Prop 2_ Prop 3 1. Recall that a 2-model is just a 1-model with an additional function countE. N 's interpretation of every symbol of the language of 1-models except for pop is the same 31 as M's interpretation, and the definition of N is completed by defining ppoppcqqN " ` countEpcq  M for every civilization c We are free to set the parameters of Theorem 1, but we have made them all the same as they are for Corollary 1, so clearly N satisfies N ą LBpd,~kq. Also observe that pavgpopEăNq M " pavgpopăNq N and pavgpopEěNq M " pavgpopěNq N and so M's satisfying Patch 2 implies N 's satisfying Patch 1. We can now apply Theorem 1 to get that N satisfies Prop 1_ Prop 2_ Prop 3. Observe that the meaning of Prop 1 and Prop 2 is the same for 1-models and 2-models, and by the way we defined N it is clear that pProp 1qN Ø pProp 1qM and pProp 2qN Ø pProp 2qM. If we can show pProp 31qN Ø pProp 31qM, then we're done. For that, simply note that p#SEqM " p#SqN and p#UEqM " p#UqN , so pfEsimq M " pfsimq N . This model translation requires the permissibility in Definition 1 of civilizations that run at least one ancestral simulation but have cumulative pre-PH population size 0; such civilizations in N are produced from civilizations in M with no E-observers that run at least one ancestral simulation. That makes little sense according to the informal interpretation of C as a set of "civilizations". However, it is important to note that this is not a weakness. That Theorem 1 works with such models is just a mathematical fact, and we can exploit that fact to get this easy proof of Corollary 1. Alternatively, we could copy the proof of Theorem 1 and superficially modify it to get a proof of Corollary 1. 32 6.3 Proofs that Theorem 1 and Corollary 1 are optimal 6.3.1 Proof of Theorem 2 Recall the statement of Theorem 2: Let (1 denote entailment with respect to Definition 1 (models for the Simulation Argument). For all settings of d P N, ~k P pN`q3: N ě LBpd,~kq,Patch 1 *1 Prop 1_ Prop 2_ Prop 3 Proof. We give a model (Definition 1) that satisfies N " LBpd,~kq and Patch 1 and falsifies each of Prop 1,Prop 2,Prop 3. We specify exactly one PH civilizationxxv cěN that does N ancestor simulations. We specify k2 PH civilizations that do fewer than N simulations (in fact they do none), so |CěN | " k2|C PH XCăN | and Prop 1 is falsified. Note that |CPH| " 1 ` k2. We specify k1p1 ` k2q civilizations that never reach a PH state, so |CPH| " k1|C PH |, and Prop 2 is falsified. Note that |CăN | " |C PH XCăN | ` |CPH| " k2 ` k1p1` k2q. cěN has cumulative population size one, so avgpopěN " 1. xxvi For the other civilizations CăN , we specify that each has cumulative population size d, so avgpopăN " d, and Patch 1 is satisfied. Observe that the total number of simulated observers #S is N , and the total number of non-simulated observers #U is avgpopěN |CěN | ` avgpopăN |CăN | " 1 ` dpk2` k1p1` k2qq. We'll show #S " k3#U so that Prop 3 is falsified, and then we're done. Indeed the reader can check that the right hand sides of the following equations are equivalent. #S " N " dk3pk1 ` k2 ` k1k2q ` k3 xxvThis construction generalizes for |CěN | equal to any positive integer. xxviThe construction generalizes for avgpopěN equal to any positive integer. 33 k3#U " k3p1` dpk2 ` k1p1` k2qqq 6.3.2 Proof of Corollary 2 Recall the statement of Corollary 2: Let (2 denote entailment with respect to Definition 1 (models for the Simulation Argument). For all settings of d P N, ~k P pN`q3: N ě LBpd,~kq,Patch 2 *2 Prop 1_ Prop 2_ Prop 3 1 Proof. The proof is almost identical to that of Theorem 2. Use the same construction of a 1-model M, and then additionally specify countEpcq " poppcq for every civilization c. Note that this makes |CEě1ěN | " |CěN | " HspEq and |C Eě1 ăN | " |CăN | " HnpEq (recall HnpEq is [BK10]'s notation for the average number of E-observers in C Eě1 ăN civilizations, and similarly for HspEq and C Eě1 ěN ), in which case Patch 2 and Patch 1 are equivalent. In an earlier draft of this paper, we made the fractions |CEě1 ěN | |CěN | and |CEě1 ăN | |CăN | be parameters of the argument. This makes it more tedious to prove the natural analog of Corollary 2, in which "all settings" includes setting those fractions to arbitrary rational numbers in r0, 1s, and letting d be any rational number greater than 0. One must construct a model with civilizations that have exactly the right ratios of E-observers to observers. It does however work out fine, after some minor adjustments to the statement of Corollary 2 (e.g. the LB term gets put inside t u).