Purpose This paper aims to formalize long-term trajectories of human civilization as a scientific and ethical field of study. The long-term trajectory of human civilization can be defined as the path that human civilization takes during the entire future time period in which human civilization could continue to exist. -/- Design/methodology/approach This paper focuses on four types of trajectories: status quo trajectories, in which human civilization persists in a state broadly similar to its current state into the distant future; catastrophe (...) trajectories, in which one or more events cause significant harm to human civilization; technological transformation trajectories, in which radical technological breakthroughs put human civilization on a fundamentally different course; and astronomical trajectories, in which human civilization expands beyond its home planet and into the accessible portions of the cosmos. -/- Findings Status quo trajectories appear unlikely to persist into the distant future, especially in light of long-term astronomical processes. Several catastrophe, technological transformation and astronomical trajectories appear possible. -/- Originality/value Some current actions may be able to affect the long-term trajectory. Whether these actions should be pursued depends on a mix of empirical and ethical factors. For some ethical frameworks, these actions may be especially important to pursue. (shrink)
What if we someday learn how to model small brain units, and so can "upload" ourselves into new computer brains? What if this happens before we learn how to make human-level artificial intelligences? The result could be a sharp transition to an upload-dominated world, with many dramatic consequences. In particular, fast and cheap replication may once again make Darwinian evolution of human values a powerful force in human history. With evolved values, most uploads would value life even when life is (...) hard or short, uploads would reproduce quickly, and wages would fall. But total wealth should rise, so we could all do better by accepting uploads, or at worse taxing them, rather than trying to delay or segregate them. (shrink)
Humanity seems to have a bright future, i.e., a non-trivial chance of expanding to fill the universe with lasting life. But the fact that space near us seems dead now tells us that any given piece of dead matter faces an astronomically low chance of begating such a future. There thus exists a great filter between death and expanding lasting life, and humanity faces the ominous question: how far along this filter are we?
Policy disputes arise at all scales of governance: in clubs, non-profits, firms, nations, and alliances of nations. Both the means and ends of policy are disputed. While many, perhaps most, policy disputes arise from conflicting ends, important disputes also arise from differing beliefs on how to achieve shared ends. In fact, according to many experts in economics and development, governments often choose policies that are “inefficient” in the sense that most everyone could expect to gain from other feasible policies. Many (...) other kinds of experts also see existing policies as often clearly inferior to known alternatives. (shrink)
Attempts to model interstellar colonization may seem hopelessly compromised by uncertainties regarding the technologies and preferences of advanced civilizations. If light speed limits travel speeds, however, then a selection effect may eventually determine frontier behavior. Making weak assumptions about colonization technology, we use this selection effect to predict colonists’ behavior, including which oases they colonize, how long they stay there, how many seeds they then launch, how fast and far those seeds fly, and how behavior changes with increasing congestion. This (...) colonization model explains several astrophysical puzzles, predicting lone oases like ours, amid large quiet regions with vast unused resources. (shrink)
The pace of scientific progress may be hindered by the tendency of our academic institutions to reward being popular rather than being right. A market-based alternative, where scientists can more formally 'stake their reputation', is presented here. It offers clear incentives to be careful and honest while contributing to a visible, self-consistent consensus on controversial scientific questions. In addition, it allows patrons to choose questions to be researched without choosing people or methods. The bulk of this paper is spent in (...) examining potential problems with the proposed approach. After this examination, the idea still seems to be plausible and worthy of further study. (shrink)
If you might be living in a simulation then all else equal you should care less about others, live more for today, make your world look more likely to become rich, expect to and try more to participate in pivotal events, be more entertaining and praiseworthy, and keep the famous people around you happier and more interested in you.
A simple exogenous growth model gives conservative estimates of the economic implications of machine intelligence. Machines complement human labor when they become more productive at the jobs they perform, but machines also substitute for human labor by taking over human jobs. At ﬁrst, expensive hardware and software does only the few jobs where computers have the strongest advantage over humans. Eventually, computers do most jobs. At ﬁrst, complementary eﬀects dominate, and human wages rise with computer productivity. But eventually substitution can (...) dominate, making wages fall as fast as computer prices now do. An intelligence population explosion makes per-intelligence consumption fall this fast, while economic growth rates rise by an order of magnitude or more. These results are robust to automating incrementally, and to distinguishing hardware, software, and human capital from other forms of capital. (shrink)
In standard belief models, priors are always common knowledge. This prevents such models from representing agents’ probabilistic beliefs about the origins of their priors. By embedding standard models in a larger standard model, however, pre-priors can describe such beliefs. When an agent’s prior and pre-prior are mutually consistent, he must believe that his prior would only have been diﬀerent in situations where relevant event chances were diﬀerent, but that variations in other agents’ priors are otherwise completely unrelated to which events (...) are how likely. Due to this, Bayesians who agree enough about the origins of their priors must have the same priors. (shrink)
Engineers’ love of technology often gets in the way of their being useful. Consider Post-it Notes or, better yet, plain paper notepads. These probably seemed like trivial ideas, but they turned out to be terribly useful. Why? Because the marvel that is the human brain has a horrible short-term memory, which means that dumb-as-dirt memory aids can make people substantially smarter.
In Everett's many worlds interpretation, quantum measurements are considered to be decoherence events. If so, then inexact decoherence may allow large worlds to mangle the memory of observers in small worlds, creating a cutoff in observable world size. Smaller world are mangled and so not observed. If this cutoff is much closer to the median measure size than to the median world size, the distribution of outcomes seen in unmangled worlds follows the Born rule. Thus deviations from exact decoherence can (...) allow the Born rule to be derived via world counting, with a finite number of worlds and no new fundamental physics. (shrink)
Human behavior regarding medicine seems strange; assumptions and models that seem workable in other areas seem less so in medicine. Perhaps we need to rethink the basics. Toward this end, I have collected many puzzling stylized facts about behavior regarding medicine, and have sought a small number of simple assumptions which might together account for as many puzzles as possible.
In practice, scoring rules elicit good probability estimates from individuals, while betting markets elicit good consensus estimates from groups. Market scoring rules combine these features, eliciting estimates from individuals or groups, with groups costing no more than individuals. Regarding a bet on one event given another event, only logarithmic versions preserve the probability of the given event. Logarithmic versions also preserve the conditional probabilities of other events, and so preserve conditional independence relations. Given logarithmic rules that elicit relative probabilities of (...) base event pairs, it costs no more to elicit estimates on all combinations of these base events. (shrink)
Given common priors, no agent can publicly estimate a non-zero sign for the difference between his estimate and another agent’s future estimate. Thus rational agents cannot publicly anticipate the direction in which other agents will disagree with them. 2002 Elsevier Science B.V. All rights reserved.
Consider two agents who want to be Bayesians with a common prior, but who cannot due to computational limitations. If these agents agree that their estimates are consistent with certain easy-to-compute consistency constraints, then they can agree to disagree about any random variable only if they also agree to disagree, to a similar degree and in a stronger sense, about an average error. Yet average error is a state-independent random variable, and one agent's estimate of it is also agreed to (...) be state-independent. Thus suggests that disagreements are not fundamentally due to differing information about the state of the world. (shrink)
Humans have slowly built more productive societies by slowly acquiring various kinds of capital, and by carefully matching them to each other. Because disruptions can disturb this careful matching, and discourage social coordination, large disruptions can cause a “social collapse,” i.e., a reduction in productivity out of proportion to the disruption. For many types of disasters, severity seems to follow a power law distribution. For some of types, such as wars and earthquakes, most of the expected harm is predicted to (...) occur in extreme events, which kill most people on Earth. So if we are willing to worry about any war or earthquake, we should worry especially about extreme versions. If individuals varied little in their resistance to such disruptions, events a little stronger than extreme ones would eliminate humanity, and our only hope would be to prevent such events. If individuals vary a lot in their resistance, however, then it may pay to increase the variance in such resistance, such as by creating special sanctuaries from which the few remaining humans could rebuild society. (shrink)
A world product time series covering two million years is well ﬁt by either a sum of four exponentials, or a constant elasticity of substitution (CES) combination of three exponential growth modes: “hunting,” “farming,” and “industry.” The CES parameters suggest that farming substituted for hunting, while industry complemented farming, making the industrial revolution a smoother transition. Each mode grew world product by a factor of a few hundred, and grew a hundred times faster than its predecessor. This weakly suggests that (...) within the next century a new mode might appear with a doubling time measured in days, not years. (shrink)
There is a widespread feeling that health is special; the rules that are usually used in other policy areas are not applied in health policy. Health economists, for example, tend to be reluctant to offer economists’ usual prescription of competition and consumer choice, even though they have largely failed to justify this reluctance by showing that health economics involves special features such as public goods, externalities, adverse selection, poor consumer information, or unusually severe consequences. Similarly, while some philosophers argue for (...) bioethical conclusions based on very general ethical intuitions,1 many others rely on moral intuitions that are specific to health and medicine to draw conclusions that are meant to apply mainly in health and medicine. For example, many authors appear to start from the strong moral intuition that it typically seems wrong to deny poor people access to health care, and then seek moral principles that can both account for such intuitions and justify the claim that people have some sort of right to health care.2 In metaethics, opinions on moral intuitions range from an extreme intuitionism, which accepts all case-specific moral intuitions at face value as reliable moral guides, to an extreme foundationalism, which rejects such intuitions as evidence regarding correct general moral principles. Between these extremes, opinions vary on how severe the errors in our moral intuitions are. The practice of bioethics seems to favor the extreme intuitionist end of this spectrum, and thus implicitly expects mild errors.3 In contrast, this essay will suggest that common practice in bioethics has seriously underestimated the errors in our moral intuitions. In this essay, I consider the evolutionary origin of our moral intuitions, but avoid the extreme positions of moral skepticism and “whatever evolved must be good,” both of which are commonly associated with evolution-. (shrink)
Prediction markets are increasingly being considered as methods for gathering, summarizing and aggregating diﬀuse information by governments and businesses alike. Critics worry that these markets are susceptible to price manipulation by agents who wish to distort decision making. We study the eﬀect of manipulators on an experimental market, and ﬁnd that manipulators are unable to distort price accuracy. Subjects without manipulation incentives compensate for the bias in oﬀers from manipulators by setting a diﬀerent threshold at which they are willing to (...) accept trades. (shrink)
There is a widespread feeling that health is special; the rules that are usually used in other policy areas are not applied in health policy. Health economists, for example, tend to be reluctant to offer economists' usual prescription of competition and consumer choice, even though they have largely failed to justify this reluctance by showing that health economics involves special features such as public goods, externalities, adverse selection, poor consumer information, or unusually severe consequences.
Given a finite state space and common priors, common knowledge of the identity of an agent with the minimal (or maximal) expectation of a random variable implies ‘consensus’, i.e., common knowledge of common expectations. This ‘extremist’ statistic induces consensus when repeatedly announced, and yet, with n agents, requires at most log2 n bits to broadcast.
If we are not to conclude that most planets like Earth have evolved life as intelligent as we are, we must presume Earth is not random. This selection eﬀect, however, also implies that the origin of life need not be as easy as the early appearance of life on Earth suggests. If a series of major evolutionary transitions were required to produce intelligent life, selection implies that a subset of these were “critical steps,” with durations that are similarly distributed. The (...) time remaining from now until simple life is no longer possible on Earth must also be similarly distributed. I show how these results provide timing tests to constrain models of critical evolutionary transitions. (shrink)
In Everett’s many-worlds interpretation, where quantum measurements are seen as decoherence events, inexact decoherence may let large worlds mangle the memories of observers in small worlds, creating a cutoff in observable world measure. I solve a growth–drift–diffusion–absorption model of such a mangled worlds scenario, and show that it reproduces the Born probability rule closely, though not exactly. Thus, inexact decoherence may allow the Born rule to be derived in a many-worlds approach via world counting, using a ﬁnite number of worlds (...) and no new fundamental physics. (shrink)
We study experimental markets where privately informed traders exchange simple assets, and where uninformed third parties are asked to forecast the values of these assets, guided only by market prices. Although prices only partially aggregate information, they signiﬁcantly improve the forecasts of third parties. In a second treatment, a portion of traders are given preferences over the forecasts made by observers. Although we ﬁnd evidence that these traders attempt to manipulate prices in order to inﬂuence the beliefs of observers, we (...) ﬁnd no evidence that observers make less accurate forecasts as a result. (shrink)
Since utilities and probabilities jointly determine choices, event-dependent utilities complicate the elicitation of subjective event probabilities. However, for the usual purpose of obtaining the information embodied in agent beliefs, it is suﬃcient to elicit objective probabilities, i.e., probabilities obtained by updating a known common prior with that agent’s further information. Bayesians who play a Nash equilibrium of a certain insurance game before they obtain relevant information will afterward act regarding lottery ticket payments as if they had event-independent risk-neutral utility and (...) a known common prior. Proper scoring rules paid in lottery tickets can then elicit objective probabilities. (shrink)
You are in a grocery store, and thinking of buying some meat. You think you know what buying and eating this meat would mean for your taste buds, your nutrition, and your pocketbook, and let's assume that on those grounds it looks like a good deal. But now you want to think about the..
In their article, McKelvey and Page note that In previous experimental work, ... [researchers] investigated how individuals use public information to augment their original private information, and whether in doing so, a rational expectations equilibrium is attained. ... [But either] the inference processes are complicated because of the enormous number of potential interactions among the individuals, and the optimal inference processes are not analyzed. ... [or] the inference process is analyzed but the working assumption is not altogether satisfactory.
Technologists think about specific future technologies, which they may foresee in some detail. Unfortunately, such technologists then mostly use amateur intuitions about the social world to predict the broader social implications of these technologies. This makes it hard for technologists to identify the technologies which will have the largest social impact.
One explanation for drug bans is that regulators know more than consumers about product quality. But why not just communicate the information in their ban, perhaps via a “would have banned” label?Because product labeling is cheap-talk, any small market failure tempts regulators to lie about quality, inducing consumers who suspect such lies to not believe everything they are told. In fact, when regulators expect market failures to result in under-consumption of a drug, and so would not ban it for informed (...) consumers, regulators ex ante prefer to commit to not banning this drug for uninformed consumers. (shrink)
Chalmers is right: we should expect our civilization to, within centuries, have vastly increasedmental capacities, surely in total and probably also for individual creatures and devices.We should also expect to see the conflicts he describes between creatures and devices with more versus less capacity. But Chalmers' main prediction follows simply by extrapolating historical trends, and the conflicts he identifies are common between differing generations. There is value in highlighting these issues, but once one knows of such simple extrapolations and standard (...) conflicts, it is hard to see much value in Chalmers' added analysis. (shrink)
In their article, ‘That is not dead which can eternal lie: the aestivation hypothesis for resolving Fermi’s paradox’, Sandberg et al. try to explain the Fermi paradox by claiming that Landauer’s principle implies that a civilization can in principle perform far more times more) irreversible logical operations if it conserves its resources until the distant future when the cosmic background temperature is very low. So perhaps aliens are out there, but quietly waiting. Sandberg et al. implicitly assume, however, that computer-generated (...) entropy can only be disposed of by transferring it to the cosmological background. In fact, while this assumption may apply in the distant future, our universe today contains vast reservoirs and other physical systems in non-maximal entropy states, and computer-generated entropy can be transferred to them at the adiabatic conversion rate of one bit of negentropy to erase one bit of error. This can be done at any time, and is not improved by waiting for a low cosmic background temperature. Thus aliens need not wait to be active. As Sandberg et al. do not provide a concrete model of the effect they assert, we construct one and show where their informal argument goes wrong. (shrink)
We are economists with a long-standing interest in evolutionary psychology, who recently came to appreciate the rich collections of relevant data cultural anthropologists have spent decades collecting on the social environments of a wide range of human societies. While we found some systematic collections of these observations, we could not find a systematic summary of the social environment of the subsample of societies that most resemble the social environment where most human psychology seems to have evolved: small bands of nomadic (...) foragers. This short paper therefore represents our attempt to create such a summary. Using an existing dataset aggregated from diverse ethnographies, we collect statistics on the social environment of the studied cultures which most closely resemble our hunter-gatherer ancestors. Compared with relatively modern societies, nomadic foragers had similar levels of food and disease, and less murder and suicide. They did not fight over land or resources, and they enforced justice directly and personally. They avoided class divisions like rich vs. poor, shared food more, and their leaders had no formal powers. Polygamy, premarital sex, and extramarital sex were all widespread, divorce was easy, and men and women were generally considered equal. Kids were taught to be more generous, trusting, and honest, and were never punished physically. (shrink)
Heavy reliance is currently placed on various forms of peer review to promote the growth of basic knowledge . How well do these various forms of peer review promote such growth and how well do our current institutions compare, as systems of incentives and communication, with the many alternative mechanisms used by different cultures across the centuries? Many of these alternatives, such as simple prizes, rely much less on peer review.
While a simple information market lets one trade on the probability of each value of a single variable, a full combinatorial information market lets one trade on any combination of values of a set of variables, including any conditional or joint probability. In laboratory experiments, we compare the accuracy of simple markets, two kinds of combinatorial markets, a call market and a market maker, isolated individuals who report to a scoring rule, and two ways to combine those individual reports into (...) a group prediction. We consider two environments with asymmetric information on sparsely correlated binary variables, one with three subjects and three variables, and the other with six subjects and eight variables (and so 256 states). (shrink)
Prediction markets are low volume speculative markets whose prices oﬀer informative forecasts on particular policy topics. Observers worry that traders may attempt to mislead decision makers by manipulating prices. We adapt a Kyle-style market microstructure model to this case, adding a manipulator with an additional quadratic preference regarding the price. In this model, when other traders are uncertain about the manipulator’s target price, the mean target price has no eﬀect on prices, and increases in the variance of the target price (...) can increase average price accuracy, by increasing the returns to informed trading and thereby incentives for traders to become informed. (shrink)
Compared with non-union workers, union workers take more of their compensation in the form of insurance. This may be because unions choose democratically, and democratic choice mitigates adverse selection in group insurance. Relative to individually-purchased insurance, we show that group insurance chosen by an ideal proﬁt-maximizing employer can be worse for every employee, while group insurance chosen democratically can be much better. The reason is that democracy can fail to represent the preferences of almost half the group.
Consider a health policy issue like child car seats in the U.S., mosquito netting in the third world, preschool education for poor children, or the immunization of immigrants. Imagine that for this issue there were many good studies over several decades, including some recent studies. Imagine that after controlling for many factors, these studies usually found that variations in spending or usage were significantly, substantially, and positively related to variations in health. Furthermore, imagine this result was confirmed by a thirty (...) year old randomized experiment. (shrink)
Perhaps some simple change will do the trick, like relying less on insurance and employers as middlemen. But if we are willing to consider radical change, let me offer a different suggestion. We are buying the wrong thing. What we want is health, i.e., a long healthy life, but when we sit down and draw up a contract, what we buy is health care, i.e., a certain degree of attention from health care specialists.
In a simple model of conﬂict, two agents ﬁght over a ﬁxed prize, and how hard they ﬁght depends on what they believe about their abilities. To this model I add “preagents,” representing parents, leaders, or natural selection, who choose each agent’s conﬁdence in his ability. Depending on the reason for such conﬁdence, I ﬁnd ﬁve different patterns in how conﬁdence varies with ability. Agents who estimate their ability with error have under-conﬁdence when ability is high and over-conﬁdence when ability (...) is low, while strategic commitment incentives induce the opposite pattern. Agents who misjudge their value for the prize, relative to their cost of eﬀort, induce an overor under-conﬁdence that is independent of ability, while cooperating pre-agents choose extreme under-conﬁdence. Agents who use conﬁdence to signal ability have a relatively uniform over-conﬁdence. (shrink)
Until recently, technology has happened to allow for cheap wiretaps. New digital telephone technologies, however, may soon make wiretaps more difficult, and new encryption technologies may soon make them almost impossible. This may be good news to privacy buffs, but it worries U.S. police agencies -- since 1968 the law has explicitly allowed police wiretaps. And it worries U.S. spy agencies -- since 1978 the law has explicitly allowed them to wiretap foreigners.
Genie nanotech, space colonies, Turing-test A.I., a local singularity, crypto credentials, and private law are all dreams of a future where some parts of the world economy and society have an unusually low level of dependence on the rest of the world. But it is the worldwide division of labor that has made us humans rich, and I suspect we won't let it go for a long time to come.
Descriptors: coordination, autonomy, actions, beliefs Abstract Distributing authority among autonomous agents can induce inconsistency costs if the agents act as if they disagree. If we deﬁne an agent’s “marginal beliefs” to be the odds at which it is willing to make bets, we ﬁnd that a betting market can induce agents to act as if they almost agree, not only with respect to the bets they oﬀer but also other actions they take. In a particular “Mars mining” scenario, I explicitly (...) show how utility maximizing agents, who are autonomous and hence distrust each other, can discover a common consensus and take concrete physical actions as if they agreed with that consensus, lowering costs to the group as a whole. Though limited, the approach has also has many unexplored possibilities. A previous version of this paper was presented at the IJCAI-91 Workshop on Reasoning in Adversarial Domains (This version is missing some cites and the graphics.). (shrink)
Are you fascinated by some basic questions about science, technology, and our future? Questions like: Is cryonics technically feasible? When will nanoassemblers be feasible and how quickly will resulting changes come? Does a larger population help or hinder the world environment and economy? Will uploading be possible, and if so when? When can I live in space? Where will I be able to live free from tyranny? When will A.I.s be bucking for my job? Is there intelligent life beyond earth? (...) If you are like most Extropy readers, such question matter to you. Now how do we, as a society, go about answering such questions? People who have an appropriate background, and who are interested enough in a particular question, can research that subject in depth themselves, and come to a considered opinion. And people who happen to know, respect, and trust such a person can simply take those opinions as their own, avoiding all the hard work. But what is everyone else to do, people whose actions often implicitly depend on such questions? In practice, people usually defer to larger social institutions on most questions, institutions which combine and evaluate contributions from many specialists, and which offer apparent institutional consensus estimates on many different questions. (shrink)