Human extinction

Abstract: risks are those that threaten the entire future of humanity. Many theories of value imply that even relatively small reductions in net existential risk have enormous expected value. Despite their importance, issues surrounding human-extinction risks and related hazards remain poorly understood. In this article, I clarify the concept of existential risk and develop an improved classification scheme. I discuss the relation between existential risks and basic issues in axiology, and show how existential risk reduction (via the maxipok rule) can serve as a strongly action-guiding principle for utilitarian concerns. I also show how the notion of existential risk suggests a new way of thinking about the ideal of sustainability. Policy Implications • Existential risk is a concept that can focus long-term global efforts and sustainability concerns. • The biggest existential risks are anthropogenic and related to potential future technologies. • A moral case can be made that existential risk reduction is strictly more important than any other global public good. • Sustainability should be reconceptualised in dynamic terms, as aiming for a sustainable trajectory rather than a sus- tainable state. • Some small existential risks can be mitigated today directly (e.g. asteroids) or indirectly (by building resilience and reserves to increase survivability in a range of extreme scenarios) but it is more important to build capacity to improve humanity's ability to deal with the larger existential risks that will arise later in this century. This will require collective wisdom, technology foresight, and the ability when necessary to mobilise a strong global coordi- nated response to anticipated existential risks. • Perhaps the most cost-effective way to reduce existential risks today is to fund analysis of a wide range of existen- tial risks and potential mitigation strategies, with a long-term perspective.

Abstract: Acknowledgements Foreword Introduction I BACKGROUND Long-term astrophysical processes Evolution theory and the future of humanity Millenial tendencies in responses to apocalyptic threats Cognitive biases potentially affecting judgement of global risks Observation selection effects and global catastrophic risks Systems-based risk analysis Catastrophes and insurance Public policy towards catastrophe II RISKS FROM NATURE Super-volcanism and other geophysical processes of catastrophic import Hazards from comets and asteroids Influence of Supernovae, gamma-ray bursts, solar flares, and cosmic rays on the terrestrial environment III RISKS FROM UNINTENDED CONSEQUENCES Climate change and global risk Plagues and pandemics: past, present, and future Artificial Intelligence as a positive and negative factor in global risk Big troubles, imagined and real IV RISKS FROM HOSTILE ACTS Catastrophe, social collapse, and and human extinction The continuing threat of nuclear war Catastrophic nuclear terrorism: a preventable peril Biotechnology and biosecurity Nanotechnology as global catastrophic risk The totalitarian threat Author's biographies Index

Abstract: Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the propects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from a human to a "posthuman" society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could case our extinction or destroy the potential of Earth - originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies.

Abstract: In this century a number of events could extinguish humanity. The probability of these events may be very low, but the expected value of preventing them could be high, as it represents the value of all future human lives. We review the challenges to studying human extinction risks and, by way of example, estimate the cost effectiveness of preventing extinction-level asteroid impacts.