An existential risk is a global catastrophic risk that threatens to exterminate humanity or severely curtail its potential. Existential risks are unique because current institutions have little incentive to mitigate them, except as a side effect of pursuing other goals. There is little to no financial return in mitigating existential risk. Bostrom (2001) argues that because reductions in existential risks are global public goods, they may be undervalued by the market. Also, because we have never confronted a major existential risk before, we have little to learn from, and little impetus to be afraid. For more information, see this reference.
There are three main categories of existential risk — threats from biotechnology, nanotechnology, and AI/robotics. Nuclear proliferation itself is not quite an existential risk, but widespread availability of nuclear weapons could greatly exacerbate future risks, providing a stepping stone into a post-nuclear arms race. We’ll look at that first, then go over the others.
Nuclear risk. The risk of nuclear proliferation is currently high. The United States is planning to spend $100 billion on developing new nuclear weapons, and reports suggest that the President is not doing enough to curtail nuclear proliferation, despite the emphasis on the War on Terror. Syria, Qatar, Egypt, and the United Arab Emirates met to announce they their desire to develop nuclear technology. North Korea successfully tested a nuclear weapon in October. Iran continues enriching uranium against the will of the United Nations, and an Iranian official hinted that the country may be obtaining nuclear weapons. Last night, President Bush used the most confrontational language yet towards Iran, accusing it of directly providing weapons and funds to combatants killing US soldiers. The geopolitical situation today with respect to nuclear technology is probably the worst it has been since the Cold War.
Biotechnological risk. The risk of biotechnological disaster is currently high. An attempt among synthetic life researchers to formulate a common set of ethical standards, at the International Conference on Synthetic Biology, has failed. Among the synthetic biology and biotechnology communities, there is little recognition of the risk of genetically engineered pathogens. President Bush’s plan to spend $7.1 billion on bird flu vaccines was decreased to $2.3 billion by Congress. There is little federal money being spent on research to develop blanket countermeasures against unanticipated biotechnological threats. There are still custom DNA synthesis labs that fill orders without first scanning for harmful sequences. Watch-lists for possible bioweapon sequences are out of date, and far from comprehensive. The cost of lab equipment necessary to make bioweapons has decreased in cost and increased in performance, putting it within the financial reach of terrorist organizations. Until there is more oversight in this area, the risk will not only remain, but increase over time. For more information, see this report.
)