Existential risk from artificial general intelligence information
Hypothesized risk to human existence
Part of a series on
Artificial intelligence
Major goals
Artificial general intelligence
Recursive self-improvement
Planning
Computer vision
General game playing
Knowledge reasoning
Machine learning
Natural language processing
Robotics
AI safety
Approaches
Symbolic
Deep learning
Bayesian networks
Evolutionary algorithms
Situated approach
Hybrid intelligent systems
Systems integration
Applications
Projects
Deepfake
Machine translation
Generative AI
Art
Audio
Music
Healthcare
Mental health
Government
Industry
Earth sciences
Bioinformatics
Physics
Philosophy
Chinese room
Friendly AI
Control problem/Takeover
Ethics
Existential risk
Turing test
Regulation
History
Timeline
Progress
AI winter
AI boom
AI era
Glossary
Glossary
v
t
e
Existential risk from artificial general intelligence is the idea that substantial progress in artificial general intelligence (AGI) could result in human extinction or an irreversible global catastrophe.[1][2][3]
One argument goes as follows: human beings dominate other species because the human brain possesses distinctive capabilities other animals lack. If AI were to surpass humanity in general intelligence and become superintelligent, then it could become difficult or impossible to control. Just as the fate of the mountain gorilla depends on human goodwill, so might the fate of humanity depend on the actions of a future machine superintelligence.[4]
The plausibility of existential catastrophe due to AI is widely debated, and hinges in part on whether AGI or superintelligence are achievable, the speed at which dangerous capabilities and behaviors emerge,[5] and whether practical scenarios for AI takeovers exist.[6] Concerns about superintelligence have been voiced by leading computer scientists and tech CEOs such as Geoffrey Hinton,[7] Yoshua Bengio,[8] Alan Turing,[a] Elon Musk,[11] and OpenAI CEO Sam Altman.[12] In 2022, a survey of AI researchers with a 17% response rate found that the majority of respondents believed there is a 10 percent or greater chance that our inability to control AI will cause an existential catastrophe.[13][14] In 2023, hundreds of AI experts and other notable figures signed a statement that "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war".[15] Following increased concern over AI risks, government leaders such as United Kingdom prime minister Rishi Sunak[16] and United Nations Secretary-General António Guterres[17] called for an increased focus on global AI regulation.
Two sources of concern stem from the problems of AI control and alignment: controlling a superintelligent machine or instilling it with human-compatible values may be difficult. Many researchers believe that a superintelligent machine would resist attempts to disable it or change its goals, as that would prevent it from accomplishing its present goals. It would be extremely difficult to align a superintelligence with the full breadth of significant human values and constraints.[1][18][19] In contrast, skeptics such as computer scientist Yann LeCun argue that superintelligent machines will have no desire for self-preservation.[20]
A third source of concern is that a sudden "intelligence explosion" might take an unprepared human race by surprise. Such scenarios consider the possibility that an AI that is more intelligent than its creators might be able to recursively improve itself at an exponentially increasing rate, improving too quickly for its handlers and society at large to control.[1][18] Empirically, examples like AlphaZero teaching itself to play Go show that domain-specific AI systems can sometimes progress from subhuman to superhuman ability very quickly, although such systems do not involve altering their fundamental architecture.[21]
^ abcRussell, Stuart; Norvig, Peter (2009). "26.3: The Ethics and Risks of Developing Artificial Intelligence". Artificial Intelligence: A Modern Approach. Prentice Hall. ISBN 978-0-13-604259-4.
^Bostrom, Nick (2002). "Existential risks". Journal of Evolution and Technology. 9 (1): 1–31.
^Turchin, Alexey; Denkenberger, David (3 May 2018). "Classification of global catastrophic risks connected with artificial intelligence". AI & Society. 35 (1): 147–163. doi:10.1007/s00146-018-0845-5. ISSN 0951-5666. S2CID 19208453.
^Bostrom, Nick (2014). Superintelligence: Paths, Dangers, Strategies (First ed.). Oxford University Press. ISBN 978-0-19-967811-2.
^Vynck, Gerrit De (23 May 2023). "The debate over whether AI will destroy us is dividing Silicon Valley". Washington Post. ISSN 0190-8286. Retrieved 27 July 2023.
^Metz, Cade (10 June 2023). "How Could A.I. Destroy Humanity?". The New York Times. ISSN 0362-4331. Retrieved 27 July 2023.
^"'Godfather of artificial intelligence' weighs in on the past and potential of AI". www.cbsnews.com. 25 March 2023. Retrieved 10 April 2023.
^"How Rogue AIs may Arise". yoshuabengio.org. 26 May 2023. Retrieved 26 May 2023.
^Turing, Alan (1951). Intelligent machinery, a heretical theory (Speech). Lecture given to '51 Society'. Manchester: The Turing Digital Archive. Archived from the original on 26 September 2022. Retrieved 22 July 2022.
^Turing, Alan (15 May 1951). "Can digital computers think?". Automatic Calculating Machines. Episode 2. BBC. Can digital computers think?.
^Parkin, Simon (14 June 2015). "Science fiction no more? Channel 4's Humans and our rogue AI obsessions". The Guardian. Archived from the original on 5 February 2018. Retrieved 5 February 2018.
^Jackson, Sarah. "The CEO of the company behind AI chatbot ChatGPT says the worst-case scenario for artificial intelligence is 'lights out for all of us'". Business Insider. Retrieved 10 April 2023.
^"The AI Dilemma". www.humanetech.com. Retrieved 10 April 2023. 50% of AI researchers believe there's a 10% or greater chance that humans go extinct from our inability to control AI.
^"2022 Expert Survey on Progress in AI". AI Impacts. 4 August 2022. Retrieved 10 April 2023.
^Roose, Kevin (30 May 2023). "A.I. Poses 'Risk of Extinction,' Industry Leaders Warn". The New York Times. ISSN 0362-4331. Retrieved 3 June 2023.
^Sunak, Rishi (14 June 2023). "Rishi Sunak Wants the U.K. to Be a Key Player in Global AI Regulation". Time.
^Cite error: The named reference :12 was invoked but never defined (see the help page).
^ abYudkowsky, Eliezer (2008). "Artificial Intelligence as a Positive and Negative Factor in Global Risk" (PDF). Global Catastrophic Risks: 308–345. Bibcode:2008gcr..book..303Y. Archived (PDF) from the original on 2 March 2013. Retrieved 27 August 2018.
^Russell, Stuart; Dewey, Daniel; Tegmark, Max (2015). "Research Priorities for Robust and Beneficial Artificial Intelligence" (PDF). AI Magazine. Association for the Advancement of Artificial Intelligence: 105–114. arXiv:1602.03506. Bibcode:2016arXiv160203506R. Archived (PDF) from the original on 4 August 2019. Retrieved 10 August 2019., cited in "AI Open Letter - Future of Life Institute". Future of Life Institute. January 2015. Archived from the original on 10 August 2019. Retrieved 9 August 2019.
^Dowd, Maureen (April 2017). "Elon Musk's Billion-Dollar Crusade to Stop the A.I. Apocalypse". The Hive. Archived from the original on 26 July 2018. Retrieved 27 November 2017.
^"AlphaGo Zero: Starting from scratch". www.deepmind.com. 18 October 2017. Retrieved 28 July 2023.
Cite error: There are <ref group=lower-alpha> tags or {{efn}} templates on this page, but the references will not show without a {{reflist|group=lower-alpha}} template or {{notelist}} template (see the help page).
and 27 Related for: Existential risk from artificial general intelligence information
since 2005 on identifying and managing potential existentialrisksfromartificialgeneralintelligence. MIRI's work has focused on a friendly AI approach...
regulation of artificialintelligence is the development of public sector policies and laws for promoting and regulating artificialintelligence (AI); it is...
called for the creation of "friendly AI" to mitigate existentialriskfrom advanced artificialintelligence. He explains: "The AI does not hate you, nor does...
Artificialgeneralintelligence (AGI) is a type of artificialintelligence (AI) that can perform as well or better than humans on a wide range of cognitive...
Effective altruism Existentialriskfromartificialgeneralintelligence Future of Humanity Institute Global catastrophic risk (existentialrisk) Government...
whether, human intelligence will likely be surpassed. Some argue that advances in artificialintelligence (AI) will probably result in general reasoning systems...
towards benefiting life and away from large-scale risks, with a focus on existentialriskfrom advanced artificialintelligence (AI). FLI's work includes grantmaking...
drastically curtail humanity's existence or potential is known as an "existentialrisk." Over the last two decades,[when?] a number of academic and non-profit...
philosopher Nick Bostrom in 2003. It illustrates the existentialrisk that an artificialgeneralintelligence may pose to human beings were it to be successfully...
topic of transhumanist research is how to protect humanity against existentialrisks, such as nuclear war or asteroid collision.[better source needed]...
that unrestricted technological progress (especially driven by artificialintelligence) is a solution to universal human problems like poverty, war and...
Cyborgs – Mind uploading – Existentialriskfromartificialgeneralintelligence Global catastrophic risk § Artificialintelligence AI takeover – point at...
Compatible: ArtificialIntelligence and the Problem of Control is a 2019 non-fiction book by computer scientist Stuart J. Russell. It asserts that the risk to...
and control the behavior of AI systems, including proposed artificialgeneralintelligences (AGIs), in order to reduce the danger they might pose if misaligned...
catastrophic risks Ethics of uncertain sentience Existentialriskfromartificialgeneralintelligence Human Compatible Personhood Philosophy of artificial intelligence...
malicious actors, technological unemployment, and existentialriskfromartificialgeneralintelligence. Hinton was educated at Clifton College in Bristol...
systems; the risk is compounded in the case of a race to artificialgeneralintelligence, which may present an existentialrisk. A third risk of an AI arms...
Generative artificialintelligence (generative AI, GenAI, or GAI) is artificialintelligence capable of generating text, images, videos, or other data...
and artificialgeneralintelligence. In expecting the largest risks to stem from future technologies, and from advanced artificialintelligence in particular...
This is a timeline of artificialintelligence, sometimes alternatively called synthetic intelligence. Timeline of machine translation Timeline of machine...
prone to s-risks. AI control problem Ethics of artificialintelligence Ethics of terraforming Existentialriskfromartificialgeneralintelligence Global...
partly motivated by concerns about AI safety and the existentialriskfromartificialgeneralintelligence. OpenAI states that "it's hard to fathom how much...