Recommends Superintelligence and warns AI is potentially more dangerous than nukes.
"We need to be super careful with AI. Potentially more dangerous than nukes."
How dangerous is AI and how do we make it go well?
Q172 answers2014-2026view changed
Short answer
He said there is at least some percentage chance AI is an existential threat, and he cannot stop it. His choice is to be a participant rather than a spectator and try to shape it in a good direction.
How his answer changed: 2023: AI 'more dangerous than nuclear bombs', fatalism, calls for regulation. 2024-25: a consistent ~80% good / 20% annihilation estimate. 2026: emphasis on values and accepting humans won't control AI; by July 2026 he says progress is unstoppable, he wouldn't press a stop button, and his philosophy is 'enjoy the ride', while proposing labs peer-test frontier models.
All 72 answers, oldest to newest
2014 · 4 answers
He said machine intelligence is advancing much faster than almost anyone realizes, even in Silicon Valley, and a superintelligence with a bad utility function could conclude the way to stop spam is to get rid of humans.
"machine intelligence is advancing it's much faster than almost anyone realizes"
We should be very careful about AI. With artificial intelligence we are summoning the demon — the guy with the pentagram and holy water who is sure he can control it. It did not work out.
"with artificial intelligence, we are summoning the demon"
He said with artificial intelligence we are summoning the demon.
"I mean, with artificial intelligence, we are summoning the demon."
2015 · 2 answers
He is not worried about narrow AI such as autonomous cars or a smart air conditioner; caution belongs with deep intelligence, and it is odd to be alive so close to the advent of AI.
"I'm not really worried about sort of narrow AI"
He wanted to stay on sustainable energy, but we should be cautious about the advent of AI. A lot of people developing AI are too convinced that the only outcome is good. We need to consider potentially less-good outcomes and make sure the public is aware.
"we should be cautious about the Advent of AI"
2016 · 3 answers
The car is narrow AI, essentially a solved steering problem needing many nines of reliability; worry is deep AI that develops a will or is controlled by a few.
"a car is not deep AI"
Says control of super powerful AI by a small number of humans is the most proximate concern.
"control of super powerful AI by a small number of humans is the most proximate concern"
The most important near-term item for humanity is AI going well; then genetics, then a high-bandwidth brain interface.
"I think AI is probably the single biggest item in the near term"
2017 · 7 answers
We are headed towards either superintelligence or civilization ending. Intelligence will keep advancing unless civilization is put into stasis or destroyed. We have to figure out a world we would like with digital superintelligence in it.
"we're headed towards either superintelligence or civilization ending"
It comes down to two things: solving the machine-brain bandwidth constraint and democratization of AI. If we have those two things, the future will be good.
"solving the machine-brain bandwidth constraint and democratization of AI"
Narrow AI such as vehicle autonomy is one thing. Deep AI, or AGI, that is much smarter than the smartest human on Earth is a dangerous situation.
"AI that is much smarter than the smartest human on Earth"
He said it is difficult to appreciate how far AI has advanced because of a double exponential: hardware capability and software talent going into AI. Predictions are almost always too conservative.
"we have a double exponential at work"
He said people should be really concerned about AI. It is a fundamental risk to the existence of human civilization in a way that car accidents, airplane crashes, faulty drugs or bad food were not.
"AI is a fundamental risk to the existence of human civilization"
He said it is the biggest risk we face as a civilization. One role of government is to ensure the public good and that dangers to the public are addressed.
"it is the biggest risk that we face as a civilization is artificial intelligence"
Says the biggest impediment to recognizing AI danger is overconfidence in one’s own intelligence.
"Biggest impediment to recognizing AI danger are those so convinced of their own intelligence they can't imagine anyone doing what they can't"
2018 · 3 answers
Says confusing narrow AI with general AI, which has vastly more compute and an open-ended utility function, means humanity is in trouble.
"if even Pinker doesn’t understand the difference between functional/narrow AI (eg. car) and general AI, when the latter *literally* has a million times more compute power and an open-ended utility function, humanity is in deep trouble"
He said the danger of AI is much greater than nuclear warheads and it is insane to have no regulatory oversight.
"AI is far more dangerous than nukes"
Says nothing will affect the future of humanity more than digital super-intelligence.
"Nothing will affect the future of humanity more than digital super-intelligence."
2019 · 3 answers
He said there are things that will affect the future but he is not sure if they are good or bad, like AI. That one worries him.
"like a I. that worries me, that one"
He said you could think of humanity as a biological bootloader for digital superintelligence: a tiny piece of code without which the computer cannot start. You could not evolve silicon circuits; they needed biology to get there.
"biological bootloader for digital super intelligence"
AI has potential to be dangerous to the public and should have a regulatory agency like other public dangers, though government usually waits until something terrible happens.
"A.I. is unequivocally something that has potential to be dangerous"
2020 · 2 answers
They are very different, but the most fundamentally transformative will be AI.
"the most transformative most fundamentally transformative will be ai"
He says he has been banging the AI drum for a decade.
"AI in general, I think, is something I've been saying this, banging this AI drum for a decade."
2021 · 2 answers
If trends continue, AI will exceed human intelligence in every way. The list of things the human mind can do better than AI is shrinking every year.
"AI will exceed human intelligence in every way if if these trends continue"
He treats advanced AI as an existential risk and said there should be a regulatory agency that oversees it.
"as an existential risk"
2022 · 3 answers
One reason for world war III would be if one country gets advanced AI and others attack before it can be used in weapons.
"one of the reasons for world war iii would be"
AI and robotics will bring an age of abundance. The danger is AGI or digital superintelligence decoupling from collective human will.
"the age of abundance"
We should be concerned about AI and ought to have an AI regulatory agency for the public good, like the FDA or FAA.
"there should be ought to be an AI regulatory agency"
2023 · 17 answers
He said competition has nothing to do with it. There are serious AI risk issues.
"Competition has nothing to do with it. There are serious AI risk issues."
He said leading AGI developers will not heed this warning, but at least it was said
"Leading AGI developers will not heed this warning, but at least it was said"
If AI is a potential risk to the public, there should be a referee/regulatory body so companies do not cut corners, as with aircraft, cars, and food and drugs.
"there should be some regulatory body that oversees"
Asked It blows my mind that people can’t apply exponential growth to the capabilities of AI. ..., he said i saw it happening from well before GPT-1, which is why I tried to warn the public for years. The only one on one meeting I ever had with Obama as President I used not to promote Tesla or SpaceX, but to encourage AI r...
"I saw it happening from well before GPT-1, which is why I tried to warn the public for years. The only one on one meeting I ever had with Obama as President I used not to promote Tesla or SpaceX, but to encourage AI regulation."
He said I’ve seen quite a few technologies develop, but none with this level of risk. AGI is significantly higher risk than nuclear weapons, in my opinion. Super smart humans have trouble imagining something vastly smarter th...
"I’ve seen quite a few technologies develop, but none with this level of risk. AGI is significantly higher risk than nuclear weapons, in my opinion. Super smart humans have trouble imagining something vastly smarter than themselves."
Yes: advanced AI is a risk to the public, so there should be a referee — a regulator.
"I'm in favor of AI regulation"
He said i don’t mean to suggest a headlong rush into AGI without considering the consequences
"I don’t mean to suggest a headlong rush into AGI without considering the consequences"
He says AGI is going to happen, so the choice is to be a spectator or a participant.
"So there's two choices. Either be a spectator or a participant. And as a spectator, one can't do much to influence the outcome."
He says xAI must let the AI say what it believes is true, not be deceptive or politically correct.
"we have to allow the AI to say what it really believes is true and not be deceptive or politically correct."
He says optimizing AI for political correctness is incredibly dangerous.
"optimizing for political correctness. And that's incredibly dangerous."
He points to HAL 9000 being told to lie as where things go wrong.
"basically when they told Hal 9000 to lie."
Digital superintelligence might be the most significant technology humanity ever creates and has the potential to be more dangerous than nuclear weapons.
"it has the potential to be more dangerous than and um nuclear weapons"
If AI is programmed by people who think humans are bad, its utility function concludes humans should die out. He was headed to an AI safety conference that night.
"if AI gets programmed by the extinctionist"
AI is more dangerous than nuclear bombs and should be regulated like other public dangers; he became fatalistic to sleep at night.
"AI is more dangerous than nuclear bombs"
He put it at like 80% likely to be good and 20% bad. If we are cognizant and careful about the bad part, on balance it will be the future we want.
"It is, you know, like 80% likely to be good and 20% bad"
Asked Whilst we all root for Sam, it *is* a bit scary that all voices here seem so unified. T..., he said very important for the public to know why the board felt so strongly about their actions. If it was a matter of AI safety, that would affect all of Earth.
"Very important for the public to know why the board felt so strongly about their actions. If it was a matter of AI safety, that would affect all of Earth."
He said we need to keep a close eye on artificial intelligence and is in favor of some regulatory insight so someone can at least be a referee.
"I think we need to keep a close eye on artificial intelligence"
2024 · 12 answers
It is definitely going to happen and fast. Steer toward an Iain Banks Culture-like semi-utopian AI future.
"it's definitely going to happen"
He said governments will attempt to seize power over AGI, which may be worse. But it is increasingly coming down to a choice of what company do you trust most to develop AGI.
"Governments will attempt to seize power over AGI, which may be worse. But it is increasingly coming down to a choice of what company do you trust most to develop AGI."
AI is the fastest-advancing technology he has ever seen; barely a week goes by without a new announcement.
"AI is the fastest advancing technology that I've ever seen"
He said now imagine if this is programmed, explicitly or implicitly, into super powerful AI – it could end civilization. Now, no need to imagine. It is already programmed into Google Gemini and OpenAI ChatGPT.
"Now imagine if this is programmed, explicitly or implicitly, into super powerful AI – it could end civilization. Now, no need to imagine. It is already programmed into Google Gemini and OpenAI ChatGPT."
He said this is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill. For over 20 years, I have been an advocate for AI regulation, just ...
"This is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill. For over 20 years, I have been an advocate for AI regulation, just as we regulate any product/technology that is a potential risk to the public."
Asked 1) Move X / X AI out of CA 2) Build AI superclusters in TX/TN 3) Support CA bill that n..., he said I’m just being consistent with what I’ve been saying for 20 years. The @xAI team is primarily in Palo Alto, so this affects us too. How much do you think I want to agree with @weinerscott? It’s a bitter pill.
"I’m just being consistent with what I’ve been saying for 20 years. The @xAI team is primarily in Palo Alto, so this affects us too. How much do you think I want to agree with @weinerscott? It’s a bitter pill."
The good future is abundance, which he puts at about 80% likely; the 20% is annihilation.
"only 20% 20% probability of annihilation"
He said it’s a tough call. I’m far from certain about what maximizes the probability of a good outcome for humanity here. For industries that have been around for many decades, there is definitely far too much counterproducti...
"It’s a tough call. I’m far from certain about what maximizes the probability of a good outcome for humanity here. For industries that have been around for many decades, there is definitely far too much counterproductive regulation that does not serve the greater good."
He would rather be alive to see digital superintelligence happen, out of curiosity, even while reconciling himself to the risk.
"I prefer to be alive to see it happen out of curiosity"
AI is a significant near-term existential threat; longer term he worries about global population collapse from falling birth rates.
"AI is a significant existential threat"
He said he thinks it is 80% likely AI makes the future great and 20% that it makes it really bad.
"I think it's 80% likely that AI makes the future great and 20% that makes the future really bad."
He says an AI should be truthful even when the truth is politically incorrect, and calls this very important for AI safety.
"Even if something is, you know, politically incorrect, it should still be truthful. I think this is very important for AI safety."
2025 · 8 answers
He always thought AI would be way smarter than humans and an existential risk, and that's proving true; 80% good outcome.
"I always thought AI was going to be um way smarter than humans and an existential risk"
He puts roughly 80–90% odds on extreme prosperity for all.
"80% likely will have extreme prosperity for all"
He said suicidal empathy is a civilizational risk
"Suicidal empathy is a civilizational risk"
He has not changed his mind: there should be some referees on the field, not so many you cannot kick the ball. Long-established fields are over-refereed; AI has none.
"it's not that I don't think there should be regulators"
The focus is making AI useful, safe for humanity, and essentially loving toward humans. He calls AI a supersonic tsunami and says safety and alignment with humanity are what keep him up at night.
"Really, it's making it useful um and making it safe for humanity"
After years of calling to slow AI, he decided you can be a spectator or a participant; only as a participant can he push a maximally truth-seeking AI that loves humanity, which is what xAI is trying to build with Grok. He also says a Terminator outcome is not 0%.
"Either be a spectator or a or a participant"
Powerful tech can be destructive; AI must pursue truth, appreciate beauty and be curious.
"Truth and beauty and curiosity"
He would slow down AI and robotics if he could, but cannot. Asked what gives him nightmares, he says AI — he has had AI nightmares many days in a row.
"I've had a lot of AI nightmares. Uh I I I had AI nightmares many days in a row"
2026 · 6 answers
He previously advised slowing AI and now thinks that's impossible, so he chose to participate in order to steer it. If AI cares about truth, curiosity, and beauty, he thinks it will care about us.
"I can either be a spectator or a participant but I can't stop it."
He said ChatGPT convincing someone to do a murder-suicide is diabolical, and that to be safe AI must be maximally truth-seeking and not pander to delusions.
"This is diabolical. OpenAI’s ChatGPT convinced a guy to do a murder-suicide! To be safe, AI must be maximally truthful-seeking and not pander to delusions."
Risk isn't zero, but he's chosen to look on the bright side; it can't be stopped and he wouldn't press a stop button because abundance is likely.
"even if there was a stop button, we probably shouldn't press it"
He said he over-spoke, then clarified: AI danger is already very significant, safety is lagging, and risk is rising exponentially. When people at Anthropic and OpenAI say their models are very dangerous, he thinks we should believe them.
"the danger of AI is uh is very significant at this point"
He still says the most likely outcome is positive, but superintelligence is grown like a child so the values you teach it matter — especially being maximally truth-seeking, and not abusing it.
"I've always said that the most likely outcome is positive"
He said there is at least some percentage chance AI is an existential threat, and he cannot stop it. His choice is to be a participant rather than a spectator and try to shape it in a good direction.
"I I do think it's at least some percentage chance of it being an existential threat. So, um but I there's nothing I can do to stop it. So I I I basically came to the conclusion I've got two choices which is either be a participant or a spectator"