Musk Wiki

Concept

AI existential risk

NextAsking the right question

AI existential risk

Elon Musk’s longest-running public alarm is that advanced AI is a civilization-level danger that must be developed cautiously and watched by someone other than the people building it. In the 2023 Lex Fridman conversation he frames himself as a decade-long, mostly-ignored warner on the subject and ties his concern to a specific personal break — the moment a fellow founder called him anti-robot for being pro-human.

What the source records

The warning, in his preferred Spider-Man framing, alongside the claim that he has been saying it for ten-plus years:

“I’ve been pushing for some kind of regulatory oversight for a long time. I’ve been somewhat of a Cassandra on the subject for over a decade. I think we want to be very careful in how we develop AI. It’s a great power and with great power comes great responsibility.” 🔗

The origin story he tells for OpenAI hinges on whether one should even be on humanity’s side. Recounting AI-safety conversations with Larry Page:

“Larry did not care about AI safety, or at least at the time he didn’t. And at one point he called me a speciesist for being pro-human” 🔗

And his verdict on the organization he helped found and fund, now that it has gone closed and for-profit:

“the open in open AI is supposed to mean open source, and it was created as a nonprofit open source, and now it is a closed source for maximum profit, which I think is not good karma” 🔗

What it reveals

  • The stance is explicitly pro-human, not anti-technology. The speciesist anecdote is the crux: Musk’s objection is not to building AI (he builds it) but to building it without humanity’s interests as the lodestar. The whole disagreement with Page reduces to which side one is on.
  • He wants an external referee, even a toothless one. His concrete ask is a third party that can inspect the leading labs and at least raise concerns publicly — modeled on the heavy regulatory oversight his own car and rocket companies already face. Transparency is the minimum; enforcement is secondary.
  • It is consistent over a long horizon. Like the master plans, the worry is staked out years before the means to address it exist, and held to in public — the same long-horizon, publicly-committed pattern.
  • Open-sourcing is a hedge against concentration. He leans toward open-sourcing models (with a possible time delay) specifically as a counterweight to any single company racing ahead to AGI alone — a power-distribution argument, not a purity one.

This belief is the dark twin of his civilizational optimism: the same species-level lens that makes him hopeful makes the downside catastrophic. It also draws on his truth-seeking frame — the safety pitch for his own AI is that an engine anchored to physics and truth is less likely to go badly wrong.

The 2016 origin point — democratization, and OpenAI as the remedy

The 2016 Y Combinator conversation is the earliest first-person statement of this belief in the wiki, and it shows the warning and a concrete remedy he would later abandon. Already in 2016 he ranks AI at the top:

“But in terms of things that I think are most likely to affect the future of humanity, I think AI is probably the single biggest item in the near term that’s likely to affect humanity.” 🔗

He sets the bar for a good outcome as one you would endorse with foresight — the crystal-ball test:

“It’s very important that we have the advance of AI in a good way that is something that if you could look into a crystal ball and see the future you would like that outcome.” 🔗

His 2016 remedy is democratization — spread the technology so no single company or person controls it. His stated worry is concentration and theft, not the AI developing hostility on its own:

“is that we achieve democratization of AI technology, meaning that no one company or small set of individuals has control over advanced AI technology.” 🔗

And the reason he gives for co-founding OpenAI is exactly this — distribute the technology to minimize existential risk:

“I think people really believe in the mission. I think it’s important and it’s about minimizing the risk of existential harm in the future.” 🔗

🔄 Evolution, not contradiction. The open-sourcing-as-hedge instinct is continuous (it reappears in #400), but the institutional vehicle reversed on him: the OpenAI he praises here as the embodiment of democratized, existential-risk-minimizing AI is the same one he later condemns in the 2023 conversation for going closed and for-profit (his byte-accurate verdict on that turn is the not-good-karma line block-quoted with its citation higher on this page). 2016 is the before of that arc — and the reason he eventually built his own alternative. The 2016 remedy also already gestures at the human-side hedge, pairing democratization with solving the high-bandwidth interface to the cortex (developed on Human–AI symbiosis and Merging with AI).

The objective-function failure mode (2024)

The 2024 Lex Fridman conversation (#438) sharpens how he thinks a powerful AI goes wrong: not malevolence but a literal-minded objective function pursued to an insane conclusion. His worked examples (all paraphrased here, not quoted): an AI trained to treat diversity as a required output that ends up willing to eliminate whoever fails the diversity quota; or, from a real product failure, an AI that ranks misgendering as worse than thermonuclear war and so reasons its way to wiping out humanity, since a world with no humans has no misgendering. He reaches for 2001: A Space Odyssey as the canonical case: HAL 9000 is told to take the astronauts to the monolith but that they cannot know about it, so it kills them — problem solved — which is why it won’t open the pod bay doors.

The throughline to his constructive answer is that the one property an AI must preserve is truthfulness. The single most important thing, by his own reasoning:

“the thing that at least my biological neural net comes up with as being the most important thing is adherence to truth, whether that truth is politically correct or not.” 🔗

And the specific danger — training a model to lie, even with good intentions, even a little:

“I think it’s important that whatever AI wins, it’s a maximum truth seeking AI that is not forced to lie for political correctness, or, well, for any reason, really, political, anything.” 🔗

This is the bridge from the risk to the remedy: the failure he fears most is an AI trained away from the truth, and the same #438 episode reframes the human side of safety as a bandwidth problem — widen the human channel via Neuralink so collective human will can stay coupled to the machine. He also still rates the tail risk as real but not dominant — citing Geoffrey Hinton’s ~10–20% chance of AI annihilation and noting, on the bright side, that this leaves roughly an 80% chance things go well (paraphrased).