Ban Superintelligence? The Open Letter With 800+ Signatories, Reported Straight

Abstract illustration of a pause symbol formed from interlocking hands, a halt motif, on deep navy

On September 22, 2025, the Future of Life Institute published a Statement on Superintelligence calling for a prohibition on developing superintelligent AI — and the public push that October drew more than 800 prominent signatories, from AI pioneers Geoffrey Hinton and Yoshua Bengio to Steve Wozniak, Richard Branson, Steve Bannon, Susan Rice, and Prince Harry and Meghan Markle. It is the most forceful mainstream call yet to halt the race toward machines that outthink humanity. This post reports what the letter says, who signed it, and the honest objections — including from people who share its worries.

Quick Takeaways

  • The Future of Life Institute's Statement on Superintelligence (September 22, 2025) calls for prohibiting superintelligence development until there is broad scientific consensus it can be done safely and strong public buy-in.
  • The October 2025 public push drew 800+ prominent signatories across the political spectrum — Hinton, Bengio, Wozniak, Branson, Bannon, Susan Rice, former Joint Chiefs chairman Mike Mullen, Prince Harry and Meghan.
  • The demand is categorical, not a pause: a ban, not a slowdown — going further than FLI's March 2023 six-month-pause letter.
  • Critics — including AI safety sympathisers — argue a ban is unenforceable, unverifiable, and would simply move development to whoever ignores it.
  • The letter's real significance may be political rather than practical: it made "should we build this at all?" a mainstream question.
  • What the Letter Actually Demands

    Illustration of a formal document with a quill pen and red wax seal, in white and red on dark background

    The statement's core sentence, as reported across outlets: "We call for a prohibition on the development of superintelligence, not lifted before there is broad scientific consensus that it will be done safely and controllably, and strong public buy-in." Note the structure — it is a ban with two unlock conditions, not a moratorium with a date. The first condition (scientific consensus on safe, controllable superintelligence) is a bar the field has never cleared for any transformative technology in advance. The second (strong public buy-in) is a political condition, not a technical one. Together they amount to: not until we collectively decide otherwise, with no deadline.

    The letter acknowledges AI's potential benefits — "unprecedented health and prosperity" in FLI's framing — but argues the labs' stated goal of building superintelligence within the coming decade raises concerns ranging from economic obsolescence and loss of control to national security risks and, at the extreme, human extinction.

    Who Signed — and Why the Breadth Matters

    Illustration of multiple hands signing a document, in red and black

    The signatories' breadth is the message: opposition to unconstrained superintelligence does not map onto a left-right axis. When Steve Bannon and Susan Rice sign the same document, the disagreement being papered over is not partisan — it is about whether the thing should be built at all.

    Why Even Sympathisers Call It Unenforceable

    The sharpest objections come not from the labs but from people who share the letter's worries:

    Verification is nearly impossible. A ban on "developing superintelligence" requires defining the threshold — and as Post 4's timeline survey shows, the field cannot agree on what counts as AGI, let alone superintelligence. A ban on an undefined thing is a press release, not a policy.

    It moves development, not stops it. Training a frontier model requires compute that is increasingly trackable, but "development" spans algorithms, data and ideas that cross borders frictionlessly. Critics argue a Western ban would primarily relocate the race to jurisdictions that decline to sign — the same objection that has haunted every technology-control regime, now applied to the one technology where the objectors believe the stakes are highest.

    The unlock conditions may never be met. "Broad scientific consensus that it will be done safely" is arguably unachievable in principle for a system smarter than its evaluators — Post 16's alignment discussion explains why safety proofs for superintelligence are a genuinely hard problem. A ban whose exit conditions can never be satisfied is permanent prohibition by another name, which is either the point or the problem depending on your view.

    The Letter in Context

    This was not FLI's first letter. In March 2023 the institute asked labs to pause training systems more powerful than GPT-4 for six months — a letter that drew over 1,100 signers, including Elon Musk, and which the labs effectively declined. In September 2025, Hinton and Bengio were among the signers of a separate Global Call for AI Red Lines. The superintelligence statement escalates the demand from pause to prohibition, and its timing — as labs openly reorganised around superintelligence as the explicit goal — reads as a response to the race accelerating, not slowing.

    Abstract illustration of red and black lines radiating outward in a burst pattern

    And here is the irony the letter's own supporters acknowledge: one year after the 800+ signed, the US government moved in the opposite direction — not banning superintelligence but rebranding around it (see Post 15, on the September 2026 executive order that made "Super Intelligence" the federal government's official term). The ban letter and the EO are the two poles of the current moment: one says stop until proven safe, the other says the era has already begun.

    For what the labs themselves are doing about the safety problem today, see Post 17 on superalignment.

    The Bottom Line

    The superintelligence ban letter is best understood as a political event, not a policy proposal: 800+ prominent figures, across every political divide, putting their names to the claim that building superintelligence should require affirmative collective consent. Whether a ban could work is a separate, harder question — and the honest answer, even from sympathisers, is that nobody has shown how. What the letter proves is that "should we build this?" is now a mainstream question. What it does not prove is that anyone has an enforceable answer.


    Subscribe to receive free email updates:

    0 Response to "Ban Superintelligence? The Open Letter With 800+ Signatories, Reported Straight"

    Post a Comment