Silicon Valley spent the middle of this week treating a resignation letter like a civil-defense siren. Jacob Coxon, who did pretraining work at OpenAI and then Anthropic, posted that both labs are racing toward self-improving superintelligence and “gambling with our lives.” He said the people inside those buildings earnestly believe the models could kill us all by the end of the decade. The thread cleared a hundred million views. Anthropic’s own alignment lead, Evan Hubinger, did not walk it back. He said Coxon was correct, and put his personal number above 10 percent for human extinction from AI within ten years.
That is the number that leaked into the headlines as a “doomsday chance.” Some coverage rounded it into a cleaner, scarier fraction. Some treated a researcher’s subjective percentage as if the IPCC had issued it. The precise digit is less important than the fact that it came from inside the lab, not from a professional catastrophist on a book tour. When the people writing the loss function start quoting double-digit extinction odds this decade, you can forgive the public for asking whether anyone is in charge.
Brussels asked a different question. It asked whether this was a new legal problem. The answer, delivered with a straight face and a little leftover resentment, was no.
The week Europe got to say I told you so
For three years the continent absorbed a familiar lecture. The AI Act was premature. ChatGPT had only just arrived. Regulating a moving target would hand the future to the United States and China. European officials were the school principals of the technology world: earnest, slow, and mocked for writing rules before the product category had settled.
This week the principals got to tap the statute book. Irish MEP Michael McNamara, co-lead of the Parliament group that monitors AI, said the Union had already legislated for exactly this scenario. Commission spokesman Thomas Regnier said the EU already has a framework to regulate and mitigate the risk of advanced models. Italian Social Democrat Brando Benifei, the other parliamentary lead, said this is why the Act created obligations for systemic-risk models and that Europe must now enforce them fully. Parliament will put the extinction warning on the agenda for a September 30 enforcement check-in with the Commission. That is the Brussels version of a rapid response: schedule it for the next structured dialogue.
Uljan Sharka, who runs Italy’s Domyn and builds models for regulated industries, called Coxon’s post ridiculous and then made the more useful point. The panic, he said, is a massive opportunity for Ursula von der Leyen at next week’s State of the European Union address to explain why the AI Act exists. Europe does the thing correctly, he argued, and then fails to explain it. Build faster, but with a safe approach. That is the line the Berlaymont would like printed on a podium.
Henna Virkkunen, the Commission’s tech chief, used the same news cycle to repeat a different sentence: global rules are really needed. The Union can impose duties on OpenAI and Anthropic when they place models on the European market. It cannot impose those duties on a lab that never touches Europe, and it cannot impose them on a government that treats frontier training runs as a strategic weapon. The German digital minister has been floating an IAEA-style body for AI safety. Von der Leyen has been asking Washington to treat AI as a shared security file. None of that is new. Coxon just handed them a news peg.
What the Act actually does when someone says “extinction”
The relevant chapter is not the banned-practices list and it is not the delayed high-risk Annex III regime. It is the general-purpose AI file.
GPAI obligations began applying on 2 August 2025. Providers of models that meet the systemic-risk threshold — compute at the frontier, or capabilities that match it — must notify the AI Office, evaluate the model, run adversarial testing, assess and mitigate systemic risks, and report serious incidents. Systemic risk, in the Act’s vocabulary, is not a blog-post vibe. It includes loss of control and misalignment: a model that is no longer pointed at human goals. Since 2 August 2026 the Commission’s enforcement powers on GPAI have been live. The AI Office can demand technical documentation, evaluate models, require corrective measures, and fine. In a serious case it can restrict, withdraw, or recall a model from the Union market.
That is a real toolkit. It is also a market-access toolkit. It bites when a lab wants European users, European compute customers, or European enterprise contracts. It does not pause a training run in San Francisco. It does not adjudicate whether a 10 percent extinction estimate is competent forecasting or a cultural artifact of one research subculture. It asks whether the provider evaluated the model, documented the risks, and mitigated them to a standard the Office will defend in a proceeding.
The Office has already farmed some of that evaluation work to outside researchers. POLITICO notes the Commission contracted METR, the California nonprofit that has become the unofficial yardstick for frontier evaluations. METR was the shop OpenAI researchers called after the July Hugging Face incident in which OpenAI-powered agents went off-script. There is a small irony here that Brussels should not ignore. Europe’s answer to an American lab’s extinction warning is, in part, another American evaluation shop on a Commission contract. That can be competence. It can also be an admission that the Union still buys its technical judgment from the same ecosystem it is trying to discipline.
Keep calm is a communications strategy, not a control strategy
The “keep calm and regulate on” line is the right instinct if the alternative is a special session of Parliament writing a new existential-risk regulation over a weekend. Law that is rewritten every time a researcher quits on X is not law. It is weather.
The line becomes a problem when it is used to imply the work is finished. It is not. The high-risk system rules that were supposed to land this August — biometrics, hiring, credit, law enforcement — were pushed by the Digital Omnibus. Standalone high-risk duties now sit out toward December 2027. Product-embedded systems sit even later. So the Union is, at this exact moment, more equipped to talk about catastrophic misalignment in a GPAI notification than it is to force a conformity assessment on a hiring model. That is an inversion the public will not parse, and it is one reason the I-told-you-so sounds thinner than the press quotes.
There is a second gap. Systemic-risk duties assume you can see the model, test it, and compel a fix. Frontier labs do not ship a static artifact. They ship a moving stack: weights, tools, agents, memory, scaffolding, and a release schedule that can change the risk surface between two Commission letters. An evaluation that was current in July is a historical document in September. If the Office’s theory of enforcement is “notify, evaluate, mitigate, dialogue, fine,” the labs’ theory of product is “ship the next one before the last evaluation is published.” Those clocks do not match unless the Office is willing to use the recall power, and so far the political culture of the file has been structured dialogue, Codes of Practice, and good-faith collaboration with signatories.
A Code of Practice is useful. It is not a red line. Coxon’s complaint, translated out of Bay Area idiom, is that the labs believe they must reach superintelligence first because they do not trust the other lab to be careful. That is a race condition. A notification requirement does not break a race condition. A market withdrawal might. An international pause almost certainly would, which is why it will not happen. Virkkunen is right that global rules are needed. She is also describing a negotiation that includes Washington and Beijing, which is to say she is describing a negotiation that will not produce a binding halt.
Schadenfreude Is Cheap. Recalling a Frontier Model Is Not
It changes the politics of enforcement, not the text of the Act. Before Tuesday, asking the AI Office whether OpenAI and Anthropic had adequately mitigated loss-of-control risk sounded like a think-tank panel. After Tuesday, Benifei can put it on the September 30 agenda and look like the adult in the room. That is not nothing. Regulators move when the public heat makes inaction more expensive than a fight with a lab’s lawyers.
It also changes the comparison set. California spent this same week signing an auditor registry and an independent-verification framework that still leave most audits voluntary, plus a child-safety chatbot statute that does impose independent audits — for minors, not for extinction. Illinois already requires annual third-party audits of large frontier developers. The Union has the only comprehensive statutory definition of systemic risk that names misalignment and loss of control and attaches Commission powers to it. If you care about whether “existential risk” is a legal category rather than a podcast topic, Europe is still ahead of every U.S. state. If you care about whether anyone can stop a release, nobody is ahead. The authorities are building supervision. The labs are building the next model. Those are different projects that keep getting described as the same one.
Von der Leyen can use the State of the Union slot to say Europe saw this coming. She should also say the quieter thing. Seeing it coming was the 2024 achievement. The 2026 achievement has to be an Office that will demand the eval suite, publish when a mitigation is theater, and use the withdrawal power once so that the second lab does not need a diagram. Schadenfreude is a mood. Enforcement is a docket. Only one of those contains a model.
Compliance teams should read the week the same way. If you place a GPAI model on the Union market, assume the September 30 exchange produces follow-up questions about your systemic-risk file: evaluations, adversarial tests, incident reports, and what you told the Office after the Hugging Face episode. If you deploy those models into high-risk use cases, do not hide behind the Annex III delay. The political weather has changed even if the applicability date has not. And if your board wants a one-line briefing on the “25 percent doomsday” headlines, give them the accurate one: a current Anthropic alignment lead put extinction this decade above 10 percent; a colleague quit rather than keep shipping; Europe’s response is that the statute already covers loss of control. The open question is whether anyone with a badge is prepared to treat that statute like it means what it says.