In February 2026, Anthropic told the Pentagon no. The Trump administration wanted unrestricted access to Claude. Anthropic drew two lines: no mass domestic surveillance, no fully autonomous weapons. They got labeled a "supply chain risk" and lost all federal contracts.

They held those lines. Claude was still used in Iran strikes that killed hundreds of people, potentially including a school full of girls. Target selection was never one of Anthropic's conditions.

This is the problem. The conditions were real and held under genuine pressure. They were also drawn where pressure forced them, not where the harm stopped.

Why Ethics Frameworks Don't Work Here

Responsible AI frameworks identify the right harms. But they assume companies can just choose to do the right thing. The AI race is a prisoner's dilemma: every company has an incentive to defect, because one-sided restraint just means someone else takes your contract and does it with fewer guardrails.

That's exactly what happened. OpenAI signed a Pentagon contract after Anthropic's standoff. If Anthropic had drawn broader lines, it would have lost the contract sooner. The harm would have been the same or worse. RAND has modeled this: to make cooperation rational, all players must be bound at once.

How Products Actually Get Built

Products get built toward success metrics: activation, retention, engagement. Legal is supposed to sign off, but legal is a bottleneck, and slow legal review is a tax on shipping speed. Many people working on a subproject assume someone upstream already considered the broader implications. That assumption is often wrong, and verifying it is rarely anyone's explicit job.

A Stanford study found that ethics teams at major tech companies are under-resourced, lack authority to mandate reviews, and are consulted too close to launch to require meaningful changes. Managers with more authority overrode recommended fixes to ship on schedule. A UC Berkeley study had similar findings. All the incentives point one direction: launch.

The Answer Is Oversight

The conclusion is straightforward, even if the execution is hard.

At the product level: mandatory third-party audits before AI models get integrated into anything, with authority to block deployment. Defense procurement already does this for hardware. AI models have been getting a pass.

At the government level: the US DoD's AI ethics principles are voluntary and self-assessed. No one checks. The EU AI Act requires mandatory assessments applied to all companies at once. Whether enforcement holds up is an open question, but the structure is right: compliance stops being a competitive disadvantage when everyone has to comply.

At the international level: 166 countries voted in 2024 to negotiate a treaty on lethal autonomous weapons. Russia, North Korea, and Belarus voted against. China and Israel abstained. The countries that would actually use these weapons without constraints are the ones refusing to be bound. We still need the treaty. The Nuclear Non-Proliferation Treaty is imperfect but beats any voluntary alternative. There's no AI equivalent yet.

The Point

The structure of the AI industry rewards narrow ethics and punishes broad ones. Any framework that ignores this is just a preference that holds until it gets expensive.

The fix is mandatory oversight with real authority, applied to everyone at once. Building it requires people who understand both how institutions work and how products ship.