| Along with OpenAI and Google, Anthropic is one of the world’s three frontier artificial intelligence laboratories and maker of the popular chatbot Claude. It has been a key supplier of AI products to the United States government, but after a standoff over contractual terms, late last week President Donald Trump announced that federal agencies would cease using Anthropic’s products following a six-month phase-out. Secretary of War Defense Pete Hegseth turned up the pressure by “designat[ing] Anthropic a Supply-Chain Risk to National Security,” which, he claimed, prevents Anthropic from doing business with any other firms that do business with the U.S. military. |
| In a response posted on its website, Anthropic took issue with Hegseth’s extraordinary action, noting that the supply chain risk designation has been “historically reserved for US adversaries, never before publicly applied to an American company.” Anthropic also contested Hegseth’s authority to ban the company from doing any business with other military contractors. Rather, Anthropic contended, the key statute forbids those other contractors from doing business with a company designated as a supply chain risk only in connection with their work for the military. In this view, if a company does business with civilian clients as well as the Pentagon, it may continue to use Anthropic’s products on its civilian work. |
| Anthropic is correct about the statute it cited, but the government may have other authorities to impose the broader ban. Whether it can do so successfully could be resolved in future litigation. Although there are obstacles to Anthropic suing the government, it is possible that litigation between Anthropic and one or more of its customers could tee up the issue in a posture that a court can resolve. |
| In the balance of this essay, I address the issues at the root of the dispute between Anthropic and the Pentagon. President Trump announced his ban on using Anthropic’s products in a social media post decrying what he called a “WOKE COMPANY” led by “Leftwing nutjobs.” What inspired this characterization? Was Claude chastising military intelligence officers for failing to use gender-inclusive language in their queries? Did Claude call out the 2025 re-renaming of Fort Bragg as a racist dog whistle? |
| Hardly. Anthropic had only two reservations. Whereas the Department of Defense wished to use its products for “any lawful purpose,” Anthropic sought a further contractual stipulation that its products would not be used to conduct mass surveillance or in autonomous weapons. Apparently, those were deal breakers for the Pentagon. |
| Is Mass Surveillance a Lawful Purpose? |
| In its response to the supply chain risk designation, Anthropic stated its belief “that mass domestic surveillance of Americans constitutes a violation of fundamental rights.” That is no mere belief. It is a constitutional fact. |
| In Riley v. California in 2014, the Supreme Court unanimously construed the Fourth Amendment to require that government agents must generally obtain a warrant based on probable cause to access the contents of someone’s mobile phone. Writing for the Court, Chief Justice John Roberts underscored the depth and breadth of information available to the government from a person’s mobile phone—including intensely private matters such as emails, texts, financial information, browsing history, and virtually every aspect of life. Surveillance tools that would permit the government to gain access to people’s phones, computers, tablets, wearables, and other devices pose a qualitatively similar risk to privacy. Thus, mass surveillance—which necessarily would not be preceded by individualized judicial assessment of probable cause for everyone subject to such surveillance—would be a mass violation of the Fourth Amendment. |
| But if that is so, then why would Anthropic need a special carveout? Mass surveillance is not a lawful purpose in the first place, so in authorizing the Defense Department to use its products for any lawful purpose, Anthropic would not be authorizing their use for mass surveillance. |
| There are at least two possible reasons why Anthropic nonetheless sought the mass surveillance carveout. First, some kinds of mass surveillance would not violate the Fourth Amendment. That includes surveillance of non-U.S. persons outside the United States. In addition, the government can learn a great deal by using open-source collection methods that may not implicate constitutional or statutory protections. Meanwhile, the so-called third-party doctrine sometimes allows the government to obtain information from firms (such as internet service providers and mobile phone companies) without a warrant, even when a warrant would be required to obtain the same information directly from individuals. |
| Hence, it is possible that the “fundamental rights” of the American people that Anthropic seeks to guard by refusing permission for its products to be used for mass surveillance extend beyond the protections afforded by current law. Anthropic was founded in 2021 by former OpenAI researchers to center AI safety in its mission, paying special attention to how its products are used, so much so as to have created a constitution that places limits on what Claude may do. Claude’s constitution can include limits that the U.S. Constitution as construed by the Supreme Court lacks. |
| Second, even if, in principle Anthropic would object only to the kind of mass surveillance that violates the Fourth Amendment or other legal restrictions, the precise scope of those limits is under-determined by existing precedents. By objecting to the use of its products for all mass surveillance, Anthropic could have been aiming to close a potential loophole or avoid an argument about exactly which forms of mass surveillance are lawful. |
| Autonomous Weapons |
| Similar analysis applies to Anthropic’s insistence that its products not be used in or for autonomous weapons. Its statement expresses doubts “that today’s frontier AI models are reliable enough to be used in fully autonomous weapons. Allowing current models to be used in this way would endanger America’s warfighters and civilians.” Although stated as a policy concern, this statement also has implications for legality—especially the worry about endangering civilians. |
| Article 57 of the 1977 Additional Protocol (I) of the Geneva Conventions forbids targeting civilians and requires that any collateral civilian casualties be proportionate to the military advantage secured by attacking legitimate targets. The U.S. signed but never ratified the Additional Protocol, but the principle of Article 57 has long been an element of customary international law that the United States accepts. Indeed, members of the U.S. armed forces have been successfully prosecuted for killing civilians in violation of this norm. Deployment of an autonomous weapons system that failed to adequately distinguish between combatants and civilians or attacked military targets while paying inadequate attention to the disproportionate impact on civilians would thus be unlawful. |
| Anthropic’s worries about autonomous weapons are not hypothetical. Israel has deployed several AI systems in selecting targets in Gaza. Although humans are supposed to approve each targeting decision, there is evidence that in many circumstances they simply trust the AI, which seems to be the point of such systems: they generate more targets more quickly than humans can. The fact that Hamas uses the civilian population as human shields undoubtedly has contributed to the large number of civilian deaths in Gaza, but the fact that Israel has made extensive use of AI likely has also played a role. |
| But now we come to the same puzzle as with mass surveillance. If deployment of autonomous weapons would be unlawful anyway—because it would create too great a risk of attacks on or disproportionate harm to civilians—then why did Anthropic insist on a specific carveout? The answer parallels our answer for mass surveillance. |
| First, Anthropic might have qualms about supporting some uses of autonomous weapons that the international law of war permits. Autonomous weapons are sufficiently novel that it is not entirely clear the extent to which they violate or comply with international law. Even if commanders in the field who make the decision to deploy autonomous weapons are not implicated in any civilian casualties those weapons cause, Anthropic may understandably wish to avoid moral complicity that does not rise to the level of legal complicity. Moreover, Anthropic cited the safety of autonomous weapons for U.S. service members who use them, which would not implicate international law but is nonetheless a legitimate concern for the company. |
| Second, and again paralleling the analysis of mass surveillance, Anthropic can better protect its interest in avoiding complicity by obtaining a categorical carveout than by arguing on a case-by-case basis that some particular use of its AI systems in autonomous weapons is unlawful. |
| * * * |
| Accordingly, Anthropic had understandable reasons not to be satisfied by the government’s assurances that its products would be used only for lawful purposes. The remaining question is why the Defense Department was unwilling to allow the carveouts Anthropic sought. The most obvious explanation is deeply troubling: President Trump and Defense Secretary Hegseth may plan to conduct mass surveillance and to deploy autonomous weapons. |
Have friends who like law? Forward this email. 
Like Verdict for legal discussions on Facebook. 
Follow @verdictjustia for news and updates on Twitter. 
