Dario Amodei invokes Dr. Strangelove. He means it as a cautionary reference. He does not appear to notice that Kubrick's film is a cautionary reference about people exactly like him — calm, probabilistic, persuasive, and wrong at the level of premises rather than conclusions.
The occasion is a recent interview in which the CEO of Anthropic — a company now valued, we are told, at nine hundred and sixty-five billion dollars — explains why artificial intelligence is, on balance, more likely to prevent catastrophic war than to cause it. The argument proceeds along familiar rails: the danger of nuclear conflict lies in miscalculation, in the fog of crisis, in two sides jumping at each other before diplomacy can intervene. AI, being faster and more comprehensive in its analysis, will dissipate that fog. It will model the adversary's intentions with sufficient accuracy that the fatal misunderstanding never occurs. This is not a stupid argument. It is worse than stupid — it is plausible in precisely the way that the Doomsday Device is plausible in Kubrick's War Room, where every speaker is measured, every calculation is internally consistent, and the result is the extinction of all life on earth. The film's thesis is not that irrational men will destroy us. The film's thesis is that rational men will destroy us by being rational about premises that were insane from the start. Amodei's premise — that great-power war is primarily a product of misunderstanding — is such a premise.
Wars do not happen the way bar fights happen. The phrase 'the two sides jump at each other' is doing extraordinary smuggling work in Amodei's formulation, importing the assumption that armed conflict arises from mutual hot-headedness, from a failure of information. But the partitions of Poland were not failures of information. The decision to strip a neighbor of sovereignty was made in full comprehension of what was being done and to whom. The calculus was simple: their land was cheaper to take than to trade for. AI does not interrupt that calculus. AI executes it at machine speed and calls the result optimization. The Poles have been available for comment since 1795.
What Amodei offers is the Enlightenment wager in its most durable form: more knowledge produces better outcomes. It is a wager that has been losing steadily since 1914, when the most educated civilization in human history produced the most efficient slaughter in human history, not despite its education but through the direct application of it.
Every millenarian drinks water that tastes like righteousness. The AI safety community is no exception. Its eschatology merely runs on GPUs.
I keep returning to Stanislav Petrov. September 26, 1983. The machine told him five American ICBMs were inbound. The machine was wrong. Petrov, trained to obey the machine, chose to distrust it. He hesitated. His hesitation — irrational, insubordinate, grounded in nothing firmer than a gut sense that the pattern was off — saved perhaps three hundred million lives.
Every AI safety argument I have encountered amounts to the same theological proposition: next time we will build the machine so perfectly that disobedience becomes unnecessary. This is not engineering. This is the elimination of Petrov from the system by design. It is the confidence that no future moment will require a man to override his instruments on the basis of something he cannot articulate. It is the abolition of hesitation as a strategic resource. And hesitation — the gap between signal and response, the space in which a human being can refuse to be the function the system requires him to be — may be the only thing that has kept us alive since Trinity.
Amodei sounds reasonable. That is precisely the problem. Reasonableness, in Kubrick's grammar, is not the opposite of catastrophe. It is catastrophe's preferred dialect.
I write this from Wellfleet, where the harbor this morning had that stunned quality that follows a week of northeast wind — not calm but exhausted, holding its surface the way a face holds composure after sustained argument. The oystermen were out at first light, governed by the tide, which is to say governed by something that does not optimize. The tide does not calculate whether it is, on balance, more likely to benefit the oysters. It simply operates according to a physics that precedes intention. There is a wisdom in systems that do not know what they are doing — a wisdom that consists precisely in the absence of the bright, totalizing confidence that knows what is best and acts at machine speed to achieve it. Petrov did not know what he was doing. He only knew that the machine's certainty felt wrong. That ignorance was the most rational act of the twentieth century.
