A line in Dario Amodei’s new essay caught my eye (emphasis mine):
It is possible to have an even stronger version of this position, which is that because the possibilities of AI-enabled totalitarianism are so dark, autocracy is simply not a form of government that people can accept in the post-powerful AI age. Just as feudalism became unworkable with the industrial revolution, the AI age could lead inevitably and logically to the conclusion that democracy (and, hopefully, democracy improved and reinvigorated by AI, as I discuss in Machines of Loving Grace) is the only viable form of government if humanity is to have a good future.
I agree democracy would be the most desirable post-AGI form of government. But that it is the only viable form of government does not clearly follow from Dario’s essay. He praises democracy while doubting decentralized antibodies, like using AI to check AI. He condemns authoritarianism while elevating centralized defenses, like drafting Claude’s constitution. If I were not reading carefully, I might have thought his arguments implied the following claim instead:
[Not a real quote!] Just as feudalism became unworkable with the Industrial Revolution, democracy could become unworkable in the AI age.
To be clear, I don’t think Dario thinks democracy will become unworkable. In my experience, Anthropic has only ever acted in good faith, and I’m sure Dario has already thought deeply about the questions I’m about to raise. Rather, the essay surfaces a tension between the urgency of preparing for powerful AI, and the desirability of collective deliberation in that preparation.
For Dario, preparing for powerful AI is urgent indeed. He writes that “we have no time to lose,” compares it to “the single most serious national security threat we’ve faced in a century, possibly ever,” and he “can feel the pace of progress, and the clock is ticking down.”
The greater the urgency, the greater the strain on democracy. At least, people have engaged with this tension before. The Roman Republic sometimes appointed a dictator with extraordinary powers in wartime. Lincoln suspended the writ of habeas corpus to detain Confederates. During the Cuban Missile Crisis, Kennedy blockaded Cuba and negotiated with Khrushchev without asking Congress. In a crisis, for better or worse, leaders represent people at a greater distance.
In preparing for powerful AI too, we have to confront this tension plainly. Otherwise, I’m left with many questions about the risks and our defenses against them in The Adolescence of Technology.
First, a few of the defenses Dario raises against AI risks don’t seem more democratic than other alternatives at first blush. In response to autonomy and misuse risks, he cites Claude’s constitution, which bets that training AI with principles and values will make it more robust to alignment traps. He finds Constitutional AI more promising than the strategy of “[keeping] AIs in check with a balance of power between many AI systems, as we do with humans,” as a small number of base models and similar alignment methods industrywide may cause AI to fail in a correlated way. Does Constitutional AI prevent these correlated failures?
Beyond that, the more important question is who gets to participate in designing an AI’s constitution. Today, frontier labs unilaterally design their own models. But there’s no reason why they wouldn’t broaden their inputs to include more people, as Anthropic has explored. Dario does, however, call out “some AI companies” that “have shown a disturbing negligence towards the sexualization of children.” Presumably, the values behind this negligence are less welcome in an aligned constitution. How should we feel about the proliferation of diverse constitutions, or competing values in the same constitution?
Dario is obviously aware of the question. Against bioweapon-related misuse, he cites costly classifiers that Anthropic and DeepMind maintain to block dangerous outputs, and agrees that the problem “can’t be solved by the voluntary actions of Anthropic or any other single company alone.” It strikes me, though, that of the fivefold defenses against labor market disruption (get better job data, work with traditional enterprises, take care of your own employees, large-scale philanthropy, and government intervention), all but one right now primarily depend on unilateral action from private companies or individuals. Democratic ideas, or will, are not yet there.
This prisoner’s dilemma brings to the fore the biggest question for me. In the foreseeable future many actors will decide how to develop powerful AI, not least among them actors who don’t share Dario’s views. He concludes that “the last few years should make clear that the idea of stopping or even substantially slowing the technology is fundamentally untenable.” I agree. But it seems similarly untenable to me that all the actors controlling the direction of AI will agree on what to do.
Would Dario accept this characterization? If so, how would he take setbacks to his efforts? This could range from the relatively harmless “overly prescriptive legislation” to catastrophic prisoner’s dilemma defections that would erase prior work. And if those decisions are democratically legitimate? One wonders if the “some AI companies” he calls out for being negligent includes one which just integrated with the Pentagon. I also can’t help but notice that one of his unambiguous calls to exclude participation from an actor he considers risky—don’t sell chips to China—was resoundingly rejected by a President democratically empowered to do so.
Dario asks us to imagine we are a head of state treating “a country of geniuses in a datacenter” as a possible national security threat. I find it hard to see how we get there. Intelligence today is diffuse; we’re a country of PhD-level knowledge on the smartphone of every citizen. How did they all fit into one datacenter? Perhaps one company unilaterally decided not to release their genius-level intelligence. But then, why didn’t other companies compete to do so? Did the government intervene to stop them? What if there was not the political will, the awareness of the urgency, or even the value alignment to do so? Our present democracy is less likely to produce a country of geniuses in a datacenter than it is to endow a country of voters, exactly where they are, with ever more agency.
Again, I’m sure Dario is no stranger to these concerns. The fact that some defenses are unilateral and easy to implement doesn’t mean democracy will become unworkable. And just because there’s tension between urgency and deliberation doesn’t mean it’s a zero-sum tradeoff. The more encouraging and constructive Straussian reading of The Adolescence of Technology is that we have to enrich our set of participatory defenses. We have to scale them to be robust to substantive disagreement on AI risk. We can’t default to unilateral means to achieve democratic ends, using urgency as an excuse.
If we have or can buy more time, existing democratic processes will work. Everyone will be better educated about AI and its risks (and Anthropic is already hugely contributing to this). The work is slow, but it is the right way to do it. I think powerful AI will probably take longer than Dario does, so this is my solution. But if powerful AI arrives in the next year or two, then we have to expand the frontier to be both responsive and inclusive. For those very different approaches to AI risk to disappear is as unlikely as stopping AI progress entirely. We’ll have to learn to live with something Hayekian like AI checking other AI, the main defense I think Dario underrates. Believing in democracy means having faith in its decentralized wisdom.
I recently watched No Country for Old Men (2007) for the first time, and I get why it’s a classic about change (yes, I’m making this about AI). The antagonist, a hitman played by Javier Bardem, thinks he’s fate incarnate, but the film subverts that. There’s a scene where the Sheriff, played by Tommy Lee Jones, unable to catch the hitman, visits his uncle and laments that he feels “overmatched” by how evil is changing. His uncle refuses to validate him:
What you got ain’t nothin’ new. This country’s hard on people. You can’t stop what’s coming. It ain’t all waiting on you. That’s vanity.
So too for democracy in the AI age.