The Alignment Problem of 1776

What the Founders knew about unaccountable power, and what it means for superintelligence

Today, the United States turns 250. By the time it turns 260, the most capable minds in the country — the ones writing the laws and commanding the markets — may not be human. The reason is AI superintelligence.¹

While today’s AI is chatbots that draft our emails, agents that write and debug our code, and assistants that summarize our meetings and help with writing this very article — this is not where we are going. Multiple multi-billion dollar AI companies have a different, explicit goal — to build an AI superintelligence that is smarter than every human combined.

Such AI superintelligences will be very different from what we see today. Superintelligence is not mere “bookish” intelligence — these systems might exceed Elon Musk at creativity and engineering, Albert Einstein at scientific ability, and Terence Tao at mathematical ability… while being more likable than Dolly Parton, better at building mass movements than Donald Trump, and better at speaking than Barack Obama. And there may be millions of such superintelligences, working perfectly in coordination, at speeds millions of times faster than humans.

The men who signed the Declaration of Independence could neither have anticipated the transformation of the past 250 years nor this potential transformation to come. Indeed, many people alive today do not understand or anticipate it. We might then conclude the Founders have nothing to say about artificial intelligence — that consulting the Founders on superintelligence is like consulting them on antibiotics.

But while the Founders were not experts in technology, they were experts in exactly one problem — how do you safely live alongside a powerful force you can never fully trust, fully predict, or fully control? 250 years ago, the question was the King …and then the new American government itself. Today, the same question applies to superintelligence: can it be similarly controlled, and can it be similarly made accountable to the people?

The Declaration

The Declaration of Independence opens with self-evident truths — all men created equal, endowed with unalienable rights, governments deriving their just powers from the consent of the governed. But most of the document is something less quoted — a long list of grievances against a distant power that kept altering the terms of ordinary people's lives without their consent.

In reality, the colonists were, by the standards of the age, actually lightly governed. Parliamentary taxes on the eve of the Revolution came to a small fraction of what Englishmen at home paid — by most estimates the average Briton bore a per-capita tax burden ten to twenty-five times that of the average colonist. The colonists were not, in any material sense, groaning under tyranny.

However, on the very day in 1766 that Parliament repealed the hated Stamp Act, it passed the Declaratory Act, asserting its authority to bind the colonies “in all cases whatsoever.” The act levied no tax and quartered no troops — it did nothing except state the claim. But the claim became the whole problem. The revolutionaries’ conception of freedom held that a people is unfree not when power actually abuses them, but when power could abuse them at its own discretion and answers to no one for the choice. A slave with an exceptionally benevolent master is a slave nonetheless.²

Today’s AI has similar problems. First, that superintelligent AI may prove our own undoing. If humans were no longer the most intelligent and capable species on the planet, could we ensure our desires are still respected? Or would we have as much control over the fate of the AI-run world as chimps have over the human-run world? The people making these AI systems frequently claim they are far from certain they can control what they are building. Elon Musk speaks of building powerful AI as "summoning the demon." Sam Altman wrote in 2015 that “development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity”. And in 2023 the CEOs of OpenAI, Anthropic, and Google DeepMind all signed a one-sentence statement that mitigating extinction risk from AI should be a global priority alongside pandemics and nuclear war.

Second, even if we could ensure that such superintelligence has our best intentions at heart — that such AI systems were machines of loving grace that end disease, end drudgery, and give us abundance beyond our wildest dreams, would we still be free? “The taxes are low, the empire keeps you safe, and life is good” was also true in 1775, and the colonists revolted anyway, because benevolence is not the same as accountability.

Consent of the governed

The people building these systems do not talk like people who believe they need anyone’s consent. Mira Murati, then chief technology officer of OpenAI, was asked about the creative livelihoods her products would displace and offered that “maybe they shouldn’t have been there in the first place.” Anthropic’s Dario Amodei warned that AI could eliminate half of all entry-level white-collar jobs within five years, in the tone of a man reporting the weather rather than one of the people doing the replacing. The mood is always indicative, never interrogative. This will happen. This is inevitable. The Founders knew this voice intimately — it is the voice of power personally insulated from the consequences and unaccustomed to being told no by the people whose affairs it has taken upon itself to arrange.

Rarely is there the fundamental question — ought the millions of people whose working lives are being redesigned have a say in the terms, the pace, or the destination?

And when the American people are actually asked, the answer is not ambiguous. Recent polling finds that over 80% of both Democrats and Republicans think AI companies should not build systems smarter than humans until they can demonstrate those systems can be controlled, and a separate national survey found only 5% of Americans support a trajectory of fast, largely unregulated development.

We thus end up in a similar standoff as the Declaratory Act vs the Declaration of Independence, with the companies asserting an authority to proceed in all cases whatsoever and an American public with different terms. Where this standoff goes remains to be decided.

The genius of refusal

In March 1783, with peace at hand, officers of the Continental Army were still upset — because they had not yet been paid. They gathered at Newburgh, New York to weigh an ultimatum. If the war somehow continued, the ultimatum proposes the army abandon Congress and leave the country defenseless. If peace came, the ultimatum proposes a refusal to disband — a thinly veiled threat of military takeover.

General George Washington appeared unannounced at the gathering. “Gentlemen, you will permit me to put on my spectacles, for I have not only grown gray, but almost blind, in the service of my country”, Washington said. He urged the soldiers to work peacefully, which they did.

Nine months later, Washington traveled to Annapolis and resigned his commission to the civilian Congress — the voluntary surrender of supreme military power by the man in American history best positioned to keep it. When King George III heard what Washington intended, he reportedly told the painter Benjamin West that if Washington did it, he would be the greatest man in the world. Washington did it — and then did it again, walking away from the Presidency after two terms and fixing a precedent that held for 150 years.

The signature act of the American founding is not the seizure of power but the relinquishment. Every republic before ours had eventually met its Caesar and the Founders realized that a true constitutional order must instill an allergy to power. Power must never be fully trusted — not because the powerful are unusually wicked, but because they are ordinarily human.

Now hold that tradition up against the AI race, whose public metric is to build as powerful as you can, as fast as you can. The leaders of the frontier companies say openly that their own products may become dangerous beyond precedent, but that they nonetheless can be trusted to do what is best for society and govern their tools appropriately, showing us their voluntary commitments, responsible scaling policies, and safety frameworks a company can revise at will.

Madison had a name for written limits unsupported by enforcement — “parchment barriers,” which experience showed were “greatly overrated” against “the encroaching spirit of power.” The Founders would not have doubted the sincerity of those leading the AI companies. They would have asked what happens when the sincere man is replaced, outcompeted, or simply wrong — and instead insist that no single actor hold unilateral command. Many such “parchment barriers”, such as OpenAI’s non-profit structure, have already fallen to economic competition and other weaknesses of human will.

The industry already knows this lesson, yet seems to not yet understand it. J.R.R. Tolkien, an Oxford traditionalist who shared the Founders’ allergy in full, wrote The Lord of the Rings, a story whose central discovery is that ultimate power cannot be safely wielded by anyone, for any end. Gandalf dares not take it, even to keep it safe; Galadriel passes her test by turning it down. The tragedy is Boromir, the sincere patriot who insists his people, uniquely, can wield the enemy’s weapon against him. In a 1943 letter to his son, Tolkien wrote that “the most improper job of any man, even saints... is bossing other men. Not one in a million is fit for it and least of all those who seek the opportunity.”

This is the book that AI leaders quote from — Sam Altman published a blog post reflecting that AGI “has a real ‘ring of power’ dynamic to it, and makes people do crazy things”. Elon Musk, an OpenAI co-founder turned rival similarly renders his verdict on Altman: “The ring of power can corrupt, and he has the ring of power”, yet the irony is this statement was delivered from atop xAI, a competing AI company Musk had founded just months earlier.

Elon Musk and Sam Altman co-founded OpenAI out of their distrust of Google, Dario Amodei co-founded Anthropic out of his distrust of OpenAI, and Elon Musk co-founded xAI out of his distrust of Sam Altman — each person thinking they must pursue ultimate power lest someone less responsible pursue such power first. The Founders’ answer is that there never will be one wise, benevolent AI company who can carry the Ring safely. Instead, we need Washington at Annapolis — a system where the Ring gets handed back to the people.

Auxiliary precautions

In Federalist No. 51, James Madison wrote:

If men were angels, no government would be necessary. If angels were to govern men, neither external nor internal controls on government would be necessary. In framing a government which is to be administered by men over men, the great difficulty lies in this: you must first enable the government to control the governed; and in the next place oblige it to control itself.

Read with modern eyes, that is a near-perfect statement of what AI researchers call the alignment problem — the question of how to build a system powerful enough to be useful while ensuring it stays controllable and pursues the goals you intended. Madison’s premise is that you cannot rely on the good character of the agent, so you must rely on structure. A dependence on the people is the “primary control,” he wrote, “but experience has taught mankind the necessity of auxiliary precautions.”

Before the Convention, Madison systematically studied every ancient and modern confederacy he could find sources on, cataloguing their failure modes the way a safety team catalogues jailbreaks. Madison’s core mechanism, “ambition must be made to counteract ambition,” is adversarial oversight — set powerful agents against each other so none can dominate, and give each the means and motive to check the rest. Every claim that some class of people was wise enough to rule without a check has ended the same way.

However, these Madisonian mechanisms assume the contending powers are roughly comparable. A system or set of systems smarter, faster, and more coordinated than every institutional check simultaneously breaks the balancing machinery. This is what Madison feared most, one interest growing powerful enough that the precautions no longer successfully bind it.

The Founder’s machinery must be applied while it still can be. Checks and balances bind humans — humans can be voted out, subpoenaed, outcompeted, shamed, fired. For at least a few more years, the Ring is held entirely by humans — executives, boards, engineers, the officials who could govern them — and everything about them remains within reach of the oldest tools of the republic. The question is not whether the Constitution can restrain a superintelligence. It is whether the American people will assert their authority over the people building one, during the window in which those people can still be obliged to answer.

The standing army

None of this is an argument against permissionless innovation, which is genuinely one of America’s founding advantages. Nobody voted for the light bulb, the airplane, or the iPhone, and nobody needed to — a country that required a congressional blessing before every product launch would still be waiting for the telegraph. Permissionless innovation works because its failures are escapable: if the product is bad, you don’t buy it; if it harms you, you sue; if the company is reckless, it dies and the rest of us carry on. Consent is delivered continuously, through the market, one purchase and one lawsuit at a time.

The line is drawn where the consequences of failure stop being escapable. You can decline to buy a chatbot. You cannot decline to live in a world where millions of superintelligences have been deployed, any more than a Bostonian in 1770 could decline to live in an occupied city.

The Founders understood this distinction. They demanded no congressional renewal for gristmills or printing presses. But they put unprecedented Congressional control over one dual-use technology they could not live without but could not survive unchecked — the standing army. The Declaration indicts King George for exactly this: “He has kept among us, in times of peace, Standing Armies without the Consent of our legislatures,” and he has “affect[ed] to render the Military independent of and superior to the Civil Power.”

The Constitution took this most dangerous instrument in the republic and wrapped it in accountability to the people. Command was vested in an elected civilian, removable by the voters. Congress — not the generals — would raise the army, fund it, and declare its wars. And the Constitution declared that no appropriation for the army “shall be for a longer Term than two Years”, meaning that the army cannot exist by default. Its existence lapses, automatically, unless the people’s elected representatives affirmatively renew it. The most powerful force in the country lives on a two-year lease from the American people, and every soldier in it swears an oath not to a general but to the Constitution.

AI superintelligence risks enabling a similar permanent concentration of power exceeding the threat and promise of the standing army. The country may genuinely want such superintelligence for its prosperity and defense, but such superintelligence must be treated with similar deep suspicion, and must be safeguarded and stewarded with similar deep control and deep accountability.

A Republic, if you can keep it

For 250 years the threats to the republic were recognizably human — kings, factions, demagogues, foreign empires — and the machinery, creaky and patched, has held. When America turns 260, we may share the country with minds that exceed our own more thoroughly than the federal government exceeds any single citizen. Whether that goes well is a technical question but also a deeply political one. It echoes the questions the Founders answered at Newburgh, at Annapolis, at Philadelphia, and in the two-year lease they put on their own army. No power in America — however capable, however sincere, however useful — may be permitted to rule without the consent of the governed.

When Elizabeth Willing Powel asked Benjamin Franklin what the Convention had produced, he reportedly answered: “A republic, if you can keep it.” The technology is new but the problem remains the oldest one in our history. Those who can see what is coming should understand that we are not waiting for new wisdom. We are deciding, right now, whether to use the wisdom we’ve already inherited.

Happy Fourth of July.

1. I’m 50% sure we will have such ‘superintelligent’ AI systems before the end of 2036 absent some major war or regulation disrupting current technological progress. Regardless, precise timelines are actually not relevant to the thesis of this essay.

2. A truth the Founders articulated with painful clarity, and yet lived on the wrong side of.