How the AI Wave Could Drive Some Parts of the World Toward "Neo-fuedalism"
And how some (the e/accs) would like to see it
This is the third post on The Coming Wave: Technology, Power and the 21st Century's Greatest Dilemma, by Mustafa Suleyman.1 It was released Sept. 5, 2023. The first post, explaining why I’m reading it, is here. The second post, on why the coming wave is so, so big, is here. This post, the third, is on the threat that the coming wave poses to the nation-state, our basic building block of civilization for the last several hundred years. The fourth post, my final one, covers Suleyman's proposed solutions and can be read here.
In April, I will be going through John Inazu's forthcoming book Learning to Disagree: The Surprising Path to Navigating Differences with Empathy and Respect.
In February, I went through Nick Troiano’s The Primary Solution.
In January, I went through Michael Wear’s The Spirit of Our Politics.
The book of the month schedule is here.
The Coming Wave’s Threat to Free Societies
Mustafa Suleyman's book does something impressive. It situates the coming wave of technologies — AI, advanced biotech, quantum computing and robotics — in the context of world history over the last few centuries.
"Over the last five hundred years," Suleyman writes, "centralizing power in a singular authority has been essential to keeping the peace, unleashing the creative talents of billions of people to work hard, seek out education, invent, trade, and in doing so, drive progress."
"The grand bargain of the nation-state, therefore, is that not only can centralized power enable peace and prosperity, but this power can be contained using a series of checks, balances, redistribitutions, and institutional forms," he adds. "Today, more so than at any time in history, the technologies of the coming wave threaten to unsettle this fragile equilibrium" (147).
The Coming Wave is a book that hopes that humanity can figure out a way to contain and constrain AI. But there is a sense, too, that this wave is so formidable that it may fundamentally alter the shape of government and society in the coming decades.
"In this scenario rapid fragmentation of power could accelerate a kind of 'turbo-balkanization' that gives nimble and newly capable actors unprecedented freedom to operate," Suleyman writes, acknowledging that this is speculation as of now, but also possible. "Something more like the pre-nation-state world emerges in this scenario, neo-medieval, smaller, more local, and constitutionally diverse, a complex, unstable patchwork of polities" (201).
"Taxation, law enforcement, compliance with norms: all under threat," he says, in this scenario.
The Self-Declared Tech Overlords
And there are some very powerful people who want this future, and are pushing to see it happen, Suleyman argues.
"Hyper-libertarian technologists like the PayPal founder and venture capitalist Peter Thiel celebrate a vision of the state withering away, seeing this as liberation for an overmighty species of business leaders or 'sovereign individuals,' as they call themselves" (201).
There is a quote in a dystopian novel by Andrew Yang and Stephen Marche, released last year, where a billionaire character clearly based on Thiel says the following:
"The engineers are the real power now, as is our right, as is our destiny ... Everyone talks about conspiracies. You know what the real conspiracy is? Democracy. Democracy means mediocrity. Sheep built democracy, out of fear. Fear of talent. Fear of the builders. Fear of the creators. They need us but they hate us. We have a real chance this election to end the administrative state, so that we may live by what we will for ourselves."
Suleyman clearly states that this future, of a neo-fuedal world without strong nation-states, "would be a disaster" (151).
But he is also clear-eyed that our governments are beleaguered by problems and weakened by infighting and dysfunction. Our citizens do not trust our leaders, and our incentive structures are pushing our leaders to act out rather than knuckle down and solve problems.
This is especially the case in democracies. Our openness and emphases on deliberation and equality have become stress points, while authoritarian states like China act quickly and decisively in response to a quickly changing and evolving threat environment. Even here in the West, some of the growing appetite for a strongman leader is that he will get stuff done, no matter how it happens.
The coming years and decades will present a bigger menu of challenges than we've already seen. Labor will be under assault from robotics and AI. That will further roil the working class and beyond. Public safety threats will proliferate. The future of war is quickly changing.
A ‘new arsenal of repression’ for authoritarian states
Meanwhile, in places like China, the government in Beijing is running AI policy on two-tracks: "a regulated civilian path and a freewheeling military industrial one" (231).
Suleyman is frank that the AI wave poses a massive threat to free nations, and could throw them into chaos while strengthening the control of authoritarian regimes.
"Without a major shift in focus, many open democratic states face a steady decay of their institutional foundations ... At the same time, authoritarian states are given a potent new arsenal of repression" (185).
Wealth, too, will see massive sea changes, and concentration. In fact, one of the threats to the nation-state has already become some of the most powerful tech companies, Suleyman writes.
"I do think we have to confront the sheer scale and influence that some boardrooms have," he writes. "They are empires of a sort, and with the coming wave their scale, influence and capability are set to radically expand" (186).
To amplify this idea, Suleyman makes a brilliant point about how too often we compare AI to the individual human mind or intelligence. "What this misses is that the most powerful forces in the world are actually groups of individuals coordinating to achieve shared goals" he writes.
"Indeed, machine intelligence resembles a massive bureaucracy far more than it does a human mind" (186).
I loved this point, because it really drives home how profoundly important it is for our society to rediscover the value of institutionalism in an age of hyperindividualism.
“A Wave Full of Collisions”
In closing, we are heading toward a time of centralization in some ways and places, and decentralization in others. "Herein lies the key to understanding the coming wave of contradictions: a wave full of collisions" (202).
The question for next week's final post is: what can be done to avoid these darker outcomes, "to constrain technologies to ensure they continue to do far more good than harm" (228).
Suleyman was in the news this week. He was hired by Microsoft away from the startup he founded, Inflection. Microsoft has brought Suleyman on to run their consumer AI business. Inflection had raised $1.5 billion and was developing its own chatbot.