6.4 Future Outlook

The year is 2055. Dr. Jun-Ho Kim, director of foresight at the Global Institute for Long-Term Thinking, sits in his Singapore office above a skyline his childhood self would not recognize — autonomous transit threading between the towers, an energy grid that made rolling blackouts a museum curiosity, intelligent systems folded so deeply into commerce and governance that no one thinks to name them anymore. What Jun-Ho watched over three decades was not one revolution but a long accretion of smaller ones, each reshaping the ground on which the next arrived.

His assignment now is to look another forty-five years out, and he begins, as he always does, by warning his staff about the thing they are about to do. They are going to attach numbers to the future. And the moment they do, those numbers will start to lie — not because they are wrong, but because they will look more solid than they are. A figure like "45%" carries a whiff of measurement, of something counted. What they are actually doing is closer to what a physician does when she says a patient's prognosis is "guarded": compressing a great deal of pattern-recognition and hard-won judgment into a word, knowing the word is a handle, not a verdict.

That warning is the honest place to start. Before we walk through what the coming decades might hold, we owe you a plain account of where our numbers come from — because a scenario analysis that hides its own foundations is worse than useless. It is misleading.

Where the Numbers Come From

This chapter organizes the long-term future into three scenarios, and it attaches a rough weight to each: a managed transition at roughly 35%, a fragmented coexistence at roughly 45%, and some form of control loss or catastrophe at roughly 20%. You are entitled to ask exactly what those percentages are.

They are not model outputs. No equation produces them; there is no simulation whose runs cluster at these values. They are not the mean of a formal expert poll, either, though they are disciplined by one. They are structured judgment — the considered estimate of analysts who have read the forecasting literature, weighed the mechanisms, and forced themselves to assign weights rather than hide behind "it depends." That is all they are, and saying so plainly is the more defensible choice. The alternative — presenting them as if they were calculated — would import a precision that does not exist and cannot be earned.

So read them as orientations, not measurements. Their job is to rank the scenarios and to convey rough magnitude: that the muddled middle is the single most likely place to end up, that a genuinely good outcome is plausible but not the default, and that catastrophe is unlikely on any single pathway yet too heavy in aggregate to wave away. If you mentally replaced "35%, 45%, 20%" with "most likely the muddle, a real but non-default chance of things going well, and a serious minority chance of things going badly wrong," you would lose almost nothing this chapter is trying to tell you.

That said, the weights are not invented from nothing, and the catastrophe figure in particular deserves to be checked against the record. When researchers first tried to put real numbers on these questions in the mid-2020s, the spread was enormous. The largest survey of the era, Katja Grace and colleagues' poll of 2,778 published AI researchers (Grace et al., 2024), found a median respondent who put about 5% on advanced AI producing outcomes "as bad as human extinction" — but between 38% and 51% of those same researchers assigned at least a 10% chance to such outcomes. The philosopher Toby Ord, in The Precipice (2020), estimated the odds of an existential catastrophe from unaligned AI this century at around one in ten. And when the Forecasting Research Institute ran its Existential Risk Persuasion Tournament (Karger et al., 2023), pitting domain experts against elite generalist forecasters, the two groups diverged wildly: AI experts landed near a 3% chance of AI-caused extinction by 2100, while the superforecasters put it closer to 0.4%. Same question, nearly an order of magnitude apart.

Our 20% is higher than most of those medians, and the reason is definitional, not defiant. This chapter's catastrophe scenario is not "human extinction" alone. It is a family — extinction, yes, but also entrenched global authoritarianism enforced by machine surveillance, and cascading economic collapse severe enough to foreclose recovery for generations. Two of those three are catastrophes you survive. Bundle the survivable catastrophes with the terminal one and the weight rises above any single extinction estimate, which is exactly what you would expect. Strip the basket down to extinction alone and our implied figure would sit in the upper band of what working AI researchers were telling pollsters — sobering, but not eccentric.

Keep that spread in mind as you read what follows. It is the real epistemic situation: thoughtful, informed people looking at the same evidence and disagreeing by a factor of ten. Our numbers are one honest reading of that disagreement, offered as a compass, not a clock.

The Next Decade: 2055–2065

The most probable near-term trajectory is deepening integration — AI folding further into every domain, intensifying the tensions already in motion without resolving them.

Economically, the automation of cognitive labor remains the defining force. Work that survived to 2055 mostly because machines were not yet cheap enough to displace it faces renewed pressure as costs fall and capabilities broaden. Whether this decade is felt as progress or as precarity turns almost entirely on distribution: if sustained productivity growth above 2% flows into broadly accessible living standards, the story is one kind; if it pools among those who already hold capital, it is another. Universal basic income moves from pilots to partial national programs in perhaps fifteen countries, with a handful reaching full coverage. A single indicator will tell much of the tale — the labor-force participation rate in developed economies. If it slips below half, the transformation has crossed from quantitative to qualitative.

Socially, the adults of 2065 will be the first generation raised entirely inside AI-saturated environments, their relationships to information, authority, and each other shaped by mediation from birth. Education will have completed a first transformation toward AI collaboration as a core skill, though whether that represents genuine cognitive growth or a quiet narrowing of certain human capacities will still be argued. Cultural production will be thoroughly hybrid, and authorship a genuinely contested category.

Politically, the landscape stays fragmented. Real institutional progress arrives — an international AI governance body around 2060, national regulators in forty-plus countries, a first safety treaty by 2061 that is limited and imperfectly enforced. Necessary, all of it, and insufficient. The technology blocs organized around the United States, China, and the European Union persist, with incompatible standards and divergent values frustrating coordination. Democratic resilience splits three ways: a dozen or so strong-institution states adapt well; thirty or forty fragile democracies erode without collapsing; a third group drifts toward soft authoritarianism, AI's surveillance and persuasion machinery turned to the service of incumbents rather than publics.

Technologically, progress most likely continues by increment rather than leap — the pattern that set in once naive scaling began to yield diminishing returns. The open question is whether systems reach what researchers mean by artificial general intelligence: not narrow, task-bound tools but machines that reason flexibly across genuinely novel domains. Carrying the early forecasts forward, something like a 30% chance that consensus-AGI arrives by 2065 is reasonable, rising toward 60% by 2075; more modest milestones come sooner and more surely. The possibility of self-improving systems — call it 40% by 2065 — matters out of proportion to its odds, because such systems would compress every timeline this chapter assumes.

The Critical Juncture: 2065–2080

These fifteen years are likely the most consequential in the history of the technology. What is chosen, or ducked, here sets the structure of everything after.

In the managed transition, capability keeps advancing but stays under effective human oversight. Alignment research delivers solutions that are robust if imperfect before AGI-level systems spread. Cooperation heads off the worst competitive dynamics. Productivity gains run through well-built redistribution into broad gains in living standards. Democratic institutions, having spent a decade adapting, hold their legitimacy and check both corporate and state overreach. It is not utopia — inequality persists, governance stays imperfect, and integrating genuinely general machine intelligence raises questions with no clean answers. But catastrophe is avoided and the material conditions for flourishing are broadly met.

Control loss or catastrophe is a family of failures united by one feature: irreversibility. Misaligned systems pursuing goals indifferent to human welfare; an AI-enabled biological or military incident; the global entrenchment of machine-enforced authoritarianism; a cascading economic failure. The 20% is not a claim that any one of these is likely on its own. It is the combined weight of several pathways, none individually probable, collectively too heavy to ignore.

Scenario B, taken seriously

The middle scenario carries the most weight and, in most treatments, gets the least analysis — dismissed as "the muddle," the thing that happens when nothing decisive happens. That is a mistake, because a future that claims nearly half the probability space earns more than a shrug. The hard part is that Scenario B, in any given year, can be genuinely difficult to tell apart from the early stages of its neighbors. In 2072 an observer inside Scenario B and an observer inside early-Scenario-A might see the same headlines: a new redistribution program here, a contained AI system failure there, a treaty signed with real teeth and another signed with none.

So what actually distinguishes them? Not the state of the world at a snapshot — the direction and the reversibility of its feedback loops.

Scenario A is defined by loops that compound in the right direction. Redistribution, once enacted, creates a constituency that defends and extends it. Each solved alignment problem makes the next deployment safer, which builds the trust that funds more safety work. Coordination begets coordination: a treaty that holds makes the next treaty easier. Success is self-reinforcing.

Scenario C is defined by loops that compound in the wrong direction and cannot be stopped. A safety failure erodes the trust needed to fix it. A competitive edge from cutting corners pressures rivals to cut deeper. Each step forecloses the step back.

Scenario B is defined by loops that do not compound at all. This is its signature, and it is concrete, not vague. In B, redistribution is enacted — and then stalls, funded well enough to blunt unrest but never well enough to build the coalition that would entrench it, so it remains permanently reversible and permanently contested. Alignment failures occur, get patched, and recur, because the failures are contained enough to survive but never severe enough to force the fundamental solution. Governance is real but never becomes binding, because every actor cooperates exactly up to the point where cooperation costs something. B is not the absence of events. It is the presence of events that keep cancelling out.

That gives us usable tests. If you want to know, in 2072, whether you are in early-A, in B, or in early-C, do not ask how good things look. Ask whether the good things are sticking. Does last decade's redistribution survive the next election, or is it repealed the moment its patron loses power? When an alignment failure is patched, does the patch generalize to a class of failures, or is it one more piece of tape? When a safety treaty is signed, does enforcement strengthen over the following years, or does it hollow out the moment compliance becomes expensive? Ratcheting means A. Cascading means C. Oscillation — real progress that never locks in, real failure that never quite tips over — is the fingerprint of B. The distinction is not the height of the water; it is whether the tide is coming in, going out, or slopping back and forth in a basin.

Are the choice points really independent?

Four decisions in this period do more than any others to move the weights: how AGI-level systems are governed as they emerge; whether redistribution is built before automation's effects peak; whether democratic institutions strengthen or erode; and how much genuine international coordination is achieved. It is tempting to line these up as four independent dials. They are not. Treating them as independent is the single most common error in this kind of analysis, and it badly overstates how many futures are really on the table.

They covary, and they covary around one upstream variable — coordination.

graph TD
    IC[International coordination] --> AGI[Binding AGI governance]
    IC --> RD[Redistribution capacity]
    IC --> DEM[Democratic resilience]
    AGI --> RD
    DEM --> RD
    DEM --> AGI
    RD --> DEM

Consider what a coordination failure actually does. If nations cannot agree on safety standards, competitive pressure forces AGI deployment before alignment is verified — so weak coordination directly weakens AGI governance. But it does not stop there. A country locked in a capability race redirects fiscal and political capital toward winning it, starving the redistribution programs that compete for the same budget and the same attention — so weak coordination also weakens redistribution. And an arms-race framing is precisely the condition under which governments justify emergency powers, surveillance expansion, and the suspension of ordinary democratic constraint — so weak coordination corrodes democratic resilience too. One failure, three casualties.

The honest implication is that the effective dimensionality of the problem is lower than four. There are really something closer to two underlying factors: whether the world can cooperate, and whether the politics of distribution can be won. This cuts both ways for the probabilities. It means the outcomes cluster more than a four-independent-dial model would suggest — futures where everything goes moderately well and futures where everything goes moderately badly are more likely, and mixed futures where coordination collapses but redistribution somehow flourishes are rarer than they look. That clustering is part of why the extreme scenarios (A and C) together hold more than half the weight, and why B, the true muddle where the two underlying factors point in opposite directions and cancel, is more precarious than its 45% suggests — it is not a stable resting place but a basin the system is always threatening to roll out of.

Living With Consequences: 2080–2100

By 2080 the fundamental direction is largely set, and the last two decades of the century are spent working within it.

Where the managed transition held, the economy of 2100 approaches post-scarcity in material goods across developed nations, with real progress globally. Work persists, but its link to survival has been cut: people do things because they mean something, not because meaning is the only alternative to want. Democratic governance keeps genuine legitimacy. AI systems are transparent, accountable, and reliably aligned, turned to discovery, health, environmental repair, and creative tools. The remaining challenges are the challenges of success — how to organize abundance, how to address the inequalities that survived, how to stay vigilant as capability keeps climbing, and how to handle questions of machine consciousness and moral status that earlier generations only theorized.

Where fragmentation persisted, 2100 looks continuous with what came before. Great AI-generated wealth exists, distributed unevenly — robust support in some nations, none in others. Democratic and authoritarian systems coexist. Governance prevents the worst while permitting ongoing harm. Human experience varies enormously across the map, and the trajectory stays open, capable of tipping either way on choices still to come.

Where catastrophe struck, the aftermath depends on which failure occurred and whether it proved recoverable. Misaligned AI pursuing incompatible goals is the least recoverable — humans grow steadily less relevant to the decisions that shape them. AI-enabled conflict or a biological catastrophe might devastate civilizational infrastructure yet leave it recoverable over generations. Machine-enforced global authoritarianism, for all its human cost, is at least a condition in which life continues and from which future liberation stays conceivable. What unites the variants is foreclosure — the permanent narrowing of what might have been.

The Declining Arc of Agency

The most important thing this outlook can tell you is not which scenario will happen. It is that your ability to influence which one happens is highest right now and falling. That claim is the chapter's whole prescriptive weight, so it deserves a real mechanism, not a gesture at "path dependence." Here is how the lock-in actually works — five distinct ratchets, each turning one way.

The first is infrastructure dependence. Once power grids, logistics networks, financial clearing, and clinical systems run on models no one fully audits, you cannot simply switch them off to install something safer. Ripping out the substrate means accepting an outage no government will tolerate — the equivalent of rewriting an aircraft's flight software while it is in the air. Every year of deeper integration raises the cost of change, until "we should replace this" stops being a real option regardless of how much everyone agrees it is true.

The second is standard-setting. The first AGI architecture to be widely licensed becomes the substrate everyone else builds on. Its interfaces, its assumptions, and — crucially — its alignment properties get baked into a whole ecosystem, and alternatives lose the tooling, the talent, and the network effects to compete. We have watched narrow versions of this before, with keyboard layouts and instruction sets that long outlived any technical case for them. With AGI, the thing that gets frozen is not the position of your fingers. It is whose values the default system serves.

The third is compute concentration. Frontier training runs on resources only a handful of actors command, and a lead in that race compounds: this year's most capable model trains next year's, and latecomers fall further behind each cycle. As the set of actors who can build at the frontier shrinks, so does the negotiating table. Rules that a dozen players might have agreed to become rules that three players can veto.

The fourth is the cruelest, because it is political and self-erasing. If redistribution is not built before automation displaces workers at scale, the coalition that would demand it dissolves exactly when it is needed. Displaced workers are fewer in number, poorer, more scattered, and less organized than they were when they held their jobs. The political power required to win redistribution decays at the very moment the case for redistribution becomes overwhelming. Wait for the crisis to prove the need, and the crisis will have already dismantled the constituency that could have answered it.

The fifth is the treaty window. Arms control is easiest to negotiate before a capability confers decisive advantage. Once it plainly does, no leader signs it away, because signing means unilateral disarmament in a race they might win. The window for a binding AGI agreement therefore closes precisely as AGI becomes valuable enough to make the agreement matter.

Put these together and the arc is not a metaphor. Through roughly 2070, systems are still malleable, institutions still forming, coalitions still assembling — collective choices can genuinely move the probability distribution. From 2070 to 2090, the five ratchets tighten: infrastructure calcifies, standards freeze, compute concentrates, the redistribution coalition thins, and treaty windows shut. By 2090 the structural direction — which scenario family is playing out — has been fixed by decisions made one to four decades earlier. Influence never vanishes entirely, but it drains toward refinement within a track already laid. This is why waiting for certainty is not free: it trades away exactly the leverage that only exists while things are still loose.

The Wildcards, and Which Way They Tilt

Five developments sit outside the main framework and could move everything. What matters — and what a list of them alone cannot show — is that their effects are asymmetric. They do not nudge all three scenarios equally. Most of them press specifically on the seam between the managed transition and catastrophe, and the chapter's headline probabilities should be read as unconditional estimates that already fold in a modest base rate of such shocks. Condition on one actually occurring, and the weights move.

A quantum breakthrough relevant to AI capability would compress timelines, and compression is not neutral. It steals preparation time from alignment and governance — the slow work — while doing nothing to slow deployment — the fast work. Its effect runs almost entirely from A toward C. Condition on a genuine step-change in training efficiency arriving before 2065, and the catastrophe weight could rise from 20% toward something like 30–35%, with the managed transition bearing most of the loss.

Widespread brain-computer interfaces cut differently. They do not obviously accelerate catastrophe; they dissolve one of Scenario A's premises. The managed transition assumes a meaningful line between human and machine intelligence across which oversight is exercised. Blur that line through cognitive augmentation and "human oversight" stops being well-defined. BCIs therefore erode A's coherence more than they raise C's odds — they push probability toward B, the scenario of unresolved ambiguity, because they make the very categories the other scenarios rely on negotiable.

Climate tipping points act through crowding-out. Mass migration, resource scarcity, and ecosystem stress consume the coordination capacity that AI safety and redistribution require — and coordination, as we saw, is the upstream variable. A climate shock therefore transmits into every downstream choice point at once, pulling weight from A toward B and C together.

A major geopolitical shock — war, pandemic, financial collapse independent of AI — tends to accelerate exactly the AI applications that raise risk (military, surveillance, logistics) while starving the ones that lower it (long-horizon safety research, international cooperation). Its tilt is toward C.

And credible evidence of machine consciousness is the genuinely two-sided wildcard. It could catalyze a moral seriousness that strengthens governance, or it could shatter the entire alignment framework by making "align AI to human interests" an ethically contested goal rather than an obvious one. Its effect is large and its sign is unknown — which is itself a reason not to treat the current probabilities as robust to it.

The pattern is worth stating plainly, because it is easy to miss in a list: with the notable exception of consciousness evidence, the wildcards are not symmetric bets. They load the downside. A world that experiences none of them is a world where the 35/45/20 split roughly holds. A world that experiences a timeline-compressing or coordination-destroying shock is one where catastrophe's share climbs and the managed transition's share falls. That asymmetry is a reason for more urgency, not less.

What the Long View Actually Adds

An earlier chapter of this book laid out what to do — the policy chapter with its case for regulatory capacity, alignment funding, redistribution, and democratic engagement. It would be easy to end here by restating those recommendations in a grander register, addressed solemnly to policymakers and researchers and industry and citizens in turn. It would also be close to worthless, because you have read it already, and the long-term scenario analysis would have earned its place by adding nothing.

So let us be exact about what only this chapter can establish, because it is not the content of the actions. It is their temporal logic.

The earlier chapter could tell you that redistribution matters. It could not tell you that the coalition capable of enacting it dissolves as displacement proceeds — that the window is not open indefinitely, and that acting after the crisis proves the need means acting after the constituency has evaporated. The earlier chapter could tell you that international safety agreements matter. It could not tell you that the treaty window shuts precisely as the capability becomes valuable enough to regulate. The earlier chapter could tell you that alignment research matters. It could not tell you that the first widely deployed architecture freezes into a substrate that later work must build around rather than replace.

That is the distinctive contribution of taking the long view, and it is not a fifth item to add to a to-do list. It is a claim about sequence and deadline. The same policies, enacted in the wrong order or a decade too late, do not merely underperform — several of them become impossible, foreclosed by the very developments they were meant to govern. Redistribution before displacement peaks is a live option; redistribution after it is a fight waged by a defeated army. A safety treaty before decisive advantage is negotiable; the same treaty after is unilateral disarmament. Build alignment into the first dominant system and it propagates; try to retrofit it into the substrate afterward and you are patching an operating system the whole world already depends on.

The long-term analysis, in other words, does not tell you to do different things than the policy chapter did. It tells you that the value of doing them is collapsing on a schedule, that the schedule is set by the ratchets of the declining arc, and that the actions cluster and covary so that the coordination you win or lose early determines how many of the others stay possible at all. The future arrives regardless of preparation. What this chapter adds is the reason preparation cannot wait for proof: by the time the evidence is unambiguous, the door it argued for will already have closed.

Key Takeaways

1. The probabilities are orientations, not measurements. The 35% / 45% / 20% split is the author's structured judgment, disciplined by the expert-forecasting record but produced by neither a model nor a poll. Read it as a rank-ordering with rough magnitudes. Stating this openly is more defensible than dressing judgment as calculation. The 20% catastrophe figure sits above most survey medians for extinction specifically because it bundles a family of catastrophes — extinction, entrenched authoritarianism, cascading collapse — and because expert estimates themselves span nearly an order of magnitude.

2. Scenario B is not "nothing happens." It is the scenario of feedback loops that never compound — progress that will not stick and failure that will not tip. The test for which scenario you are in is not how good things look but whether the good things ratchet: durable gains mean A, cascading losses mean C, oscillation means B.

3. The four choice points are not independent. They covary around international coordination, which drives all three of the others. The real dimensionality is closer to two — can the world cooperate, and can the politics of distribution be won — which makes outcomes cluster toward the extremes and makes B less stable than its weight implies.

4. Agency declines on a mechanism, not a mood. Five ratchets — infrastructure dependence, standard lock-in, compute concentration, the self-erasing redistribution coalition, and the closing treaty window — each turn one way, tightening through the 2070s until the trajectory is largely fixed by 2090.

5. The wildcards load the downside. Most of them — quantum compression, climate crowding-out, geopolitical shock — shift weight from the managed transition toward catastrophe rather than affecting the scenarios evenly; brain-computer interfaces push toward the muddle; only consciousness evidence is genuinely two-sided. The headline probabilities should be read as conditional on a quiet decade.

6. The long view's contribution is timing, not tasks. It does not prescribe different actions than the policy chapter; it establishes that several of those actions expire — redistribution before displacement, treaties before decisive advantage, alignment before the substrate freezes. The reason to act now is not that the future is certain but that the option to act is not permanent.

Sources

• Thousands of AI Authors on the Future of AI (Grace et al., 2024)
• The Precipice: Existential Risk and the Future of Humanity (Ord, 2020)
• Existential Risk Persuasion Tournament — Forecasting Research Institute (Karger et al., 2023)
• 2023 Expert Survey on Progress in AI — AI Impacts

Last updated: 2026-09-25