Trust and Authenticity

In January 2024, a finance clerk at the Hong Kong office of the British engineering firm Arup joined a video call. The company's chief financial officer was on screen, dialing in from the UK, flanked by several colleagues the clerk recognized. The faces moved the way faces move. The voices carried the right accents and cadences. Over the course of the meeting, the CFO walked the clerk through a series of urgent, confidential transfers. The clerk did as instructed, moving roughly 200 million Hong Kong dollars — about 25 million US dollars — across fifteen separate transactions.

Every other person on that call was a fabrication. The CFO, the colleagues, the meeting itself: assembled from publicly available footage into a live video conference convincing enough that a trained professional never thought to reach for doubt. The clerk only understood what had happened after checking in with head office. By then the money was somewhere else.

For the whole of human history, the senses were the court of last resort. If you saw it happen, it happened. If you heard your mother's voice on the line, it was your mother. That ancient contract between perception and reality is the thing artificial intelligence has quietly voided. This chapter is about how badly, how fast, and whether anything can be built to stand where that trust used to.

The Shape of the Wave

Begin with the trajectory, because the numbers point the same direction even when the individual figures wobble. DeepMedia, a firm that tracks synthetic media, estimated that voice and video deepfakes shared online climbed from roughly 500,000 in 2023 to a projected 8 million in 2025 — sixteenfold in two years. Identity-verification companies report a parallel curve: deepfake fraud attempts multiplying by roughly an order of magnitude annually since 2022.

Hovering over all of it is a single, endlessly repeated forecast. In 2022, Europol's Innovation Lab warned that "as much as 90 percent of online content may be synthetically generated by 2026." The number gets quoted constantly, almost always stripped of its caveats. It is worth being precise about what it is: a loose projection, not a measurement — and "synthetically generated" folds in autocomplete, translation, and the ordinary machine assistance now baked into every keyboard and camera, not just deceptive deepfakes. The honest reading is not that 90 percent of what you see is a lie. It is that the default has flipped. The safe assumption used to be that a piece of media was authentic unless you had reason to doubt it. Increasingly, the safe assumption runs the other way.

That inversion — from presumed-real to presumed-synthetic — is the actual event. The exact percentage matters far less than the direction, and the direction is not in dispute.

What the Machines Can Do Now

The capability curve is the engine under the numbers. Voice fell first. Commercial tools now build a convincing replica of a specific person's voice from somewhere between three seconds and a minute of audio — enough to capture not just pitch and accent but the idiosyncratic texture that makes a voice recognizably one person's: the particular rasp, the way a sentence trails off, the catch in the throat under stress. A birthday clip on Facebook, a voicemail greeting, a few seconds of a podcast — any of these is now sufficient raw material.

Video followed, and the Arup case shows how far. No longer just a doctored recording after the fact, but synthesis fast enough to sustain a live conversation. Real-time face-swapping now rides on top of a video call. The remaining tells — a flicker at the hairline, an odd blink rate, lips a half-beat out of sync — shrink with each model generation, and they vanish entirely in the compressed, low-resolution conditions of an ordinary phone call or social feed, which is precisely where most people meet most content.

The threshold that matters is not that experts can never catch a fake. Given clean footage, time, and forensic tools, they often can. The threshold is that for the everyday sensory diet of a normal person — a voice on the phone, a clip in a feed, a face on a call — synthetic media has become routinely indistinguishable from the real thing. Seeing is no longer believing. Neither, now, is hearing.

What Has Actually Been Deployed

Here the book's discipline about registers earns its keep, because the gap between what is technically possible and what has been demonstrated in the wild is wide, and the honest picture is more complicated than the frightening one.

The clearest documented cases are real and unsettling. Two days before Slovakia's 2023 parliamentary election, fabricated audio of a leading candidate discussing how to rig the vote spread across social media during the pre-election silence period, when he could not easily respond. In January 2024, thousands of New Hampshire voters received a robocall in a cloned voice of US President Joe Biden telling them not to bother voting in the primary; the operative behind it drew a proposed multimillion-dollar FCC fine, and the telecom carrier that transmitted the calls settled. Coordinated networks — the ones platform researchers track under names like Spamouflage and Doppelganger — have been caught using AI to manufacture fake personas, complete with synthetic profile photos, invented posting histories, and machine-written commentary, to push state-aligned narratives at scale.

The other half of the story gets less airtime. When OpenAI, Meta, and independent researchers have dissected these operations, the striking finding has often been how little traction most of them got. OpenAI's 2024 review of influence operations run through its models found campaign after campaign generating fluent content and almost no real engagement — fake accounts talking mostly to other fake accounts. The most consequential political deepfakes to date have tended to be low-tech or narrowly targeted, and the feared "October surprise" — a single fabricated video swinging a national election — has not, as of this writing, been documented as decisive anywhere. The capability is fully operational. The demonstrated strategic impact has so far run below the technology's ceiling. Both things are true at once, and a serious account has to hold them together rather than collapse into either complacency or panic.

Why the Economics Favor the Fake

Beneath the incidents sits a structural asymmetry no amount of goodwill can wish away: authenticity does not scale, and synthesis does.

Genuine content is expensive in the one currency that cannot be manufactured — human time. Reporting an article, filming a documentary, sitting for an interview: each costs hours, expertise, and effort per unit produced. Generating synthetic content costs seconds and cents, and the marginal cost of the ten-thousandth item is essentially the cost of the first. Put the two curves side by side and the outcome is not a close call.

That last row is where the asymmetry turns dangerous. Platforms do not reward truth; they reward attention, and their ranking systems are built to surface whatever keeps people scrolling. Outrage travels faster than nuance. A clean, simple, emotionally charged narrative outruns a complicated true one. Synthetic media is not merely cheaper to make — it is more tunable, shapeable with precision toward whatever provokes the strongest reaction, unburdened by any obligation to have actually happened. Content farms already spin up thousands of AI-written articles a day, search-optimized and ad-supported. The incentive gradient runs downhill, toward the fake, and the platforms' own machinery gives it a shove.

Why the Old Defenses Buckle

For two decades, the answer to online falsehood rested on three pillars: fact-checkers to debunk it, content moderation to remove it, and media literacy to inoculate the public against it. All three were designed for a world of scarce, expensive lies, and all three are failing against abundant, cheap ones — each for a specific, structural reason.

Fact-checking is fundamentally slower than fabrication. It takes a professional hours or days to verify and debunk a single claim; it takes a model seconds to generate a hundred fresh ones. This is not a resourcing gap that more funding closes. It is an arithmetic mismatch, and the arithmetic only worsens as generation gets cheaper.

Content moderation depends on detection, and detection is losing ground to generation. You cannot remove at scale what you cannot reliably identify at scale.

Media literacy carries the deepest problem of the three. The entire pedagogy assumes a stable background of authentic media against which a suspicious item can be measured — telltale signs, consider-the-source heuristics, reverse-image checks. That model quietly presumes most of what the student encounters is real. Flip the default and the lesson curdles. Teaching people to distrust what they see does not reliably produce careful discernment at scale; it produces blanket cynicism, which is its own kind of damage. The tool built to defend truth ends up corroding the belief that truth is knowable at all.

The Detection Arms Race, and Who Wins It

Can technology police its own output? A large research effort is betting yes — building classifiers that spot the statistical fingerprints of generated media, forensic tools that catch impossible physics in lighting and blood flow, provenance systems that track a file's history.

The trouble is measurable. Detectors that score above 90 percent accuracy in the lab routinely slide toward coin-flip territory on real-world content — compressed, cropped, re-uploaded, and produced by a generation method the detector never trained on. This "generalization gap" is the crux: a detector learns the signatures of yesterday's models, and today's models no longer leave them. Worse, detection and generation are two faces of the same process. The standard way to train a better generator is to pit it against a detector until it learns to slip past. Every advance in spotting fakes becomes, almost immediately, training material for making better ones.

That points to an uncomfortable structural conclusion. The arms race between generation and detection is probably not winnable in the sense people want — a stable end state where fakes are reliably caught. The incentives are lopsided. Enormous commercial value flows into making generation better, cheaper, and more realistic; comparatively little flows into detection, which produces no product anyone lines up to buy. Money, talent, and compute pool on the generation side of the ledger. As long as that holds, detection lags by construction, not by accident. Betting the defense of shared reality on winning this race is betting against the economics.

The Half-Built Infrastructure of Proof

If you cannot reliably detect fakes after the fact, the alternative is to certify the real at the moment of creation — to sign authentic content the way a mint stamps a coin. Two efforts carry most of this hope.

The C2PA standard — the Coalition for Content Provenance and Authenticity, surfaced to users as "Content Credentials" — embeds tamper-evident metadata recording where a file came from, when, and what was done to it. Its backing is genuinely heavyweight: Adobe, Microsoft, Google, Sony, Nikon, Leica, and Canon among others have signed on, building it into cameras and editing software. And the European Union's AI Act supplies the legal stick: its Article 50 transparency rules, whose obligations to label AI-generated and manipulated content apply from August 2026, make disclosure a requirement rather than a courtesy for providers operating in the European market.

These are real steps. But they hit obstacles that are structural, not teething. Provenance metadata is a badge of authenticity, not a lock — screenshot a credentialed image, or run it through a tool that doesn't preserve the data, and the credential is simply gone, leaving an unmarked file that looks like any other. The system only works inside the walled garden of tools and platforms that choose to honor it; a bad actor operating outside that garden meets no friction at all. And even where the plumbing exists, it helps only if ordinary people check it — which today means knowing an inspectable credential exists and going looking for it. Verification is opt-in, buried, and invisible to the very users who most need it.

The deeper asymmetry is this: for authentication to function, it must be universal, automatic, and legible to a distracted person glancing at a phone. The absence of a credential has to reliably mean suspect this. We are nowhere close. Authentication marks the honest; it does nothing to the file that was never marked, and most of the internet is unmarked files.

Reality Fatigue and the Liar's Dividend

What does living inside all this do to a person? Researchers and clinicians have begun describing something they call "reality fatigue" — the low-grade exhaustion of chronic vigilance, the mental tax of asking, again and again, is this real? Is this photo staged, is this voice cloned, is this stranger who they say they are. Here the book has to be candid about its own register: reality fatigue is an emerging description, not a settled clinical diagnosis, and whether it names a passing adjustment or a durable change in how humans process information is genuinely unknown. We are early. The honest answer to "what will this do to us long-term" is that no one yet has the longitudinal evidence to say.

What can be observed are the behavioral responses, and they run in two directions, both corrosive. Some people retreat into epistemic bunkers, trusting only sources that already flatter their beliefs — if everything mediated might be fake, the familiar and the emotionally resonant become a stand-in for the true, which pours fuel on polarization. Others drift toward a weary nihilism: if nothing can be verified, why try, and they withdraw from the whole civic business of caring what is true.

That second response has a name and a beneficiary. The legal scholars Bobby Chesney and Danielle Citron called it the "liar's dividend": once the public knows that any recording could be fake, the guilty gain a fresh escape hatch — dismiss the authentic as fabricated. The deepfake's most corrosive effect may not be the fake that fools you, but the real footage a wrongdoer can now wave away as a deepfake. Genuine atrocity footage, a real damning recording, an authentic confession — each can be laundered into "probably AI." When any evidence can be denied, accountability itself springs a leak. Democratic deliberation rests on a floor of shared, presumed-real facts to argue about. Pull up that floor and citizens are no longer disagreeing about what to do — they are disagreeing about what happened, a fight with no available resolution.

Reaching for Anchors

When mediated trust breaks down, people grope for something solid, and it is worth being clear-eyed about what each handhold offers and where it gives way.

Physical presence is the strongest anchor and the least scalable. In a shared room, cloned voices and fabricated video are simply not available to a deceiver — which is why high-stakes dealings, sensitive conversations, and relationships that genuinely matter are quietly drifting back toward in-person contact, not because it is efficient but because it cannot yet be faked. Its limit is obvious: presence does not scale to the volume of modern life, and most of what we need to trust reaches us through a screen.

Long-term relationships offer a subtler shield. A model working from a scraped minute of audio can nail a voice for thirty seconds; it cannot reproduce the accumulated private context of years — the in-jokes, the behavioral tics, the things only the two of you know. This is why the oldest countermeasure to the family-emergency voice scam is still the best: a family safe-word, a question the caller could not possibly answer. But this anchor works only where a shared history already exists. It offers nothing for trusting a stranger, a new institution, an unfamiliar source — which is most of public life.

Institutional verification — biometric identity checks, government documentation, in-person enrollment — can supply real assurance for banking, legal agreements, and credentialed settings. But it is invasive by design, pooling exactly the biometric and identity data that becomes a catastrophic target the moment it leaks, and it fits the informal torrent of everyday communication about as well as a passport check fits a hallway chat.

None of these fully substitutes for the thing that is eroding: a generalized, ambient social trust that let us take most of what we encountered at face value. Together they amount to a society improvising toward a new equilibrium whose foundations no one has yet agreed on.

We Have Been Here Before — At a Cost

There is a case for calm, and it deserves a fair hearing. Humans have absorbed reality-warping media before and come out the other side. Photography arrived, and soon after, so did photographic manipulation — Stalin airbrushing the disgraced out of official portraits is only the most famous instance. The public eventually internalized that a photo could lie. Photoshop turned image manipulation into a desktop commodity in the 1990s, and within a generation "that's photoshopped" hardened into reflexive common sense. Email brought phishing; most people learned, painfully, not to trust a message on the strength of its sender line. Each time, society recalibrated. New heuristics formed. The sky did not fall.

This is the strongest reason for hope, and it should not be brushed aside. But the historical pattern carries a warning the optimistic version usually omits: every one of those adaptations cost something, and the bill was never paid evenly. The adjustment to each new deception took years, sometimes a generation. During the lag — before the new heuristic hardened — real people were defrauded, defamed, and deceived, and those harms landed first and hardest on those with the least defense: the elderly, the isolated, the less technically fluent, the poor. The photograph's loss of innocence cost us something permanent too — a default trust we do not get back. Adaptation is real. It is also slow, incomplete, and expensive, and the deepfake transition is arriving faster and cutting deeper than any of its predecessors.

What Should Be Done, and Who Should Pay

The policy questions follow directly, and the book takes positions while showing the reasoning.

On authentication, the responsibility to prove origin belongs upstream, with the tools and platforms of creation and distribution, not with the individual squinting at a phone. Loading the burden onto the ordinary viewer — "just check the credentials" — is a design that guarantees failure, because the people most vulnerable to deception are the least equipped to perform verification. Mandatory disclosure, of the kind the EU AI Act is beginning to require, is the right instrument, applied at the point where content enters wide distribution.

On platform liability, the calibration is genuinely hard, and false confidence here would be dishonest. Hold platforms strictly liable for every synthetic harm and you incentivize sweeping, automated over-removal that buries satire, art, and legitimate speech alongside fraud — a censorship problem wearing a safety costume. Hold them to nothing and the incentive to police their own distribution evaporates. The defensible middle ties liability to conduct: what a platform knew, how fast it acted on credible notice, and whether it deployed available provenance and detection at all. Liability for negligence in distribution, not strict liability for the mere existence of a fake.

And the governing principle beneath all of it — consistent with this book's central thesis that distribution is the whole question — is that the costs of this transition are already falling, and will keep falling, disproportionately on people with the fewest resources to protect themselves. The affluent will buy verification services, biometric vaults, and the time to be careful. The retiree, the immigrant, the low-wage worker will absorb the scams and the confusion. A policy response that only builds better tools for those who can afford them does not solve the trust crisis; it privatizes safety and socializes the damage. Minimizing the transition's cost, and distributing that cost fairly, is not a footnote to the technical problem. It is the problem.

How We Will Know

Because so much here is genuinely unsettled, it helps to name in advance what success and failure would look like — so we are not left arguing after the fact about what happened, which is exactly the trap the technology sets.

Successful adaptation would show up as functional new heuristics that most people, not just the tech-savvy, actually use: provenance checking that feels as automatic as a padlock icon once did, verification defaults quietly built into the tools everyone already holds, fraud losses plateauing rather than climbing, and a public grown appropriately skeptical without tipping into believing nothing. Failed adaptation would look like the opposite, and may already be faintly visible: skepticism curdling into blanket cynicism, the liar's dividend maturing into a routine and successful defense for the powerful, elections and courts and public-health authorities repeatedly unable to establish agreed facts, and a slow civic disengagement as people conclude that finding out what is true costs more effort than it is worth.

Which trajectory we are on is, right now, an open empirical question. That uncertainty is not a rhetorical dodge; it is the accurate state of knowledge. The perceptual gap between real and fake will keep closing — that much is near-certain. Whether our institutions and habits close the trust gap fast enough, and at a cost we distribute justly, is not determined by the technology. It is a set of choices still in front of us. The most important thing to understand about the age of synthetic media is that its worst outcome is not yet written.

Key Takeaways

1. The default has flipped. Synthetic media has gone from rare and expensive to abundant and nearly free. Deepfakes shared online rose roughly sixteenfold between 2023 and 2025 (DeepMedia), and Europol's much-cited projection that up to 90 percent of online content could be synthetic by 2026 — a loose forecast, not a measurement — captures the real shift: the safe assumption has inverted from presumed-real to presumed-synthetic.

2. The technology is fully capable; its demonstrated impact is more mixed. Voice cloning needs seconds of audio; real-time video deepfakes fooled trained staff into a $25 million transfer at Arup. But dissections of political influence operations (by OpenAI, Meta, and others) have repeatedly found fluent AI content generating little real engagement. Both facts are true; honest analysis holds them together.

3. The economics permanently favor the fake. Authenticity costs human hours and scales linearly; synthesis costs cents and scales without limit — and is more tunable for the outrage that platform engagement systems reward. The incentive gradient runs downhill toward fabrication.

4. The old defenses buckle for structural reasons. Fact-checking is arithmetically slower than generation; moderation depends on failing detection; media-literacy pedagogy assumes an authentic baseline that no longer holds and can curdle into blanket cynicism.

5. The detection arms race is probably not winnable. Lab-accurate detectors collapse on real-world content, generation trains directly against detection, and far more money flows into making fakes than into catching them. Authentication (C2PA, the EU AI Act's Article 50, applying from August 2026) certifies the honest but does nothing to the unmarked files that make up most of the internet.

6. The deepest harm is the liar's dividend. Beyond any single convincing fake lies the ability to dismiss genuine evidence as fake. "Reality fatigue" — chronic authenticity-vigilance — is real but not yet a settled diagnosis; its long-term psychological effects are genuinely unknown.

7. Anchors exist but none scale. Physical presence, long-term relationships (the family safe-word), and institutional verification each restore trust in narrow domains and each fail outside them; none replaces the generalized ambient trust that is eroding.

8. The costs are the policy question. History shows humans adapt to deceptive media — slowly, incompletely, and at a price paid first by the vulnerable. Responsibility to verify belongs upstream with platforms and tools, liability should track negligent conduct rather than the mere existence of a fake, and the central task is ensuring the transition's costs do not fall hardest on those least able to bear them. The worst outcome is not yet written.

Sources

• C2PA — Coalition for Content Provenance and Authenticity
• EU AI Act — Transparency Obligations (Article 50)
• Voice Cloning Scams on the Rise | FBI

Last updated: 2026-07-27