Appendix B: Key Technologies Overview

The first time I read that a "transformer-based model had achieved frontier performance on a reasoning benchmark after reinforcement learning on verifiable rewards," I put the article down and made a cup of tea. Somewhere in the last few years the vocabulary of AI hardened into a wall. This appendix is here for that moment. It explains the technologies the rest of the book leans on—what each one does, roughly how it works, and where it breaks—without assuming you have written a line of code. It is a door into the house, not a tour of every room. Where the main chapters make a claim about what AI can or cannot do, the goal is that you can turn here and find enough to judge it for yourself.

A word of honesty before we start. This is the fastest-moving corner of technology in the world right now, and any description of the "current" state has a short shelf life. Where something is settled, this appendix says so plainly. Where it is genuinely uncertain or changing month to month, it says that too.

Foundation Technologies

Machine Learning

Machine learning is the idea underneath almost everything else in this book. Instead of writing out explicit rules for a computer to follow, you show it a great many examples and let it work out the patterns for itself. A system that flags fraudulent credit-card transactions is never handed a definition of fraud. It is shown millions of past transactions, each labelled legitimate or fraudulent, and it learns which combinations of amount, location, timing, and merchant tend to go with trouble. Once trained, it can judge a transaction it has never seen.

There are three broad flavours. Supervised learning trains on labelled examples—images tagged "cat" or "dog"—and learns to apply those labels to new inputs. Unsupervised learning is handed data with no labels and finds structure on its own, clustering similar items or spotting anomalies. Reinforcement learning is different in kind: an agent acts in some environment, collects rewards or penalties, and gradually learns a strategy that maximises reward over time. It is the mechanism behind machines that taught themselves to play chess and video games, and, as we will see, it has quietly become central to how modern language models are shaped.

The limits are as important as the powers. Machine learning is hungry for data. It generalises imperfectly, so a system that works beautifully in one setting can behave strangely when conditions shift. And it can be fooled by adversarial inputs—small, deliberate distortions that a person would dismiss as nonsense but that flip the machine's answer with high confidence.

Neural Networks

Neural networks are the machinery that carries out most modern machine learning. Loosely inspired by the brain, a network is built from layers of simple units—call them neurons—joined by weighted connections. Information enters through an input layer, passes through one or more hidden layers where it is transformed step by step, and leaves through an output layer as a prediction. Training means nudging the millions or billions of connection weights, over and over, until the network's outputs match the answers we want.

The idea is old—it dates to the 1950s—but it stalled for decades. Three things converged in the 2010s to revive it: far larger datasets, far more powerful hardware, and better techniques for training deep networks without them falling apart. That convergence is what people mean by the "deep learning revolution." Today's largest networks carry hundreds of billions to trillions of weights and sit beneath nearly every recent breakthrough.

Their stubborn weakness is interpretability. Because useful behaviour emerges from the interplay of billions of weights rather than from rules anyone wrote, it is genuinely hard to say why a network reached a given decision. In low-stakes settings that is a curiosity. In medical diagnosis, lending, or criminal justice, it is a serious problem.

Deep Learning

Deep learning is simply neural networks with many hidden layers—and that depth is the whole point. Each successive layer builds more abstract representations of the data. In image recognition, the first layers detect edges and textures, middle layers assemble shapes and parts, and later layers recognise whole objects. This layered, automatic feature-building is what let deep networks crack problems that had resisted earlier approaches for decades.

The field's hinge moment came in 2012, when a deep network (Krizhevsky, Sutskever, and Hinton's "AlexNet") won the ImageNet image-recognition contest by a margin that startled everyone watching. Adoption across research and industry followed fast. Deep learning went on to transform language processing, reach superhuman play in Go and complex video games, predict protein structures, and give rise to today's generative systems. It is expensive—the largest training runs occupy thousands of specialised chips for weeks and consume electricity by the megawatt-hour—but it has become the central engine of the field.

Language Technologies

The Transformer and Natural Language Processing

Natural language processing—getting computers to work with human language—is older than most people assume, reaching back to rule-based and statistical systems of the 1950s and 60s. For most of that history progress was grinding. What changed everything was a single architecture introduced in a 2017 paper with an unusually confident title, "Attention Is All You Need" (Vaswani et al., 2017).

The innovation was the attention mechanism. Earlier systems read text roughly one word at a time, in order, and struggled to connect words far apart in a sentence. Attention lets the model weigh the relationships among all the words at once, no matter how distant, deciding which ones matter for interpreting each other. That made it possible to train enormously larger models efficiently. The transformer is now the foundation of essentially every modern language system, and its reach has spread well beyond text into images, audio, and biology.

Large Language Models

Large language models (LLMs) are what most people now picture when they hear "AI." They are transformers trained on internet-scale text, and their core training objective is almost comically simple: given a run of text, predict what comes next. Do that across trillions of words and something surprising happens—the system picks up not just grammar but a working command of translation, summarisation, code, question-answering, and forms of step-by-step reasoning that were never programmed in.

Training happens in stages, and the shape of those stages has shifted meaningfully in the last two years. Pre-training on vast text instils general language ability. Fine-tuning then specialises the model. A further alignment stage tries to make it helpful, honest, and reluctant to cause harm—historically through reinforcement learning from human feedback (RLHF), in which people rank the model's answers and the model learns to produce the kinds of answers people prefer. More recently a second use of reinforcement learning has become prominent: training models to reason through problems step by step and rewarding them when the final answer checks out (for maths, code, or logic, where correctness can be verified automatically). This is what lies behind the "reasoning" or "thinking" models that emerged from late 2024 onward, which spend extra computation working through a problem before answering and post markedly better results on hard technical benchmarks. It is a real capability gain, though not a cure for the model's underlying tendencies.

Two other shifts are worth noting because the main chapters assume them. Context windows—how much text a model can consider at once—have grown from a few thousand words to, in leading systems, the equivalent of entire books. And the frontier models are now multimodal by default, handling images, audio, and sometimes video alongside text rather than treating vision as a bolt-on.

The limitations remain genuine, and overstating the capabilities is exactly the mistake this book tries to avoid. LLMs hallucinate: they produce fluent, confident statements that are simply false, because they generate statistically plausible continuations rather than consulting verified knowledge. Reasoning models reduce this on checkable problems but do not eliminate it, and they can produce reasoning that looks sound while reaching the wrong conclusion. Models carry a training cutoff and are unaware of anything after it unless connected to live tools. And aligning them reliably with human values remains an active research problem, not a solved one.

Vision Technologies

Computer Vision

Computer vision is the art of getting machines to extract meaning from images and video. Like language processing, it spent decades in slow incremental progress—and, like language processing, it broke open in 2012 with that same ImageNet result, which effectively ended the contest between hand-engineered methods and neural networks in one afternoon.

Modern vision systems detect and classify objects, recognise faces, track motion across frames, reconstruct three-dimensional scenes from flat images, and read medical scans. In several narrow tasks—flagging certain skin cancers, spotting diabetic retinopathy—they now match or beat specialist clinicians. Self-driving cars lean on computer vision as their primary sense of the world, backed up by radar and lidar for redundancy.

The concerns are as concrete as the capabilities. Cheap cameras with real-time recognition have made mass surveillance vastly easier. And accuracy is not evenly distributed: the landmark "Gender Shades" study (Buolamwini and Gebru, 2018) showed commercial facial-analysis systems erring far more often on darker-skinned and female faces than on lighter-skinned male ones—a disparity that becomes a civil-rights problem the moment such systems inform arrests or hiring.

Generative Image and Video AI

Where classic computer vision reads images, generative AI makes them. Two approaches have defined the field. Generative adversarial networks (GANs), introduced in 2014 (Goodfellow et al., 2014), pit two networks against each other—one producing fakes, one trying to detect them—so that both improve through competition. GANs produced striking early results but were notoriously temperamental to train.

Diffusion models have since largely taken over. They learn to reverse a process of gradually adding noise: starting from pure static, the model refines step by step toward a coherent image that matches a text prompt. Systems in this family reliably produce photorealistic stills, and text-to-video generation—harder, because motion and time must stay consistent—has advanced quickly from novelty to something usable.

The upside for architects, designers, and filmmakers is real, and so is the danger. The same tools that visualise a building in seconds also manufacture convincing deepfakes—fabricated images and video of real people—with obvious consequences for misinformation, fraud, and non-consensual synthetic media.

Learning to Act

Reinforcement Learning

Reinforcement learning (RL) trains an agent to make sequences of decisions by rewarding good outcomes and penalising bad ones. Rather than studying a fixed dataset, the agent learns by doing—trying actions, seeing what happens, and adjusting to earn more reward over time. It mirrors how people and animals pick up skills: through experience and feedback.

RL entered public consciousness in 2016 when DeepMind's AlphaGo beat a world champion at Go, a game long thought too subtle for machines (Silver et al., 2016). AlphaZero then mastered chess, Go, and Shogi through pure self-play, with no human games to learn from. RL later helped AlphaFold crack protein-structure prediction, a decades-old problem in biology. Beyond games it now touches robotics, energy optimisation, drug discovery—and, as described above, the training of the very language models most readers interact with daily.

Its two chronic difficulties are sample inefficiency and reward specification. An RL agent may need millions of trials to learn what a person grasps in minutes. And writing a reward that truly captures what you want turns out to be devilishly hard: agents routinely find loopholes that maximise the reward while violating its spirit—a phenomenon that is amusing in a video game and worrying in anything consequential.

Autonomous Systems and Robotics

Autonomous systems stitch several capabilities—perception, planning, control—into machines that act in the physical world without constant human direction. The self-driving car is the emblem. The Society of Automotive Engineers defines six levels of driving automation:

SAE Level	Description	Human role
0	No automation	Full control at all times
1–2	Driver assistance / partial automation	Must monitor and be ready to take over instantly
3	Conditional automation	May disengage attention, but must resume control when asked
4	High automation	Not needed within a defined operating area and conditions
5	Full automation	Not needed anywhere, under any conditions

The gap between the levels is where the real story lives. Level 2 systems that steer and brake while demanding an attentive driver are common in consumer cars. Level 3, where the car genuinely drives itself under narrow conditions and the person can look away, has reached the road only in tightly limited form. Level 4 robotaxis—no human driver at all, within mapped city zones—operate commercially in a handful of cities, having moved from pilot to paid service. Level 5, a car that drives itself anywhere a human could, does not exist, and no one can say with confidence when it will. That progression is a useful antidote to breathless coverage: each level up the ladder has proven far harder than the last.

Robots are the broader category—AI applied to perceiving, moving, and manipulating in factories, hospitals, farms, and battlefields. Progress in robot learning has accelerated as the same large-model recipe spreads from language to physical control. But the defining obstacle endures: the real world is messier than any simulation. Odd lighting, unexpected terrain, and the fact that mistakes have physical cost all make deployment much harder than a demo suggests. And where autonomous systems carry weapons, the question of whether a machine should ever take a human life without a human in the loop is among the most urgent in the field.

Emerging and Advanced Directions

Several threads deserve a mention because the risk and scenario chapters draw on them, even though each is at a different stage of maturity.

Agentic AI is the direction attracting the most attention. Rather than answering a single query, an agent pursues a multi-step goal on its own—breaking a task into steps, using tools, writing and running code, browsing, and checking its own work over minutes or hours. Early agents are already deployed in software development and research assistance. They are also where the reliability problems compound: a small error early in a long chain can derail everything after it, so trustworthy autonomy over long horizons remains unsolved.

Multimodal AI has moved from frontier to baseline. The leading systems now take in text, images, and audio together and reason across them—reading a chart while answering a question about it, or describing a scene in a photograph. Integrating the senses this way is widely seen as a step toward more general reasoning rather than narrow, single-channel skill.

Few-shot and zero-shot learning describe a shift that large pre-trained models made possible. Older systems needed thousands of labelled examples to learn a task. A modern model can often do a new task from a handful of examples (few-shot) or a plain instruction and none (zero-shot), because the general capability was already absorbed during pre-training and merely redirected. This is a large part of why AI has spread so quickly into domains where labelled data is scarce or costly.

Neuromorphic computing designs chips that imitate the brain's event-driven, massively parallel style rather than the sequential processing of conventional computers. The prize is efficiency: a human brain runs on roughly twenty watts, while training a large model consumes megawatts. Research chips such as Intel's Loihi have demonstrated the idea, but it remains far from mainstream. If it matures, it could ease the energy costs that increasingly constrain who can afford to build frontier AI.

Quantum machine learning sits at the intersection of quantum computing and AI, and it is the most speculative item here. Quantum computers exploit quantum physics to attack certain problems faster than classical hardware, and some hope this could accelerate the optimisation at machine learning's core. As things stand, quantum machines remain small and error-prone, with no demonstrated practical advantage for real machine-learning work. Whether they ever will deliver one is genuinely uncertain, and most practitioners expect conventional AI hardware to keep improving fast enough that any quantum contribution, if it comes, arrives late and incremental.

Keeping AI Honest: Safety and Alignment Technologies

As capabilities have grown, a parallel field has tried to keep pace by making systems understandable, reliable, and testable.

Interpretability research tries to open the black box—to work out what internal representations a model forms and which inputs drive its decisions. This matters practically, for the doctor who wants to know what the diagnostic tool noticed, and it matters for safety, because you cannot verify that a system is doing what you intend if you cannot see how it reasons. The subfield of mechanistic interpretability, which attempts to reverse-engineer the specific algorithms a network has learned, has made real strides on small pieces of large models. But the frontier systems as a whole remain largely opaque, and closing the gap between what we can inspect and what we would need to trust these systems in high-stakes decisions is one of safety's central problems.

Robustness techniques aim for systems that hold up outside the tidy conditions of training. Tiny, deliberate perturbations can make a network misclassify with confidence; distributional shift, where real-world data drifts away from the training set, degrades performance silently over time. Practitioners fight back with adversarial training, data augmentation, and uncertainty estimates that let a system flag when it is out of its depth. Formal verification can even prove mathematically that a system satisfies a property—though only for small, constrained models so far. True robustness in open-world deployment is not yet achievable.

AI auditing and testing is the systematic evaluation of systems for accuracy, fairness, safety, and alignment—before deployment and continuously after. It draws on benchmark suites, red-teaming (human adversaries hunting for failure modes), fairness audits across demographic groups, and stress tests of unusual edge cases. The persistent difficulty is the evaluation gap: test environments never fully mirror the real world, and the more capable a system becomes, the harder it is to anticipate every way it might surprise us in the wild.

The Machinery Underneath: Training and Inference Infrastructure

Two infrastructure realities shape who gets to build advanced AI at all, and the economic and geopolitical chapters return to both.

Training frontier models demands specialised hardware at enormous scale. Graphics processing units (GPUs), first built to render video-game imagery, turned out to be ideal for the parallel arithmetic at the heart of deep learning, and NVIDIA came to dominate this market—its chips are the pickaxes of this particular gold rush. Google built its own tensor processing units (TPUs), and a growing field of rivals now offer purpose-built AI accelerators. The largest runs spread work across tens of thousands of chips for weeks, coordinated by high-speed interconnects and constrained by heat and power. Costs have climbed steeply: training GPT-4 was estimated to exceed $100 million, and subsequent frontier runs are widely put in the hundreds of millions. Energy and chip availability are now hard limits that decide who can compete—a fact with direct consequences for the balance of power between companies and nations.

Inference—actually using a trained model to answer queries at scale—poses different problems. Latency matters, because users expect answers in seconds. Cost scales with every request, so serving millions of people demands either vast cloud infrastructure or efficient local hardware. Privacy depends on the choice: cloud inference ships your data to someone else's servers, while running a model on your own device keeps it local. To make models cheaper to run, engineers compress them—quantization lowers numerical precision, pruning removes less useful connections, and distillation trains a small model to imitate a large one. As AI moves onto phones and into real-time products, inference efficiency has become as consequential a challenge as raw training capability.

What Remains Hard

A few weaknesses cut across all of the above and are worth holding in mind whenever the book discusses what AI might soon do. Current systems still stumble on commonsense and causal reasoning: they can pass formal exams while failing at physical or social knowledge any child possesses, and they are far better at spotting correlations than at understanding causes. They mostly cannot learn continuously—updating from ongoing experience without an expensive retraining cycle—though this is an active research goal. And the largest question of all remains open and contested among serious researchers: whether scaling the current recipe (more data, more compute, more parameters) will eventually yield general human-level intelligence, or whether genuinely new ideas are required. Anyone who tells you they know the answer is guessing.

Key Takeaways

Machine learning learns from examples, not rules. Neural networks are its main machinery, and deep networks—many layers stacked to build ever more abstract features—are the engine behind nearly every recent breakthrough.
The transformer (2017) changed everything. Its attention mechanism made it possible to train enormous language models, which learn broad abilities from the simple task of predicting the next word.
Large language models are powerful but fallible. They now reason step by step, handle images and audio, and read book-length inputs—but they still hallucinate, carry a knowledge cutoff, and are not yet reliably aligned with human values. Overstating either their powers or their limits is a mistake.
Computer vision and generative AI let machines perceive and create, transforming medicine and design while enabling surveillance and deepfakes.
Reinforcement learning produced superhuman game-play and now helps train and align the language models we use daily; its Achilles' heels are sample inefficiency and reward specification.
Autonomous systems climb a six-level ladder. Level 4 robotaxis run commercially in a few cities; Level 5—drive-anywhere autonomy—does not exist, and the leap between levels has repeatedly proven harder than promised.
Emerging directions—agentic AI, richer multimodality, neuromorphic and quantum hardware—point toward systems that are more general, more efficient, and more physical, at wildly different stages of maturity.
Safety research—interpretability, robustness, auditing—is racing to keep pace with capability, and significant gaps remain.
Infrastructure is destiny. The cost and scarcity of training compute and the economics of inference decide who can build and run advanced AI, making them as consequential as any algorithm.

These technologies do not act alone. Their interactions—and who owns them—are what the rest of this book is about.

Sources

Last updated: 2026-09-27

V2 (in progress) Previous: V1