Inside the CPU

10.0 What this chapter gives you#

1. You will be able to name every part inside a processor and say what it does.
2. You will be able to walk one instruction from memory to result, step by step, and say which wire carries what at each step.
3. You will be able to read a line of x86-64 assembly and the same line in ARM64 assembly, and say what the machine does with it.
4. You will be able to explain what an instruction set architecture is, and why it is a contract rather than a design.
5. You will be able to explain pipelining with real timing tables, and say exactly why a very deep pipeline made the Pentium 4 slower, not faster.
6. You will be able to explain out-of-order execution, register renaming and branch prediction without any magic, with real accuracy figures.
7. You will be able to explain why processors stopped getting faster in megahertz around 2005 and started getting wider and more numerous instead.
8. You will be able to walk the MESI cache coherence protocol between two cores by hand, and calculate Amdahl’s law for a real program.
9. You will be able to read a desktop processor spec sheet and a phone system-on-chip spec sheet and decode every single number on the page.
10. You will be able to say why gigahertz alone tells you almost nothing, and what number to look at instead.

10.1 What a CPU is: a box that repeats one loop forever#

PLAIN10.1.1 in simple words#

1. A CPU is a small square of silicon that does one thing over and over.
2. The one thing is: get the next order, work out what it means, do it.
3. Then it does it again. And again. Billions of times each second.
4. Nothing else. There is no cleverness hidden anywhere. It is a loop.
5. The orders are called instructions. Each is just a number in memory.
6. An instruction is tiny. “Add these two numbers.” “Copy this to there.” “If that number is zero, jump somewhere else.”
7. A program is a long list of these tiny orders, one after another.
8. Everything you have ever seen a computer do is millions of these orders running fast enough that the steps blur into one smooth thing.
9. Earlier chapters built the pieces: switches, gates, an adder, an arithmetic unit, and cells that remember a bit. This chapter wires them into a CPU.

PLAIN10.1.2 a picture in your head#

1. Picture a very fast, very obedient cook alone in a kitchen.
2. On the wall is a long shelf of recipe cards, numbered 1, 2, 3, and so on.
3. The cook has a small clip that holds one card at a time. That clip is the instruction register.
4. The cook has a counter on a string round the neck showing which card is next. That is the program counter.
5. The cook has a few small bowls on the bench, always within reach. Those are the registers.
6. There is a big cold store at the far end of the building. That is main memory. Walking there takes a long time.
7. Near the bench there is a small fridge holding whatever was fetched recently, so the cook does not have to walk. That is the cache.
8. The cook does exactly what the card says, moves the counter to the next number, and picks up the next card. Forever.

Where this comparison breaks: the cook understands the recipe. The CPU does not understand anything. The bit pattern of the instruction physically opens and closes switches, and the result falls out. There is no reading and no deciding, only voltage arriving at gates.

PLAIN10.1.3 a worked example#

1. Say memory holds the number 7 at address 64 and the number 5 at address 65.
2. We want their sum written to address 66.
3. The CPU does this in four orders, and each order is one trip round the loop.

4. Notice that steps 1, 2 and 4 move data around and do no maths at all.
5. Only step 3 does arithmetic. That is normal. Most instructions in a real program are moving things, not calculating things.
6. Notice also that the CPU never added “7 plus 5”. It added whatever happened to be in bowl 0 and bowl 1. It has no idea what those numbers mean.

PLAIN10.1.4 what is really happening inside#

1. Split the CPU into two halves: the part that carries numbers, and the part that decides what happens to them.
2. The carrying part is the datapath: registers, the arithmetic unit, and the wires between them.
3. The deciding part is the control unit. It reads the bit pattern of the instruction and switches on exactly the right set of wires.
4. Think of the control unit as a room full of light switches. An instruction pattern is a hand that flips a particular set of switches on.
5. Those control wires say things like: “let register 1 out onto bus A”, “tell the arithmetic unit to add, not subtract”, “write the answer back into register 0”, “increase the counter by one”.
6. Everything else is plumbing. Buses are shared bundles of wires that carry an address, or data, or control signals.
7. A memory controller sits at the edge of the chip and talks the exact electrical language that memory chips expect.
8. All of it moves in step with a clock, a square wave that ticks billions of times a second and says “now everyone move”.

TECHNICAL10.1.5 the engineer’s version#

1. The functional blocks of any general-purpose CPU core are as follows.
2. Control unit: decodes the instruction and drives the control signals. In a modern x86 core it is itself a small program in read-only memory, plus hardwired fast paths. See section 10.6 on microcode.
3. ALU (arithmetic logic unit): add, subtract, AND, OR, XOR, shift, compare. Built from the adders and multiplexers of Chapter 5.
4. Register file: a small multi-ported SRAM array. Multi-ported means several reads and writes can happen in the same clock cycle.
5. Program counter (PC), called RIP on x86-64 and PC on ARM64: holds the address of the next instruction.
6. Instruction register (IR): holds the instruction word currently being decoded. On out-of-order cores this is not one register but a queue.
7. Caches: L1 instruction, L1 data, L2 per core, L3 shared. Section 10.11.
8. Buses and interconnect: on-die, these are now networks, not buses. AMD uses Infinity Fabric, Arm uses CMN mesh interconnects.
9. Memory controller: generates DDR5 command sequences, refresh, training. On AMD desktop parts it lives on a separate I/O die, not on the core die.

10. The Intel 4004 shipped on 15 November 1971. Federico Faggin led the design with Ted Hoff, Stan Mazor and Masatoshi Shima of Busicom.
11. An AMD Zen 4 core complex die holds 6.57 billion transistors in 70 mm2 on TSMC N5. The Zen 5 equivalent is roughly 8.3 billion in about 70.6 mm2 on TSMC N4P. Treat the Zen 5 figure as approximate.
12. To observe the blocks on Linux: lscpu, lscpu -C for caches, cat /proc/cpuinfo. On macOS: sysctl -a | grep machdep.cpu.

WORDS10.1.6 remember these#

1. Instruction — one tiny order — an encoded operation in the ISA.
2. Datapath — the pipes numbers flow through — registers, ALU, buses.
3. Control unit — the switchboard — logic that drives control signals from the decoded opcode.
4. Program counter — a bookmark — register holding the next instruction address, RIP on x86-64, PC on ARM64.
5. Instruction register — the clip holding the current card — latch holding the instruction word being decoded.
6. Register file — the small bowls on the bench — multi-ported SRAM array of architectural or physical registers.
7. Memory controller — the translator at the door — on-die block generating DDR command and timing sequences.

10.2 The stored-program idea#

PLAIN10.2.1 in simple words#

1. Here is the single biggest idea in computing, and it is very simple.
2. Instructions are numbers. Data is numbers. Memory holds numbers.
3. So instructions and data can live in the same memory, side by side.
4. Nothing in memory has a label saying “this one is an order”. It is only an order because the program counter pointed at it.
5. That is why one machine can run any program. You do not rewire it. You just put different numbers in memory.
6. Before this idea, changing a computer’s job meant physically replugging it.
7. It also means a program can write numbers that are then run as orders. That is enormously useful and enormously dangerous, and both are covered below.

PLAIN10.2.2 a picture in your head#

1. Picture a huge wall of numbered pigeonholes, all identical.
2. Some hold shopping lists. Some hold instructions for the shop assistant.
3. Nothing on the outside of a pigeonhole tells you which is which.
4. The assistant simply starts at pigeonhole 100 and treats whatever is there as an instruction, because that is where they were told to start.
5. If someone slips a shopping list into pigeonhole 100, the assistant will try to obey the shopping list as if it were an order.
6. That mistake is the root of a whole class of security holes.

Where this comparison breaks: real hardware now does add labels. Modern memory pages carry permission bits saying “may be executed” or “may be written”, and in most cases not both. The pigeonholes are still identical, but the shelf now has rules about which ones the assistant may read orders from.

PLAIN10.2.3 a worked example#

1. Take the four bytes 48 83 C0 01 sitting in memory.
2. Read as data, that is just four numbers: 72, 131, 192, 1.
3. Read as an x86-64 instruction, it is add rax, 1, meaning “add one to register RAX”.
4. Both readings are correct. The bytes do not choose. The CPU chooses, by where the program counter points.
5. Now take the two bytes EB FE. As an x86-64 instruction that is jmp -2, a jump back to itself: an infinite loop in two bytes.
6. If a program accidentally jumps into a block of text, the CPU will happily try to execute the letters. H is 0x48, which starts many valid x86 instructions. That is why crashes from bad jumps look so random.

PLAIN10.2.4 what is really happening inside#

1. There are two ways to arrange memory, and they have names.
2. Von Neumann: one memory, holding both instructions and data, reached through one set of wires.
3. Harvard: two separate memories with separate wires, one for instructions and one for data.
4. Von Neumann is flexible. You can load a program as if it were data, then run it. That is exactly what an operating system does when it starts an app.
5. Von Neumann has a bottleneck. One set of wires must carry both the next instruction and the data it needs, so they take turns.
6. Harvard has no such traffic jam, because the two roads are separate.
7. Real CPUs use a mixture called modified Harvard. Deep down there is one memory, von Neumann style. But right next to the core there are two separate small caches: one for instructions, one for data.
8. So the core sees Harvard, with two roads, and the system sees von Neumann, with one memory. You get the speed of one and the flexibility of the other.

TECHNICAL10.2.5 the engineer’s version#

1. John von Neumann’s “First Draft of a Report on the EDVAC” circulated on 30 June 1945 and described a single addressable store for code and data. The design work was joint with J. Presper Eckert and John Mauchly, whose names were left off the draft, a dispute that is still argued about.
2. The Harvard Mark I, completed in 1944 under Howard Aiken at Harvard, read instructions from punched paper tape and data from separate counters. That physical separation is where the name comes from.
3. Modified Harvard in practice: separate L1 instruction and L1 data caches fed from a unified L2. On AMD Zen 5 that is 32 KB L1i and 48 KB L1d per core, backed by a unified 1 MB L2.
4. This split means self-modifying code needs explicit help. Writing bytes goes into L1d. The fetcher reads L1i. The two can disagree.
   1. On x86-64 the hardware keeps them coherent for you, but a serializing instruction is still required before the new bytes are guaranteed run.
   2. On ARM64 you must do it by hand: DC CVAU to clean the data cache to the point of unification, IC IVAU to invalidate instruction cache, then DSB and ISB barriers. Skipping this is a classic JIT bug.
5. The danger side. A stack buffer overflow writes past the end of an array and overwrites the saved return address, so the CPU returns into attacker bytes. The Morris worm of 2 November 1988 used exactly this against the Unix fingerd service.
6. The defence is the no-execute page permission bit. AMD shipped NX with Athlon 64 in 2003, Intel called it XD, and Microsoft enabled it as Data Execution Prevention in Windows XP Service Pack 2 in August 2004.
7. Attackers answered with return-oriented programming: chain together fragments of code that already exists and is already marked executable. No new code is written, so NX does not stop it.
8. The useful side. A just-in-time compiler writes machine code into a data buffer at run time, flips the page permission from writable to executable, and jumps into it. The JavaScript engines in every browser do this thousands of times while a page loads.

9. Tools: readelf -l shows segment permissions, R E for read-execute and RW for read-write. On Linux cat /proc/self/maps shows the same live.

WORDS10.2.6 remember these#

1. Stored program — orders live in memory like any other number — code and data share one address space.
2. Von Neumann architecture — one memory for everything — unified instruction and data store, with a shared bus bottleneck.
3. Harvard architecture — two separate memories — separate instruction and data address spaces and buses.
4. Modified Harvard — one memory but two front doors — split L1i and L1d over a unified lower hierarchy.
5. Buffer overflow — writing past the end of a box — overwriting adjacent stack or heap state, classically the saved return address.
6. NX bit — a page marked “not runnable” — no-execute permission bit in the page table entry, XD on Intel.
7. JIT — writing new code while running — just-in-time compilation into a writable page later marked executable.

10.3 The fetch-decode-execute cycle#

PLAIN10.3.1 in simple words#

1. The loop the CPU repeats forever has three parts and standard names.
2. Fetch: read the instruction at the address in the program counter.
3. Decode: work out what it is and where its inputs are.
4. Execute: do it, and write the answer somewhere.
5. Then increase the program counter and start again.
6. Some instructions add a fourth part, a memory access, and a fifth, writing the result back into a register. We will use the five-part version later.
7. If the instruction was a jump, the program counter is not increased by one. It is loaded with a new address. That is how loops and decisions work.
8. That is the entire behaviour of a processor. Everything after this in the chapter is a trick to run this loop faster.

PLAIN10.3.2 a picture in your head#

1. Picture a postal sorting clerk at a desk.
2. The clerk reads the number on a slip of paper: “go to shelf 100”.
3. They walk to shelf 100 and bring back the envelope. That is fetch.
4. They open it and read what kind of job it is. That is decode.
5. They do the job at their desk. That is execute.
6. They cross out 100 and write 101 on the slip, then start again.
7. If the envelope said “next, go to shelf 240”, they write 240 instead.

Where this comparison breaks: a real CPU does not finish one envelope before starting the next. It has several envelopes open at once, in different stages, and it often guesses which shelf comes next before it knows. Those are pipelining and branch prediction, sections 10.7 and 10.8.

PLAIN10.3.3 a worked example#

1. Let us invent a small machine so nothing is hidden. Call it KB-1.
2. It has four registers R0 to R3, 16-bit words, and 16-bit instructions.
3. An instruction splits into fields like this.

 bit 15..12   11..10   9..8    7..0
 +--------+--------+--------+----------+
 | opcode |   rd   |   rs   | imm/addr |
 +--------+--------+--------+----------+

4. Five opcodes are enough for our program.

5. Memory starts with 7 at address 0x40, 5 at 0x41, and 0 at 0x42.
6. The program, with its exact machine words, is this.

 addr  word    assembly
 0x00  0x1040  LOAD  R0, [0x40]
 0x01  0x1441  LOAD  R1, [0x41]
 0x02  0x3100  ADD   R0, R1
 0x03  0x2042  STORE R0, [0x42]

7. Check the encoding of 0x1441 by hand. In binary it is 0001 01 00 01000001. Opcode 0001 is LOAD, rd is 01 meaning R1, the low eight bits are 0x41. So: load from address 0x41 into R1.
8. Now run it and watch every value change.

9. Three kinds of instruction did all the work: a load, an arithmetic operation, and a store. Almost all real code is these three kinds.

PLAIN10.3.4 what is really happening inside#

1. Now walk one cycle at the level of the wires. Take instruction 3, ADD.
2. Tick 1, fetch. The program counter holds 0x02. Its value is driven onto the address bus. The read line is raised.
3. Memory answers with the 16 bits 0x3100 on the data bus.
4. The write-enable line of the instruction register is raised, so on the next clock edge the instruction register latches 0x3100.
5. Tick 2, decode. The top four bits, 0011, go into the control unit. The control unit is a lookup that turns 0011 into a set of raised wires.
6. Bits 11 to 10 pick R0 as the destination and as source A. Bits 9 to 8 pick R1 as source B. Those go to the register file’s address inputs.
7. The register file drives 7 onto bus A and 5 onto bus B. No clock needed: reading is combinational, the values just appear after a small delay.
8. Tick 3, execute. The control unit puts the code for “add” on the ALU’s function input. The adder settles and 12 appears at its output.
9. The write-back line for R0 is raised. On the next clock edge, R0 latches 12.
10. In parallel, the program counter’s own adder computes 0x02 plus 1, and the program counter latches 0x03 on the same edge.
11. Every one of these steps is switches opening and closing, as in Chapter 4. Nothing is interpreted. The bit pattern is the command.

  PC --> [ address bus ] --> MEMORY
                               |
                          [ data bus ]
                               v
                        INSTRUCTION REG
                               |
                    +----------+----------+
                    |                     |
              CONTROL UNIT          REGISTER FILE
                    |                  |      |
              control wires          busA    busB
                    |                  |      |
                    +--------------> [   ALU   ]
                                         |
                                    write back

TECHNICAL10.3.5 the engineer’s version#

1. The same program in real x86-64, with RDI holding the base address.

  8B 07        mov  eax, dword ptr [rdi]
  03 47 04     add  eax, dword ptr [rdi+4]
  89 47 08     mov  dword ptr [rdi+8], eax

2. Note the byte counts: two, three, three. x86-64 instructions are variable length, from 1 to 15 bytes, with 15 a hard architectural limit.
3. Note also that x86-64 can add straight from memory. One instruction does a load and an add. That is the classic CISC habit.
4. The same program in ARM64, with X0 holding the base address.

  B9400001     ldr  w1, [x0]
  B9400402     ldr  w2, [x0, #4]
  0B020021     add  w1, w1, w2
  B9000801     str  w1, [x0, #8]

5. Every ARM64 instruction is exactly four bytes. Always. That is the classic RISC habit, and it makes the decoder far simpler.
6. ARM64 cannot add from memory. Loads and stores are the only instructions that touch memory. This is called a load-store architecture.
7. Decode 0xB9400402 by hand. Bits 31 to 22 are 1011100101, the 32-bit LDR unsigned-offset form. Bits 21 to 10 are the scaled immediate, value 1, and the scale for a 32-bit load is 4, so the offset is 4 bytes. Bits 9 to 5 are Rn equals X0. Bits 4 to 0 are Rt equals W2.
8. On a modern core, none of this happens in three tidy ticks. The x86-64 add eax, [rdi+4] is split into two internal micro-operations, an address generation plus load, and an add. Section 10.6.
9. To watch real instructions: objdump -d ./a.out disassembles a binary, gdb with layout asm and stepi single-steps them, and perf stat ./a.out counts how many actually retired.

WORDS10.3.6 remember these#

1. Fetch — go and get the next order — read memory at the PC into the instruction register.
2. Decode — work out what it says — map opcode and operand fields onto control signals and register file ports.
3. Execute — do it — drive the ALU or address unit and produce a result.
4. Write-back — put the answer away — latch the result into the destination register on the clock edge.
5. Load-store architecture — only two instructions touch memory — arithmetic operates on registers only, as in ARM64, RISC-V and MIPS.
6. Combinational — settles by itself, no clock — logic whose output depends only on current inputs after a propagation delay.

10.4 Instruction set architecture#

PLAIN10.4.1 in simple words#

1. An instruction set architecture, or ISA, is the list of orders a CPU understands, and exactly how each one is written as a number.
2. It also says how many registers there are, what they are called, how big a number is, and what happens when something goes wrong.
3. It is a promise, not a design. It says what the chip must appear to do. It says nothing about how the chip does it inside.
4. That promise is why a program compiled in 2003 still runs today. The inside of the chip changed completely. The promise did not.
5. Two chips can keep the same promise with wildly different insides. An Intel and an AMD desktop chip share almost nothing internally, yet run the same programs, because they honour the same ISA.
6. The three ISAs that matter today are x86-64, ARM64 and RISC-V.

PLAIN10.4.2 a picture in your head#

1. Think of an ISA as the menu in a restaurant, not the kitchen.
2. The menu says: item 14 is soup, item 22 is rice, item 30 is bread.
3. Any kitchen that produces the right dish for each number is a valid kitchen. One may use a wood fire, another an induction hob.
4. A customer who learned the menu twenty years ago can still order item 14 and get soup, even though the kitchen was rebuilt three times.
5. If a new kitchen decided item 14 now means salad, every old customer breaks. That is why ISAs almost never remove things.
6. That is also why x86-64 still carries instructions from 1978.

Where this comparison breaks: menus are short, and ISAs are not. A full x86-64 manual set from Intel runs to several thousand pages across its volumes, and nobody knows all of it. Also, some menu items are far slower to cook than others, and the menu does not tell you which.

PLAIN10.4.3 a worked example#

1. Every instruction is a number split into fields. Take RISC-V, which is the tidiest example, with a fixed 32-bit length.
2. The R-type format, used for register-to-register arithmetic, is this.

 bits 31..25  24..20 19..15 14..12 11..7  6..0
 +---------+------+------+------+------+-------+
 | funct7  | rs2  | rs1  |funct3|  rd  |opcode |
 +---------+------+------+------+------+-------+

3. For add x5, x6, x7: opcode 0110011, funct3 000, funct7 0000000, rs1 is x6, rs2 is x7, rd is x5. The whole word comes out as 0x007302B3.
4. Change funct7 to 0100000 and the same opcode becomes sub. One bit flip in a field turns add into subtract. That is what a field is for.
5. An immediate is a constant baked into the instruction itself, rather than read from a register. addi x5, x6, 100 carries the 100 in 12 bits.
6. Now addressing modes: the different ways an instruction names where its data is. Here they are with worked values, using x86-64 notation.

7. Work one through. If RBX holds 0x1000 and RCX holds 3, then mov rax, [rbx+rcx*4] reads the 8 bytes at 0x1000 plus 12, which is 0x100C. That single instruction indexes an array of 4-byte elements.
8. That scale factor of 1, 2, 4 or 8 exists precisely because arrays of bytes, shorts, ints and longs are so common.

PLAIN10.4.4 what is really happening inside#

1. The decoder is a big lookup. It takes the instruction bits and produces control signals, plus register numbers, plus a constant.
2. With fixed-length instructions, the decoder knows where every field is before it has read anything. It can slice all fields in parallel.
3. With variable-length instructions it cannot. It must work out how long the first instruction is before it knows where the second begins.
4. On x86-64 the length depends on prefixes, on the opcode, on the ModRM byte, on the SIB byte and on the displacement size. Length decoding is a serial chain, and it is genuinely hard.
5. Intel and AMD solve it by brute force: try to decode at many byte offsets at once and throw away the wrong answers, plus cache the decoded results so the work is not repeated. Section 10.6.
6. That is a real, permanent cost of x86-64. It costs transistors and power that an ARM64 decoder simply does not spend.
7. It is also, in 2026, a survivable cost. It is a fixed overhead at the front of a core that is otherwise dominated by caches and execution units.

TECHNICAL10.4.5 the engineer’s version#

1. The RISC idea began at IBM with John Cocke’s 801 project, which ran from 1975 onward and was not published widely at the time.
2. David Patterson’s group at Berkeley built RISC-I in 1981 and coined the term. John Hennessy’s group at Stanford built MIPS from 1981.
3. Hennessy and Patterson shared the 2017 ACM Turing Award for this work.
4. The original argument: compilers do not use complex instructions, so spend the transistors on registers, pipelines and caches instead.
5. What is the real difference today. Honest answer: at the level of the execution core, almost none. Both x86-64 and ARM64 cores decode into internal micro-operations, rename registers, execute out of order and retire in order. The differences that survive are these.
   1. Decode complexity. Fixed 32-bit versus 1 to 15 bytes.
   2. Memory operands in arithmetic instructions, allowed on x86-64, not on ARM64.
   3. Memory ordering model, strong on x86-64, weak on ARM64. Section 10.10.
   4. Legacy baggage: x86-64 must still support 16-bit real mode at power-on.
6. Where experts disagree: some argue variable-length decode caps how wide an x86 front end can practically get, others point to Intel and AMD shipping 8-wide decode in 2024 and say the ceiling keeps moving. Both sides have a point and the argument is not settled.

7. Two footnotes to that table. RISC-V’s embedded variant has 16 registers rather than 32, and its 2-byte instruction length comes from the optional C compressed extension, not the base set.
8. Dates to keep straight. AMD published the x86-64 specification in 2000 and shipped Opteron in April 2003. Intel shipped its compatible EM64T in the Nocona Xeon in June 2004. Arm announced ARMv8-A with AArch64 in October 2011. RISC-V began at Berkeley in 2010 under Krste Asanovic and David Patterson, its unprivileged ISA was ratified as version 20191213, and RISC-V International moved to Switzerland, with the rename completed in March 2020.
9. Tools: gcc -march=native -Q --help=target lists the ISA extensions the compiler will use, and on Linux the flags line in /proc/cpuinfo lists what the chip actually supports.

WORDS10.4.6 remember these#

1. ISA — the list of orders and how they are written — architectural contract defining instructions, registers, memory model and exceptions.
2. Opcode — the part that says what to do — the operation field of an instruction word.
3. Operand — the part that says what to do it to — register number, memory address or constant.
4. Immediate — a number written into the order itself — constant encoded in the instruction, not fetched.
5. Addressing mode — the recipe for finding the data — the rule turning instruction fields into an effective address.
6. RISC — few simple orders, all the same size — reduced instruction set computer, load-store, fixed length.
7. CISC — many orders, many sizes — complex instruction set computer with memory operands and variable length.
8. Microarchitecture — how one chip actually implements the promise — the internal design, invisible to software.

10.5 Registers in real CPUs#

PLAIN10.5.1 in simple words#

1. A register is a tiny box inside the CPU that holds one number.
2. There are very few of them. A modern CPU has a few dozen that programs can name, not thousands.
3. They are the only storage the arithmetic unit can reach in the same tick of the clock. Everything else is further away.
4. Some registers are general: you can put anything in them.
5. Some have a fixed job. One always points at the next instruction. One always points at the top of the stack. One holds yes or no answers about the last calculation.
6. Almost all real work happens in registers. Data is pulled in from memory, chewed on in registers, and pushed back out.
7. A good compiler spends most of its effort deciding which values live in registers and which have to spill out to memory.

PLAIN10.5.2 a picture in your head#

1. Think of a carpenter at a bench.
2. The registers are the two or three tools actually in their hands.
3. The L1 cache is the tool tray at the edge of the bench, an arm’s reach away.
4. Main memory is the tool cupboard on the other side of the workshop.
5. Storage is the hardware shop down the road.
6. Every step out costs time. Hands are instant. The tray takes a second. The cupboard takes a minute. The shop takes a day.
7. So the carpenter arranges work to keep what is needed in their hands. That is exactly what a compiler’s register allocator does.

Where this comparison breaks: hands hold whole tools, and registers hold only fixed-size numbers, typically 64 bits. Also a carpenter can put a tool down anywhere, while a register value must be explicitly written somewhere or it is simply overwritten and lost.

PLAIN10.5.3 a worked example#

1. On x86-64 the general purpose registers have odd names, because they grew from 1978 by accident rather than by plan.
2. The 8086 of 1978 had eight 16-bit registers, and each name meant something.

3. In 1985 the 80386 widened them to 32 bits and put an E in front: EAX, EBX, and so on. E stands for extended.
4. In 2003 x86-64 widened them to 64 bits and put an R in front: RAX, RBX, and so on. It also added eight plain-numbered ones, R8 to R15.
5. So RAX, EAX, AX, AH and AL are all the same physical register, seen through windows of 64, 32, 16, 8 and 8 bits.
6. ARM64 refused all of that. It has 31 registers with no story attached: X0 to X30 for the 64-bit view, W0 to W30 for the low 32 bits.
7. There is no X31. That slot means either the constant zero or the stack pointer, depending on the instruction. Reading the zero register always gives 0, and writing to it throws the value away.

PLAIN10.5.4 what is really happening inside#

1. Why are registers so fast. Four reasons, all physical.
2. They are physically next to the arithmetic unit, often microns away. Signal travel time is close to nothing.
3. They are addressed by a tiny number, three to five bits. That decode is a few gates, not a lookup in a table of tags like a cache.
4. They are built from fast, wide SRAM cells with many ports, so several registers can be read and written in the same cycle.
5. There are very few of them, so the array is small, so its wires are short, so it settles quickly.
6. The cost of all that is area and power per bit. A register file bit costs far more silicon than a cache bit, which is why you cannot have thousands.
7. There is one more thing hiding here. The names in the program are not the real boxes. A modern CPU has hundreds of real registers and shuffles which physical box currently holds RAX. That is register renaming, section 10.8.

TECHNICAL10.5.5 the engineer’s version#

1. x86-64 architectural integer state: 16 general purpose registers of 64 bits, RIP as the instruction pointer, RFLAGS as the status register.
2. Writing a 32-bit sub-register zero-extends into the full 64 bits. Writing an 8-bit or 16-bit sub-register does not, and leaves the top bits alone. That asymmetry is a standard, written in the architecture manual, and it causes real partial-register stalls if you get it wrong.
3. RFLAGS bits worth knowing: CF is bit 0 carry, PF bit 2 parity, ZF bit 6 zero, SF bit 7 sign, IF bit 9 interrupt enable, DF bit 10 direction, OF bit 11 overflow.
4. ARM64 architectural integer state: X0 to X30, plus SP, plus PC which is not a general register and cannot be written directly. Condition flags live in PSTATE as N, Z, C, V.
5. By convention, not by hardware rule, ARM64 uses X30 as the link register holding the return address, and X29 as the frame pointer.
6. Calling conventions are conventions, set by an ABI document, not by the chip. The System V ABI used on Linux and macOS passes integer arguments in RDI, RSI, RDX, RCX, R8, R9 and returns in RAX. Microsoft’s x64 ABI uses RCX, RDX, R8, R9. ARM64’s AAPCS64 passes in X0 to X7 and returns in X0.
7. Physically, the architectural names map onto a much larger physical register file. AMD Zen 5 has a floating point physical register file of 384 entries, doubled from 192 in Zen 4, and an integer file of a couple of hundred entries.

8. Those latency figures are for AMD Zen 5 on the Ryzen 9000 series, 2024. At 5.7 GHz one cycle is 0.175 nanoseconds, so an L1 hit is about 0.7 ns and a memory access is over 400 times slower.
9. Tools: gdb command info registers prints them live. On Linux, perf stat -e ld_blocks.no_sr and similar counters expose partial register and store-forwarding stalls on Intel parts.

WORDS10.5.6 remember these#

1. Register — a tiny box for one number inside the CPU — architecturally named fast storage, typically 64 bits wide.
2. General purpose register — one you can use for anything — an integer register with no fixed hardware role.
3. Stack pointer — marks the top of the working pile — RSP on x86-64, SP on ARM64.
4. Flags register — a strip of yes or no answers — RFLAGS on x86-64, the NZCV bits of PSTATE on ARM64.
5. Zero register — a tap that always gives zero — XZR and WZR on ARM64, no equivalent on x86-64.
6. Spill — running out of hands and putting something down — the compiler storing a value to the stack because registers ran out.
7. ABI — the agreement about who passes what where — application binary interface, a convention, not part of the ISA.

10.6 Microcode#

PLAIN10.6.1 in simple words#

1. Some instructions are simple, and the hardware can do them directly.
2. Some are not. “Copy this whole block of memory to there” is one order for the programmer but hundreds of steps for the hardware.
3. So inside the CPU there is a second, smaller, more private instruction set.
4. A complicated outside instruction is turned into a short program of these small private steps. Those steps are called micro-operations.
5. The little program that produces them is the microcode.
6. This is why one x86 instruction can take one clock cycle or one hundred. Some are a single step, and some are a whole routine.
7. The important consequence is this: some of the behaviour of a CPU is stored, not wired. Stored things can be changed later.
8. That is how a firmware update can change how a chip behaves without anyone touching the silicon.

PLAIN10.6.2 a picture in your head#

1. Picture a restaurant kitchen again. The menu is the instruction set.
2. Some menu items are one action: “pour a glass of water”.
3. Some are not: “make a three-course meal for six”. The chef has a card file of standard sub-recipes and works through them in order.
4. The card file is the microcode. It sits in the kitchen, not on the menu.
5. Customers never see the cards. They only see the menu, which never changes.
6. If a sub-recipe turns out to be wrong, the owner can replace those cards overnight. The menu still says the same words the next morning.

Where this comparison breaks: the chef can improvise, and the microcode sequencer cannot. Also, most modern instructions skip the card file entirely and are handled by hardwired fast decoders, because going through microcode is slow. Microcode is the exception path, not the normal path.

PLAIN10.6.3 a worked example#

1. Take the x86-64 instruction add [rax], rbx, which means “add RBX to the number in memory at the address in RAX”.
2. To a programmer that is one instruction, four bytes.
3. Inside, it becomes roughly three micro-operations.

  1. load  t0 <- MEM[rax]        (read 8 bytes)
  2. add   t0 <- t0 + rbx        (do the arithmetic)
  3. store MEM[rax] <- t0        (write 8 bytes back)

4. t0 is a temporary that has no name in the instruction set. Programs cannot see it. It exists only for the duration of the instruction.
5. Now take rep movsb, which copies a whole block of bytes. That is a microcoded loop. It may issue thousands of micro-operations from one instruction, and the exact number depends on the block length.
6. Compare with ARM64. The same job needs three separate instructions written out by the compiler, one load, one add, one store. The work is identical. The difference is who wrote the three steps down: the microcode, or the compiler.

PLAIN10.6.4 what is really happening inside#

1. The front end of an x86 core has two paths.
2. The fast path: several simple hardware decoders, each turning one instruction into one to four micro-operations directly.
3. The slow path: the microcode sequencer, a small read-only memory holding canned sequences, used for anything complicated.
4. Decoding x86 is expensive, so cores keep a micro-op cache. Once an instruction has been decoded, the resulting micro-operations are stored.
5. The next time round a loop, the front end reads finished micro-operations straight out of that cache and skips decoding entirely.
6. That saves a lot of power, and it also delivers more operations per cycle than the decoders can.
7. A microcode update is a signed blob of data. Firmware loads it into a small patch memory very early in boot. It is not stored on the chip permanently.
8. It disappears at every power-off and must be loaded again next boot, by the motherboard firmware or by the operating system.

TECHNICAL10.6.5 the engineer’s version#

1. Maurice Wilkes proposed microprogramming in 1951, at the Manchester University Computer Inaugural Conference. EDSAC 2, running from 1958, was the first microprogrammed machine.
2. IBM made it famous with System/360 in 1964: one ISA implemented by many physically different machines, each with its own microcode. That is the clearest early proof that an ISA is a contract, not a design.
3. On Intel cores, simple decoders handle one-to-one and one-to-four cases. Anything longer goes to the MSROM, the microcode sequencer ROM.
4. Micro-op cache sizes, all implementation details rather than architecture.

5. Micro-op fusion and macro-op fusion muddy instruction counting. A compare followed by a branch is commonly fused into one internal operation, so counting retired instructions is not the same as counting work.
6. Microcode update mechanics. On Intel, software writes the physical address of the update to model-specific register 0x79 and the load happens. On AMD the equivalent is MSR 0xC0010020. Updates are encrypted and signed. A CPU will reject an unsigned patch.
7. Limits, and this matters. Microcode cannot add execution units, cannot change cache sizes, and cannot rewrite the hardwired fast decoders. It patches sequences and it can flip configuration bits. The number of patch slots is finite.
8. Spectre and Meltdown were disclosed on 3 January 2018. CVE-2017-5753 is Spectre variant 1, bounds check bypass. CVE-2017-5715 is Spectre variant 2, branch target injection. CVE-2017-5754 is Meltdown.
9. Intel’s microcode updates added entirely new architectural controls: IA32_SPEC_CTRL exposing IBRS and STIBP, and IA32_PRED_CMD exposing IBPB. A firmware update added new registers to a shipped ISA. That is unusual.
   1. IBRS restricts indirect branch prediction across privilege levels.
   2. IBPB flushes predictor state at a chosen barrier point.
   3. STIBP stops one hardware thread steering the other thread’s predictor.
10. Google published retpoline on 4 January 2018, a purely software fix for variant 2 that replaces indirect jumps with a return trampoline the predictor cannot usefully steer.
11. Meltdown was not fixed by microcode at all. It was fixed in software by kernel page-table isolation, which unmaps kernel memory while user code runs, based on the 2017 KAISER work from TU Graz.
12. The rollout went badly. Intel’s January 2018 microcode caused unexpected reboots, and Microsoft shipped an update on 29 January 2018 that disabled it. Measured performance drops of 2 to 14 percent were reported on eighth-generation Core platforms, depending heavily on workload.
13. Observe it: grep microcode /proc/cpuinfo on Linux gives the loaded revision, dmesg | grep -i microcode shows the load at boot, and iucode_tool -l lists available Intel updates.

The honest version: saying “the CPU runs microcode” suggests every instruction goes through an interpreter. It does not. On a modern x86 core the large majority of instructions never touch the microcode sequencer, and on ARM64 cores the sequencer is far smaller still. Microcode is the escape hatch for complicated and rare cases, plus a patching mechanism.

WORDS10.6.6 remember these#

1. Micro-operation — one small private step — the internal RISC-like operation an instruction is broken into, often written uop.
2. Microcode — the card file of stored sub-recipes — ROM sequences that expand complex instructions into micro-operations.
3. Micro-op cache — remembering the decode so you need not redo it — a cache of decoded micro-operations, also called the op cache or DSB.
4. Microcode update — an overnight replacement of some cards — a signed, volatile patch loaded by firmware at every boot.
5. IBRS, IBPB, STIBP — three new switches added by a patch — speculation controls introduced by 2018 microcode for Spectre variant 2.
6. KPTI — hiding the kernel’s map from user code — kernel page-table isolation, the software fix for Meltdown.

10.7 Pipelining#

PLAIN10.7.1 in simple words#

1. A simple CPU wastes almost all of itself, almost all of the time.
2. While it is fetching an instruction, the arithmetic unit sits idle. While it is doing arithmetic, the fetch hardware sits idle.
3. Pipelining fixes that. Break the work into stages, and keep every stage busy on a different instruction at the same time.
4. It does not make any one instruction faster. Each instruction still takes the same number of stages to get through.
5. It makes instructions come out more often. That is throughput, not latency.
6. If the pipeline has five stages and nothing goes wrong, you finish one instruction every clock cycle instead of one every five.
7. Every CPU made since the 1980s is pipelined. It is not optional.

PLAIN10.7.2 a picture in your head#

1. You have four loads of washing. Each load needs washing, drying, folding.
2. Each of the three jobs takes 30 minutes.
3. The naive way: wash, dry and fold load 1 completely, then start load 2. That is 90 minutes each, 360 minutes in total.
4. The pipelined way: as soon as load 1 leaves the washer, load 2 goes in.
5. Now the timeline looks like this, in 30-minute slots.

 slot   1     2     3     4     5     6
 wash  L1    L2    L3    L4
 dry         L1    L2    L3    L4
 fold              L1    L2    L3    L4

6. Total: 6 slots, 180 minutes, not 360. Twice as fast for four loads.
7. With 100 loads it approaches three times as fast, the number of stages.
8. Notice load 1 still took 90 minutes. Nothing got faster. Things just overlapped.

Where this comparison breaks: laundry loads are independent, and instructions are not. Load 2 might need the result of load 1. Worse, sometimes you do not know which load comes next until load 1 finishes. Those two problems are data hazards and control hazards, and they are the whole difficulty.

PLAIN10.7.3 a worked example#

1. The classic teaching pipeline has five stages, from the Berkeley and Stanford RISC designs of the early 1980s.

2. Now run five instructions and watch the pipeline fill and drain.

 cycle    1   2   3   4   5   6   7   8   9
 instr 1  IF  ID  EX  MEM WB
 instr 2      IF  ID  EX  MEM WB
 instr 3          IF  ID  EX  MEM WB
 instr 4              IF  ID  EX  MEM WB
 instr 5                  IF  ID  EX  MEM WB

3. Five instructions finish in 9 cycles, not 25. The first takes 5 cycles to appear, then one appears every cycle.
4. The formula for n instructions in a k-stage pipeline is k plus n minus 1 cycles, against n times k without pipelining.
5. For n equals 5 and k equals 5: 9 cycles against 25. Speedup 2.78.
6. For n equals 1,000 and k equals 5: 1,004 cycles against 5,000. Speedup 4.98, which is almost the full 5.
7. There is a second, bigger win. Each stage now does one fifth of the work, so each stage settles in one fifth of the time, so the clock can run roughly five times faster too. Pipelining is why clocks got fast at all.

PLAIN10.7.4 what is really happening inside#

1. Between every pair of stages there is a row of flip-flops, from Chapter 6, holding that stage’s result until the next clock edge. These are pipeline registers, and they are the physical dividers.
2. Three things can go wrong. They are called hazards.
3. Structural hazard: two instructions want the same piece of hardware in the same cycle. For example, one instruction fetching while another reads data, when there is only one memory port. Fixed by splitting instruction and data caches, which is the modified Harvard idea from section 10.2.
4. Data hazard: an instruction needs a result that is not written yet.

  add r1, r2, r3     ; result of r1 written in stage WB
  sub r4, r1, r5     ; needs r1 in stage EX, two cycles earlier

5. The fix is forwarding, also called bypassing. Extra wires take the answer straight from the output of the arithmetic unit back into its own input, before it has been written anywhere.
6. Forwarding cures most data hazards but not all. If the producer is a load from memory, the value does not exist until the MEM stage, so the consumer must wait one cycle. That wasted cycle is a stall, and the empty slot pushed through the pipeline is a bubble.
7. Control hazard: a branch. Until the branch is resolved, the fetch unit does not know which instruction comes next.
8. In a five-stage pipeline you lose two or three cycles per branch if you just wait. Since roughly one instruction in five or six is a branch, that is a large loss, and it is why branch prediction exists.

TECHNICAL10.7.5 the engineer’s version#

1. The IBM 7030, known as Stretch and delivered in 1961, was the first heavily pipelined machine, and it already fetched past unresolved branches.
2. Pipeline depth by generation, all measured in stages of the integer pipeline, which is the usual quoted figure.

3. The cautionary tale is Intel NetBurst. Willamette shipped in November 2000 with a 20-stage pipeline. Prescott followed in February 2004 with 31 stages. The design goal was very high clock speed, and Intel spoke publicly about reaching 10 GHz.
4. It did not work, for three reasons that all reinforce each other.
   1. Deeper pipelines make every branch misprediction more expensive, because more work must be thrown away. Prescott’s penalty was well over 30 cycles.
   2. Every extra stage adds pipeline registers, and those flip-flops burn power and add setup and hold overhead that eats the timing gain.
   3. Leakage current at 90 nm rose faster than expected, so the promised clock speeds never arrived. Intel stopped at 3.8 GHz in November 2004.
5. Intel cancelled the successors, Tejas and Jayhawk, in 2004. They were reported to target between 40 and 50 stages. Top NetBurst parts had a thermal design power of 115 W, which was extreme for the time.
6. Intel abandoned NetBurst and shipped the Core microarchitecture in July 2006, which went back to a 14-stage pipeline at lower clocks and was substantially faster in real work.
7. The lesson, stated properly: pipeline depth trades clock frequency against misprediction cost and power. There is an optimum, and it is well short of 30 stages for general-purpose code. Modern cores sit at roughly 14 to 20 stages from fetch to retire.
8. Measurement: perf stat -e cycles,instructions,stalled-cycles-frontend, stalled-cycles-backend on Linux separates front-end stalls, which are usually fetch and branch problems, from back-end stalls, which are usually memory and dependency problems.

WORDS10.7.6 remember these#

1. Pipelining — overlapping the stages of different instructions — splitting the datapath into stages separated by pipeline registers.
2. Throughput — how often results come out — instructions completed per unit time.
3. Latency — how long one thing takes end to end — cycles from issue to completion for a single instruction.
4. Hazard — a reason the overlap cannot happen — structural, data or control conflict preventing the next stage from proceeding.
5. Forwarding — passing the answer straight back — bypass paths routing an ALU result to a dependent instruction before write-back.
6. Stall — waiting — holding earlier stages while a dependency resolves.
7. Bubble — an empty slot moving down the pipe — an injected no-operation filling a stalled stage.
8. Pipeline depth — how many stages there are — the count of stages from fetch to retire, a trade-off with clock speed.

10.8 Going wider and smarter#

PLAIN10.8.1 in simple words#

1. Pipelining gets you to about one instruction per cycle. Then what.
2. You build more of everything. Two adders, three adders, six adders. Fetch several instructions at once. Finish several at once.
3. That is superscalar: more than one instruction completed per cycle.
4. But instructions often wait for each other. If instruction 2 needs the answer from instruction 1, doubling the hardware helps nothing.
5. So the CPU looks further ahead. If instruction 5 is ready and instruction 2 is not, it runs instruction 5 first. That is out-of-order execution.
6. It still has to pretend everything happened in order, in case something goes wrong. So results are held back and released in the original order.
7. And when it reaches a branch and does not yet know the answer, it guesses, and carries on as if the guess were right. That is speculation.
8. If the guess was right, the work is kept. If wrong, it is all thrown away and the CPU restarts down the other road.
9. Modern CPUs guess correctly the great majority of the time, which is the only reason this works at all.

PLAIN10.8.2 a picture in your head#

1. Picture a busy kitchen with six cooks instead of one.
2. Orders arrive on a rail in strict order. A dispatcher reads them.
3. Order 2 needs a sauce that order 1 is still making, so the dispatcher skips it and gives order 5, which needs nothing, to a free cook.
4. Finished dishes go to a shelf, not to the customer. The shelf is served strictly in the original order, so the customers never notice the reshuffle.
5. That shelf is the reorder buffer.
6. Now the head waiter sees a customer approaching the till and guesses they will order the usual. The kitchen starts making it before they speak.
7. Nine times out of ten the waiter is right and the food is already there. One time in ten the food is binned.

Where this comparison breaks: binned food in a real kitchen is wasted money, and binned speculative work in a CPU is only wasted time and energy. But it is not perfectly invisible. The traces speculation leaves in the caches are exactly what Spectre reads, which is why guessing turned out to be a security problem in 2018.

PLAIN10.8.3 a worked example#

1. Look at why renaming is needed. Take this sequence.

  1: mul  rax, rbx      ; slow, takes several cycles
  2: mov  rcx, rax      ; needs the result of 1
  3: mov  rax, 5        ; writes rax again, needs nothing
  4: add  rdx, rax      ; needs the new rax

2. Instruction 3 does not depend on anything. It could run immediately.
3. But it writes RAX, and instruction 2 still needs the old RAX. If 3 ran early, it would destroy the value 2 is waiting for.
4. This is a false dependency. The two instructions do not share data, only a name. It is a collision of labels, not of information.
5. The fix: give them different physical boxes. Say RAX currently lives in physical register P17. Instruction 3 is told to write P42 instead, and from then on RAX means P42.
6. Now instruction 2 reads P17 and instruction 3 writes P42. They are independent, and both can run at once.
7. That relabelling is register renaming, and modern cores do it for every single instruction, every cycle.
8. Now count the win. Suppose the multiply takes 4 cycles. Without renaming the sequence takes about 7 cycles. With renaming, instructions 3 and 4 run in parallel with the multiply and the sequence takes about 5.

PLAIN10.8.4 what is really happening inside#

1. Here is the shape of a modern core’s back end, in order.
2. Fetch and predict. The branch predictor supplies a stream of addresses; the fetcher pulls bytes from the L1 instruction cache.
3. Decode into micro-operations, or read them from the micro-op cache.
4. Rename: map each architectural register name onto a free physical register, using a register alias table.
5. Allocate: give each operation a slot in the reorder buffer, which keeps program order, and a slot in a scheduler.
6. Schedule and issue: reservation stations, or a unified scheduler, hold operations until every input is ready, then fire them at a free unit.
7. Execute: the arithmetic units, address generation units, load and store units, and vector units all work in parallel.
8. Write back the result into the physical register, and mark the reorder buffer slot as complete.
9. Retire, in strict program order. Only at retirement do results become architecturally real. If an exception happened, everything after it is discarded and never becomes visible.
10. That last rule is what makes exceptions precise: the software sees a clean, ordered machine even though the hardware ran a mess.

TECHNICAL10.8.5 the engineer’s version#

1. History. The CDC 6600 of 1964, designed by Seymour Cray, had ten functional units and a scoreboard to track dependencies. Robert Tomasulo’s 1967 algorithm for the IBM System/360 Model 91 added reservation stations and register renaming, and it is still the basis of every design today.
2. The Intel Pentium of March 1993 was the first superscalar x86, 2-wide with its U and V pipes. The Pentium Pro of November 1995 was the first out-of-order x86.
3. Modern structure sizes, all implementation details, not architecture.

4. Branch prediction, in order of invention.
   1. Static: always predict not taken, or the backward-taken and forward-not-taken heuristic, which suits loops. No storage needed.
   2. One-bit: remember the last outcome per branch. A loop of N iterations mispredicts twice, at entry and exit.
   3. Two-bit saturating counter: James E. Smith, 1981. Four states, from strongly not taken to strongly taken. It takes two wrong outcomes to change the prediction, so a loop mispredicts once, not twice.
   4. Two-level adaptive: Tse-Yu Yeh and Yale Patt, 1991. Index a table of counters using a history register of recent branch outcomes.
   5. gshare: Scott McFarling, 1993. Combine the branch address with the global history by XOR, which uses the table far better.
   6. Perceptron: Daniel Jimenez and Calvin Lin, 2001. A simple neural predictor, able to use very long histories.
   7. TAGE: Andre Seznec and Pierre Michaud, 2006. Several tagged tables indexed with geometrically increasing history lengths, so short and long correlations are both captured. TAGE-SC-L, with a statistical corrector and a loop predictor, won the Championship Branch Prediction contests and is the basis of what ships today.
5. Real accuracy, in three honest lines.
   1. On many workloads modern predictors get above 99 percent of conditional branches right.
   2. On hard integer code the figure is lower. The 2019 paper “Branch Prediction Is Not A Solved Problem” measured about 95 percent average accuracy for an 8 KB TAGE-SC-L across SPEC CPU 2017 integer benchmarks.
   3. That same paper argues the remaining misses are worth an 18.5 percent instructions-per-cycle opportunity at current core sizes, rising to 55.3 percent for a core four times wider.
6. Real cost of one miss. Measured on Intel Skylake: 16.5 cycles average when the correct path hits in the micro-op cache, 19 to 20 cycles when it does not. At 5 GHz that is roughly 3.3 to 4 nanoseconds. On a core that can retire 6 instructions per cycle, a single misprediction throws away on the order of 100 instruction slots.
7. Two-bit counter state machine, worth memorizing.

   taken ->      taken ->      taken ->
 [00]--------->[01]--------->[10]--------->[11]
 strong NT     weak NT       weak T       strong T
   <--------    <--------     <--------
   not taken    not taken     not taken
 predict NT    predict NT    predict T    predict T

8. Measurement: perf stat -e branches,branch-misses ./program gives the real misprediction rate for your code. A rate above about 2 percent on hot code usually means a data-dependent branch that should be turned into branchless arithmetic or a conditional move.

WORDS10.8.6 remember these#

1. Superscalar — more than one instruction finished per cycle — multiple issue, with duplicated execution units.
2. Out-of-order execution — run whatever is ready — dynamic scheduling with in-order retirement.
3. Register renaming — giving the same name different boxes — mapping architectural registers to a larger physical register file.
4. Reorder buffer — the shelf that restores order — a circular buffer holding results until in-order retirement.
5. Reservation station — a waiting bay for an operation — a scheduler entry holding a micro-op until its operands are ready.
6. Speculation — acting on a guess — executing past an unresolved branch and discarding on a miss.
7. Branch predictor — the thing that guesses — hardware predicting direction and target, today TAGE-style.
8. Precise exception — software sees a tidy machine — the guarantee that on a fault, all earlier instructions completed and none later did.

10.9 SIMD and accelerators#

PLAIN10.9.1 in simple words#

1. Very often a program does the same sum to a long list of numbers.
2. Brighten every pixel. Add every element of two arrays. Scale every audio sample.
3. Doing them one at a time wastes the machine, because the operation is identical each time. Only the data changes.
4. So CPUs got wide registers that hold several numbers side by side, and instructions that do the same sum to all of them at once.
5. This is SIMD: single instruction, multiple data.
6. A 256-bit register holds eight 32-bit numbers. One add instruction adds all eight pairs in one go. Eight times the work, one instruction.
7. That is not the same as having eight cores. It is one core doing one instruction that happens to be eight-wide.
8. Later, chips added units for specific jobs: encryption, and now matrix multiply for neural networks.

PLAIN10.9.2 a picture in your head#

1. A normal instruction is a single hole punch: one hole per press.
2. A SIMD instruction is a row of eight punches on one bar. One press, eight holes, all in line.
3. The catch is obvious. The paper must be lined up. If your eight numbers are scattered around memory, you spend more time gathering them into a row than you save by punching them together.
4. Also, the bar punches all eight or none. If you only want six of them, you need a mask that blocks two of the punches.

Where this comparison breaks: the punch bar is fixed at eight, but Arm’s SVE deliberately does not fix the width. The same program runs on hardware with 128-bit or 512-bit vectors without recompiling. That is a genuinely different design idea, not just a bigger bar.

PLAIN10.9.3 a worked example#

1. Add two arrays of eight floating point numbers, using AVX on x86-64. RDI points to array a, RSI to array b, RDX to the result.

  vmovups ymm0, [rdi]        ; load 8 floats from a
  vmovups ymm1, [rsi]        ; load 8 floats from b
  vaddps  ymm0, ymm0, ymm1   ; 8 adds in one instruction
  vmovups [rdx], ymm0        ; store 8 results

2. Four instructions do 8 additions plus 24 element moves. The scalar version would need a loop of 8 iterations with about 4 instructions each.
3. Same idea in ARM64 NEON, which is 128 bits wide, so four floats at a time.

  ld1  {v0.4s}, [x0]         ; load 4 floats
  ld1  {v1.4s}, [x1]         ; load 4 floats
  fadd v0.4s, v0.4s, v1.4s   ; 4 adds at once
  st1  {v0.4s}, [x2]         ; store 4 results

4. Read v0.4s as “register v0 treated as 4 single-precision values”. Change it to v0.2d and the same register is 2 doubles. Change it to v0.16b and it is 16 bytes. The bits do not move. Only the interpretation changes, which is the numbers-and-meaning idea from Chapter 7.
5. With AVX-512, ZMM registers are 512 bits, so the same loop handles 16 floats per instruction.
6. Real speedups are usually below the theoretical multiple. Memory bandwidth, alignment and loop overhead all take a cut. Two to six times is typical for an eight-wide operation on real code.

PLAIN10.9.4 what is really happening inside#

1. Physically, a SIMD unit is several copies of the same arithmetic circuit, fed from one wide register and controlled by one decoded instruction.
2. There is only one instruction fetch, one decode, one schedule. That saving is a large part of why SIMD is efficient in energy, not just in time.
3. Wide units are hot. Running 512-bit operations flat out draws so much current that some Intel chips reduced their clock frequency while doing it, a well-documented behaviour on Skylake-based Xeon parts from 2017.
4. Then there are fixed-function accelerators. Instead of a general unit, put an entire algorithm in silicon.
5. AES encryption is the classic case. One instruction performs one complete round of the cipher, something that takes dozens of ordinary instructions.
6. Now the same idea has arrived for neural networks. The core operation there is multiplying matrices, so chips now include matrix units and separate neural processing units on the same die.
7. Those units are not general. They do one shape of arithmetic extremely efficiently and are useless for anything else.

TECHNICAL10.9.5 the engineer’s version#

1. The term SIMD comes from Michael Flynn’s 1966 taxonomy of parallel machines.
2. The x86 vector history, with widths and dates.

3. MMX arrived with the Pentium MMX in January 1997 and reused the x87 floating point registers, so you could not mix the two.
4. SSE arrived with the Pentium III in February 1999 and finally gave separate XMM registers. AVX arrived with Sandy Bridge in January 2011 and introduced the three-operand VEX encoding, so the destination need not also be a source.
5. AVX-512 first shipped in Knights Landing in 2016 and in Xeon Skylake-SP during 2017. Consumer support has been inconsistent: present on Rocket Lake, disabled on Alder Lake, and supported on AMD Zen 4 from 2022 and Zen 5 from 2024, with Zen 5 desktop parts having a full 512-bit wide data path. Intel announced AVX10 in July 2023 to tidy this up.
6. Arm’s line runs in parallel.
   1. Advanced SIMD, marketed as NEON, is part of ARMv7-A and mandatory in ARMv8-A, with 32 registers of 128 bits.
   2. SVE was announced at Hot Chips in August 2016. It is vector-length agnostic, permitting 128 to 2048 bits in 128-bit steps.
   3. Fujitsu’s A64FX, used in the Fugaku supercomputer from 2020, implements 512-bit SVE. SVE2 arrived with Armv9-A, announced in March 2021.
7. AES-NI arrived with Intel Westmere in January 2010. Six instructions: AESENC, AESENCLAST, AESDEC, AESDECLAST, AESIMC and AESKEYGENASSIST. One AESENC performs a complete AES round. Beyond speed, the crucial property is that it is constant time and uses no lookup tables, which removes the cache-timing side channels that plagued table-based software AES. Armv8 has the equivalent AESE and AESD instructions.
8. Matrix and neural units on the same die.
   1. Intel Advanced Matrix Extensions, AMX, shipped in Sapphire Rapids Xeon in January 2023: eight tile registers of 1 KB each with a TMUL unit.
   2. Arm’s Scalable Matrix Extension, SME, is part of Armv9 and appears in recent Apple silicon.
   3. Separate NPUs: Intel Meteor Lake in December 2023 was the first Core part with one; Lunar Lake in September 2024 raised it to 48 TOPS. Microsoft’s Copilot+ PC badge requires at least 40 TOPS.
   4. In phones, the Snapdragon 8 Elite Gen 5, announced 24 September 2025, has a Hexagon NPU that Qualcomm states is 37 percent faster than the previous generation. Apple’s A19 Pro, from September 2025, has a 16-core Neural Engine.
9. Separate what is what, as the honesty rules require.
   1. Established fact: the silicon exists, it multiplies matrices at low precision far more efficiently than the general CPU cores, and it is shipping in ordinary phones and laptops today.
   2. Active research: how to split a model across CPU, GPU and NPU, which number formats to use, and how to keep memory bandwidth from becoming the limit.
   3. Marketing claim: TOPS figures. A TOPS number depends entirely on the number format assumed, is usually a peak that no real workload reaches, and is not comparable between vendors. Treat it as a rough class, not a measurement.
10. Chapters 46 onwards return to this: what those matrix units actually compute, and why a neural network reduces to matrix multiply.
11. Tools: on Linux cat /proc/cpuinfo lists avx2, avx512f, aes and so on in the flags line. On macOS, sysctl hw.optional lists the Arm features present.

WORDS10.9.6 remember these#

1. SIMD — one order, many numbers — single instruction, multiple data, a lane parallel execution model.
2. Vector register — a wide box holding several numbers — XMM at 128, YMM at 256, ZMM at 512 bits on x86-64.
3. Lane — one slot in the row — one element position within a vector register.
4. Mask register — a stencil saying which lanes count — k0 to k7 on AVX-512, predicate registers on SVE.
5. Vector-length agnostic — the same code on any width — the SVE model where the hardware vector length is discovered at run time.
6. AES-NI — encryption built into the metal — x86 instructions performing AES rounds directly, constant time.
7. NPU — a chunk of silicon for neural maths — neural processing unit, a fixed-function matrix and activation engine.
8. TOPS — a marketing speed number — tera-operations per second, precision dependent and not comparable across vendors.

10.10 Many cores#

PLAIN10.10.1 in simple words#

1. Until about 2005, each new chip simply ran at a higher clock speed.
2. Then that stopped. Clocks have barely moved since. A fast desktop chip in 2004 ran at 3.8 GHz, and a fast desktop chip in 2025 boosts to 5.7 GHz. That is not much in twenty years.
3. The reason is heat. Pushing the clock higher needs more voltage, and the power a chip burns rises with the square of the voltage. The heat became impossible to remove.
4. So instead of one faster CPU, chipmakers put several CPUs on one piece of silicon. Each one is a core.
5. Two cores can do two things at once, genuinely, in parallel.
6. That only helps if the software can be split into pieces that run at the same time. Many programs cannot be split much, and for those, more cores do very little.
7. Cores also have to agree about memory. If two cores both hold a copy of the same number and one changes it, the other must find out.
8. Keeping them in agreement is called cache coherence, and it is one of the hardest parts of building a modern chip.

PLAIN10.10.2 a picture in your head#

1. One cook working faster has a limit: hands only move so quickly, and past a point the kitchen catches fire.
2. So you hire eight cooks. Now eight dishes can be made at once.
3. But if the recipe says “wait for the stock to reduce for an hour”, eight cooks do not make the stock reduce in less than an hour. That step is stubbornly serial.
4. Worse, the cooks share one fridge. If two cooks both take out the butter, and one of them salts it, the other is holding stale butter.
5. So the kitchen needs a rule: before you change something, shout, and everyone else throws their copy away.
6. Shouting takes time, and the more cooks there are, the more shouting.

Where this comparison breaks: cooks can talk to each other in any order, but cores must agree on a strict order of memory events, or programs break in ways that only show up once in a million runs. That is memory ordering, and it is much stricter and much more subtle than shouting.

PLAIN10.10.3 a worked example#

1. Amdahl’s law, from Gene Amdahl in 1967, tells you the ceiling.
2. Say 95 percent of your program can run in parallel and 5 percent cannot.
3. The formula for speedup with n cores is 1 divided by ((1 minus p) plus p divided by n), where p is the parallel fraction.
4. With p equal to 0.95, work it out.

5. Read the last line again. Even with infinite cores, a program that is 5 percent serial can never go more than 20 times faster.
6. At 16 cores you already get 9.14 of a possible 20. Doubling to 32 gets you to 12.6. The returns collapse quickly.
7. If only 50 percent is parallel, the ceiling is 2 times, no matter what.
8. This single calculation explains why buying more cores often does nothing.

PLAIN10.10.4 what is really happening inside#

1. Take two cores, each with its own L1 cache, sharing memory. Both want the variable X at address A. Follow the standard four-state rule set, MESI.
2. Each cached line is in one of four states: Modified (I changed it and nobody else has it), Exclusive (I have the only copy and it is clean), Shared (others may have it too, clean), Invalid (I have nothing).

3. Step by step. At step 1 core 0 misses, fetches the line, and since no other core has it, it takes it Exclusive.
4. At step 2 core 0 writes. Because it already had it Exclusive, no message is needed at all. It flips silently to Modified. Memory is now out of date.
5. At step 3 core 1 asks for the line. Core 0 sees the request, notices it holds a Modified copy, and supplies the fresh data. Both settle at Shared.
6. At step 4 core 1 wants to write. It must first broadcast a request for ownership. Core 0 drops its copy to Invalid. Core 1 becomes Modified.
7. At step 5 core 0 wants to read again, and the whole dance repeats the other way round.
8. Each of those transfers costs tens of cycles. Two cores fighting over one variable in a tight loop is one of the slowest things a modern CPU can do.

TECHNICAL10.10.5 the engineer’s version#

1. The physics. Dynamic power is approximately P equals alpha times C times V squared times f. Alpha is switching activity, C capacitance, V supply voltage, f frequency.
2. Dennard scaling, from Robert Dennard’s 1974 IBM paper, said that as transistors shrink you can lower V and C in step, so power per square millimetre stays constant while frequency rises. That held for thirty years.
3. It broke down around 2005 to 2006. Threshold voltage could not keep falling without leakage current exploding, so supply voltage stalled near 1 volt, so power density started rising with every shrink.
4. Intel cancelled its 4 GHz Pentium 4 in October 2004 and topped out at 3.8 GHz in November 2004. IBM’s POWER4 in 2001 was the first mainstream dual-core chip; AMD’s Athlon 64 X2 and Intel’s Pentium D both arrived in May 2005. That is the exact moment the industry turned.
5. Simultaneous multithreading, stated honestly. SMT duplicates the architectural state, meaning registers, program counter and flags, so one core presents two logical processors to the operating system. It does not duplicate execution units, caches or TLBs. Those are shared.
   1. The gain comes from filling issue slots that one thread leaves empty while it waits for memory. Typical real throughput gain is 10 to 30 percent, not 100 percent.
   2. It can be negative. Two threads with large working sets thrash the shared L1 and L2 and both run slower than one would alone.
   3. Intel shipped Hyper-Threading on Xeon in February 2002 and on the 3.06 GHz Pentium 4 in November 2002.
   4. It has a security cost. Shared resources leak timing information, which produced attacks including PortSmash in 2018 and the MDS family in
      2019. OpenBSD disabled SMT by default in 2018.
   5. Intel dropped SMT entirely on Arrow Lake in October 2024. The Core Ultra 9 285K has 24 cores and exactly 24 threads.
6. Cache coherence. MESI comes from Mark Papamarcos and Janak Patel, 1984, and is often called the Illinois protocol. Real chips extend it: MOESI adds Owned, used by AMD, and MESIF adds Forward, used by Intel. Large chips use a directory rather than broadcast snooping, because broadcast does not scale past a few dozen cores.
7. Memory ordering is a separate problem from coherence. Coherence is about one address. Ordering is about the visible order of accesses to different addresses.
   1. x86-64 uses Total Store Order. Loads are not reordered with loads, stores are not reordered with stores, but a store followed by a load to a different address may appear reordered because of the store buffer.
   2. ARM64 is weakly ordered. Almost any reordering is allowed unless you write a barrier.
   3. Barriers: MFENCE, LFENCE and SFENCE on x86-64; DMB, DSB and ISB on ARM64. In portable code, use the C11 or C++11 atomics with an explicit memory order rather than inline assembly.
   4. This is why lock-free code that works on an Intel laptop can fail on an Apple silicon Mac. The bug was always there. x86-64’s stronger model was hiding it.
8. False sharing. Two threads write two different variables that happen to sit in the same 64-byte cache line. There is no logical sharing at all, but the line ping-pongs between cores at tens of cycles per bounce.

struct bad  { long a; long b; };            /* same line */
struct good {
    long a; char pad[56];                   /* 8 + 56 = 64 */
    long b;
};

9. In C++, alignas(64) or std::hardware_destructive_interference_size expresses this properly. On Linux, perf c2c record and perf c2c report find false sharing directly by address.
10. Amdahl’s law was presented at the 1967 AFIPS Spring Joint Computer Conference. John Gustafson’s 1988 rebuttal points out that in practice people grow the problem to fit the machine, and for a fixed time budget rather than a fixed problem size the scaling looks much better. Both are correct; they answer different questions.
11. Tools: nproc and lscpu show cores and threads, taskset pins a process to specific cores, and numactl --hardware shows memory locality on multi-socket machines.

WORDS10.10.6 remember these#

1. Core — one complete CPU on the die — an independent instruction stream with its own registers, L1 and L2.
2. Power wall — the point where heat stopped the clock rising — the end of frequency scaling around 2005 due to power density.
3. Dennard scaling — shrinking used to be free — constant power density under scaling, broken since roughly 2006.
4. SMT — one core pretending to be two — simultaneous multithreading, duplicated architectural state over shared execution resources.
5. Cache coherence — everyone agrees what a given address holds — MESI and its relatives, enforced by snooping or a directory.
6. Memory ordering — the agreed order of unrelated accesses — TSO on x86-64, weak on ARM64, controlled by barriers.
7. False sharing — accidental fighting over one line — unrelated variables in the same 64-byte cache line bouncing between cores.
8. Amdahl’s law — the serial part sets the ceiling — speedup equals 1 / ((1 - p) + p/n).

10.11 Caches from the CPU side#

PLAIN10.11.1 in simple words#

1. Main memory is desperately slow compared to the CPU. Fetching from it takes the time of hundreds of instructions.
2. So the CPU keeps small, fast copies of recently used memory close by. Those are caches.
3. There are usually three levels. L1 is tiny and instant, L2 is bigger and slower, L3 is much bigger and slower again.
4. Nothing is copied one byte at a time. Memory is moved in blocks of 64 bytes, called a cache line.
5. So if you read one byte, you get its 63 neighbours for free. That is why walking through an array in order is enormously faster than jumping about.
6. Caches work because of two habits real programs have. If you used something recently you will probably use it again soon, and if you used something you will probably use the thing next to it.
7. Those two habits have names: temporal locality and spatial locality. Almost all of computer performance rests on them.

PLAIN10.11.2 a picture in your head#

1. Picture a library with a huge basement and a small desk.
2. You cannot work in the basement. You fetch books up to the desk.
3. Fetching one book takes twenty minutes, so you never fetch one book. You fetch the whole shelf section it sits in. That section is the cache line.
4. Your desk holds four books. The table behind you holds fifty. The room next door holds a thousand. Those are L1, L2 and L3.
5. When your desk is full and you need a new book, you must put one back. Which one you choose is the replacement policy.
6. If you always ask for books that are near each other, you almost never go to the basement. If you ask for random books, you go every time.

Where this comparison breaks: you decide what to fetch, and the CPU mostly does not. The hardware guesses your next request and fetches it before you ask. That guessing is prefetching, and when it works you never see the basement at all.

PLAIN10.11.3 a worked example#

1. Real figures for AMD Zen 5, the Ryzen 9000 desktop series from 2024.

The L1 and L2 cycle counts are AMD’s published figures. The L3 figure is approximate: AMD quoted 50 cycles for Zen 4, and independent measurements of Zen 5 land close to it but vary with the part and the memory clock.

2. Read the last two rows together. Going to memory instead of L3 costs about nine times longer. Going to memory instead of L1 costs over a hundred times longer.
3. Now measure it yourself. Walk a large array in order, then in random order.

/* sequential: hardware prefetch works, ~1 to 2 ns per element */
for (i = 0; i < n; i++) sum += a[i];

/* random: every access is a cache miss and a TLB risk */
for (i = 0; i < n; i++) sum += a[idx[i]];   /* idx shuffled */

4. On a current desktop, with an array well over the L3 size, the sequential loop typically runs at roughly 1 to 2 nanoseconds per element and the random loop at roughly 70 to 120 nanoseconds per element. Those are approximate and machine dependent, but the ratio, somewhere around 50 to 100 times, is consistent everywhere.
5. Same instructions. Same number of additions. Same array. The only difference is the order of the addresses.
6. This is the single most important performance fact in the chapter.

PLAIN10.11.4 what is really happening inside#

1. When the core wants an address, the cache must answer “do I have it” in one or two cycles. It cannot search. It must go straight to one place.
2. So the address is cut into three fields.
   1. The offset: which byte inside the 64-byte line.
   2. The index: which set of the cache to look in.
   3. The tag: the rest of the address, stored alongside the data to prove which line this actually is.
3. Direct-mapped means one place per index. Fast and simple, but two hot addresses that share an index knock each other out forever.
4. Fully associative means a line may sit anywhere. No conflicts, but you must compare every tag, which is impossibly expensive at any size.
5. Set-associative is the compromise, and it is what everyone uses. Each index selects a small set of, say, 8 or 12 slots, and only those tags are compared. You get most of the conflict resistance for a small cost.
6. When a set is full, something must go. The replacement policy picks the victim, usually an approximation of least recently used.
7. On a write, two choices. Write-through sends every write onward immediately, which is simple but floods the bus. Write-back marks the line dirty and only writes it out when it is evicted. Every modern data cache is write-back.

TECHNICAL10.11.5 the engineer’s version#

1. Work the address split by hand for Zen 5’s L1 data cache: 48 KB, 12-way set associative, 64-byte lines.
   1. Lines total: 48 x 1024 / 64 = 768.
   2. Sets: 768 / 12 = 64.
   3. Offset bits: log2(64) = 6.
   4. Index bits: log2(64) = 6.
   5. Tag bits: everything above bit 11.
2. Take the address 0x12345678 and split it.

 0x12345678 = 0001 0010 0011 0100 0101 0110 0111 1000
              |<--------- tag ------->|index | off |
 tag   = 0x12345   (bits 31..12)
 index = 011001b = set 25   (bits 11..6)
 offset= 111000b = byte 56  (bits 5..0)

3. Now a direct-mapped counter-example: a 32 KB direct-mapped cache with 64- byte lines has 512 lines, so 9 index bits. Any two addresses exactly 32 KB apart share an index. A loop touching two arrays that happen to be 32 KB apart will miss on every single access. That is a conflict miss, and it is why associativity exists.
4. The three kinds of miss, called the three Cs after Mark Hill’s 1987 thesis: compulsory (first ever touch), capacity (working set exceeds the cache) and conflict (fits, but the sets collide).
5. Replacement policies in shipping hardware: true LRU up to about 4 ways; tree-based pseudo-LRU beyond that; RRIP and its dynamic variant from a 2010 ISCA paper for last-level caches, which resist thrashing far better; pure random in some Arm L2 designs, chosen for its tiny cost.
6. Write policy detail: write-back with write-allocate is standard for data caches. A dirty bit per line tracks whether a write-back is needed. Stores to memory marked write-combining, typically video memory, bypass the cache through separate combining buffers.
7. Hardware prefetchers, per level and multiple per level. Next-line, stride detectors that spot a constant step, stream prefetchers that follow several sequences at once, and region-based prefetchers. They can be counter productive on pointer-chasing code, and they can be disabled through model-specific registers on both Intel and AMD parts.
8. Cache line size is 64 bytes on x86-64 and on most Arm cores. Apple’s M series uses 128 bytes, which is a real portability trap for anyone padding structures by hand.
9. Inclusion policy is an implementation detail, not a standard. Intel’s ring-based designs historically used an inclusive L3; AMD’s L3 is a victim cache, holding only lines evicted from L2, which is why AMD’s L3 is quoted per core complex die rather than per chip.
10. Tools: lscpu -C prints every level with size, ways and line size; getconf LEVEL1_DCACHE_LINESIZE prints the line size; perf stat -e cache-references,cache-misses,LLC-load-misses measures the real miss rate; valgrind --tool=cachegrind simulates it per line of source code.

WORDS10.11.6 remember these#

1. Cache line — the block memory moves in — 64 bytes on x86-64 and most Arm, 128 on Apple silicon.
2. Tag, index, offset — proof, shelf, and position on the shelf — the three fields an address is split into for lookup.
3. Set-associative — a few slots per index — N-way lookup comparing N tags in parallel.
4. Conflict miss — two hot addresses on the same shelf — a miss that would not happen in a fully associative cache of the same size.
5. Write-back — write later, when evicted — a dirty bit marks lines needing a write-out, as opposed to write-through.
6. Prefetching — fetching before you ask — hardware or software issuing loads ahead of demand based on detected patterns.
7. Locality — you reuse things, and their neighbours — temporal and spatial locality, the reason caches work at all.

10.12 Interrupts, exceptions and privilege#

PLAIN10.12.1 in simple words#

1. The loop of fetch, decode, execute never stops on its own. Something must be able to break into it.
2. Two kinds of thing break in.
3. An interrupt comes from outside: a key was pressed, a network packet arrived, a timer expired. It has nothing to do with the current instruction.
4. An exception comes from inside: a divide by zero, an instruction the chip does not know, a memory address that is not mapped.
5. Either way the CPU stops, remembers exactly where it was, jumps to a fixed handler routine, runs it, and then goes back.
6. It also changes mode. Handlers run with more power than ordinary programs.
7. That mode change is the whole basis of operating system security. Your program cannot touch the disk directly. It must ask, and asking means deliberately triggering a controlled jump into the more powerful mode.

PLAIN10.12.2 a picture in your head#

1. Picture a receptionist working through a pile of forms.
2. The phone rings. They put a bookmark in the pile, note exactly where they were, answer the phone, deal with it, and return to the exact same form.
3. The phone number is not decided on the spot. There is a printed list on the wall: “for fire, call this room; for deliveries, call that room”. That list is the interrupt vector table.
4. Some rooms need a key the receptionist does not have. To get in, they must go through a specific door with a guard, who checks them and hands over the key on the way in and takes it back on the way out.
5. That guarded door is a system call.

Where this comparison breaks: the receptionist chooses when to look up. A CPU can be interrupted between any two instructions, whether it wants to be or not, and the hardware, not the software, decides when.

PLAIN10.12.3 a worked example#

1. Print thirteen characters to the screen. On Linux, x86-64.

  mov  rax, 1          ; 1 = the write system call
  mov  rdi, 1          ; file 1 = standard output
  lea  rsi, [msg]      ; address of the text
  mov  rdx, 13         ; how many bytes
  syscall              ; cross into the kernel

2. The same thing on Linux, ARM64.

  mov  x8, #64         ; 64 = the write system call
  mov  x0, #1          ; file 1 = standard output
  adr  x1, msg         ; address of the text
  mov  x2, #13         ; how many bytes
  svc  #0              ; cross into the kernel

3. Notice the shape is identical. Put a number saying which service, put the arguments in agreed registers, execute one special instruction.
4. Notice the register names differ, the call numbers differ, and the instruction differs. All three are conventions of that operating system on that architecture, not laws of the machine.
5. The syscall and svc instructions do almost nothing themselves. They save the return address, raise the privilege level, and jump to one fixed address the kernel installed at boot. The kernel does the rest.

PLAIN10.12.4 what is really happening inside#

1. Between instructions, the core checks whether an interrupt is pending. If interrupts are enabled and one is pending, it acts.
2. It finishes the instructions already in flight and discards the speculative ones, so the machine has a clean, precise state.
3. It looks up the vector number in a table of handler addresses whose location is held in a special register.
4. It pushes enough state to come back: at minimum the return address and the flags, and on x86 also the stack and code segment selectors and, for some faults, an error code.
5. It raises the privilege level and jumps to the handler.
6. The handler immediately saves everything else it will use, because the hardware only saved the minimum.
7. When finished, a return-from-interrupt instruction restores the saved state and drops the privilege level in one atomic step.
8. All of this costs real time. A modern system call is on the order of tens to a few hundred nanoseconds, and the Meltdown mitigations of 2018 made it noticeably more expensive by forcing a page-table switch on every crossing.

TECHNICAL10.12.5 the engineer’s version#

1. x86-64 uses the Interrupt Descriptor Table, located by the IDTR register. It has 256 entries. Vectors 0 to 31 are architecturally defined exceptions; 32 to 255 are available for devices and software.

2. Exceptions are classified as faults, which re-run the instruction after the handler, traps, which continue after it, and aborts, which do not return. A page fault is a fault; that is exactly how demand paging works.
3. ARM64 uses a vector table located by VBAR_EL1, with 16 entries of 128 bytes each.
   1. The 16 entries are four exception types, which are synchronous, IRQ, FIQ and SError, crossed with four sources.
   2. The four sources are the current exception level using SP0, the current level using SPx, a lower level in AArch64, and a lower level in AArch32.
   3. State is saved into ELR_EL1 for the return address, SPSR_EL1 for the saved processor state, and ESR_EL1 for the syndrome describing why.
4. Privilege levels. x86-64 defines rings 0 to 3. In practice only ring 0, the kernel, and ring 3, user code, are used; ring 1 and 2 are historical. Virtualization adds VMX root mode below ring 0, informally called ring -1, and System Management Mode sits outside all of it.
5. ARM64 defines exception levels instead: EL0 for applications, EL1 for the kernel, EL2 for a hypervisor, EL3 for the secure monitor. Higher number means more privilege, the opposite direction to x86 rings.
6. SYSCALL on x86-64 was introduced by AMD and is deliberately minimal.
   1. It saves RIP into RCX and RFLAGS into R11, then masks RFLAGS using IA32_FMASK.
   2. It loads the code and stack selectors from IA32_STAR and jumps to the address held in IA32_LSTAR.
   3. It performs no memory reads and no descriptor table lookup. That is precisely why it is fast compared with the old int 0x80 gate.
7. Because SYSCALL destroys RCX, the Linux x86-64 system call ABI passes its fourth argument in R10 rather than RCX. The number goes in RAX, arguments in RDI, RSI, RDX, R10, R8, R9, and the result comes back in RAX. On ARM64 the number goes in X8, arguments in X0 to X5, result in X0.
8. Kernel page-table isolation, the Meltdown fix, unmaps almost all kernel memory while user code runs, so each crossing writes CR3 and disturbs the TLB. On processors without PCID tagging the cost is severe. This is a concrete case where a security fix has a permanent, measurable performance price.
9. The vDSO exists to avoid the crossing entirely for cheap, common calls. Linux maps a small shared page into every process so that clock_gettime and similar can read a kernel-updated timestamp without a system call.
10. Tools: strace ./program traces every system call and its arguments, cat /proc/interrupts shows interrupt counts per CPU per device, and perf trace gives timing per call.

WORDS10.12.6 remember these#

1. Interrupt — something outside wants attention — an asynchronous event from a device, delivered between instructions.
2. Exception — the current instruction went wrong — a synchronous fault, trap or abort caused by execution itself.
3. Vector table — the printed list of who to call — IDT on x86-64, the table at VBAR_EL1 on ARM64.
4. Context switch — putting a bookmark in and picking up another book — saving and restoring register state to change what is running.
5. Privilege ring — how much the running code is allowed to do — rings 0 to 3 on x86-64, exception levels EL0 to EL3 on ARM64.
6. System call — knocking on the kernel’s door on purpose — SYSCALL on x86-64, SVC on ARM64, with the number in a register.
7. Page fault — the address had no memory behind it — a fault that lets the kernel map a page and re-run the instruction.

10.13 Reading a real spec sheet#

PLAIN10.13.1 in simple words#

1. A CPU page is a wall of numbers. Almost all of them are simple once you know what each one is measuring.
2. Cores means how many complete CPUs are on the chip. Threads means how many programs the operating system can schedule at once, which may be the same number or twice it.
3. Base clock is what the chip guarantees under a heavy load on all cores. Boost clock is the best it will ever do, briefly, on one core, when cool.
4. Cache numbers tell you how much fast memory sits between the cores and main memory, at each of three levels.
5. TDP is a cooling target in watts. It is not the power the chip draws.
6. Process node, such as “3 nm”, is a marketing name for a manufacturing generation, not a measurement of anything on the chip.
7. Socket says which motherboard it physically fits.
8. Memory channels and PCIe lanes say how much data can get in and out.

PLAIN10.13.2 a picture in your head#

1. Compare it to reading a car’s specification.
2. Cores are cylinders. Threads are how many drivers the dashboard pretends there are.
3. Base clock is the speed you can hold all day up a hill. Boost clock is what the speedometer reaches downhill with a tailwind, for thirty seconds.
4. Cache is the fuel already in the lines. TDP is the radiator’s rating.
5. PCIe lanes are the number of doors for luggage.

Where this comparison breaks: a car’s top speed is a property of the car, but a CPU’s boost clock is a property of the car, the road, the weather and the driver together. Cooling, motherboard power delivery and even the individual silicon sample all change it.

PLAIN10.13.3 a worked example#

1. Take a current desktop chip: the AMD Ryzen 9 9950X3D, launched on 12 March 2025. Every number decoded.

2. The 128 MB of L3 is not one pool. This chip has two eight-core dies. One has a 64 MB cache die stacked on top of it, giving that die 96 MB. The other has the normal 32 MB.
3. So threads on the wrong die do not get the big cache. That is why the operating system scheduler has to be told which die is which.
4. Now the phone side: the Qualcomm Snapdragon 8 Elite Gen 5, announced on 24 September 2025.

5. Notice what is missing from the phone sheet: no TDP, no socket, no memory channels, no PCIe lanes. A phone chip is soldered, its memory is stacked on the package, and its power budget is a thermal envelope of a few watts, not a published number.

PLAIN10.13.4 what is really happening inside#

1. TDP deserves a paragraph of its own, because it is widely misread.
2. TDP is the heat, in watts, that the cooler must be able to remove for the chip to hold its rated behaviour. It is a design target for the cooling system.
3. AMD’s socket AM5 parts will draw up to 1.35 times the TDP in package power. So a 170 W part draws up to about 230 W.
4. Intel now publishes two numbers instead: base power and maximum turbo power. On the Core Ultra 9 285K those are 125 W and 250 W.
5. Boost clock is a ceiling, not a promise. It requires a light load, a good core, headroom in the power budget, and cool silicon. All-core sustained clocks under a heavy load are typically several hundred megahertz lower.
6. Cores are no longer identical. Intel’s desktop parts mix performance cores and efficient cores with different clocks and different capabilities. Phone chips have done this since Arm’s big.LITTLE arrangement in 2011.
7. “Cores plus threads” on a mixed chip is genuinely confusing. The Core Ultra 9 285K has 8 performance cores plus 16 efficient cores, which is 24 cores, and because SMT was removed it has exactly 24 threads.

TECHNICAL10.13.5 the engineer’s version#

1. The two current desktop parts side by side, from vendor specifications.

2. Process nodes: the 9950X3D uses TSMC N4P for its core dies with a separate 6 nm class I/O die; the 285K uses a TSMC N3B compute tile in a multi-tile package. In both cases the number is a process generation name. Chapter 3 covered why no dimension on the chip actually measures 3 or 4 nanometres.
3. The mobile parts for comparison, from vendor material.

4. Note that Apple publishes core counts and little else; Qualcomm publishes clocks; neither publishes sustained power. Comparing them properly needs measurement, not spec sheets.
5. Memory bandwidth is worth computing rather than reading. Dual-channel DDR5-6000 gives 2 channels times 8 bytes times 6000 million transfers per second, which is 96 GB/s peak. Real achieved bandwidth is typically 70 to 85 percent of that.
6. PCIe bandwidth likewise: PCIe 5.0 runs at 32 gigatransfers per second per lane, and with 128b/130b encoding that is about 3.94 GB/s per lane per direction, so a 16-lane graphics slot is about 63 GB/s each way.
7. Tools: lscpu, lscpu -C, dmidecode -t processor for socket and package data, sudo lspci -vv for actual negotiated PCIe link width and speed, and turbostat for live clock, power and temperature.

WORDS10.13.6 remember these#

1. Base clock — the speed it can always hold — guaranteed frequency at rated power with all cores loaded.
2. Boost clock — the best it ever does — opportunistic maximum for a light load with thermal and power headroom.
3. TDP — how much heat the cooler must remove — thermal design power, a cooling target, not consumption.
4. PPT, PL1, PL2 — the real power limits — package power tracking on AMD, base and turbo power limits on Intel.
5. Socket — the physical fit — LGA 1718 for AM5, FCLGA1851 for Arrow Lake.
6. Memory channel — one independent 64-bit path to DRAM — two channels is standard on desktop, more on workstation and server.
7. PCIe lane — one serial link pair for expansion — PCIe 5.0 gives about 3.94 GB/s per lane per direction.
8. System on chip — everything in one package — CPU, GPU, NPU, modem, memory controller and I/O on one piece of silicon.

10.14 How fast is a CPU really#

PLAIN10.14.1 in simple words#

1. Speed is not gigahertz. Gigahertz is how many times per second the clock ticks, and that is only half the story.
2. The other half is how much work the chip gets done per tick.
3. A chip at 3 GHz that does two instructions per tick beats a chip at 4 GHz that does one.
4. Work per tick is called instructions per cycle, or IPC.
5. Time taken equals the number of instructions, times cycles per instruction, divided by the clock speed. Those three numbers are the whole of it.
6. A compiler that emits fewer instructions helps. A wider chip that runs more per cycle helps. A higher clock helps. All three multiply together.
7. So you cannot compare two different chips by clock speed alone, and you certainly cannot compare two different instruction sets that way.

PLAIN10.14.2 a picture in your head#

1. Think of a factory line. The clock is how often the conveyor advances.
2. IPC is how many items each advance actually delivers.
3. A line that advances four times a second but usually delivers nothing is worse than one that advances three times a second and always delivers two.
4. Turbo is the line running fast for a short burst while everything is still cool, then slowing down once the machines heat up.
5. Thermal throttling is the safety cut-out that slows the line before anything melts.

Where this comparison breaks: a factory delivers nothing when starved of parts, and so does a CPU, but the starvation is invisible. A core waiting on memory still ticks its clock and still reports full frequency. It just retires almost no instructions. That is why frequency graphs look healthy while programs run slowly.

PLAIN10.14.3 a worked example#

1. Take a program of exactly 1,000,000,000 instructions.
2. CPU A: 4.0 GHz, IPC 1.0. Time = 1e9 / (4e9 x 1.0) = 0.250 seconds.
3. CPU B: 3.0 GHz, IPC 1.8. Time = 1e9 / (3e9 x 1.8) = 0.185 seconds.
4. CPU B is 26 percent faster while being 25 percent slower in gigahertz.
5. This is not a made-up case. It is exactly what happened in 2006. The Pentium 4 670 ran at 3.8 GHz. The Core 2 Duo E6600 ran at 2.4 GHz and beat it comfortably per core, because its IPC was far higher.
6. Now add the third term. If a better compiler removes 20 percent of the instructions, CPU A takes 0.200 seconds without any hardware change.
7. Real IPC values worth carrying around: pointer-chasing code that misses cache constantly runs at 0.1 to 0.3; ordinary integer application code runs at roughly 1 to 2.5 on a modern wide core; tight vectorized numeric loops can exceed 4.

PLAIN10.14.4 what is really happening inside#

1. Turbo works like a budget. The chip watches its power draw, its current draw and its temperature, thousands of times a second.
2. While all three are under their limits, it raises the clock. When any one hits its limit, it lowers it.
3. So the same chip in a small case with a weak cooler is genuinely a slower chip than in a big case with a good one. The number on the box does not change; the achieved clock does.
4. Not all cores are equal even within one chip. Manufacturing variation means some cores tolerate higher clocks, and the firmware knows which, and sends single-threaded work there.
5. Thermal throttling is the last-resort mechanism. If temperature reaches the limit, typically around 95 to 100 degrees Celsius, the clock drops sharply until it recovers.
6. This is why a benchmark run for 10 seconds and a benchmark run for 10 minutes give different answers, and why laptop reviews measure both.

TECHNICAL10.14.5 the engineer’s version#

1. The performance equation, often called the iron law of processor performance, is: CPU time = instruction count x cycles per instruction x clock period.
2. IPC is the reciprocal of CPI. Each of the three terms is owned by a different party: the compiler and program own instruction count, the microarchitecture owns CPI, and the process and power budget own the clock.
3. Measuring it, on Linux.

perf stat -e cycles,instructions,branch-misses ./program
# prints, among other lines:
#   1,842,113,904  cycles
#   3,120,884,551  instructions   #  1.69  insn per cycle

4. That “insn per cycle” figure counts retired instructions, meaning the ones that actually completed. Speculative work that was thrown away is not counted, which is the correct definition.
5. IPC is not comparable across instruction sets. One x86-64 instruction may do a load, an arithmetic operation and an addressing calculation, while the ARM64 equivalent needs two or three instructions. Comparing IPC between them measures encoding density, not speed.
6. IPC is also not comparable across programs. A memory-bound program has low IPC because it is waiting, and no amount of core width fixes it.
7. Benchmark categories, and what each is honest about.
   1. SPEC CPU 2017: real compiled applications, with separate integer and floating point suites and separate rate and speed variants. Rate measures throughput with many copies; speed measures one job’s time. It is the closest thing to a standard, and results must be submitted with full disclosure of compiler flags.
   2. Geekbench 6 and Cinebench 2024: quick, widely quoted, and dominated by short bursts, so they favour high boost clocks over sustained ability.
   3. Microbenchmarks: measure one instruction’s latency or throughput. Useful for engineers, useless as a purchase guide.
   4. Your own workload, timed: always the best answer if you can get it.
8. Turbo control is exposed as real, documented limits. Intel defines PL1, the long-term power limit, PL2, the short-term limit, and tau, the time constant over which the average is taken. AMD defines PPT for package power, TDC for sustained current and EDC for peak current, with Precision Boost 2 adjusting clocks continuously against all three plus temperature.
9. Where experts disagree: whether single-number benchmark scores are useful at all. One camp says a composite score is the only practical way for a buyer to compare; the other says composites hide the fact that different workloads rank chips in different orders. Both are right in their own context, and the safe habit is to look at the sub-scores.
10. Tools: turbostat on Intel and zenmonitor or ryzen_monitor on AMD show live per-core clock, package power and temperature. s-tui shows throttling under load. stress-ng generates the load.

WORDS10.14.6 remember these#

1. IPC — how much gets done per tick — instructions retired per cycle, measured by hardware performance counters.
2. CPI — the same thing upside down — cycles per instruction, the reciprocal of IPC.
3. Iron law — the only performance formula you need — time equals instruction count times CPI times clock period.
4. Retired instruction — one that really counted — an instruction that committed architectural state, excluding discarded speculation.
5. Turbo — a temporary speed rise — opportunistic frequency increase within power, current and thermal limits.
6. Thermal throttling — slowing down to survive — automatic clock reduction when a temperature limit is reached.
7. SPEC CPU 2017 — the serious benchmark — a suite of real applications with mandatory disclosure of build settings.

10.98 Common wrong ideas#

1. Wrong: a CPU understands your program. Right: it matches bit patterns to control signals. Nothing is understood at any point.
2. Wrong: a higher clock speed means a faster chip. Right: time equals instruction count times cycles per instruction divided by clock speed, and the middle term differs by more than 2 times between designs.
3. Wrong: instructions and data are stored differently. Right: they are the same bytes in the same memory. Only permission bits and where the program counter points make the difference.
4. Wrong: more cores means proportionally more speed. Right: Amdahl’s law caps you at 1 divided by the serial fraction, so 5 percent serial code can never go more than 20 times faster.
5. Wrong: hyper-threading doubles performance. Right: it duplicates architectural state only, and the typical real throughput gain is 10 to 30 percent, sometimes negative.
6. Wrong: RISC chips are simple and CISC chips are complex. Right: both decode into internal micro-operations and both run out of order. The real surviving differences are decode complexity, memory operands and the memory ordering model.
7. Wrong: a deeper pipeline is always faster. Right: the Pentium 4 went from 20 to 31 stages, was cancelled at 3.8 GHz, and was beaten by a 14-stage design in 2006.
8. Wrong: microcode is what runs your program. Right: most instructions never reach the microcode sequencer. It handles complicated cases and provides a patching mechanism.
9. Wrong: cache makes memory faster. Right: cache makes repeated and nearby access faster. Truly random access over a large array gets almost no benefit at all.
10. Wrong: TDP is how much power the chip uses. Right: TDP is a cooling design target. An AMD 170 W part draws up to about 230 W at the package.

10.99 Chapter summary in 20 lines#

1. A CPU is one loop repeated forever: fetch, decode, execute, repeat.
2. Its parts are a control unit, an ALU, a register file, a program counter, an instruction register, caches, interconnect and a memory controller.
3. The stored-program idea puts instructions and data in the same memory, which is why one machine runs any program.
4. That same idea permits buffer overflows, self-modifying code and JIT compilation, and it is why the NX permission bit had to be invented.
5. Real CPUs are modified Harvard: one memory underneath, split L1 instruction and data caches on top.
6. An instruction is a number split into fields: opcode, operands, immediates. Addressing modes are the rules that turn those fields into an address.
7. An ISA is a contract, not a design. x86-64 dates from 2000, ARM64 from 2011, RISC-V began at Berkeley in 2010 and is open.
8. Registers are the fastest storage that exists because they are tiny, near, many-ported and addressed by three to five bits.
9. Microcode turns one complex instruction into several micro-operations, and because it is stored rather than wired, it can be patched at boot.
10. That patching mechanism carried the 2018 Spectre mitigations, which added new control registers to a shipped architecture by firmware update.
11. Pipelining overlaps stages so one instruction completes per cycle instead of one every five, and it is why clock speeds could rise at all.
12. Hazards are structural, data and control. Forwarding fixes most data hazards; branch prediction fixes most control hazards.
13. Very deep pipelines lose more on every mispredict than they gain in clock. The 31-stage Pentium 4 Prescott is the proof.
14. Superscalar, out-of-order execution, register renaming and the reorder buffer extract parallelism while still showing software a tidy, ordered machine.
15. Modern TAGE-style branch predictors are right the great majority of the time, and each miss still costs about 16 to 20 cycles.
16. SIMD does one instruction to many numbers at once, from MMX in 1997 to AVX-512 and Arm SVE. Fixed-function units such as AES-NI and NPUs go further and put whole algorithms in silicon.
17. Clocks stopped rising around 2005 when Dennard scaling ended, so chips went multicore, and Amdahl’s law then set the ceiling.
18. Cache coherence keeps cores agreeing about one address; memory ordering and barriers govern the visible order across different addresses.
19. Caches work only because programs reuse data and touch neighbours. Sequential access can be 50 to 100 times faster than random access.
20. Judge a CPU by instruction count times cycles per instruction times clock period, measured on your own workload, not by the number on the box.