Storage - Disks, SSDs, Flash and Filesystems

12.0 What this chapter gives you#

1. You will be able to say exactly what makes storage different from memory, and why every computer needs both.
2. You will be able to draw a hard disk drive and name every moving part in it, and say how one bit is written and read back.
3. You will be able to compute how long a spinning disk takes to reach a random piece of data, from the drive’s rotation speed alone.
4. You will be able to explain a flash memory cell, why it wears out, and why erasing works on big blocks but writing works on small pages.
5. You will be able to explain why an SSD contains its own small computer, and what that computer spends its time doing.
6. You will be able to say why a big file copy to a cheap SSD starts fast and then suddenly slows down.
7. You will be able to trace a byte from a program down through the filesystem, the interface and the drive, and name each standard on the way.
8. You will be able to say what happens on disk when you create, write, rename and delete a file, step by step.
9. You will be able to explain why a 4 GB video will not copy onto a FAT32 USB stick, and what to do about it.
10. You will be able to explain journaling, RAID, backups and secure erase well enough to make real decisions with real money.

12.1 Storage versus memory: the real difference#

PLAIN12.1.1 in simple words#

1. Memory is where a computer keeps what it is thinking about right now.
2. Storage is where a computer keeps what it wants to still have tomorrow.
3. The big difference is what happens when the power goes off.
4. Memory forgets. Pull the plug and the contents are gone in under a second.
5. Storage remembers. Pull the plug, come back in a year, the file is still there.
6. A thing that keeps its contents without power is called non-volatile.
7. A thing that loses its contents without power is called volatile.
8. Memory is fast and expensive and small. Storage is slow and cheap and huge.
9. Both exist because no single technology is fast, cheap, huge and permanent all at once. Nobody has found one. Engineers pick two or three and stack the results in layers.

PLAIN12.1.2 a picture in your head#

1. Think of a person working at a desk in a library.
2. The papers spread on the desk are memory. They are within arm’s reach and you can grab any of them instantly.
3. The shelves around the room are storage. Getting a book takes you a walk.
4. The desk is small. You cannot fit the library on it.
5. At the end of the day the cleaner clears the desk completely. That is power loss. Anything not filed on a shelf is gone.
6. So you work fast on the desk, and you file to the shelves anything you want tomorrow.
7. There is also a basement archive, far away, cheap, enormous, and slow to reach. That is tape.

Where this comparison breaks: the desk in a real computer is not merely small, it is thousands of times faster than the shelves, not two or three times. Also, a librarian never has to rewrite a whole shelf to change one book, but a flash chip has almost exactly that problem, which we meet in section 12.5.

PLAIN12.1.3 a worked example#

1. Suppose you edit a photo. The photo file lives on your SSD.
2. You double-click it. The computer copies the file from storage into memory.
3. Every brush stroke changes the copy in memory. The file on disk is untouched.
4. You press Save. Now the memory copy is written back down to storage.
5. If the power fails between step 3 and step 4, your strokes are gone and the old file is still there, unharmed.
6. That single sentence explains why software nags you to save.

PLAIN12.1.4 what is really happening inside#

1. Memory in a normal computer is DRAM. Each bit is a tiny capacitor, a bucket holding a little charge.
2. The bucket leaks. It has to be topped up thousands of times per second. That topping up is called refresh, and it needs power.
3. Cut the power and every bucket empties within milliseconds. That is why RAM is volatile: not because of a rule, but because of a leak.
4. Storage stores a bit as something physical that does not need topping up: the direction of a magnet on a disk, or a pocket of trapped electrons in flash.
5. Nothing has to actively hold that state. It just sits there.
6. The price of permanence is speed. Flipping a magnet or pushing electrons through an insulator takes far longer than charging a tiny capacitor.
7. So the whole machine is built as a ladder. Tiny and instant at the top, huge and slow at the bottom, with each level acting as a cache for the one below.

TECHNICAL12.1.5 the engineer’s version#

1. The memory hierarchy spans roughly eight orders of magnitude in latency and six in price per byte. Figures below are typical for mid-2026 desktop and server parts.

2. Prices move fast and 2025 to 2026 was unusual. AI datacentre demand pulled NAND and DRAM capacity away from consumers and prices roughly doubled.

3. The honest version: “non-volatile” is not “eternal”. NAND flash loses charge over years when unpowered, magnetic domains slowly relax, and tape binders hydrolyse. Non-volatile means “does not need power to hold state for a useful period”, not “permanent”.
4. Byte-addressable persistent memory sat between DRAM and SSD for a few years. Intel Optane, based on 3D XPoint, shipped from 2017 with roughly 300 ns latency. Intel wound the business down in 2022. The idea is not dead, but no mainstream product occupies that rung today.
5. Tools that show the ladder: lscpu and getconf -a for cache sizes, free -h and vm_stat for memory, lsblk and diskutil list for block devices, smartctl -a for drive internals, fio for measured latency.

WORDS12.1.6 remember these#

1. Volatile — forgets without power — state decays once supply is removed.
2. Non-volatile — remembers without power — retains state with supply removed.
3. DRAM — the fast forgetful memory — dynamic RAM, one capacitor and one transistor per bit, requiring periodic refresh.
4. Latency — the wait before anything arrives — time from request issue to first byte returned.
5. Throughput — how much arrives per second once it starts — sustained transfer rate, usually MB/s.
6. Memory hierarchy — the ladder of speeds — cache levels plus main memory plus backing store, each caching the level below.

12.2 The old ways: cards, tape, drums, cores and floppies#

PLAIN12.2.1 in simple words#

1. Before disks, computers stored data as holes in card, as magnetized spots on ribbon, and as rings of metal threaded on wire.
2. A punched card is a stiff paper card. A hole means one thing, no hole means the other. A machine feels for the holes with brushes or light.
3. Paper tape is the same idea on a long roll instead of separate cards.
4. Magnetic tape is a long plastic ribbon coated with rust-like powder. You magnetize small patches of it to record bits.
5. A magnetic drum is a spinning metal cylinder with heads along its side.
6. Core memory is a mesh of wires with tiny magnetic rings at every crossing. Each ring stores one bit by which way round it is magnetized.
7. A floppy disk is a thin flexible plastic disc, coated like tape, spun inside a sleeve.
8. All of these are gone from daily life except tape, which is quietly bigger than ever.

PLAIN12.2.2 a picture in your head#

1. Picture a cassette tape and a vinyl record sitting next to each other.
2. The cassette must be wound forward to reach the middle of a song. To hear track eight you wait for tracks one to seven to go past.
3. The record lets you drop the needle straight onto track eight.
4. That is the whole difference between sequential access and random access, and it is the difference between tape and disk.
5. Tape is not slow because the ribbon moves slowly. Modern tape moves fast. Tape is slow to start because the right spot may be half a kilometre away along the ribbon.

Where this comparison breaks: a record needle can reach any groove in a second, while a disk head reaches any track in about ten milliseconds, a thousand times quicker. And nobody streams a record at 400 megabytes per second, which a modern tape drive does easily once it gets going.

PLAIN12.2.3 a worked example#

1. The standard IBM punched card, in its final form from 1928, has 80 columns.
2. One column holds one character. So one card holds 80 characters, that is 80 bytes.
3. A standard box holds 2,000 cards. That is 160,000 bytes, about 160 kB.
4. A single modern phone photo of 4 MB would need 25 boxes of cards.
5. Stacked, those 50,000 cards stand about nine metres tall and weigh roughly 130 kilograms.
6. Now the modern end. An LTO-9 cartridge holds 18 TB of raw data in a plastic box the size of a thick paperback.
7. That is 18,000,000 MB, or about 4.5 million phone photos, or 225 million punched cards.

PLAIN12.2.4 what is really happening inside#

1. On tape and on disk the recording trick is identical. A coil of wire carries a current, the current makes a magnetic field, and the field lines up the magnetic particles in the coating underneath.
2. Reverse the current and the particles line up the other way. Two directions, two symbols, one bit.
3. Nothing holds them there. They stay because the coating is chosen to be hard to demagnetize once set.
4. Core memory is the same physics with a different shape. Each ring can be magnetized clockwise or anticlockwise. Wires threaded through the ring set it and sense it.
5. Reading a core is destructive: to find out which way it was, you force it one way and see whether it flipped. If it flipped, it was the other way. The machine must then write the value back.
6. Every core in early machines was threaded by hand, by workers with needles and microscopes. Memory was literally woven.
7. A drum stores data on the outside of a spinning cylinder, with one fixed head per track. No head has to move, so the only wait is for the drum to come round. That made drums fast for their day and small in capacity.

TECHNICAL12.2.5 the engineer’s version#

1. Gustav Tauschek patented magnetic drum memory in Austria in 1932. His drum held about 500,000 bits, roughly 62.5 kB.
2. Jay Forrester at MIT developed practical coincident-current magnetic core memory around 1949 to 1953; the Whirlwind I machine received core in 1953. An Wang and Way-Dong Woo filed related work in 1949.
3. The UNIVAC I UNISERVO of 1951 was the first tape drive on a commercial computer. Its tape was nickel-plated phosphor bronze, 128 characters per inch, about 12,800 characters per second.
4. IBM’s 726 tape unit of 1952 moved the industry to plastic tape at 100 bits per inch and about 7,500 characters per second.
5. Floppy disk lineage: IBM 23FD, 8-inch and read-only, 1971, about 80 kB. IBM 33FD, 8-inch read/write, 1973, 242.9 kB. Shugart SA400, 5.25-inch, 1976, about 110 kB. Sony’s 3.5-inch design from 1981 settled at 720 kB double-density and 1.44 MB high-density from 1987.

6. Linear Tape-Open (LTO) is the surviving tape standard, an open format run by HP, IBM and Quantum. It is a genuine multi-vendor standard, not one company’s product.

7. LTO-9 and LTO-10 both run at 400 MB/s native. Marketing figures of 45 TB and 75 TB assume 2.5 to 1 hardware compression, which real data rarely reaches. Compressed figures are a marketing claim; native figures are the standard.
8. LTO-10 was announced on 13 August 2025 at 30 TB. In November 2025 the program raised it to 40 TB using an aramid-based tape substrate, and simultaneously cut its long-range roadmap: LTO-11 to 70 TB, LTO-12 to 120 TB, LTO-13 to 210 TB, LTO-14 to 365 TB.
9. Tape survives because of cost, air gap and shelf life. Media runs roughly 6 to 10 USD per TB against 21 to 32 USD for disk, a cartridge on a shelf cannot be reached by ransomware, and vendors rate media for about 30 years. The catch is the drive, which costs several thousand US dollars, so tape only pays off above roughly 100 TB of archive.
10. Tools: mt and mt-st to control a tape device, tar and LTFS to write it. LTFS, standardized as ISO/IEC 20919, makes a tape look like a filesystem.

WORDS12.2.6 remember these#

1. Sequential access — you must go past everything in between — media where access time is proportional to distance along the medium.
2. Random access — you can jump straight there — access time roughly independent of the previous position.
3. Core memory — memory woven from magnetic rings — coincident-current ferrite-core store, destructive read with write-back.
4. LTO — the modern tape standard — Linear Tape-Open, open multi-vendor format, generation 10 at 30 to 40 TB native.
5. Air gap — the archive is physically unplugged — offline copy unreachable by any network attacker.
6. Native capacity — the honest number — raw bytes stored with no compression assumed.

12.3 The hard disk drive, in full#

PLAIN12.3.1 in simple words#

1. A hard disk is a stack of smooth metal or glass discs that spin very fast.
2. Each disc is coated with a magnetic film. That film is where the bits live.
3. A small arm swings across the discs, like the arm of a record player.
4. On the tip of the arm sits a head, which both writes and reads.
5. To write, the head makes a tiny magnetic field that flips a patch of the coating to point one way or the other.
6. To read, the head senses those patches as they rush past underneath.
7. The head never touches the disc. It flies above it on a cushion of air that the spinning disc drags along with it.
8. The gap is a few nanometres. That is smaller than a virus, thousands of times thinner than a hair.
9. If the head touches the disc at full speed, it gouges the coating. That is a head crash, and the data in that path is gone forever.

PLAIN12.3.2 a picture in your head#

1. Imagine a jumbo jet flying at full cruising speed, but only one millimetre above the ground, counting blades of grass as it goes.
2. Now scale that picture down to the size of a coin, and you have a disk head.
3. The head is a small block, roughly the size of a grain of salt cut in four.
4. The disc under it moves at around 100 kilometres per hour at the outer edge.
5. The arm can swing from the middle to the edge and stop on a chosen track in about eight thousandths of a second.
6. The whole thing is sealed, because a single speck of dust is a boulder at that height.

Where this comparison breaks: an aircraft is held up by its own wings and its own engines, while a disk head is a passive glider held up entirely by air the disc drags along. Slow the disc and the head lands. And the head is not merely looking at the ground, it is also repainting it as it goes.

PLAIN12.3.3 a worked example#

1. Take a common 7,200 RPM desktop drive. RPM means revolutions per minute.
2. 7,200 revolutions per minute is 120 revolutions per second.
3. One revolution therefore takes 1 divided by 120, which is 8.33 milliseconds.
4. Suppose the data you want is on the track under the head already. On average you wait half a turn for it to come round: 4.17 milliseconds.
5. Now suppose it is on a different track. The arm must move. That takes about 8.5 milliseconds on average for a desktop drive.
6. Total wait: about 12.7 milliseconds before a single byte arrives.
7. In those 12.7 milliseconds a modern CPU at 4 GHz executes roughly 50 million clock cycles. The processor is waiting an eternity.

PLAIN12.3.4 what is really happening inside#

1. The magnetic film is made of grains. Each grain is a tiny permanent magnet with a preferred direction it likes to point.
2. A group of grains all pointing the same way is a magnetic domain. One recorded bit uses tens of grains, not one. Using many grains averages out the noise so the read-back is reliable.
3. Writing: current through a coil in the head makes a field at a narrow gap. The field is strong enough to force the grains under it to flip direction.
4. Reading: here is the part that surprises people. The head does not sense “north” or “south”. It senses changes.
5. As the disc moves, wherever the magnetization reverses, there is a small stray field. The read sensor detects those reversals.
6. So the drive does not store 1 as north and 0 as south. It stores a stream of reversals, and an encoding scheme turns reversal patterns into bits.
7. That is why a long run of identical bits is a problem: no reversals means nothing to sense, and the drive loses its place. Encodings deliberately guarantee frequent reversals.
8. The head assembly is moved by a voice coil: a coil of wire sitting in a strong permanent magnet, exactly like the motor in a loudspeaker. Push current through it and the arm swings. Reverse it and the arm swings back.
9. The disc stack is turned by a spindle motor running on a fluid bearing, which is quieter and more precise than balls.

TECHNICAL12.3.5 the engineer’s version#

1. The IBM 350 Disk Storage Unit, part of the IBM 305 RAMAC, was announced on 14 September 1956. It held 5 million six-bit characters across fifty 24-inch discs, with an average access time of 600 ms. It weighed over a ton and rented for about 3,200 US dollars a month.
2. In 1956 that is roughly 3.75 MB in modern eight-bit bytes, in a cabinet the size of two large refrigerators, delivered by cargo aircraft and moved with forklifts.
3. Flying height fell from about 20 micrometres in 1956 to roughly 1 to 3 nanometres today. Head sliders shrank through “pico” (1.25 by 1.0 by 0.3 mm) to “femto” (0.85 by 0.7 by 0.23 mm) classes.
4. The read sensor changed three times. Inductive coils gave way to anisotropic magnetoresistance, then to giant magnetoresistance (GMR), discovered independently in 1988 by Albert Fert in France and Peter Grunberg in Germany. IBM shipped the first GMR-head drive, the Deskstar 16GP, in 1997. Fert and Grunberg shared the 2007 Nobel Prize in Physics for the discovery. Modern heads use tunnelling magnetoresistance, a further refinement.
5. GMR is a genuine case of a physics discovery reaching a shipping product in nine years, and it is why disk capacity grew so fast in the late 1990s.
6. Recording orientation changed too. Longitudinal recording laid magnets flat, in the plane of the disc. Perpendicular magnetic recording (PMR) stands them on end, packing them tighter and resisting thermal decay. Toshiba shipped the first PMR drive in 2005 and Seagate followed in 2006. All modern drives are perpendicular.
7. Shingled magnetic recording (SMR) overlaps write tracks like roof tiles, because a write head is wider than a read head. It buys about 20 percent capacity at a severe cost: rewriting one track means rewriting the whole band. Drive-managed SMR hides this and can stall for seconds under sustained random writes. Western Digital shipping undisclosed SMR in Red NAS drives caused a public dispute in 2020, and the industry now labels it.
8. Heat-assisted magnetic recording (HAMR) puts a laser in the head. It heats a spot to roughly 400 to 450 degrees Celsius for about a nanosecond, which briefly makes a very stable, very small grain writable. Seagate’s Mozaic 3+ platform shipped from 2024, and Mozaic 4+ began shipping 44 TB drives with ten platters at over 4 TB each in March 2026. Western Digital pursued microwave-assisted recording (MAMR) instead.

9. Drives above about 8 TB are usually sealed and filled with helium, first shipped by HGST as the Ultrastar He6 in 2013. Helium is one seventh the density of air, so there is less turbulence and less drag, which allows more platters in the same height and cuts power.
10. Areal density is the headline metric, in bits per square inch. Production drives are now in the region of 1.5 to 2 terabits per square inch, and HAMR is the path beyond.
11. Tools: smartctl -a /dev/sda reads SMART attributes including reallocated sector count and head flying-height sensors on some models.

WORDS12.3.6 remember these#

1. Platter — one of the spinning discs — rigid aluminium or glass substrate with a sputtered magnetic film.
2. Head — the part that reads and writes — the slider carrying a write coil and a magnetoresistive read sensor.
3. Air bearing — the cushion the head flies on — the thin viscous film of gas dragged by the rotating platter, a few nanometres thick.
4. Magnetic domain — a patch of magnets all pointing the same way — a region of aligned grains representing part of one recorded symbol.
5. Voice coil actuator — the loudspeaker motor that swings the arm — rotary VCM under closed-loop servo control.
6. GMR — the effect that made big disks possible — giant magnetoresistance, Fert and Grunberg 1988, Nobel Prize 2007.
7. HAMR — heat the spot before writing it — heat-assisted magnetic recording, using a near-field laser to lower coercivity momentarily.
8. SMR — tracks overlapped like roof tiles — shingled magnetic recording, requiring band rewrites, sold as drive-managed or host-managed.

12.4 HDD geometry and performance#

PLAIN12.4.1 in simple words#

1. The surface of a disc is divided into rings called tracks.
2. Each track is chopped into short arcs called sectors. A sector is the smallest chunk the drive will read or write.
3. A sector used to hold 512 bytes. Modern drives use 4,096 bytes.
4. There are several platters stacked up, each with a surface on top and one underneath, each with its own head.
5. All the heads move together on one arm. So track 500 on every surface is reachable without moving the arm. That set of tracks is a cylinder.
6. To reach data the drive does three things: move the arm to the right track, wait for the right sector to come round, then transfer the data.
7. Moving the arm is seek time. Waiting for rotation is rotational latency. Only the third part actually moves data.
8. If your next request is right next to the last one, you skip the first two steps. That is why reading a file in order is fast.
9. If your next request is somewhere random, you pay both waits every time. That is why random access on a hard disk is brutally slow.

PLAIN12.4.2 a picture in your head#

1. Picture a very large car park laid out in concentric rings, with numbered bays around each ring.
2. You are on a moped in the middle. Someone radios you a bay number.
3. First you ride out to the right ring. That is the seek.
4. But the car park is on a turntable, spinning. You must wait for your bay to come round to you. That is the rotational latency.
5. Then you read the number plate as it passes. That is the transfer.
6. If the next twenty bays you need are the next twenty in the same ring, you just sit still and read them all as they go by. Wonderful.
7. If each next bay is in a random ring, you spend all day riding and waiting and almost no time reading.

Where this comparison breaks: a real drive also reads ahead, caches, and reorders queued requests to reduce total travel, so a clever drive is better than a moped rider taking orders one at a time. And the drive’s own controller hides a lot of this, so the outside world never sees the real geometry.

PLAIN12.4.3 a worked example#

1. Rotational latency is pure arithmetic. Average latency is the time for half a revolution.
2. Formula: average rotational latency in milliseconds equals 30,000 divided by the RPM.
3. At 5,400 RPM: 30,000 divided by 5,400 equals 5.56 ms.
4. At 7,200 RPM: 30,000 divided by 7,200 equals 4.17 ms.
5. At 10,000 RPM: 3.00 ms. At 15,000 RPM: 2.00 ms.
6. Now add a typical average seek. Take a 7,200 RPM drive with an 8.5 ms seek.
7. Average access time is 8.5 plus 4.17, about 12.7 ms.
8. Operations per second is 1 divided by 0.0127, about 79.
9. So a 7,200 RPM drive does roughly 80 random operations per second. That number has barely changed since 1995.

One random read on a 7200 RPM drive
seek 8.5 ms          rotate 4.17 ms      transfer 0.04 ms
|=================| |==========|         |
0                 8.5         12.7      12.74 ms
                                         ^ 4 KB actually moved here
99.7 percent of the time is waiting, not reading.

PLAIN12.4.4 what is really happening inside#

1. Old drives were addressed by cylinder, head, sector: tell the drive which ring, which surface, which arc. This is CHS addressing.
2. That worked while software could know the real geometry. It stopped working the moment drives started lying about their shape.
3. Drives had to lie because of a physical fact: outer tracks are longer than inner tracks. A fixed number of sectors per track wastes the outside.
4. So drives use zoned recording: more sectors per track near the edge, fewer near the middle. Geometry is no longer uniform, so CHS is fiction.
5. The fix is logical block addressing, LBA. The drive presents a simple numbered list of sectors, 0, 1, 2, and upward. The drive alone knows where block 4,000,000 physically is.
6. Zoned recording has a visible side effect: the outer tracks pass under the head faster, so they transfer more bytes per second. A drive is genuinely faster at the start of its address range than at the end, often by half.
7. Sectors are not just data. Each carries a preamble to synchronize timing, a servo pattern telling the head where it is, the data itself, and an error correcting code.
8. Moving from 512-byte to 4,096-byte sectors saves all that overhead per byte and allows a stronger error code. That is Advanced Format.

TECHNICAL12.4.5 the engineer’s version#

1. CHS addressing in the PC BIOS interrupt 13h interface allowed 1,024 cylinders, 256 heads and 63 sectors, capping capacity at 8.4 GB with 512-byte sectors. Various translation hacks pushed it further before LBA replaced it outright.
2. LBA-28 in ATA capped out at 2^28 sectors of 512 bytes, that is 128 GiB, usually quoted as 137 GB. LBA-48, introduced in ATA-6 in 2003, raises the cap to 2^48 sectors, about 128 PiB.
3. Advanced Format was defined by IDEMA and the industry transitioned from around January 2011. Two variants exist: 512e presents 512-byte logical sectors over 4,096-byte physical ones, and 4Kn presents 4,096 bytes both ways.
4. On a 512e drive, a misaligned 4 KB write triggers a read-modify-write of the physical sector, roughly halving small random write performance. Aligning partitions to a 1 MiB boundary, now the default in every modern partitioner, avoids this.
5. Sustained transfer rate depends on linear density and rotation. A modern 7,200 RPM 20 TB drive sustains roughly 260 to 290 MB/s on outer tracks and about 120 to 140 MB/s on inner tracks. Seagate quotes about 300 MB/s for the 44 TB Mozaic 4+ drives.

6. Native Command Queuing, part of SATA, lets a drive hold up to 32 outstanding commands and service them in an efficient order. It can raise random IOPS by 30 to 50 percent at high queue depth by reducing total head travel.
7. The gap between sequential and random is not subtle. A 20 TB drive doing 1 MB sequential reads delivers roughly 260 MB/s. The same drive doing 4 KB random reads at queue depth 1 delivers about 80 times 4 KB, that is roughly 0.32 MB/s. The ratio is about 800 to 1.
8. Measure it yourself:

# Sequential read throughput, 1 MB blocks
fio --name=seq --rw=read --bs=1M --size=4G \
    --filename=/dev/sdb --direct=1

# Random read IOPS, 4 KB blocks, one at a time
fio --name=rand --rw=randread --bs=4k --size=4G \
    --iodepth=1 --filename=/dev/sdb --direct=1

9. Inspect logical and physical sector size with lsblk -o NAME,PHY-SEC,LOG-SEC on Linux, or fsutil fsinfo sectorinfo C: on Windows.

WORDS12.4.6 remember these#

1. Track — one ring on the disc surface — a single concentric data path.
2. Sector — the smallest readable chunk — a physical block, 512 or 4,096 bytes plus servo, sync and ECC overhead.
3. Cylinder — the same track on every surface — the set of tracks reachable without moving the actuator.
4. Seek time — the wait while the arm moves — actuator settling time to a target track, quoted as an average over random seeks.
5. Rotational latency — the wait for the sector to come round — 30,000 divided by RPM in milliseconds, on average.
6. LBA — one long numbered list of blocks — logical block addressing, hiding real geometry behind a linear address space.
7. Advanced Format — 4K sectors instead of 512 — 512e or 4Kn physical sector formats defined by IDEMA.
8. IOPS — operations per second — input/output operations per second, always quoted with block size and queue depth or it is meaningless.

12.5 Flash memory, in full#

PLAIN12.5.1 in simple words#

1. Flash memory stores a bit by trapping electrons inside an insulator, where they cannot get out on their own.
2. A cell is a transistor with an extra hidden pocket built into it.
3. Put electrons in the pocket and the transistor becomes harder to switch on.
4. Leave the pocket empty and it switches on easily.
5. To read a cell you apply a test voltage and see whether it switches on. That tells you whether the pocket is full.
6. To write, you shove electrons through the insulator into the pocket using a high voltage.
7. To erase, you shove them all back out again, also with a high voltage.
8. The insulator is what makes it non-volatile. Electrons stay put with no power. The insulator is also what wears out, because forcing charge through it damages it a little every single time.
9. That is the whole of flash memory. Everything else is engineering around those two sentences.

PLAIN12.5.2 a picture in your head#

1. Think of a water tank with no tap and no drain, sealed on all sides.
2. Normally water cannot get in or out. That is your data sitting safe.
3. To fill it you use enormous pressure to force water straight through the wall of the tank. To empty it you do the same in reverse.
4. It works, but every time you do it the wall gets a little more damaged.
5. After enough fills and empties the wall starts to seep. Now the tank slowly loses water on its own, and a tank you filled last year no longer reads full.
6. That is exactly why flash memory has a limited number of writes, and why an old worn drive also forgets faster when left unplugged.
7. Now the reading part. You do not open the tank to check it. You judge the level from outside by how much it resists a push.
8. If the tank holds only “empty” or “full”, that judgement is easy. If you try to distinguish sixteen different water levels, it gets very hard, and small leaks matter enormously.

Where this comparison breaks: electrons are not water and do not slosh. The “pressure” is a quantum effect called tunnelling, where an electron simply appears on the far side of a thin barrier. And the damage is not a hole in a wall, it is charge getting stuck inside the insulating oxide itself.

PLAIN12.5.3 a worked example#

1. Cells are cheaper if you store more than one bit each, by using more charge levels. Here is what that costs you.

2. Going from SLC to QLC gives you four times the capacity from the same silicon. That is why every consumer drive is TLC or QLC.
3. But the voltage window is fixed. With 16 levels in the same window, each level is a narrow band, and a small charge leak pushes a cell into the neighbouring band and corrupts the value.
4. So endurance and speed collapse as bits per cell rise.

5. PLC is active research, not a shipping product as of 2026. Treat any PLC capacity claim as a roadmap, not a specification.

PLAIN12.5.4 what is really happening inside#

1. Cells are wired into a grid. Rows are called word lines, columns are called bit lines.
2. In NOR flash every cell hangs directly off a bit line, in parallel. You can read any single byte instantly, but each cell needs its own contact, so cells are large and the chip is expensive per byte.
3. In NAND flash the cells are wired in long series strings, like old Christmas lights. Far fewer contacts are needed, so cells pack much tighter.
4. The price of that is that you cannot touch one cell alone. To read one cell you must switch every other cell in its string fully on so current can pass through them.
5. So NAND works on whole rows at a time. A row of cells read together is a page. Pages are grouped into a block. Blocks are grouped into planes, and planes into a die.
6. Now the crucial asymmetry, the fact that shapes every SSD ever built:
   1. You can read one page.
   2. You can write one page, but only if it is currently erased.
   3. You cannot erase one page. You can only erase a whole block.
7. Writing pushes cells one direction only. Erasing is the only way back, and erasing is coarse.
8. So changing one byte in a page is impossible in place. The drive must write the new version of that page somewhere else entirely, and remember where.
9. That single restriction is why an SSD needs the complicated software we meet in section 12.6.
10. 3D NAND is the other big idea. Instead of shrinking cells sideways, which stopped working around 2013, manufacturers turned the strings on end and stacked layers vertically, then drilled holes through the stack.
11. This let them go back to a relaxed, reliable cell size and gain density by height instead. It is the single reason flash kept getting cheaper.

TECHNICAL12.5.5 the engineer’s version#

1. The floating-gate MOSFET was invented by Dawon Kahng and Simon Min Sze at Bell Labs in 1967. It is an ordinary MOS transistor with an electrically isolated polysilicon gate buried in the oxide between control gate and channel.
2. Fujio Masuoka at Toshiba invented flash memory around 1980. NOR flash was presented in 1984 and Intel shipped the first commercial NOR part in 1988. NAND flash was presented at the IEEE International Electron Devices Meeting in San Francisco in 1987. The name came from a colleague who thought the block erase resembled a camera flash.
3. Charge is moved by Fowler-Nordheim tunnelling for erase and either tunnelling or channel hot-electron injection for program, at internally generated voltages of roughly 15 to 20 V produced by on-chip charge pumps.
4. Trapped charge shifts the transistor threshold voltage. Reading is a sequence of sense operations at different word-line reference voltages; a TLC read needs seven reference points to separate eight states.
5. Charge trap flash replaced the conducting floating gate with an insulating silicon nitride layer that traps charge in discrete defect sites. Because the storage layer is not conductive, a single defect does not drain the whole cell, which is what makes tall 3D stacks practical. Samsung V-NAND has used charge trap from the start; Intel and Micron used a floating-gate 3D design for several generations before moving across.
6. Modern 3D NAND geometry, typical of 2024 to 2026 TLC parts:

7. Note the ratio: erasing a block costs roughly one hundred times a page read, and it destroys everything in that block. This is the reason garbage collection exists.
8. Layer counts as of 2026: SK hynix is in mainstream production at 321 layers, Samsung’s V9 generation is in the high 200s, Micron’s G9 is around 276, and Kioxia and SanDisk’s BiCS8 is 218. Samsung announced a 400-layer part for 2026 and demonstrated a 900-layer research die in May 2026 by bonding two 450-layer stacks. Stacks above roughly 128 layers are almost always built as two or more bonded decks, a technique called string stacking.
9. NOR flash did not die. It survives wherever code must be executed directly from the chip with no controller in the way: BIOS and UEFI firmware, boot ROMs, microcontrollers and automotive parts. NOR gives byte-level random read and execute-in-place; NAND gives density and cost per bit. NAND won storage, NOR kept firmware.
10. Inspect real flash geometry on Linux with nvme id-ctrl /dev/nvme0 and nvme id-ns /dev/nvme0n1, which report namespace sizes and, on drives that support it, optimal write size and NPWG or NPWA alignment hints.

WORDS12.5.6 remember these#

1. Floating gate — the hidden pocket that holds electrons — an isolated polysilicon gate whose stored charge shifts the transistor threshold.
2. Charge trap — the same job done by an insulator — a silicon nitride layer trapping charge in defect sites, standard in 3D NAND.
3. Tunnelling — electrons crossing a barrier they should not — Fowler-Nordheim tunnelling through the tunnel oxide under a high field.
4. Page — the smallest thing you can read or write — typically 16 KiB, plus spare area for error correction.
5. Block — the smallest thing you can erase — one to two thousand pages, several megabytes.
6. SLC, MLC, TLC, QLC — how many bits crammed into one cell — 1, 2, 3 and 4 bits, using 2, 4, 8 and 16 threshold levels.
7. P/E cycle — one wear event — a complete program and erase of a block, the unit in which flash lifetime is counted.
8. 3D NAND — stacking cells upward instead of shrinking them — vertical strings through a multi-layer stack, now over 300 layers.

12.6 The SSD controller and the flash translation layer#

PLAIN12.6.1 in simple words#

1. An SSD is not just flash chips in a box. It contains a small computer whose only job is to manage those chips.
2. That computer has its own processor cores, its own firmware and often its own memory.
3. It exists because of the asymmetry in the last section: the outside world wants to overwrite any block at any time, and flash cannot do that.
4. So the controller lies, politely and consistently. It tells your operating system “here is a simple numbered list of blocks you can overwrite freely”.
5. Underneath, it never overwrites anything. It writes each new version to a fresh place and updates a private map.
6. That map is the flash translation layer, or FTL. It converts the address your computer asked for into the physical page where the data really is.
7. The controller must also spread wear evenly, clean up rubbish in the background, and hide all of it from you.
8. Almost everything odd about SSD behaviour comes from that hidden work.

PLAIN12.6.2 a picture in your head#

1. Picture a large notebook where you are not allowed to use an eraser on one line. You can only tear out and rewrite a whole page at a time.
2. So when you want to change one line, you write the new version on the next blank line instead, and you keep an index at the front saying which line is the current one.
3. The old line is now rubbish, but it is still sitting there taking space.
4. Eventually the notebook fills with a mixture of current lines and rubbish lines scattered across every page.
5. To get space back you pick a page that is mostly rubbish, copy its few surviving lines onto a fresh page, update the index, and tear out the old page.
6. That tidying is garbage collection. It costs real work that nobody asked for, and it is why an SSD writes more than you told it to.
7. And you deliberately spread your writing across all pages rather than always using the front ones, so no page wears out first. That is wear levelling.

Where this comparison breaks: a real controller does this tidying while you are also writing, competing for the same chips, which is why performance can dip under sustained load. And the index itself must be kept safely, because losing it loses the whole drive, not one page.

PLAIN12.6.3 a worked example#

1. Here is write amplification with real arithmetic.
2. Take a block of 256 pages. 192 pages hold live data, 64 hold stale data.
3. You want to reclaim that block. First you must copy the 192 live pages elsewhere, then erase the block.
4. So to free 64 pages of space, the drive performed 192 page writes.
5. Now suppose your application writes 64 pages of new data into that freed space. Total flash writes: 192 copied plus 64 new equals 256.
6. Host writes: 64 pages. Flash writes: 256 pages.
7. Write amplification factor equals 256 divided by 64, which is 4.
8. Your drive is wearing out four times faster than your data volume suggests.
9. Now repeat the calculation on a drive that is only half full, where a typical block has 64 live pages and 192 stale.
10. Copy 64 live pages, free 192, then write 192 new. Flash writes 64 plus 192 equals 256, host writes 192, amplification is 1.33.
11. That is the whole explanation of why a full SSD is slow and short-lived and an empty one is fast. Free space is not idle. It is working capital.

PLAIN12.6.4 what is really happening inside#

1. The FTL keeps a table. Logical block number in, physical page location out.
2. Every write allocates a fresh page, writes there, and updates one table entry. Nothing is ever changed in place.
3. Wear levelling comes in two kinds. Dynamic levelling spreads new writes over blocks that are already free. Static levelling also moves data that has sat untouched for a long time, so that cold blocks take their share of wear. Without static levelling, a drive holding a large read-only archive would wear out the small remaining area very fast.
4. Over-provisioning is hidden spare capacity the controller keeps for itself. It never appears in your capacity figure. More spare space means garbage collection always has somewhere easy to work, which lowers write amplification.
5. TRIM solves a problem you would not guess at. When you delete a file, the filesystem just marks its blocks free in its own records. It does not tell the drive.
6. So the drive still believes every one of those pages holds live data, and dutifully copies them during garbage collection forever.
7. TRIM is a command that says “these logical blocks no longer contain anything you need to preserve”. Now the controller can drop them, and the pages become free for nothing.
8. Without TRIM, an SSD gradually behaves as if it were completely full, even when the filesystem shows it as empty.
9. SLC caching is why big copies slow down. TLC and QLC cells can be operated in one-bit mode, which is much faster. Controllers keep part of the flash in that mode as a fast landing zone.
10. Incoming writes land in the fast zone at full speed. Later, when the drive is idle, they are folded down into dense multi-bit storage.
11. If you write more than the cache holds, the cache runs out mid-copy and writes must go straight to slow storage, while folding also competes for the same chips. Speed drops sharply and visibly.
12. The FTL table is far too large to search through, so the controller keeps it in fast memory. Either a dedicated DRAM chip on the drive, or a slice of your computer’s own memory borrowed over the interface.

TECHNICAL12.6.5 the engineer’s version#

1. Typical controllers are multi-core embedded designs. Phison E26 and E28, Silicon Motion SM2508, and Samsung’s in-house controllers use two to five ARM Cortex-R class cores plus hardware engines for ECC, encryption and DMA.
2. Error correction is not optional. Modern TLC and QLC require LDPC (low-density parity-check) decoding with soft-decision reads. Raw bit error rates in the 10^-3 range are corrected down to below 10^-15.
3. Mapping granularity drives the DRAM requirement. A page-mapped FTL with 4 KiB granularity and a 4-byte entry needs about 1 GB of DRAM per 1 TB of NAND. That is the origin of the industry rule of thumb.
4. Host Memory Buffer (HMB) is an NVMe feature that lets a DRAM-less drive borrow host RAM, typically 32 to 64 MB. It holds only a hot fraction of the map, so DRAM-less drives are competitive on light desktop work and clearly worse on sustained random writes.
5. Over-provisioning has two sources. First, the decimal versus binary gap: a drive sold as 1 TB (10^12 bytes) built from 1 TiB (1.0995 times 10^12 bytes) of NAND has 7.37 percent spare for free. Enterprise parts add more deliberately, which is why capacities read 960 GB, 1,920 GB or 3,840 GB instead of round powers.

6. TRIM is the ATA DATA SET MANAGEMENT command with the deallocate bit; the SCSI equivalent is UNMAP, and NVMe uses Dataset Management with the deallocate attribute. Windows 7 (2009) was the first mainstream OS to issue it, Linux gained support in kernel 2.6.33 (2010), and macOS enabled it for Apple SSDs from 10.6.8 (2011).
7. Two ways to issue it on Linux: continuous with the discard mount option, or batched on a timer, which is now the preferred default.

# See whether the device reports discard support
lsblk -D            # DISC-GRAN and DISC-MAX columns

# Run a batched trim now, and report bytes released
sudo fstrim -av

# Check the systemd timer that normally does this weekly
systemctl status fstrim.timer

8. SLC cache sizing is an implementation detail that varies per model. A typical 2 TB consumer QLC drive might sustain 5,000 MB/s for the first 150 to 250 GB and then fall to 100 to 450 MB/s. TLC drives usually fall to 900 to 1,700 MB/s instead, which is far less painful. Reviewers call this the post-cache or steady-state write speed, and it is the number that matters for video work and large backups.
9. Power-loss protection in enterprise drives is a bank of tantalum or ceramic capacitors sized to flush the volatile write buffer and the mapping table to NAND after supply is cut. Consumer drives almost never have it. Without PLP, a drive may honestly acknowledge a write that is still in volatile buffer, which is why databases insist on explicit cache flushes.
10. Observe the drive’s own accounting with nvme smart-log /dev/nvme0, which reports data_units_written, percentage_used and unsafe_shutdowns. On drives exposing vendor logs, smartctl -a may also show host writes versus NAND writes, from which you can compute a real write amplification factor.

WORDS12.6.6 remember these#

1. FTL — the private map from your addresses to real chips — flash translation layer, maintaining logical-to-physical mapping and block state.
2. Wear levelling — spread the damage evenly — dynamic and static algorithms equalizing program/erase counts across blocks.
3. Garbage collection — tidying up to reclaim erased blocks — relocating valid pages out of victim blocks before erasing them.
4. Write amplification — the drive writes more than you asked — WAF, the ratio of NAND writes to host writes.
5. Over-provisioning — hidden spare room — reserved capacity invisible to the host, lowering WAF and raising sustained performance.
6. TRIM — telling the drive a block is now rubbish — the deallocate hint, ATA DATA SET MANAGEMENT, SCSI UNMAP, NVMe Dataset Management.
7. SLC cache — a fast landing strip made of slow flash — a region operated in pseudo-SLC mode, later folded into TLC or QLC.
8. HMB — the drive borrows your computer’s memory — Host Memory Buffer, an NVMe feature used by DRAM-less controllers.
9. Power-loss protection — capacitors that finish the job — onboard energy reserve flushing volatile buffers on supply loss.

12.7 SSD endurance and failure#

PLAIN12.7.1 in simple words#

1. Every flash block can only be erased and rewritten a limited number of times.
2. When a block’s insulator is too damaged to hold charge reliably, the controller retires it and uses a spare.
3. When the spares run out, the drive stops accepting writes.
4. Manufacturers publish this as terabytes written, or TBW: how much data you may write over the warranty period.
5. Reading is nearly free, but not entirely. Reading a page slightly disturbs its neighbours, so after a great many reads a block must be refreshed.
6. A flash cell also leaks charge slowly. An unpowered SSD gradually forgets, over months to years, and forgets faster if it is worn or hot.
7. Hard drives usually die gradually and noisily. SSDs usually die suddenly and silently.
8. That difference matters more for your backup plan than any speed figure in this chapter.

PLAIN12.7.2 a picture in your head#

1. Think of a hard drive as an old car with 300,000 kilometres on it. It squeaks, it leaks, warning lights come on, one cylinder misfires. You get weeks of notice that something is wrong.
2. Think of an SSD as a light bulb. It works perfectly, perfectly, perfectly, and then it does not work at all, with no warning of any kind.
3. Most hard drive failures leave the platters intact and let a recovery lab read the data off with a new head assembly.
4. Most SSD failures are controller failures. The flash chips are fine and completely unreadable, because only that controller knew the map.

Where this comparison breaks: plenty of hard drives also die instantly, from a seized motor or a snapped head, and plenty of SSDs fail gracefully by dropping into a read-only mode that lets you copy everything off. These are tendencies, not laws.

PLAIN12.7.3 a worked example#

1. Is a consumer SSD’s endurance rating actually a problem for you? Let us find out with arithmetic.
2. A 2 TB Samsung 990 Pro is rated at 1,200 TBW over a five-year warranty.
3. 1,200 TB divided by 5 years is 240 TB per year.
4. 240 TB divided by 365 days is about 658 GB per day, every single day, for five years.
5. Typical desktop use is 10 to 40 GB per day. At 30 GB per day you would need 1,200,000 divided by 30, that is 40,000 days, about 109 years.
6. So for ordinary use, endurance is not the thing that will kill your drive. Controller failure, firmware bugs and power events are far more likely.
7. Now the other case. A database server writing 2 TB per day reaches 1,200 TBW in 600 days, under two years. This is exactly why enterprise drives are rated in drive writes per day instead.
8. Sanity check the rating from first principles. TBW is roughly capacity times P/E cycles divided by write amplification. 2 TB times 1,500 cycles divided by 2.5 gives 1,200 TB. The published figure is consistent with TLC flash of around 1,500 cycles at a write amplification of about 2.5.

PLAIN12.7.4 what is really happening inside#

1. Wear is not uniform damage across a chip. It is charge getting trapped inside the tunnel oxide, which shifts and blurs the voltage levels.
2. As levels blur, the error correction has to work harder. Reads take longer because the controller must retry at different reference voltages.
3. A worn drive therefore gets slower before it fails, especially on reads of old data.
4. Read disturb: reading one page requires putting a raised voltage on every other word line in the string. That nudges charge into neighbouring cells. After roughly one hundred thousand to a million reads of one block, the controller must copy the block elsewhere and erase it.
5. Retention: a powered drive can refresh weak blocks in the background. An unpowered drive cannot. The clock is running on every cell.
6. Retention halves roughly for every 10 degrees Celsius of extra storage temperature, and falls sharply once the drive is near its rated cycle count.
7. So a brand new SSD in a cool drawer will hold data for many years. A heavily worn SSD in a hot loft may not survive one summer.
8. If you use SSDs for cold archives, power them on and read them through once or twice a year. Or use disks or tape, which do not have this failure mode.

TECHNICAL12.7.5 the engineer’s version#

1. JEDEC standard JESD218 defines the endurance and retention test method. Client SSDs must retain data for 1 year at 30 degrees Celsius after reaching rated endurance; enterprise SSDs must retain for 3 months at 40 degrees. Both are end-of-life figures, not new-drive figures.
2. Client drives are rated in TBW; enterprise drives in DWPD, drive writes per day, over a stated warranty term. Conversion: TBW equals DWPD times capacity in TB times 365 times years.
3. A 1 DWPD read-intensive 3.84 TB drive over 5 years is 1 times 3.84 times 365 times 5, that is 7,008 TBW. A 3 DWPD mixed-use part of the same size is 21,024 TBW.
4. SMART attributes worth watching. On SATA SSDs: 5 Reallocated Sector Count, 177 Wear Leveling Count, 231 SSD Life Left, 241 Total LBAs Written. On NVMe: percentage_used, available_spare, available_spare_threshold, media_errors, unsafe_shutdowns. percentage_used can legitimately exceed 100, which means the drive has passed its rated endurance, not that it failed.
5. Backblaze’s published quarterly drive statistics report hard drive annualized failure rates generally in the range of about 1.3 to 1.6 percent across their fleet, with SSD boot drives comparable at similar ages. The often-repeated claim that SSDs are simply more reliable than HDDs is not well supported by that data; the failure modes differ more than the rates do.
6. Google and Facebook field studies published in 2016 found that SSD uncorrectable error rates correlate more strongly with age and with the number of blocks already retired than with raw bytes written, and that early life has a distinct higher-error period. Experts disagree on how far those findings, drawn from older MLC fleets, transfer to modern TLC and QLC parts.

7. Failure mode summary. HDD: gradual, dominated by media defects, reallocated sectors and head or motor wear, usually preceded by SMART warnings and often partially recoverable. SSD: bimodal, either slow wear-out ending in a read-only state, or abrupt controller or firmware failure with total loss and essentially no consumer recovery path.
8. Firmware bugs are a real and repeated cause of sudden SSD loss. Several vendors have shipped defects that brick drives at a specific power-on-hours count, notably a family of HPE SAS SSDs in 2019 and 2020 that failed at 32,768 and again at 40,000 hours. Check for firmware updates on drives you depend on.

WORDS12.7.6 remember these#

1. TBW — how much you may write before the warranty ends — terabytes written, the client endurance rating.
2. DWPD — how many times you may rewrite the whole drive daily — drive writes per day over a stated warranty term.
3. Read disturb — reading damages the neighbours a little — cumulative disturbance requiring periodic block refresh.
4. Retention — how long it remembers unplugged — data retention time, specified by JESD218 at end of rated life and temperature dependent.
5. Available spare — how many replacement blocks are left — the NVMe SMART field that actually predicts end of life.
6. Bimodal failure — either slow decline or instant death — the two distinct SSD failure populations, wear-out and controller fault.

12.8 How storage connects: interfaces and buses#

PLAIN12.8.1 in simple words#

1. A drive is useless until it is wired to the computer. That wiring is an interface: a cable, a connector and a language.
2. The language matters as much as the cable. It sets how many requests can be in flight at once.
3. Old drives were slow, so the language only allowed one request at a time. Nobody minded, because the disk was the bottleneck anyway.
4. Then SSDs arrived and could serve thousands of requests at once, and the polite one-at-a-time language became the thing holding everything back.
5. So the industry wrote a new language, NVMe, designed for a device with no moving parts, and connected drives straight to the processor’s own fast bus.
6. That bus is PCI Express, usually shortened to PCIe. It is the same bus a graphics card uses.
7. Everything else on this page is a variation on those two ideas: how many wires, and how good the language is.

PLAIN12.8.2 a picture in your head#

1. Picture a warehouse counter with one clerk who takes one order, fetches it, comes back, and only then takes the next order. That is the old ATA and SATA model with a spinning disk behind it.
2. Now put a thousand robots in the warehouse. The clerk is still taking one order at a time, so 999 robots stand idle.
3. NVMe replaces the clerk with thousands of order slots that customers fill in themselves, and the robots pick work out of the slots as they become free.
4. Each processor core gets its own order slot, so cores never queue behind each other.

Where this comparison breaks: real NVMe queues live in the host’s memory and the drive fetches from them by direct memory access, so there is no clerk at all. And the drive can complete work out of order, which the picture of a queue does not capture.

PLAIN12.8.3 a worked example#

1. A PCIe lane is one pair of wires each way. Drives usually use four lanes, written x4.
2. Each generation roughly doubles the speed per lane.

3. Compare that with SATA 3.0 at 6 gigabits per second. After encoding overhead that is about 550 to 560 MB/s in practice.
4. So a PCIe 5.0 x4 slot carries roughly 28 times what a SATA cable carries.
5. Yet a fast NVMe drive reaching 14 GB/s is only about 90 percent of what its Gen5 x4 link allows. The drive, not the bus, is now the limit again.

PLAIN12.8.4 what is really happening inside#

1. The oldest common interface was IDE, later renamed PATA: a wide 40-pin ribbon cable carrying 16 bits side by side, with two drives per cable configured as master and slave by jumper.
2. Parallel wiring failed as speeds rose, because 16 signals never arrive at exactly the same instant. That mismatch is called skew.
3. SATA solved it by sending one bit stream very fast down a single differential pair each way, with one drive per cable and no jumpers.
4. SAS is the enterprise cousin: same cabling idea, richer command set from SCSI, two ports per drive so two servers can reach it, and expanders so one controller can address thousands of drives.
5. NVMe threw away the disk-shaped assumptions entirely. There is no cable protocol to translate, no single command queue, and no controller chip sitting between drive and CPU.
6. Physically, an NVMe drive is usually an M.2 stick: 22 mm wide, most often 80 mm long, hence the name 2280.
7. The notch cut into the M.2 edge connector is the key, and it prevents you fitting a card the socket cannot drive. An M-key socket carries PCIe x4; a B-key socket carries at most PCIe x2 or SATA.
8. In phones and cheap tablets the flash is soldered down and uses eMMC or UFS instead, which are simpler and lower powered.

TECHNICAL12.8.5 the engineer’s version#

1. IDE was created by Western Digital with Compaq in 1986 and standardized as ATA-1 in 1994. Its fastest mode, Ultra DMA 133, reached 133 MB/s and required an 80-conductor cable with ground wires interleaved to control crosstalk.
2. SATA revision 1.0 arrived in 2003 at 1.5 Gbit/s, 2.0 in 2004 at 3 Gbit/s, and 3.0 in 2009 at 6 Gbit/s. All three use 8b/10b encoding, so 6 Gbit/s yields 600 MB/s of raw payload and about 550 MB/s after protocol overhead. SATA has had no speed increase since 2009; it is a finished standard.
3. AHCI, the standard SATA host controller interface, provides one command queue of at most 32 commands, and each command costs several uncached register accesses. NVMe provides up to 65,535 I/O queues of up to 65,536 commands each, with doorbell registers written at most twice per command.
4. NVMe 1.0 was published on 1 March 2011; the current revision is NVMe 2.3, published in August 2025. The specification family split into base, command set and transport documents at NVMe 2.0 in 2021.
5. PCIe encoding changed twice. Generations 1 and 2 used 8b/10b, giving a 20 percent tax. Generations 3 to 5 use 128b/130b, about 1.5 percent. PCIe 6.0 moved to PAM4 signalling with fixed-size FLIT framing and forward error correction. PCI-SIG released the PCIe 7.0 specification in 2025 at 128 GT/s per lane, roughly 15.1 GB/s per lane; products are not yet shipping.
6. M.2 lengths in use are 2230, 2242, 2260, 2280 and 22110, the last two digits or three being the length in millimetres. Handheld consoles and thin laptops often take only 2230.
7. Larger form factors: U.2, defined by SFF-8639, puts NVMe in a 2.5-inch hot-swappable drive. U.3, defined by SFF-TA-1001, accepts SAS, SATA and NVMe in one bay. Datacentres increasingly use EDSFF rulers such as E1.S and E3.S, which cool better and pack more capacity per rack unit.

8. Thunderbolt 5 and USB4 version 2 raise the link to 80 Gbit/s symmetric, with Thunderbolt 5 able to run 120 Gbit/s in one direction for displays. External NVMe enclosures should use the UASP protocol, not the older BOT mass-storage protocol, or they lose command queuing and most of their speed.
9. Mobile storage: eMMC 5.1 uses an 8-bit parallel bus and reaches roughly 250 to 400 MB/s. UFS uses a serial MIPI M-PHY link with full command queuing. UFS 4.0, published in 2022, reaches about 4,200 MB/s sequential read, and UFS 4.1 followed in 2025. eMMC is effectively obsolete in flagship phones and survives in low-cost devices and embedded boards.
10. Useful commands: lspci -vv to see link speed and width negotiated, nvme list and nvme id-ctrl, smartctl -a, and on macOS system_profiler SPNVMeDataType.

# What PCIe generation and width did the SSD actually get?
sudo lspci -vv -s $(lspci | grep -i nvme | cut -d' ' -f1) \
  | grep -E 'LnkCap|LnkSta'
# LnkCap says what it can do, LnkSta says what it negotiated.

WORDS12.8.6 remember these#

1. PATA / IDE — the old wide ribbon cable — parallel ATA, 16-bit, two devices per channel, up to 133 MB/s.
2. SATA — one fast serial pair per direction — serial ATA, 6 Gbit/s since 2009, driven through AHCI with a 32-command queue.
3. SAS — the enterprise serial interface — Serial Attached SCSI, dual ported, expander addressable, 22.5 Gbit/s at SAS-4.
4. NVMe — the language written for flash — NVM Express over PCIe, up to 65,535 queues of 65,536 commands.
5. PCIe lane — one wire pair each way — a differential lane, doubling in rate roughly every generation.
6. M.2 — the small stick shaped drive — 22 mm wide card, keyed to advertise which signals its socket carries.
7. UFS — the phone’s fast storage — Universal Flash Storage, serial and queued, about 4,200 MB/s at UFS 4.0.

12.9 What a filesystem is#

PLAIN12.9.1 in simple words#

1. A drive does not know what a file is. It only offers a very long numbered list of fixed-size blocks.
2. Humans want named things, in folders, that can grow and shrink.
3. A filesystem is the software that turns one into the other.
4. It has to answer four questions at all times: which blocks are free, which blocks belong to which file, what each file is called, and what else we know about it.
5. The information about a file, as opposed to the file’s contents, is called metadata: size, owner, permissions, dates.
6. A folder is not a special kind of object. It is just a file whose contents are a list of names and pointers.
7. In the Unix family, the record holding a file’s metadata and the addresses of its data blocks is called an inode.
8. The surprise for most people: the file’s name is not in the inode. The name lives in the folder. The inode does not know what it is called.

PLAIN12.9.2 a picture in your head#

1. Think of a hotel. The rooms are numbered blocks on the disk.
2. The inode is the guest’s registration card. It records who they are, when they arrived, and which rooms they occupy.
3. The directory is the list at reception mapping the name “Patel” to registration card number 4,192.
4. Two names can point at the same card. That is a hard link: one guest, two names at reception, one bill.
5. Deleting a name only crosses it off the reception list. The card is only destroyed when the last name pointing to it is gone.
6. That is exactly how Unix deletion works, and why the system call is called unlink rather than delete.

Where this comparison breaks: a hotel guest knows their own name, but an inode genuinely does not contain its filename. And renaming a real guest requires nothing but a pen at reception, which is precisely why renaming a file is fast no matter how large the file is.

PLAIN12.9.3 a worked example#

1. Here is what happens on an ext4 filesystem when you run four commands.

touch notes.txt
  1. allocate a free inode, say number 4192
  2. write inode: mode, owner, size 0, timestamps
  3. add entry ("notes.txt" -> 4192) to the directory file
  4. mark inode 4192 used in the inode bitmap

write 10 KB into it
  1. allocate 3 data blocks of 4 KB from the block bitmap
  2. write the bytes into those blocks
  3. record the extent (start block, length 3) in the inode
  4. set inode size = 10240, update mtime

mv notes.txt diary.txt
  1. add ("diary.txt" -> 4192) to the directory
  2. remove ("notes.txt" -> 4192)
  3. no data block is touched at all

rm diary.txt
  1. remove the directory entry
  2. decrement the inode link count, now 0
  3. free the 3 data blocks in the bitmap
  4. free inode 4192 in the inode bitmap
  5. the 10 KB of bytes are still physically there

2. Notice step 5 of the delete. Nothing overwrote your data. Only the bookkeeping changed. That is the whole basis of file recovery.
3. Notice also that renaming a 50 GB video is instant, because a rename edits a directory entry and never touches the data.

PLAIN12.9.4 what is really happening inside#

1. Blocks on disk are 512 or 4,096 bytes, but filesystems usually group them into a larger unit, called a block in Unix and a cluster in the Windows world. 4 KB is the common default.
2. A file always occupies a whole number of clusters. A 1-byte file consumes a full 4 KB. The wasted remainder is internal fragmentation, or slack.
3. Free space must be tracked. Three classic methods:
   1. A bitmap, one bit per block. Simple and compact. Used by ext4 and NTFS.
   2. A linked chain, where each entry names the next. Used by FAT.
   3. Extents, recording start and length rather than every block, which is far more compact for big files. Used by ext4, NTFS, XFS, APFS.
4. Metadata typically includes file type, permission bits, owner and group IDs, size, link count, and three or four timestamps.
5. A directory is a file whose contents are (name, inode number) pairs. Modern filesystems index those pairs with a hash or a B-tree so that a directory with a million files is still fast to search.
6. Path lookup is therefore recursive. To open /home/ana/a.txt, the kernel reads the root directory to find home, reads that to find ana, reads that to find a.txt, then reads that inode. Each step is a real disk lookup, cached aggressively in memory.
7. Mounting attaches the root of one filesystem onto a directory of another, so the user sees one seamless tree.

TECHNICAL12.9.5 the engineer’s version#

1. A classic Unix inode is 128 or 256 bytes and holds mode, uid, gid, size, link count, timestamps and either direct block pointers or an extent tree root. ext4 uses 256-byte inodes by default, which leaves room for nanosecond timestamps and inline extended attributes.
2. The original Unix File System used 12 direct pointers plus single, double and triple indirect blocks. With 4 KB blocks that reaches about 4 TB per file but costs up to four extra reads for a distant block. Extents replaced it because one extent record can describe up to 128 MiB in ext4.
3. Inodes are allocated at format time in ext2, ext3 and ext4. mke2fs defaults to roughly one inode per 16 KB of space, so a filesystem can run out of inodes while showing free space. df -i shows this, and it is a classic incident on mail and cache servers full of tiny files.
4. Filesystem block size is fixed at format time in most designs. XFS and ext4 are limited to the CPU page size, so 4 KiB on x86-64; APFS uses 4 KiB blocks; ZFS uses a variable record size up to 1 MiB.

stat notes.txt          # inode number, links, size, blocks, times
df -i /                 # inodes used and free, not just bytes
du -sh --apparent-size .   # real bytes, versus du -sh block usage
ls -di /home            # inode number of a directory
debugfs -R "stat <4192>" /dev/sda1   # raw inode dump, ext4

5. POSIX guarantees that rename() within one filesystem is atomic: an observer sees either the old name or the new one, never neither and never both. This is the foundation of the write-to-temporary-then-rename idiom used by nearly every well-written program that updates a file safely.
6. Hard links share an inode and cannot cross filesystems or, in most systems, point at directories. Symbolic links are separate inodes containing a path string, resolved at every use, and may dangle.

WORDS12.9.6 remember these#

1. Filesystem — the software that turns blocks into files — the on-disk format plus the driver implementing it.
2. Block or cluster — the allocation unit — the smallest number of sectors the filesystem will hand to a file, usually 4 KiB.
3. Metadata — everything about a file except its contents — mode, ownership, size, timestamps, link count, block map.
4. Inode — the file’s record card — a fixed-size structure holding metadata and the map to data blocks, identified by number, not by name.
5. Directory — a file listing names — a mapping from names to inode numbers, usually indexed by hash or B-tree.
6. Extent — one run of consecutive blocks — a (start, length) record replacing per-block pointers.
7. Hard link — a second name for the same file — an additional directory entry referencing the same inode, raising its link count.
8. Slack — the wasted tail of the last cluster — internal fragmentation from rounding file size up to a whole allocation unit.

12.10 Real filesystems compared#

PLAIN12.10.1 in simple words#

1. There is no single filesystem. Every operating system grew its own, and they all still exist because removable drives have to be readable everywhere.
2. FAT is the oldest survivor. Simple, small, understood by every device from cameras to car stereos, and very limited.
3. exFAT is FAT modernized to remove the size limits, and is now the normal choice for a big memory card or a shared USB stick.
4. NTFS is the Windows filesystem: journaled, permissioned, and full of extra features.
5. ext4 is the standard Linux filesystem: reliable, fast, unexciting, and used on most servers on the internet.
6. APFS is Apple’s, built for flash, and very good at making instant copies.
7. Btrfs and ZFS are the careful ones. They check every block against a stored fingerprint and can repair damage by themselves if you gave them a spare copy.
8. The rule of thumb: use your operating system’s native filesystem for internal drives, and exFAT only for drives that must travel between systems.

PLAIN12.10.2 a picture in your head#

1. Think of a photocopier that is clever enough to be lazy.
2. You ask for a copy of a 500-page report. Instead of copying anything, it writes a note saying “this copy is the same as the original”.
3. The copy appears instantly and takes no shelf space.
4. Later you edit page 12 of the copy. Only then does it actually duplicate page 12, and only page 12. The other 499 pages stay shared.
5. That is copy-on-write, and it is what APFS, ZFS and Btrfs do. It is why duplicating a 50 GB folder on a Mac can finish instantly.
6. A snapshot is the same trick applied to a whole filesystem: freeze the current state, and only start copying blocks when something changes them.

Where this comparison breaks: the photocopier note is a single record, while a real copy-on-write filesystem must keep a reference count on every shared block and free a block only when the last reference goes. And free space reporting becomes confusing: deleting a file may free nothing, because a snapshot still references it.

PLAIN12.10.3 a worked example#

1. Why will a 4 GB video not copy to a FAT32 memory stick, even with 200 GB free?
2. In a FAT32 directory entry, the file size is stored in a 32-bit unsigned field. That is a hard part of the on-disk format, not a bug.
3. The largest number that fits in 32 bits is 2 to the power 32, minus 1, which is 4,294,967,295.
4. So the largest possible FAT32 file is 4,294,967,295 bytes, one byte short of 4 GiB.
5. There is no workaround inside FAT32. The field is not big enough to write the number down.
6. The fixes are: reformat as exFAT or NTFS, or split the file into parts.
7. The related limit people meet is that the Windows graphical format tool refuses to make a FAT32 volume larger than 32 GB. That is a policy in the tool, not a limit of the format, which reaches 2 TiB with 512-byte sectors.

PLAIN12.10.4 what is really happening inside#

1. FAT keeps one table, the File Allocation Table, with one entry per cluster. Each entry either says “free”, “bad”, “end of file”, or gives the number of the next cluster in this file. A file is a chain you follow.
2. That is why FAT is easy to implement and slow for large files: to reach the middle of a file you must walk the chain from the start.
3. NTFS keeps everything, including its own bookkeeping, in one enormous file called the master file table, or MFT. Each file gets a record, normally 1 KB.
4. A small file’s contents live inside its own MFT record, so no data block is allocated at all. This is why NTFS handles thousands of tiny files well.
5. ext4 splits the volume into block groups, each with its own inode table and bitmaps, so related metadata and data stay physically close.
6. APFS, ZFS and Btrfs never overwrite a live block. A change writes new blocks, then new parent blocks pointing at them, up to the root, and finally one atomic pointer swap at the very top.
7. That is why they can offer instant snapshots and why they are always consistent after a crash. The old tree is still intact until the moment the root changes.
8. ZFS and Btrfs additionally store a checksum of every block in that block’s parent. On read, the checksum is verified. If it fails and a redundant copy exists, the good copy is returned and the bad one is rewritten silently. That is self-healing, and it catches errors a drive never reports.

TECHNICAL12.10.5 the engineer’s version#

1. Dates and origins. FAT12 came with Microsoft’s 86-DOS work around 1980; FAT16 with PC DOS 3.0 in 1984; FAT32 with Windows 95 OSR2 in August 1996. exFAT appeared in Windows CE 6.0 in late 2006; Microsoft published the specification on 28 August 2019 and Linux gained a native driver in kernel 5.4 in November 2019.
2. NTFS shipped with Windows NT 3.1 in 1993. ext2 was written by Remy Card in 1993, ext3 added journaling in 2001 through work by Stephen Tweedie, and ext4 was marked stable in Linux 2.6.28 in December 2008. ZFS came from Sun Microsystems, principally Jeff Bonwick and Matt Ahrens, and shipped in OpenSolaris in 2005. Btrfs was started by Chris Mason at Oracle in 2007, with the on-disk format declared stable in Linux 3.10 in 2013. APFS was announced at Apple’s WWDC in June 2016, shipped in iOS 10.3 in March 2017 and in macOS High Sierra in September 2017.

3. The NTFS 8 PiB figure applies from Windows 10 version 1709 and Windows Server 2019; older releases capped volumes at 256 TiB with 64 KiB clusters. The theoretical on-disk limit is 16 EiB. Version and edition matter here, so quote the Windows release with any NTFS limit.
4. ext4’s 16 TiB per-file limit follows from 32-bit extent block counts with 4 KiB blocks. The 1 EiB volume limit follows from 48-bit block addressing.
5. APFS space sharing means several volumes in one container draw from a shared free pool, so df shows the same free figure several times. A clone is a file-level copy-on-write duplicate created by the clonefile system call and used by cp -c and the Finder. A snapshot is the container-level equivalent and is what Time Machine local backups use.
6. macOS still cannot write to NTFS without third-party software, and Windows cannot read APFS or ext4 without third-party software. exFAT is the only format that all three major systems read and write out of the box, which is why cameras and large memory cards use it. SD cards above 32 GB, the SDXC class, specify exFAT in the SD standard.
7. Btrfs RAID 5 and RAID 6 profiles are still documented by the project as having unresolved issues and are not recommended for data you care about, as of 2026. Its RAID 0, 1 and 10 profiles are considered sound. Experts disagree on Btrfs stability generally; it is the default in some enterprise Linux distributions and avoided in others.

WORDS12.10.6 remember these#

1. FAT — the simple old chain-based format — File Allocation Table, one entry per cluster forming per-file linked lists.
2. exFAT — FAT with the limits removed — Microsoft’s format for SDXC and large removable media, specification published 2019.
3. MFT — the giant index inside NTFS — Master File Table, one 1 KB record per file, small files stored resident.
4. Copy-on-write — never change a live block — write new blocks and swap the root pointer atomically.
5. Snapshot — a frozen view of the whole filesystem — a retained root pointer with reference counts preventing block reuse.
6. Clone — an instant duplicate that shares blocks — a file-level copy-on-write copy, clonefile on APFS, reflink on Btrfs and XFS.
7. Self-healing — the filesystem repairs bad blocks itself — checksum verification on read plus automatic repair from a redundant copy.

12.11 Journaling, consistency and crashes#

PLAIN12.11.1 in simple words#

1. Almost nothing a filesystem does is a single write. Most operations are three or four writes that must all happen or none.
2. Appending to a file means: mark blocks used, write the data, update the file’s record. A crash between any two leaves a half-finished mess.
3. If the blocks are marked used but the file record never learns about them, the space is lost forever.
4. If the file record claims blocks that were never marked used, the same blocks may later be handed to a second file. Now two files share bytes and both are ruined.
5. A journal fixes this by writing down what you are about to do, before you do it.
6. After a crash the system reads the journal. Anything fully written down is redone. Anything incomplete is thrown away.
7. Either way the filesystem is consistent afterwards, and it takes seconds rather than hours to check.
8. Consistent does not mean nothing was lost. It means nothing is contradictory. The last few seconds of your work can still be gone.

PLAIN12.11.2 a picture in your head#

1. Think of a bank teller moving money between two accounts.
2. Take 100 out of account A. Put 100 into account B. If the power fails in between, 100 has vanished from the world.
3. So before touching either account, the teller writes in a bound ledger: “about to move 100 from A to B”, and only then starts.
4. If the lights go out, the next morning someone reads the ledger, sees an entry, and either finishes the move or undoes it.
5. The ledger is the journal. It is small, written in order, and never has to be searched.
6. Copy-on-write filesystems solve the same problem differently. They prepare a complete new version of everything off to one side, then change one single pointer to make it live.

Where this comparison breaks: a real journal does not have to hold the file data, only the bookkeeping, so a crash can leave a consistent filesystem containing a file full of rubbish. And the ledger only helps if the pen truly touched the paper before the lights went out, which is exactly the problem covered in the technical block below.

PLAIN12.11.3 a worked example#

1. Here is how a journaled metadata update runs, in order.

1. write journal: "intend: alloc blk 5000-5002, inode 4192,
                   size 10240"
2. FLUSH  -> force the drive to commit that to media
3. write journal commit record
4. FLUSH  -> the intent is now durable
5. write the real bitmap, inode and directory blocks
6. mark the journal entry complete

Crash after step 4 -> on mount, replay steps 5 and 6.
Crash before step 3 -> on mount, discard, nothing happened.
Crash between 5 and 6 -> replay is harmless, it is idempotent.

2. Notice steps 2 and 4. Without them the drive’s own write cache may reorder everything and the whole scheme collapses.
3. Notice also that this costs two extra flushes and one extra write per operation. Journaling is not free; it buys recovery time with throughput.

PLAIN12.11.4 what is really happening inside#

1. Drives lie about writes for speed. A drive with a volatile write cache says “done” when the bytes reach its internal memory, not the platter or the flash.
2. That is safe for performance and lethal for ordering. A flush command forces the drive to actually commit everything it has acknowledged.
3. The kernel also has a cache, the page cache. A normal write() only copies bytes into memory and returns. The bytes may reach the drive seconds later.
4. fsync() is the system call that says “do not return until this file’s data and metadata are truly on stable storage”, and it triggers the drive flush.
5. Databases live or die on this. Every committed transaction must be provably on disk before the commit is reported to the client, or the database has lied about durability.
6. That is why database benchmarks care about flush latency more than about throughput, and why enterprise SSDs with capacitors are worth their price: they can honour a flush instantly because their cache is protected.
7. Journals normally protect metadata only. Your file’s bookkeeping survives; the file’s contents may be a mixture of old and new.
8. Copy-on-write designs need no journal for consistency, because the old tree remains valid until the atomic root swap. They still use a small log for speed when handling many tiny synchronous writes.

TECHNICAL12.11.5 the engineer’s version#

1. ext4 offers three journal modes, set with the data= mount option: journal writes file data through the journal too, which is safest and slowest; ordered is the default and forces data blocks out before the metadata commit that references them; writeback orders nothing and can expose stale block contents after a crash.
2. NTFS journals metadata in $LogFile using write-ahead logging with undo and redo records. It never journals file data. XFS and JFS are also metadata-only journals.
3. Barriers are implemented as FLUSH CACHE in ATA, SYNCHRONIZE CACHE in SCSI and Flush in NVMe, or per-command with Force Unit Access. Mounting with nobarrier on a drive without power-loss protection is a known way to corrupt a filesystem, and should never be done outside a battery-backed array.
4. On macOS, fsync() deliberately does not force the drive to flush its cache. Programs that need real durability must call fcntl(fd, F_FULLFSYNC, 0). This is a documented Apple-specific behaviour and a classic portability trap.
5. fdatasync() skips metadata that does not affect retrieval, such as mtime, which is measurably faster for database write-ahead logs.
6. Filesystem check tools. On a journaled filesystem a normal mount only replays the journal, taking under a second. A full structural check is a different operation: fsck.ext4 -f walks every inode, rebuilds the block and inode bitmaps, verifies link counts, and moves unreferenced-but-allocated inodes into lost+found. On a 20 TB ext4 filesystem with many files this can take hours and requires substantial memory. xfs_repair and chkdsk /f are the equivalents. ZFS deliberately has no fsck; it has zpool scrub, which verifies checksums on a live pool.

sudo tune2fs -l /dev/sda1 | grep -i 'features\|state'
sudo dumpe2fs -h /dev/sda1        # journal size and location
sudo zpool status ; sudo zpool scrub tank
sudo btrfs scrub start /mnt/data

7. The honest version: journaling protects filesystem structure, not your data. A power cut during a database write on a journaled filesystem can still lose committed-looking transactions if the application did not call fsync and the drive had no power-loss protection. Three separate parties must each do their job: the application, the kernel, and the drive.

WORDS12.11.6 remember these#

1. Journal — write down the plan before doing it — a write-ahead log of metadata changes, replayed or discarded at mount.
2. Atomicity — all of it or none of it — the guarantee that a multi-step update is never observed half-applied.
3. Write barrier — force the order — a cache flush or FUA operation preventing the device from reordering across a point.
4. fsync — really put it on the disk now — the system call forcing page cache and device cache to stable storage.
5. Ordered mode — data first, then the pointer to it — ext4’s default, preventing a new inode from exposing stale block contents.
6. fsck — the full structural rebuild — offline consistency check reconstructing bitmaps and link counts, slow on large volumes.
7. Scrub — read everything and verify it — background checksum verification and repair in ZFS and Btrfs.

12.12 Partitions, formatting and booting#

PLAIN12.12.1 in simple words#

1. One physical drive is usually cut into several independent regions called partitions.
2. The list of partitions is written in a small table at the very start of the drive.
3. There are two such table formats in use. The old one is MBR, the modern one is GPT.
4. MBR fits in one 512-byte sector, allows only four partitions, and cannot describe anything past 2 TB.
5. GPT allows a hundred and twenty-eight partitions, describes drives far larger than any that exist, and keeps a spare copy at the end of the drive in case the first is damaged.
6. Formatting a partition means writing an empty filesystem onto it: fresh bookkeeping structures saying “nothing is here”.
7. A quick format writes only those structures. It never touches your old data, which is simply no longer referenced.
8. That is why a quick format is not a way to destroy anything, and why recovery software often gets almost everything back from a quick-formatted drive.
9. Mounting is attaching a filesystem so you can see it. Windows gives it a letter; macOS and Linux graft it onto a folder in one big tree.

PLAIN12.12.2 a picture in your head#

1. Think of a drive as a large empty warehouse and partitions as internal walls.
2. The partition table is the sign by the front door listing each room and where it starts and ends.
3. Formatting a room means putting up empty shelves and a fresh index card at the door. It does not mean sweeping the floor.
4. Quick formatting replaces the index card only. Everything on the shelves is still there, just unlisted.
5. Mounting is opening a door from the corridor into that room, so people can walk in.

Where this comparison breaks: the drive does not know or care about the walls. Partitions are purely a convention agreed between software, written in a table the drive itself never reads. You can wipe the table and every byte of the data is still physically present.

PLAIN12.12.3 a worked example#

1. Why exactly does MBR stop at 2 TB?
2. An MBR partition entry records the starting sector and the length as 32-bit numbers of sectors.
3. The largest count is 2 to the power 32, minus 1, that is 4,294,967,295 sectors.
4. With traditional 512-byte sectors: 4,294,967,295 times 512 equals about 2.199 times 10 to the 12 bytes, which is 2 TiB.
5. On a 4Kn drive with 4,096-byte sectors the same arithmetic gives 16 TiB, so the limit is a property of sector size, not of storage in general.
6. GPT stores those figures as 64-bit sector numbers. 2 to the power 64 sectors of 512 bytes is about 8 zebibytes, which no drive will reach this century.

An MBR: one 512-byte sector at LBA 0
+--------------------------------------------------+
| bootstrap code                          446 bytes|
| partition entry 1                        16 bytes|
| partition entry 2                        16 bytes|
| partition entry 3                        16 bytes|
| partition entry 4                        16 bytes|
| signature 0x55 0xAA                       2 bytes|
+--------------------------------------------------+
GPT: protective MBR at LBA 0, header at LBA 1,
128 entries of 128 bytes at LBA 2..33, backup at the end.

PLAIN12.12.4 what is really happening inside#

1. When an old BIOS machine starts, it loads the first sector of the boot drive into memory and jumps into it. That is the 446 bytes of bootstrap code in the MBR. Those 446 bytes then load a bigger loader from somewhere else.
2. A UEFI machine does something far more sensible. It reads the partition table, finds a partition marked as the EFI System Partition, which is formatted FAT32, and runs a normal file from it, such as bootx64.efi.
3. That is why the EFI partition is FAT32 on a modern Mac and PC: the firmware must be able to read it before any operating system exists.
4. Secure Boot adds a signature check on that file against keys stored in firmware.
5. A logical volume manager adds a layer between partitions and filesystems. It gathers whole drives into a pool and hands out volumes from that pool, which can be grown, shrunk, moved between drives and snapshotted while in use.
6. Without it, growing a filesystem means the free space must happen to sit directly after it on the same drive. With it, you just add another drive.
7. Mount points differ by system, but the idea is identical: the filesystem’s root is grafted somewhere the user can reach.

TECHNICAL12.12.5 the engineer’s version#

1. The MBR layout dates from PC DOS 2.0 in 1983. Four primary partition entries only; the extended partition trick chains further logical partitions through a linked list of extended boot records, which is fragile and now obsolete.
2. GPT is defined by the UEFI specification, descended from Intel’s EFI work in the late 1990s for Itanium. Its header and entry array are each protected by a CRC32 checksum, and a full backup copy sits in the last sectors of the disk. LBA 0 holds a protective MBR describing one partition of type 0xEE covering the whole disk, so that MBR-only tools do not think the disk is blank and offer to initialize it.
3. Partition types are GUIDs in GPT. Common ones: EFI System Partition, Microsoft Basic Data, Linux filesystem, Linux LVM, Apple APFS Container.
4. Alignment matters. Partitions should start on a 1 MiB boundary so that filesystem 4 KiB blocks align with 4 KiB physical sectors and with SSD page and erase-block boundaries. Every current partitioner does this by default; tools from before about 2010 started at sector 63 and did not.
5. Formatting levels: a low-level format, meaning writing servo and sector structures, is a factory operation on modern drives and cannot be performed by a user. A high-level format writes filesystem metadata. Windows full format since Vista also writes zeros across the volume and runs a surface scan; Windows quick format does not.
6. Volume managers: LVM2 on Linux with physical volumes, volume groups and logical volumes; Storage Spaces on Windows; APFS containers on macOS; and ZFS, which merges volume management and filesystem into one layer deliberately.

lsblk -f                       # devices, filesystems, mountpoints
sudo parted /dev/sda print     # partition table type and entries
sudo blkid                     # UUIDs used by /etc/fstab
diskutil list                  # macOS, shows APFS containers

7. Mount by UUID or label, never by device name. Device names such as /dev/sdb are assigned in discovery order and can change between boots, which is a common cause of a machine failing to boot after adding a drive.

WORDS12.12.6 remember these#

1. Partition — one region of a drive — a contiguous LBA range described in a partition table.
2. MBR — the old 512-byte table — Master Boot Record, four primary entries, 32-bit sector counts, 2 TiB ceiling at 512-byte sectors.
3. GPT — the modern table — GUID Partition Table, part of UEFI, CRC-protected, with a backup copy at the end of the disk.
4. EFI System Partition — the small FAT32 partition the firmware boots from — holds signed .efi loader files.
5. Quick format — new bookkeeping, old data untouched — writing fresh filesystem metadata without erasing block contents.
6. Mounting — attaching a filesystem into the visible tree — associating a filesystem with a mount point or drive letter.
7. LVM — a layer that makes volumes flexible — logical volume manager pooling physical extents and serving resizable logical volumes.

12.13 RAID and redundancy#

PLAIN12.13.1 in simple words#

1. RAID means using several drives together so that they behave as one, for speed, or for safety, or both.
2. RAID 0 splits your data across drives in stripes. Twice the drives, twice the speed, all the capacity, and no safety at all. Lose one drive and you lose everything.
3. RAID 1 writes the same data to two drives. Half the capacity, and either drive alone can carry on.
4. RAID 5 spreads data across at least three drives and adds one drive’s worth of parity, a checksum that lets any single missing drive be rebuilt. You lose one drive of capacity.
5. RAID 6 does the same with two independent parities, so any two drives may die. You lose two drives of capacity.
6. RAID 10 mirrors pairs and stripes across the pairs. Half the capacity, fast, and rebuilds quickly.
7. RAID protects against exactly one thing: a drive failing. That is all.
8. It does not protect against deletion, ransomware, corrupted files, a power surge, theft, fire or flood. Every one of those hits all drives at once because they are all in one box.
9. So RAID is about staying online, not about keeping data. It is not a backup.

PLAIN12.13.2 a picture in your head#

1. Imagine four people each holding one page of a four-page document, plus a fifth person holding a page of “checksums”: for every character position, the sum of the four letters treated as numbers.
2. If one of the four is missing, you can work out their page exactly by subtracting the other three from the checksum page.
3. That is RAID 5 parity. It is not a copy of anything; it is the difference that lets you reconstruct whichever one you lost.
4. Now notice the danger. To reconstruct one page you must read every other page completely and correctly. If a second person makes even one mistake while you are reconstructing, the answer is wrong.

Where this comparison breaks: real parity uses exclusive-or rather than addition, and RAID 6 uses a second parity computed over a finite field so the two equations are independent. And the real risk is not a person making a mistake but a drive returning an unreadable sector during the rebuild.

PLAIN12.13.3 a worked example#

1. Eight 20 TB drives, and here is what you get from each layout.

2. Now rebuild time, which is where RAID 5 loses its reputation.
3. Replacing a 20 TB drive means writing 20 TB onto it. At a generous sustained 150 MB/s that is 20,000,000 MB divided by 150, which is 133,333 seconds.
4. That is 37 hours, and only if the array is otherwise completely idle. On a busy array, two to five days is realistic.
5. During all that time you have zero redundancy and every surviving drive is being read from end to end, which is the hardest work they ever do.
6. Consumer drives are specified for one unrecoverable read error per 10 to the 14 bits, which is one per 12.5 TB read. Reading seven 20 TB drives in full is 140 TB. On the specified figure, hitting at least one error is likely.
7. That arithmetic is why RAID 6 or mirroring is the normal advice above about 4 TB per drive. The honest version: real drives usually beat their specified error rate by a wide margin, so the “RAID 5 is mathematically certain to fail” claim is too strong. Experts disagree on the size of the margin. Both camps agree RAID 6 is the safer choice at these capacities.

PLAIN12.13.4 what is really happening inside#

1. Parity is computed with exclusive-or, written XOR. XOR of a set of bits is 1 if an odd number of them are 1.
2. Its useful property: if you XOR all the data blocks together to make parity, then XOR any subset including parity but missing one block, you get the missing block back.
3. RAID 5 rotates which drive holds parity for each stripe, so no single drive becomes the write bottleneck.
4. Small writes to RAID 5 are expensive. To change one block, the controller must read the old block and old parity, compute the new parity, then write both. One logical write becomes two reads and two writes. This is the RAID 5 write penalty.
5. RAID 6’s second parity uses Reed-Solomon arithmetic so that the two equations are independent and any two failures can be solved.
6. The write hole is a real hazard on parity RAID without a journal or battery: if power is lost between writing data and writing parity, the stripe is silently inconsistent, and a later rebuild will produce wrong data.
7. Hardware RAID controllers use a battery or capacitor-backed cache to close the write hole. Linux md uses a write-intent bitmap and an optional journal. ZFS avoids it entirely because RAID-Z stripes are copy-on-write and variable width.
8. RAID controllers rebuild from parity but usually cannot tell you which copy is right if the data is merely wrong rather than missing. Filesystems with checksums can, which is the argument for ZFS over classic RAID.

TECHNICAL12.13.5 the engineer’s version#

1. The taxonomy comes from the 1988 SIGMOD paper “A Case for Redundant Arrays of Inexpensive Disks (RAID)” by David Patterson, Garth Gibson and Randy Katz at the University of California, Berkeley. The industry later renamed the I to “Independent”.
2. RAID 2, 3 and 4 exist in the paper but are effectively unused: RAID 2 used Hamming codes and bit striping, RAID 3 byte striping with a dedicated parity drive, RAID 4 block striping with a dedicated parity drive. RAID 5’s rotated parity removed RAID 4’s bottleneck.
3. Nested levels are written as two digits: RAID 10 is a stripe over mirrors, RAID 01 is a mirror over stripes, and RAID 10 tolerates more failure patterns and rebuilds far faster because only one drive’s worth of data must be copied.
4. ZFS RAID-Z1, Z2 and Z3 correspond roughly to single, double and triple parity, but with variable-width stripes that eliminate the write hole. A zpool scrub verifies every checksum and repairs from redundancy. Rebuilds in ZFS, called resilvering, copy only allocated blocks, so a half-empty pool resilvers in roughly half the time. Traditional RAID rebuilds the whole drive regardless.
5. Unrecoverable read error rate is specified as 1 in 10^14 bits for most consumer SATA drives and 1 in 10^15 for enterprise SATA, SAS and NL-SAS. 10^14 bits is 12.5 TB; 10^15 is 125 TB.
6. Practical guidance widely used in 2026: mirrors or RAID 10 for databases and virtual machines where write latency matters; RAID 6 or RAID-Z2 for bulk archives; RAID 5 only on small fast drives with a short rebuild window; RAID 0 only for scratch data you can regenerate.
7. The 3-2-1 rule, popularized by the photographer Peter Krogh in his book “The DAM Book” in 2005: keep at least 3 copies of the data, on at least 2 different kinds of media, with at least 1 copy off site. A common modern extension is 3-2-1-1-0: one copy offline or immutable, and zero errors on a verified restore test.
8. The last part is the part people skip. A backup that has never been restored is a hypothesis, not a backup. Test restores on a schedule.

cat /proc/mdstat                  # Linux md array state
sudo mdadm --detail /dev/md0
sudo zpool status -v tank         # errors per device, scrub state
sudo smartctl -a /dev/sda | grep -i reallocated

WORDS12.13.6 remember these#

1. RAID — several drives behaving as one — Redundant Array of Independent Disks, from the 1988 Berkeley paper.
2. Striping — data spread across drives — RAID 0, dividing data into fixed-size chunks written in rotation.
3. Mirroring — the same data on two drives — RAID 1, full duplication.
4. Parity — the difference that rebuilds a lost drive — XOR for single parity, Reed-Solomon for the second.
5. Rebuild / resilver — reconstructing a replaced drive — a full-array read at maximum stress with no redundancy while it runs.
6. Write hole — data and parity out of step after a power cut — silent stripe inconsistency on parity RAID without a journal or backed cache.
7. URE — an unreadable sector — unrecoverable read error, specified at 1 in 10^14 bits for consumer drives.
8. 3-2-1 — the backup rule — three copies, two media types, one off site.

12.14 Data recovery, deletion and secure erase#

PLAIN12.14.1 in simple words#

1. Deleting a file does not remove the file. It removes the entry that points to the file.
2. The bytes stay exactly where they were until something else happens to be given that space.
3. That is why recovery software works, and why an empty-looking second-hand drive can be full of somebody’s private life.
4. On a hard disk, writing new data over the old data really does destroy it. One pass of zeros is enough on any drive made this century.
5. On an SSD, overwriting does not work, because the drive refuses to write in the same place. Your “overwrite” lands on a fresh page and the original is still sitting in a page you cannot address.
6. So SSDs need a different approach: a command that tells the drive itself to wipe everything, including the parts you cannot see.
7. The best approach of all is to encrypt the drive from the day you buy it. Then wiping it means destroying one key, which takes milliseconds.
8. Encrypted from the start, an unerased drive is already unreadable rubbish to anyone without the key.

PLAIN12.14.2 a picture in your head#

1. Think of a library index card system. Deleting a book means throwing away its card. The book is still on the shelf.
2. Recovery software walks the shelves ignoring the index, looking for anything that looks like a book. That is file carving.
3. Now the SSD problem. Imagine a librarian who, whenever you ask to replace a book, refuses to touch the original and instead shelves your replacement in a new spot, then quietly updates the index.
4. You can hand over replacement after replacement and the originals stay on the shelves, in a wing of the building you are not allowed to enter.
5. The only way to clear that wing is to ask the librarian, who alone has the keys, to burn everything.

Where this comparison breaks: file carving only finds file contents, not names, folders or dates, and cannot reassemble a file whose pieces were scattered. It works best on photos and videos, which have clear start and end markers and are usually written in one go.

PLAIN12.14.3 a worked example#

1. A JPEG image always starts with the bytes FF D8 FF and ends with FF D9.
2. Carving software reads the whole raw drive looking for those markers and writes out everything in between as a candidate file.
3. It needs no filesystem at all, which is why it survives a quick format.
4. Here is the difference between the erase options, on a 2 TB NVMe SSD.

5. The crypto erase is instant because it does not touch the data at all. It destroys the internal key the data was encrypted with, and the remaining bytes become noise.

PLAIN12.14.4 what is really happening inside#

1. Modern SSDs, and many hard drives, already encrypt everything they store, whether you asked for it or not. The key lives inside the drive.
2. That is called a self-encrypting drive. Normally the key is used automatically and you never notice.
3. A cryptographic erase simply replaces that key with a new random one. Every byte on the media instantly becomes undecryptable.
4. ATA Secure Erase is a command in the ATA standard telling the drive to erase its entire user area itself, including reallocated sectors that normal writes cannot reach. Enhanced Secure Erase additionally covers areas the basic command may skip.
5. NVMe has two: Format NVM, which can do a user-data erase or a cryptographic erase, and Sanitize, which is stronger and covers over-provisioned areas, caches and any unmapped data.
6. Full-disk encryption puts the key under your control rather than the drive maker’s. BitLocker on Windows, FileVault on macOS, LUKS with dm-crypt on Linux, all normally using AES in XTS mode with a 256-bit key.
7. With full-disk encryption in place, disposal is trivial. Forget the passphrase, destroy the key slot, and the drive is a brick to everyone, including you.
8. Degaussing, a strong magnetic pulse, destroys hard disks and tapes permanently, and does absolutely nothing to flash, because flash stores charge and not magnetism.

TECHNICAL12.14.5 the engineer’s version#

1. NIST Special Publication 800-88 Revision 1, “Guidelines for Media Sanitization”, published in 2014, defines three levels: Clear, Purge and Destroy. It states explicitly that for modern magnetic media a single overwrite pass is adequate for Clear, and that cryptographic erase is an acceptable Purge method for self-encrypting media.
2. The 35-pass Gutmann method comes from Peter Gutmann’s 1996 Usenix paper and targeted the specific MFM and RLL encodings of drives of that era. Gutmann himself has written that applying all 35 passes to a modern drive is pointless. Repeating it as current advice is a myth.
3. Commands and tools:

# ATA drives: check support, then issue a secure erase
sudo hdparm -I /dev/sda | grep -A6 Security
sudo hdparm --user-master u --security-set-pass p /dev/sda
sudo hdparm --user-master u --security-erase p /dev/sda

# NVMe: crypto erase (ses=2) or user-data erase (ses=1)
sudo nvme format /dev/nvme0n1 --ses=2
sudo nvme sanitize /dev/nvme0n1 --sanact=4   # crypto erase
sudo nvme sanitize-log /dev/nvme0n1          # progress

4. Drives are often left in a frozen security state by the firmware, and hdparm will report frozen. The usual workaround is a suspend and resume cycle, or hot-plugging the SATA data cable, both of which carry risk. Prefer nvme sanitize or vendor tools where available.
5. Recovery tooling: photorec and scalpel for carving, testdisk for rebuilding partition tables, ddrescue for imaging a failing drive with retry and error mapping. Always image first with ddrescue and work on the copy, never on the original.
6. On SSDs, TRIM interacts with recovery in a way that surprises people. On a TRIM-enabled drive with deterministic read-after-trim, reading a trimmed block returns zeros almost immediately after deletion. So on a modern SSD, deleting a file often does destroy it within seconds, and recovery frequently fails where it would have succeeded on a hard disk. This is a helpful property for privacy and a painful one for accidents.
7. Physical destruction remains the standard for the highest classifications: shredding to a specified particle size, disintegration, or incineration. For flash, particle size matters, because a surviving chip fragment can still hold readable pages.
8. The practical rule: encrypt at first use. Every erase question becomes easy afterwards, and the answer to “what if it fails before I can wipe it” becomes “it does not matter”, which no amount of erasing software can give you.

WORDS12.14.6 remember these#

1. File carving — finding files by their shape, ignoring the index — signature based recovery scanning raw sectors.
2. ATA Secure Erase — the drive wipes itself — an ATA command covering the whole user area including reallocated sectors.
3. Sanitize — the strongest built-in erase — NVMe or ATA operation covering over-provisioned, cached and unmapped regions.
4. Cryptographic erase — throw away the key, not the data — destroying the media encryption key of a self-encrypting drive.
5. Self-encrypting drive — it was always encrypted — media encryption performed in the controller with an internally held key.
6. Full-disk encryption — the key is yours — BitLocker, FileVault, LUKS, normally AES-XTS with a 256-bit key.
7. Degaussing — a magnetic pulse that destroys magnetic media — effective on HDD and tape, useless on flash.

12.98 Common wrong ideas#

1. Wrong: deleting a file erases it. Right: deleting removes a directory entry and frees the blocks. The bytes stay until something reuses the space, which is why recovery tools work on hard disks.
2. Wrong: an SSD has no moving parts, so it cannot fail. Right: flash wears out with every erase, leaks charge when unpowered, and SSDs commonly die suddenly from controller or firmware faults with no warning at all.
3. Wrong: defragmenting an SSD helps. Right: an SSD has no seek penalty, so defragmentation only burns write cycles. Windows deliberately runs TRIM instead when the drive is flash.
4. Wrong: RAID means my data is backed up. Right: RAID protects against a drive dying. Deletion, ransomware, corruption, fire and theft all reach every drive in the array at once.
5. Wrong: my 1 TB drive is faulty because the computer shows 931 GB. Right: the maker counted 10^12 bytes and the operating system is counting 2^40 bytes. Both numbers are correct, the units differ.
6. Wrong: a quick format wipes the drive. Right: it writes fresh empty bookkeeping and leaves every byte of old data physically in place.
7. Wrong: you must overwrite a drive seven or 35 times to be safe. Right: NIST SP 800-88 says one pass is enough on modern magnetic media, and on flash no number of overwrite passes is reliable. Use sanitize or cryptographic erase.
8. Wrong: more bits per flash cell is simply better value. Right: QLC gives more capacity per chip but roughly a tenth of the endurance of MLC and much slower sustained writes once the cache is exhausted.
9. Wrong: a filesystem journal protects my file contents. Right: most journals protect metadata only. After a crash the structure is consistent but a file may hold a mixture of old and new bytes.
10. Wrong: a 4 GB file will not fit because the USB stick is too small. Right: on FAT32 the file size field is 32 bits, so no file above 4 GiB minus one byte can be described, however much free space there is.

12.99 Chapter summary in 20 lines#

1. Memory forgets without power and storage does not; every machine needs both because no technology is fast, cheap, huge and permanent at once.
2. The storage ladder runs from registers at under a nanosecond to tape at tens of seconds, spanning about eight orders of magnitude in latency.
3. Tape is not dead. LTO-10 reached 30 TB native in August 2025 and 40 TB in November 2025, at roughly a third of disk’s cost per terabyte.
4. A hard disk writes bits by flipping magnetic domains and reads them by sensing reversals, with the head flying a few nanometres up on an air bearing.
5. The IBM 350 of 1956 held about 3.75 MB on fifty 24-inch discs and weighed over a ton; a 2026 HAMR drive holds 44 TB on ten 3.5-inch platters.
6. Giant magnetoresistance, discovered in 1988 and awarded the 2007 Nobel Prize in Physics, is what made high-capacity drives possible.
7. Disk access time is seek plus rotational latency; average rotational latency is 30,000 divided by RPM in milliseconds.
8. That arithmetic caps a 7,200 RPM drive at roughly 80 random operations per second, a figure that has barely improved in thirty years.
9. Flash stores charge behind an insulator. Reading is cheap, writing damages the insulator, and more bits per cell means far less endurance.
10. NAND’s defining rule: you read a page, you write a page, but you can only erase a whole block, so nothing is ever changed in place.
11. That rule forces an SSD to contain a small computer running a flash translation layer, doing wear levelling and garbage collection.
12. Write amplification measures the extra work; it rises sharply as the drive fills, which is why free space is working capital and not waste.
13. TRIM tells the drive which blocks are rubbish; without it an SSD slowly behaves as if permanently full.
14. NVMe over PCIe replaced one 32-command queue with up to 65,535 queues, which is why an SSD on PCIe 5.0 x4 reaches 14 GB/s and SATA stops at 550 MB/s.
15. A filesystem turns a numbered list of blocks into named files; the inode holds the metadata and the name lives in the directory, not in the inode.
16. Creating, writing, renaming and deleting are all metadata edits; renaming a 50 GB file is instant and deleting it overwrites nothing.
17. FAT32 cannot hold a file of 4 GiB or more because its size field is 32 bits wide; exFAT and NTFS remove that limit.
18. Journals make a crash leave a consistent structure, not intact contents; copy-on-write filesystems get the same guarantee by swapping one root pointer.
19. RAID keeps a service running when a drive dies, and protects against nothing else; the 3-2-1 rule, plus a tested restore, is the actual backup.
20. Encrypt every drive from first use, because then secure disposal is one discarded key rather than a hopeful overwrite that flash will quietly ignore.