What a Commit Actually Is

38.0 What this chapter gives you#

1. You will be able to say what content-addressed storage is: the name of a thing is worked out from the thing itself, so equal content always gets equal names.
2. You will be able to explain what a hash function is, name the five properties that matter, and say why SHA-1 gives 160 bits printed as 40 hexadecimal characters.
3. You will be able to state honestly that SHA-1 was broken in public on 23 February 2017 by Google and CWI Amsterdam, say what git did about it, and say why git’s use of it was never purely a security control.
4. You will be able to name git’s four object types, blob, tree, commit and tag, describe the exact bytes each one stores, and rebuild any object’s hash by hand with one shell command.
5. You will be able to draw the complete object graph of a small repository, every blob, every tree, every commit and every arrow, and say exactly which objects are created and which are reused when one file changes.
6. You will be able to explain why history is a directed acyclic graph and not a line, what a merge commit with two parents looks like inside, and why the tip hash seals everything behind it.
7. You will be able to say what a branch really is, a small text file holding 40 hexadecimal characters, and why that makes branching almost free.
8. You will be able to read and repair the state of HEAD, including detached HEAD, and recover work that looked lost after a hard reset by using the reflog.
9. You will be able to use the plumbing commands that show the machinery: git cat-file, git rev-parse, git hash-object, git ls-tree, git count-objects, git verify-pack and git rev-list.
10. You will be able to explain the reader’s own observation that origin/main was 0a95cc8, and why a rebased branch inevitably came out with a different hash.

38.1 Content-addressed storage, the central idea#

PLAIN38.1.1 in simple words#

1. Normally you give a thing a name and then put content inside it. A folder called notes holds whatever you put in it today.
2. Git does the opposite. You give it content, and git works out the name from the content.
3. The name is a long string of letters and digits, produced by feeding every byte of the content through a fixed piece of arithmetic.
4. This is called content-addressed storage. The address of a thing is computed from the thing.
5. Two consequences follow, and they are the whole reason git works the way it does.
6. First, the same content always gets the same name. So git can never store the same content twice. It looks up the name, finds it already there, and stops.
7. Second, if anyone changes the content even by one bit, the name no longer matches. So changing content without being noticed is not possible.
8. Everything else in this chapter is a consequence of those two sentences.

PLAIN38.1.2 a picture in your head#

1. Think of a very strange left-luggage office at a railway station.
2. You hand over a bag. The clerk does not give you a numbered ticket chosen from a pile.
3. Instead the clerk weighs the bag, measures it, catalogues every object inside it, and turns all of that into one long code.
4. That code is your ticket. It describes the bag rather than pointing at a shelf.
5. If you hand in an identical bag tomorrow, you get the identical ticket. The clerk says “I already have that one” and stores nothing new.
6. If somebody opens your bag and removes one sock, then re-runs the measurement, they get a different code. Your old ticket no longer matches anything. The swap is visible.

Where this comparison breaks:

7. A real clerk could be lazy and reuse a code for two nearly identical bags. A hash function cannot be lazy. It is arithmetic, and it runs the same way every time.
8. A real bag can be measured many ways. Git measures exactly one way, over exactly the bytes, in exactly one order, so the answer is never a matter of judgement.
9. And a left-luggage ticket does not tell you what is in the bag. A git hash does not either, but it is enough to prove that the bag you get back is the bag you handed in.

PLAIN38.1.3 a worked example#

1. Here is a real repository built for this chapter. Four files were created, three of them holding exactly the same line of text.
2. The command git ls-tree -r HEAD lists every file with the name git gave to its content.

100644 blob 6fe3e87371433ed417646e3cb2354cfb9cff08d3    a/one.txt
100644 blob c9330452ddba7055160034759c594f78381fd24d    a/other.txt
100644 blob 6fe3e87371433ed417646e3cb2354cfb9cff08d3    b/two.txt
100644 blob 6fe3e87371433ed417646e3cb2354cfb9cff08d3    c/three.txt

3. Three files, three different paths, one identical name: 6fe3e87....
4. Now count how many pieces of file content git actually stored:

6fe3e87371433ed417646e3cb2354cfb9cff08d3 blob 22
c9330452ddba7055160034759c594f78381fd24d blob 18

5. Two. Not four. The three identical files share one stored copy of 22 bytes.
6. Nobody asked for that. Nobody ran a de-duplication tool. It falls out of naming things by their content.
7. Now rename a/one.txt to a/renamed.txt and commit. The listing changes, but the stored content list does not change at all.

100644 blob c9330452ddba7055160034759c594f78381fd24d    a/other.txt
100644 blob 6fe3e87371433ed417646e3cb2354cfb9cff08d3    a/renamed.txt
100644 blob 6fe3e87371433ed417646e3cb2354cfb9cff08d3    b/two.txt
100644 blob 6fe3e87371433ed417646e3cb2354cfb9cff08d3    c/three.txt

8. The file name lives in the directory listing, not in the content. Renaming is free.

PLAIN38.1.4 what is really happening inside#

1. When you run git add file.txt, git reads every byte of the file.
2. It puts a small header in front of those bytes. The header is the word blob, a space, the length in bytes, and a zero byte.
3. It runs the header plus the content through the hash function. Out comes a 160-bit number.
4. It prints that number as 40 hexadecimal characters. That is the object’s name.
5. It then looks on disk for a file at .git/objects/ plus the first two characters, a slash, then the other 38 characters.
6. If that file already exists, git does nothing more. The content is already stored, by definition, because the name proves it.
7. If it does not exist, git compresses the header plus content with zlib and writes it there.
8. Reading works in reverse. Given a name, git knows the path, reads the file, decompresses it, and hands back the bytes.
9. Because the file is named after its own content, git can check itself at any time. Re-hash the content, compare with the file name, and any difference is damage or tampering.

TECHNICAL38.1.5 the engineer’s version#

1. Git implements a content-addressable object store. The key space is the output of a cryptographic hash over <type> <size>\0<content>.
2. The store is also a Merkle tree, the structure Ralph Merkle described in his 1979 doctoral work: a tree in which every parent node contains the hash of its children.
3. That is why one hash at the top, the commit hash, transitively covers every byte in the tree below it.
4. Loose objects live at .git/objects/ab/cdef.... The two-character sub-directory exists because early filesystems degraded badly with hundreds of thousands of entries in one directory.
5. Here is a real loose object, on disk, byte for byte:

00000000  78 01 4b ca c9 4f 52 30  34 61 f0 cc 2b 29 ca 4f
00000010  29 4d 2e c9 cc cf d3 e3  02 00 54 63 07 5f

6. That file is 30 bytes. The leading 78 01 is a zlib header (RFC 1950, May 1996, by Jean-loup Gailly and Mark Adler). Decompressed it is 22 bytes:

b'blob 14\x00Introduction.\n'

7. So the stored object is blob, space, 14, a NUL byte, then the 14 bytes of file content. The hash is taken over all 22 of those bytes.
8. Deduplication is exact-match only. Git does not find near-duplicates at store time. Similarity is exploited later, inside packfiles, and that is a separate mechanism covered in 38.11.

9. The honest version: content addressing gives integrity, not secrecy. Anyone with the object store can read every byte. Encryption is a separate concern that git does not provide.

WORDS38.1.6 remember these#

1. Content-addressed storage — the name comes from the content — key space is the image of a hash function over the stored bytes.
2. Blob — a stored piece of file content — a git object of type blob holding raw bytes with no name and no permissions.
3. Deduplication — storing one copy of identical things — exact-match coalescing by hash key at write time.
4. Merkle tree — a tree where each parent names its children by hash — hash tree, described by Ralph Merkle in 1979, giving transitive integrity.
5. zlib — the squeezing format git uses on disk — DEFLATE stream in a zlib wrapper, RFC 1950 and RFC 1951, both May 1996.

38.2 Hashing, and the honest truth about SHA-1#

PLAIN38.2.1 in simple words#

1. A hash function takes any amount of data and produces a short fixed-size number that stands for it.
2. Five things matter about it.
3. It is deterministic: the same input always gives the same output, on any machine, in any year.
4. It has a fixed size output: one byte in or one gigabyte in, the answer is the same length.
5. It has the avalanche effect: change one bit of input and about half the output bits flip. There is no gentle drift.
6. It is one-way: given the output you cannot work backwards to the input, other than by guessing.
7. It is collision resistant: it should be impractical to find two different inputs with the same output.
8. Git used SHA-1, which produces 160 bits. Printed four bits per character in hexadecimal, that is 40 characters.
9. The honest version: the fifth property, collision resistance, is broken for SHA-1. That is not a rumour, it is a demonstrated public fact from 2017.

PLAIN38.2.2 a picture in your head#

1. Think of a machine that turns any document into a 40-character summary code.
2. Feed it a shopping list, get a code. Feed it the complete works of a novelist, get a code of exactly the same length.
3. Change one comma anywhere in the novel and the code changes beyond recognition, not slightly.
4. You cannot run the machine backwards. Given a code, there is no way to get the novel out of it.
5. Collision resistance is the promise that nobody can make two different documents that produce the same code.
6. That promise held for SHA-1 for about twenty-two years. Then it did not.

Where this comparison breaks:

7. A summary suggests something readable. A hash output is not readable and carries no meaning at all. It is only a label.
8. Also, a real summary of a longer document loses information gently. A hash loses nearly all information at once. From 160 bits you can prove a match, but you can never reconstruct anything.

PLAIN38.2.3 a worked example#

1. Here are real SHA-1 values computed in the sandbox used to write this chapter, on an Intel Xeon at 2.10 GHz.
2. Determinism. The same five letters, hashed three times:

$ printf 'hello' | sha1sum
aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d
aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d
aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d

3. Avalanche. Change the last letter from o to p:

hello  -> aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d
hellp  -> d4baaad7a68a379b1e133e0fea0603051b0124ca

4. Of the 40 hexadecimal characters, only 3 happen to be equal. Of the 160 bits, 85 differ. The ideal for a good hash is 80. It is behaving correctly.
5. Fixed size. One byte in, and one million bytes in:

1 byte        -> 86f7e437faa5a7fce15d1ddcb9eaeaea377667b8
1,000,000 B   -> bef3595266a65a2ff36b700a75e8ed95c68210b6

6. Both answers are 40 characters. The length of the input is invisible in the output.
7. Now the git-specific part. Git does not hash the file. It hashes a header plus the file. Here is the proof, done by hand:

$ printf 'Introduction.\n' | wc -c
14
$ printf 'blob 14\0Introduction.\n' | sha1sum
0e2a325c7de3369b164d48fcba823bca9c2cec4e
$ git hash-object docs/intro.md
0e2a325c7de3369b164d48fcba823bca9c2cec4e

8. Identical. sha1sum and git agree, because git’s rule is exactly “hash the header plus the content”.

PLAIN38.2.4 what is really happening inside#

1. SHA-1 works on the input in blocks of 64 bytes.
2. It keeps five 32-bit working registers, 160 bits in total, initialized to fixed constants written into the specification.
3. For each block it mixes the block into those registers through 80 rounds of addition, rotation and bit logic, then adds the result back into the running state.
4. When the input runs out, it appends a single 1 bit, then zero bits, then the original bit length as a 64-bit number, and processes the final block.
5. The five registers, printed end to end, are the answer.
6. The reason a one-bit change scatters the output is that the mixing is designed so every input bit influences every register within a few rounds.
7. A collision means two different inputs that end with the same five registers. Because inputs are unlimited and outputs are only 160 bits, collisions must exist. The question is only whether anyone can find one.
8. In 2017 somebody did, on purpose, with two files chosen in advance to be different in a way that mattered.

TECHNICAL38.2.5 the engineer’s version#

1. SHA-1 is specified in FIPS PUB 180-1, published by the United States National Institute of Standards and Technology in 1995. It replaced SHA-0 from FIPS PUB 180 in 1993, which had a design flaw.
2. Output is 160 bits. Internal state is five 32-bit words. Block size is 512 bits. Rounds per block: 80.
3. On 23 February 2017 researchers at CWI Amsterdam and Google announced SHAttered, the first public SHA-1 collision: two different PDF files with the identical SHA-1 digest.
4. The published cost was approximately 2 to the power 63.1 SHA-1 evaluations, described as the equivalent of 6,500 CPU-years plus 110 GPU-years, and about 100,000 times cheaper than a brute-force birthday search.
5. On 5 January 2020 Gaetan Leurent and Thomas Peyrin announced SHAmbles, a chosen-prefix collision at about 2 to the power 63.4, costed at roughly 45,000 US dollars of rented GPU time. Chosen-prefix is the dangerous kind, because the attacker picks both meaningful beginnings.
6. NIST formally deprecated SHA-1 in 2011, disallowed it for digital signatures from 2013, and in 2022 set 31 December 2030 as the full phase-out date.
7. Here is what those numbers mean in the machine used for this chapter. It hashes at about 1,241 MB/s, which is about 1.95 x 10^7 blocks per second.

8. What git did. From Git 2.13, released in May 2017, git ships the collision-detecting SHA-1 implementation by Marc Stevens of CWI and Dan Shumow of Microsoft. The Git 2.13 release notes say it “has been integrated and made the default”.
9. That code computes SHA-1 normally but also watches for the internal disturbance vectors that any known collision attack must use. If it sees them, git aborts rather than storing the object. The two SHAttered PDFs cannot be committed to a modern git.
10. Separately, git has been growing a SHA-256 object format. The design document dates from 2017 and 2018, experimental support began landing from Git 2.19 in September 2018, and git init --object-format=sha256 arrived in Git 2.29 in October 2020.
11. It works. In the sandbox:

$ git init --object-format=sha256 sha256demo
$ git hash-object intro.md
22c6af3b0fccae1e36b21923804fd096c8636e54da038f04c424208
d2ecb1650

12. That is 64 hexadecimal characters, and it verifies by hand exactly the same way: printf 'blob 14\0Introduction.\n' | sha256sum gives the same value.
13. But it is still not the default in 2026, and git’s own documentation says that until the wire protocol gains SHA-256 support, using SHA-256 storage on public-facing servers is strongly discouraged. Repositories in the two formats cannot yet talk to each other.
14. The honest version, and the part people argue about: Linus Torvalds has said publicly that git’s hash was chosen mainly to guard against accidental corruption and to give a stable name, not as a signature scheme. Critics reply that in practice people do rely on commit hashes as identity, in deploy pipelines and in audit trails. Both points are true.
15. Where experts disagree: some argue the SHA-256 migration is overdue and the ecosystem cost is worth paying now. Others argue that with collision detection turned on, a real attack on a git repository is far harder than an attack on a bare hash, so the migration can proceed slowly. There is no settled answer.

WORDS38.2.6 remember these#

1. Hash function — arithmetic that turns any data into a short fixed code — a map from arbitrary-length input to fixed-length digest.
2. Avalanche effect — one small change scrambles the whole answer — a one-bit input change flips about half the output bits.
3. Collision — two different inputs with the same code — two distinct pre-images mapping to one digest.
4. Preimage resistance — you cannot work backwards from the code — finding any input for a given digest should cost about 2^160 for SHA-1.
5. SHAttered — the 2017 demonstration that SHA-1 collisions are buildable — CWI Amsterdam and Google, 23 February 2017, cost about 2^63.1.
6. sha1dc — git’s hardened SHA-1 that spots attacks — the collision-detecting SHA-1 of Marc Stevens and Dan Shumow, default since Git 2.13, May 2017.

38.3 The four object types#

PLAIN38.3.1 in simple words#

1. Git stores exactly four kinds of object. Everything in a repository is one of these four.
2. A blob is the content of one file. Just the bytes. It does not know its own name, its folder, or when it was made.
3. A tree is one directory listing. It maps names to blobs and to other trees, and records each entry’s permission mode.
4. A commit is a snapshot marker. It names exactly one tree, names zero or more parent commits, records who wrote it and who recorded it with times, and carries a message.
5. A tag object is a named, described pointer, normally to a commit, with its own author and message, and optionally a signature.
6. That is the whole vocabulary. Four nouns.
7. Notice what is missing. There is no “file” object holding a name, no “change” object, no “diff” object, and no “branch” object. Those are all built from the four.

PLAIN38.3.2 a picture in your head#

1. Think about a warehouse that stores photographs of a whole office, taken every day.
2. A blob is one printed page in a filing cabinet. It has no label on it. It is just the paper.
3. A tree is an index card for one room. It says “the page called README is card number f6da25f, the cupboard called docs is index card 80b4ad5”.
4. A commit is the front cover of one day’s survey. It says “here is the index card for the whole office, taken at 10:00 on 5 January, by this person, for this reason, and yesterday’s cover was that one over there”.
5. A tag object is a brass plaque screwed on to one particular day’s cover, saying “this one is version 1.0, blessed by this person on this date”.

Where this comparison breaks:

6. In a warehouse, tomorrow’s survey needs a whole new set of pages. In git, tomorrow’s cover reuses every index card and every page that did not change. Only the changed cards are re-cut.
7. And in a warehouse the covers are dated by the clerk. In git the cover’s own identity is computed from its contents, including the date, so you cannot alter the date and keep the identity.

PLAIN38.3.3 a worked example#

1. Here is a real repository. Three commits, two directories, five files by the end. Every hash below came out of the sandbox.
2. A blob. git cat-file -t says the type, -s says the size in bytes, -p pretty-prints the content:

$ git cat-file -t 0e2a325
blob
$ git cat-file -s 0e2a325
14
$ git cat-file -p 0e2a325
Introduction.

3. Notice: no file name anywhere. This blob does not know it is intro.md.
4. A tree. This is the top directory of the first commit:

$ git cat-file -t b39d894
tree
$ git cat-file -p b39d894
100644 blob f6da25f44e6006699370496c1d1033320df8802b    README.md
040000 tree 80b4ad5b31db187c032b03fca7a3b9a0cbdc2747    docs
040000 tree ed46dd19f6204c92947780231982716d4016b9bb    src

5. Three entries. One blob and two sub-directories. The names live here, in the tree, not in the blobs.
6. A commit. This is the third commit:

$ git cat-file -t b53a9f4
commit
$ git cat-file -p b53a9f4
tree 19f6e3d84c4955832c7f9d39e5205c273dd15d91
parent fb17f98cd37f546dfedb2f91e1b7a9762bf45d17
author KedByte Reader <reader@example.com> 1767594600 +0530
committer KedByte Reader <reader@example.com> 1767594600 +0530

Third commit: add src/util.c

7. That is the entire commit. One tree line, one parent line, two people lines with timestamps, a blank line, then the message.
8. A tag object, made with git tag -a v1.0 -m "First release of the demo":

$ git cat-file -t v1.0
tag
$ git cat-file -p v1.0
object b53a9f46d432aa53e687328841c6dd2b89963809
type commit
tag v1.0
tagger KedByte Reader <reader@example.com> 1767598200 +0530

First release of the demo

9. The tag object is a separate object with its own hash, d54ba1826a59225bc14a2b55e832536cb7a8e4ac, pointing at the commit.

PLAIN38.3.4 what is really happening inside#

1. Every one of the four objects is stored the same way: a header, then the content, hashed and compressed together.
2. The header is the type word, a space, the content length in bytes, then a single zero byte.
3. So the whole rule of the object store is one line: hash = SHA1(type + " " + size + "\0" + content).
4. Let us prove it for all four types with real numbers.
5. Blob. Content is 14 bytes.

$ printf 'blob 14\0Introduction.\n' | sha1sum
0e2a325c7de3369b164d48fcba823bca9c2cec4e

6. Tree. Content is 34 bytes of binary, so we pipe git’s own raw bytes back in:

$ git cat-file tree ed46dd19 | wc -c
34
$ { printf 'tree 34\0'; git cat-file tree ed46dd19; } | sha1sum
ed46dd19f6204c92947780231982716d4016b9bb

7. Commit. Content is 208 bytes:

$ git cat-file commit d9cbb74 | wc -c
208
$ { printf 'commit 208\0'; git cat-file commit d9cbb74; } | sha1sum
d9cbb74f1fb9bce240d20db0c8e545caf68c802b

8. Tag. Content is 156 bytes:

$ { printf 'tag 156\0'; git cat-file tag v1.0; } | sha1sum
d54ba1826a59225bc14a2b55e832536cb7a8e4ac

9. Four for four. There is no secret step. You can reproduce every hash in a git repository with printf and sha1sum.
10. One warning about the tree. git cat-file -p shows you a friendly version. The bytes on disk are not that. Here is the truth:

00000000  31 30 30 36 34 34 20 52  45 41 44 4d 45 2e 6d 64
00000010  00 f6 da 25 f4 4e 60 06  69 93 70 49 6c 1d 10 33
00000020  32 0d f8 80 2b 34 30 30  30 30 20 64 6f 63 73 00
00000030  80 b4 ad 5b 31 db 18 7c  03 2b 03 fc a7 a3 b9 a0
00000040  cb dc 27 47 34 30 30 30  30 20 73 72 63 00 ed 46
00000050  dd 19 f6 20 4c 92 94 77  80 23 19 82 71 6d 40 16
00000060  b9 bb

11. Each entry is: the mode in ASCII, a space, the file name in ASCII, a zero byte, then the 20 raw bytes of the hash. Not 40 hex characters. 20 binary bytes.
12. Also notice the directory mode is stored as 40000, five characters, even though git ls-tree prints 040000. The printed leading zero is cosmetic.

TECHNICAL38.3.5 the engineer’s version#

1. Object format, canonical for both SHA-1 and SHA-256 repositories: <type> SP <size in bytes, decimal ASCII, no leading zeros> NUL <content>.
2. Tree entry format, repeated and concatenated with no separator: <mode ASCII> SP <name> NUL <20-byte raw object id>.
3. Tree entries are sorted by name, with directory names compared as if they ended in a slash. This ordering is part of the format, not a convention: two git implementations must agree or the hashes differ.
4. Only five modes are legal in a tree. All five were produced in the sandbox:

040000 tree 0479003445f4e5a5ff25360c607ca79ffe4e4ea1    d
120000 blob dab8c79946b1756dcd7db770a986ad40d00c07f4    link.txt
100644 blob b9bca019c83a65e6d717d0b6da86215f45dde1b3    plain.txt
100755 blob 4163036efa65bd4a469e752267498f01ea36a55c    run.sh
160000 commit 0f0c4412ec5e57f084c7a39a254a1f9d67f3e965  sub

5. Git records only the executable bit. It does not record read or write permission, owner, group, or modification time. Those are deliberately outside the model.
6. A symbolic link is stored as a blob whose content is the target path. The blob above holds exactly the nine bytes plain.txt.
7. Commit object grammar, in order: one tree line, then zero or more parent lines, then author, then committer, then optional headers such as gpgsig and encoding, then a blank line, then the message.
8. Timestamps are two fields: seconds since the Unix epoch of 1 January 1970 UTC, then the author’s local UTC offset as +HHMM. The offset is stored so the original wall-clock time can be reproduced. It does not affect ordering.
9. Two universal object ids you will meet in real work, and they are the same in every SHA-1 repository on earth:

$ git hash-object -t tree /dev/null
4b825dc642cb6eb9a060e54bf8d69288fbee4904
$ git hash-object -t blob /dev/null
e69de29bb2d1d6434b8b29ae775ad8c2e48c5391

10. The first is the empty tree, useful for diffing a first commit against nothing. The second is the empty file.
11. You can construct all of this by hand. The plumbing path from bytes to a checked-out commit, run in the sandbox:

$ B=$(printf 'Hello from plumbing.\n' | git hash-object -w --stdin)
$ printf '100644 blob %s\thello.txt\n' "$B" | git mktree
ae0afea72a045c4653880d3b19ffd7a4979d8d0d
$ git commit-tree ae0afea -m "Made entirely by plumbing"
f0803fab3f189b2ed4f674bf2c63655cc7d7cfde
$ git update-ref refs/heads/main f0803fa
$ git log --oneline
f0803fa Made entirely by plumbing

12. No git add, no git commit. The high-level commands are convenience over exactly these four steps.

WORDS38.3.6 remember these#

1. Blob — stored file content with no name — an object of type blob, raw bytes, addressed by SHA of blob <size>\0<bytes>.
2. Tree — one directory listing — an object mapping sorted names to modes and object ids, using 20 raw bytes per id.
3. Commit — a snapshot marker with a message and a parent — an object naming one tree, zero or more parents, author, committer and message.
4. Annotated tag — a named marker that is itself a stored object — an object of type tag with tagger, message and optional signature.
5. Mode — the small number in front of each tree entry — the six-digit file mode; only 100644, 100755, 120000, 040000 and 160000 are valid.
6. Gitlink — a pointer to a commit in another repository — a tree entry with mode 160000 recording a submodule’s commit id.

38.4 How the four types compose#

PLAIN38.4.1 in simple words#

1. The four object types stack into one picture.
2. A commit points down at one tree. That tree points at blobs and at more trees. Those point at more blobs.
3. The whole content of the project at one moment hangs off that single commit hash.
4. A commit also points sideways, at its parent commit, which has its own tree, which shares most of the same blobs and sub-trees.
5. When you change one file, git does not copy the project. It creates one new blob for that file, one new tree for the directory containing it, one new tree for each directory above that, and one new commit.
6. Everything else is pointed at again, unchanged.
7. So the cost of a commit is proportional to the depth of the change, not to the size of the project.

PLAIN38.4.2 a picture in your head#

1. Imagine a company organization chart printed on a wall, with a card for each department and a card for each person.
2. Somebody changes their phone number. You do not reprint the wall.
3. You print one new card for that person, one new card for their team naming the new person card, one new card for their division naming the new team card, and one new card for the whole company.
4. Every other card on the wall is still valid and is still pointed at by the new cards.
5. Now you have two complete company charts on the wall, yesterday’s and today’s, and they share almost all their cards.

Where this comparison breaks:

6. Cards on a wall are pointed at by position. Git objects are pointed at by hash, so a “new card” for identical content is impossible. If two people ended up with the identical card, there would be exactly one card.
7. And a wall chart has to be walked from the top. Git can jump directly to any card, at any depth, if it knows the hash.

PLAIN38.4.3 a worked example#

1. Here is the complete object graph of the real three-commit repository, with every object and every arrow. Fifteen objects: 3 commits, 7 trees, 5 blobs.

 COMMITS  (each points left to its parent)

  d9cbb74 <----- fb17f98 <----- b53a9f4
    C1             C2             C3
    |              |              |
    | tree         | tree         | tree
    v              v              v

 ROOT TREES

  b39d894        5b9f378        19f6e3d
   |  |  |        |  |  |        |  |  |
   |  |  +--src-->|  |  +--src-->|  |  +--src--+
   |  +--docs-+   |  +--docs-+   |  +--docs-+  |
   +--README-+|   +--README-+|   +--README-+|  |
             ||             ||             ||  |
             vv             vv             vv  v

2. Rather than draw crossing lines, here is the same information written out. Each root tree has exactly three entries:

 b39d894  (root tree of C1)
   README.md -> blob f6da25f
   docs      -> tree 80b4ad5
   src       -> tree ed46dd1

 5b9f378  (root tree of C2)
   README.md -> blob f6da25f      SAME as C1
   docs      -> tree dbec85e      NEW
   src       -> tree ed46dd1      SAME as C1

 19f6e3d  (root tree of C3)
   README.md -> blob f6da25f      SAME as C1 and C2
   docs      -> tree dbec85e      SAME as C2
   src       -> tree 069dcc5      NEW

3. And the sub-trees, with their blobs:

 80b4ad5  docs at C1        dbec85e  docs at C2 and C3
   intro.md -> 0e2a325        intro.md -> bce6753

 ed46dd1  src at C1 and C2  069dcc5  src at C3
   main.c -> 78f2de1          main.c   -> 78f2de1
                              util.c   -> df76ea6

4. The five blobs, with their real sizes:

5. Now the important part. Commit 2 changed exactly one file, docs/intro.md. Here is what that cost.

 EDIT ONE FILE IN ONE DIRECTORY: C1 -> C2

 NEW objects (4)              REUSED objects (4)
 -------------------          --------------------
 blob   bce6753  intro.md     blob f6da25f README.md
 tree   dbec85e  docs/        tree ed46dd1 src/
 tree   5b9f378  root         blob 78f2de1 src/main.c
 commit fb17f98               blob 0e2a325 old intro.md
                              (kept, still in C1)

6. One blob for the new content. One tree for the directory it is in. One tree for the root above it. One commit. Four objects.
7. The src directory was not touched, so tree ed46dd1 is pointed at again by the new root tree. Not copied. Pointed at.
8. That is the rule: new objects equal one blob, plus one tree per directory level from the change up to the root, plus one commit.
9. Adding src/util.c in commit 3 cost the same shape: new blob df76ea6, new src tree 069dcc5, new root tree 19f6e3d, new commit b53a9f4.

PLAIN38.4.4 what is really happening inside#

1. When you run git commit, git works from a staging area called the index, which holds the intended next snapshot as a flat list of paths and blob ids.
2. Git writes the deepest directories first. For each directory it builds the tree bytes, hashes them, and writes the tree object if it is not already there.
3. Because it hashes before writing, an unchanged directory produces exactly the same tree bytes as last time, so exactly the same hash, so no write happens at all.
4. Then it builds the parent directory’s tree, which now contains the just-computed child hashes, and so on up to the root.
5. Finally it builds the commit object: the root tree hash, the current branch tip as parent, your identity and the time, and your message.
6. It hashes that, writes it, and moves the branch file to the new commit hash.
7. Nothing was diffed. Nothing was compared with the previous commit. The reuse is automatic because equal content produces equal names.
8. And here is the proof that git stores whole content, not changes. The second version of docs/intro.md is stored complete:

$ git cat-file -s bce6753
38
$ git cat-file -p bce6753
Introduction.
Now with a second line.

9. Thirty-eight bytes: the full new file, both lines. Not “add one line at the end”.
10. The diff you see in git show is worked out at the moment you ask, by comparing two trees:

$ git diff-tree -r d9cbb74 fb17f98
:100644 100644 0e2a325... bce6753... M  docs/intro.md

11. Git compared the two trees, saw one entry whose hash differed, and only then computed the line-by-line difference for display.

TECHNICAL38.4.5 the engineer’s version#

1. The object graph is a Merkle DAG. Nodes are objects, edges are embedded object ids, and it is acyclic because an object’s id depends on the ids it contains, so a cycle would require a hash to contain itself.
2. Write cost of a commit is O(d + 1) new objects, where d is the number of directory levels from the deepest change to the root, plus one commit and one blob per changed file.
3. That is why deep directory hierarchies cost slightly more per commit than flat ones, and why a repository with one enormous directory costs more per commit than you might expect: the whole tree object for that directory is rewritten whenever any entry in it changes.
4. Real figures from the sandbox repository at the three-commit point:

5. Only two tree objects were saved in this tiny example. In a repository with thousands of directories and a hundred thousand commits, the saving is the difference between a workable repository and an impossible one.
6. Verify the reuse yourself with git rev-parse on paths:

$ git rev-parse d9cbb74^{tree}:src
ed46dd19f6204c92947780231982716d4016b9bb
$ git rev-parse fb17f98^{tree}:src
ed46dd19f6204c92947780231982716d4016b9bb

7. Identical, from two different commits. One stored object.
8. Implementation detail, not part of the model: git also keeps a cache-tree inside .git/index so that unchanged directories do not even need their tree bytes rebuilt during commit. That is a speed optimization; the resulting hashes are identical either way.

WORDS38.4.6 remember these#

1. Snapshot — the complete state of the project at one commit — the transitive closure of objects reachable from a commit’s root tree.
2. Structural sharing — new versions reusing old parts — persistent data structure behaviour, where unchanged sub-trees are referenced not copied.
3. Index — git’s staging area — .git/index, a binary file listing paths, modes, blob ids and stat data for the next commit.
4. Merkle DAG — a graph where every node names its children by hash — a directed acyclic graph with hash-derived edges, giving whole-graph integrity.
5. diff-tree — the command that compares two snapshots — plumbing that walks two trees and reports entries whose object ids differ.

38.5 Why a commit points at its parent#

PLAIN38.5.1 in simple words#

1. Every commit except the very first one records the hash of the commit that came before it.
2. That single line, parent <hash>, is what turns a pile of snapshots into a history.
3. Follow the parent links backwards and you reach the beginning. That is what git log does.
4. It is not a line, though. It is a graph. A commit can have two parents, which is what a merge is. And two different commits can share one parent, which is what a branch is.
5. It is acyclic, meaning you can never walk forwards and end up where you started. That is guaranteed by the hashes, not by a rule someone enforces.
6. Because a commit’s hash covers its parent’s hash, and that parent’s hash covers its own parent, one hash at the tip covers the entire history behind it.
7. Change anything anywhere in the past and every hash from that point forward changes. History is therefore tamper-evident.
8. Tamper-evident is not the same as tamper-proof. Anyone can rewrite history and hand you the new version. They just cannot do it invisibly if you already know the old hash.

PLAIN38.5.2 a picture in your head#

1. Think of a family of numbered wax seals.
2. Each seal is made by melting together the day’s document and a copy of yesterday’s seal.
3. So today’s seal depends on yesterday’s, which depended on the day before, all the way to the first day.
4. If somebody quietly edits a document from three years ago, they have to make a new seal for that day. That changes the next day’s seal, and every seal after it.
5. You hold only the most recent seal in your hand. That one seal proves the whole chain.

Where this comparison breaks:

6. A wax seal can be forged by someone with the same stamp. A hash cannot be forged without solving arithmetic nobody can currently solve.
7. But the comparison also flatters git. Your seal only proves things if you wrote down what the seal looked like. If you never recorded it, someone can hand you a completely different chain of seals and you cannot tell.
8. And unlike a family seal, git history can legitimately fork. Two people can both build on yesterday’s seal and both be right.

PLAIN38.5.3 a worked example#

1. Here is a real history from the sandbox with a real merge, drawn by git itself:

$ git log --graph --oneline --all
*   7fc6cf8 Merge feature into main
|\
| * 7c7506f Fourth commit: add docs/feature.md on feature
* | 25478c1 Fifth commit: extend README on main
|/
* b53a9f4 Third commit: add src/util.c
* fb17f98 Second commit: edit docs/intro.md
* d9cbb74 First commit: readme, source and docs

2. That is a diamond, not a line. Two commits both have b53a9f4 as parent, and one commit has both of them as parents.
3. Look inside the merge commit. It has two parent lines:

$ git cat-file -p 7fc6cf8
tree b704d3e37ec1cb98ac5712b56206dbe28b5aac6e
parent 25478c16e00d681e80a0db23a6c7bca204e07404
parent 7c7506f6a3b659163ac6f5d9f155a1a9b89f298e
author KedByte Reader <reader@example.com> 1767681000 +0530
committer KedByte Reader <reader@example.com> 1767681000 +0530

Merge feature into main

4. The order of the parent lines matters and is not arbitrary. The first parent is the branch you were on. The second is the branch you merged in.
5. That is what the ^ numbers select:

$ git rev-parse HEAD^1
25478c16e00d681e80a0db23a6c7bca204e07404
$ git rev-parse HEAD^2
7c7506f6a3b659163ac6f5d9f155a1a9b89f298e

6. And the very first commit has no parent line at all. Compare:

$ git cat-file -p d9cbb74
tree b39d894c54aa6e9c3c2545eef059431d7b208ebf
author KedByte Reader <reader@example.com> 1767587400 +0530
committer KedByte Reader <reader@example.com> 1767587400 +0530

First commit: readme, source and docs

7. No parent line. That is a root commit.

PLAIN38.5.4 what is really happening inside#

1. The word “ancestor” has an exact meaning here. Commit A is an ancestor of commit B if you can get from B to A by following parent links, in any number of steps.
2. “Descendant” is the same relation read the other way.
3. Merging looks for the merge base, the closest commit that is an ancestor of both sides. In the example above that is b53a9f4.
4. Immutability is not a rule that git enforces with a lock. It is a consequence of naming.
5. If you want to change a commit’s message, git cannot edit the object, because the object’s name is derived from its bytes including the message. Changing the message produces a different object with a different name.
6. So git commit --amend does not amend anything. It builds a brand new commit object and moves the branch file to point at it. The old object is still on disk.
7. The same is true of rebase, of git filter-repo, and of every other “history rewriting” tool. They all create new objects.
8. The sealing works like this. Suppose an attacker wants to change one character in the first commit’s README.
9. That changes the README blob’s hash. That changes the root tree’s hash. That changes commit 1’s hash. Commit 2 contains commit 1’s hash, so commit 2’s hash changes. And so on to the tip.
10. If you wrote down the tip hash yesterday and it still matches today, then every byte of every file in every commit behind it is unchanged.

TECHNICAL38.5.5 the engineer’s version#

1. Git history is a directed acyclic graph, a DAG, with commits as vertices and parent references as edges. Merge commits have out-degree 2 or more.
2. Octopus merges with three or more parents are legal. The default merge strategy in modern git, ort, refuses to resolve conflicts in an octopus merge, so they are rare in practice.
3. Acyclicity is structural. A commit’s id is a hash over content that includes its parents’ ids, so creating a cycle would require finding a hash preimage, which is why nobody argues about whether cycles are possible.
4. git merge-base A B computes the lowest common ancestor. With criss-cross histories there can be several, and the ort strategy handles that by recursively merging the candidate bases.
5. Topological order is a partial order, not a total one. git log defaults to showing commits in reverse commit-date order, which can mislead when clocks disagree. Use git log --topo-order when the shape matters.
6. Signed commits. A commit signature is stored as a gpgsig header inside the commit object itself, so the signature is covered by the commit hash. Here is a real one, created and verified in the sandbox with an SSH key:

$ git cat-file -p HEAD
tree 4588a5c622c7b494a501343b782d2d928c5e447f
author KedByte Reader <reader@example.com> 1767846600 +0530
committer KedByte Reader <reader@example.com> 1767846600 +0530
gpgsig -----BEGIN SSH SIGNATURE-----
 U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgrLzsfF...
 ...
 -----END SSH SIGNATURE-----

A signed commit

7. The base64 lines are trimmed here to fit the page. The unsigned commit object was 208 bytes; the signed one is 492 bytes.
8. Verification, again real output:

$ git verify-commit HEAD
Good "git" signature for reader@example.com with ED25519 key
SHA256:32dP45eSMmVSt/G/CGvcxl/P+MO3Nwj9xeTh/GSA2wc

9. What a signature adds, precisely: it binds a key to that exact commit hash, and therefore to the whole history behind it. It answers “who asserted this tip”, which the hash alone cannot.
10. What it does not add: it does not stop rewriting. An attacker with commit access rewrites history and signs the new version with their own key. Verification then fails only if you check the key against a list you trust.
11. This was visible in the sandbox. Before an allowed-signers file was configured, git reported the signature as cryptographically good but with “No principal matched”. Good maths, unknown signer. Those are different questions and git keeps them separate.

12. The honest version: git’s tamper-evidence protects you against silent modification of a repository you already have a hash for. It does not tell you whether the history you were handed in the first place is the real one.

WORDS38.5.6 remember these#

1. Parent — the commit that came before — an object id recorded in a parent header inside the commit object.
2. Root commit — a commit with no parent — a commit object with zero parent headers; a repository may have several.
3. Merge commit — a commit joining two lines of work — a commit object with two or more parent headers, first parent being the branch merged into.
4. Ancestor — reachable by walking parents backwards — the transitive closure of the parent relation.
5. Merge base — the closest shared ancestor of two commits — the lowest common ancestor in the commit DAG, computed by git merge-base.
6. Tamper-evident — changes cannot be hidden — any modification alters the object id and propagates to every descendant hash.

38.6 What a branch really is#

PLAIN38.6.1 in simple words#

1. A branch is a file. That is the whole answer.
2. The file lives at .git/refs/heads/ followed by the branch name.
3. Inside that file are 40 hexadecimal characters and a newline. Forty-one bytes.
4. Those 40 characters are the hash of one commit: the tip of the branch.
5. A branch does not contain commits. It points at one commit. The commits it “contains” are simply the ones you reach by walking parents backwards from there.
6. Creating a branch writes one 41-byte file. Nothing is copied. No files are duplicated. That is why branching is nearly free.
7. Deleting a branch deletes one 41-byte file. The commits are untouched.
8. Committing means writing a new commit object and then overwriting those 41 bytes with the new hash.

PLAIN38.6.2 a picture in your head#

1. Think of a long shelf of books, in order, each book naming the one before it.
2. A branch is not a copy of the shelf. It is a sticky note on the spine of one book saying “main”.
3. Adding a branch means writing one more sticky note. It takes a second and costs one note.
4. Moving a branch forward means peeling the note off one book and putting it on the next.
5. Removing a branch means throwing away a sticky note. Every book is still on the shelf.

Where this comparison breaks:

6. A sticky note can fall off and be lost. Git writes the move into a log, the reflog, so you can put it back. Section 38.9 covers that.
7. And a shelf is one physical thing. In git, two people can have sticky notes with the same name on completely different books, and neither is wrong until they try to agree.

PLAIN38.6.3 a worked example#

1. Real output from a fresh sandbox repository with two commits.

$ ls -l .git/refs/heads/
-rw-r--r-- 1 root root 41 Aug 13 03:18 main
$ cat .git/refs/heads/main
bc7fc422aec1054a108922e49253ce8271b1e957

2. Forty-one bytes. Forty hexadecimal characters and a newline, confirmed by dumping the raw bytes:

0000040   7   1   b   1   e   9   5   7  \n

3. Now create two more branches, one at the tip and one at the previous commit:

$ git branch topic
$ git branch spike HEAD~1
$ ls -l .git/refs/heads/
-rw-r--r-- 1 root root 41 main
-rw-r--r-- 1 root root 41 spike
-rw-r--r-- 1 root root 41 topic

4. Three files, 41 bytes each, 123 bytes in total. Their contents:

main   bc7fc422aec1054a108922e49253ce8271b1e957
topic  bc7fc422aec1054a108922e49253ce8271b1e957
spike  1dfe2f9b9521c9a46513a095c135831a8b54ef01

5. main and topic hold the same 40 characters. They are two names for one commit, and the repository is not one byte larger for it.
6. Deleting a branch does not delete the commit:

$ git branch -D spike
Deleted branch spike (was 1dfe2f9).
$ git cat-file -t 1dfe2f9
commit

7. The commit is still there, still readable, still has its tree and its blobs. Only the pointer went.
8. You can move a branch by hand, because it really is just a file:

$ git update-ref refs/heads/topic 1dfe2f9b9521c9a4...
$ git branch -v
* main  0f0c441 three
  topic 1dfe2f9 one

PLAIN38.6.4 what is really happening inside#

1. Sometimes you look for .git/refs/heads/main and it is not there. The branch still exists. It has simply been packed.
2. Git can move all its ref files into one file called .git/packed-refs, to avoid thousands of tiny files.
3. Watch it happen for real:

$ git pack-refs --all
$ ls .git/refs/heads/
(empty)
$ cat .git/packed-refs
# pack-refs with: peeled fully-peeled sorted
bc7fc422aec1054a108922e49253ce8271b1e957 refs/heads/main
1dfe2f9b9521c9a46513a095c135831a8b54ef01 refs/heads/spike
bc7fc422aec1054a108922e49253ce8271b1e957 refs/heads/topic

4. The directory is empty and everything still works, because git looks in both places.
5. The rule is: a loose file wins over the packed file. Commit once more and the loose file comes back for that branch only:

$ git commit -m "three"
$ cat .git/refs/heads/main
0f0c4412ec5e57f084c7a39a254a1f9d67f3e965
$ grep main .git/packed-refs
bc7fc422aec1054a108922e49253ce8271b1e957 refs/heads/main

6. Two different values for refs/heads/main exist on disk at once, and that is fine. The loose file is the current one. The packed entry is stale and will be tidied later.
7. This is why you should never read ref files yourself in a script. Ask git: git rev-parse main, or git show-ref, or git for-each-ref.
8. Contrast with a system where a branch is a copy. If branching meant copying the working tree, then in the sandbox’s 200-commit repository each branch would cost 109,408 bytes of file content. A thousand branches would be about 104 MiB.
9. In git, a thousand branches cost 41,000 bytes of actual content.

TECHNICAL38.6.5 the engineer’s version#

1. A ref is a name in a hierarchical namespace, refs/heads/* for local branches, that resolves to an object id.
2. Storage back-ends, as of Git 2.43: loose files under .git/refs, plus .git/packed-refs. A newer reftable back-end exists as an option in recent releases and is not the default.
3. Measured in the sandbox on a 200-commit repository:

4. Note the gap between 41,000 bytes of content and 4.0 MB of disk. Each 41-byte file still occupies a whole filesystem block. That is the real reason packed-refs exists, and why repositories with tens of thousands of refs feel slow before it is used.
5. Ref updates are made atomic by writing a temporary file and renaming it over the target, and by taking a .lock file. This is why two concurrent commits in the same repository cannot half-write a branch.
6. Historical contrast, and it is worth knowing why git felt revolutionary in
   2005. In CVS, released in 1990, a branch was recorded inside every single file’s history, so branching a large tree touched every file. In Subversion, released in 2000, a branch is a cheap server-side copy, but it appears as a real directory path and switching a working copy is not free.
7. Git’s design goal, stated by Linus Torvalds when he began the project on 3 April 2005 after the BitKeeper licence withdrawal, was that branching and merging should be cheap enough to do casually. Writing 41 bytes achieves that.
8. Convention, not standard: the default branch name. Git shipped master as the default from 2005. GitHub changed the default for new repositories to main in October 2020, and git itself added init.defaultBranch so the choice is configurable. Neither name means anything to the software.

WORDS38.6.6 remember these#

1. Ref — a name that points at a commit — an entry in the refs/ namespace resolving to an object id.
2. Branch — a moving pointer to the tip of a line of work — a ref under refs/heads/ that is advanced automatically when you commit.
3. packed-refs — one file holding many refs — .git/packed-refs, a sorted text file consulted when no loose ref file exists.
4. Loose ref — a ref stored as its own small file — a 41-byte file under .git/refs/, which takes precedence over the packed entry.
5. update-ref — the command that sets a ref by hand — plumbing that writes a ref value with locking and reflog recording.

38.7 HEAD#

PLAIN38.7.1 in simple words#

1. HEAD is another file, at .git/HEAD, and it answers one question: where am I right now.
2. Normally it does not contain a commit hash. It contains the name of a branch.
3. Its whole content is the text ref: refs/heads/main and a newline. Twenty-one bytes.
4. That is called attached HEAD. You are “on” a branch. When you commit, git writes the new commit and then moves that branch forward.
5. Sometimes HEAD contains a commit hash directly instead of a branch name. That is detached HEAD.
6. Detached means you are looking at a commit without being on any branch. You can look, build, test, even commit.
7. It is not an error. It is a normal state. Git even prints a helpful notice when you enter it.
8. The danger is only this: if you commit while detached and then walk away, nothing points at your new commit, so it is easy to lose track of it.

PLAIN38.7.2 a picture in your head#

1. Think of the shelf of books again, with sticky notes for branch names.
2. HEAD is your finger. Normally your finger rests on a sticky note, not on a book. When the note moves, your finger goes with it.
3. Detached HEAD is when you take your finger off the notes and put it directly on a book.
4. You can still read that book, and you can still write new books after it. But no sticky note follows you.
5. If you then move your finger back to a note, the books you wrote are still on the shelf, but with nothing labelling them, they are easy to overlook.

Where this comparison breaks:

6. On a real shelf you would still see the new books. In git nothing is visible in git branch or git log unless something points at it.
7. But git is kinder than the shelf: it writes down where your finger has been. That log is the reflog, and it is how you find those books again.

PLAIN38.7.3 a worked example#

1. Attached state, real output:

$ cat .git/HEAD
ref: refs/heads/main
$ git symbolic-ref HEAD
refs/heads/main
$ git status -sb
## main

2. Now detach on purpose. Git prints a long, genuinely useful notice:

$ git checkout HEAD~2
Note: switching to 'HEAD~2'.

You are in 'detached HEAD' state. You can look around, make
experimental changes and commit them, and you can discard any
commits you make in this state without impacting any branches
by switching back to a branch.

If you want to create a new branch to retain commits you create,
you may do so (now or later) by using -c with the switch command.
Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

HEAD is now at b53a9f4 Third commit: add src/util.c

3. Look at the file now. No branch name. A raw hash:

$ cat .git/HEAD
b53a9f46d432aa53e687328841c6dd2b89963809
$ git symbolic-ref HEAD
fatal: ref HEAD is not a symbolic ref
$ git status -sb
## HEAD (no branch)

4. Commit something while detached and HEAD simply advances by itself:

$ git commit -m "Work done while detached"
$ git rev-parse HEAD
3a0a3d62188c8a51d4d914b5efd000ea8483695b
$ cat .git/HEAD
3a0a3d62188c8a51d4d914b5efd000ea8483695b

5. No branch moved, because no branch was involved.
6. Now leave. Git warns you, by name and by hash:

$ git checkout main
Warning: you are leaving 1 commit behind, not connected to
any of your branches:

  3a0a3d6 Work done while detached

If you want to keep it by creating a new branch, this may be a
good time to do so with:

 git branch <new-branch-name> 3a0a3d6

Switched to branch 'main'

7. That warning contains everything you need to recover. Copy the hash and run git branch keep-this 3a0a3d6.

PLAIN38.7.4 what is really happening inside#

1. HEAD is a symbolic reference: a ref whose value is the name of another ref rather than an object id.
2. When git needs “the current commit”, it reads HEAD. If the content starts with ref:, it follows that name and reads the branch file. Otherwise it uses the hash directly.
3. That one indirection is why committing on a branch moves the branch. Git updates whatever HEAD points to.
4. How you end up detached, in practice:
   1. git checkout <sha> or git checkout <tag> or git checkout origin/main.
   2. git rebase while it is replaying commits.
   3. A continuous integration job doing git checkout $COMMIT_SHA, which is why almost every CI build runs detached.
   4. git bisect, which walks you across commits.
5. How to recover, in order of preference:
   1. If you have not committed: git switch - or git checkout main. Nothing is at risk.
   2. If you have committed and want to keep it: git branch <name> right there, or git switch -c <name>.
   3. If you already left and lost the hash: use the reflog, section 38.9.
6. The commits you made while detached are ordinary objects. They are not marked, not special, not fragile. They are simply unreferenced, and unreferenced objects are eventually swept up by garbage collection.

TECHNICAL38.7.5 the engineer’s version#

1. .git/HEAD is 21 bytes in the attached state and 41 bytes in the detached state. Both are plain text.
2. The plumbing commands are git symbolic-ref HEAD to read or write the symbolic form, and git rev-parse HEAD to resolve either form to an object id.
3. git rev-parse --abbrev-ref HEAD prints the branch name when attached and the literal string HEAD when detached. Scripts use this to decide.
4. Other pseudo-refs live in .git/ alongside HEAD and appear during operations. All of these were observed in the sandbox:

5. advice.detachedHead controls the long notice. Setting it to false silences it, which is standard practice in CI images where every checkout is detached by design.
6. A worktree created with git worktree add gets its own HEAD, stored under .git/worktrees/<name>/HEAD. That is why two worktrees of one repository can sit on different commits while sharing one object store.
7. The honest version: people describe detached HEAD as dangerous. It is not. It is exactly as safe as any commit that no branch points at. The risk is purely about visibility, and the reflog removes most of that risk.

WORDS38.7.6 remember these#

1. HEAD — the file saying where you are — a pseudo-ref at .git/HEAD, usually symbolic.
2. Symbolic ref — a ref pointing at another ref by name — content of the form ref: refs/heads/<name>.
3. Attached HEAD — you are on a branch — HEAD is symbolic and commits advance the named branch.
4. Detached HEAD — you are on a commit with no branch — HEAD holds an object id directly; new commits are unreferenced.
5. Pseudo-ref — a special one-off ref in the git directory — names such as ORIG_HEAD and MERGE_HEAD, written by particular operations.

38.8 The rest of the refs#

PLAIN38.8.1 in simple words#

1. Branches are only one kind of ref. All refs live in the same namespace and all of them work the same way: a name that resolves to an object id.
2. refs/heads/ holds your local branches.
3. refs/tags/ holds tags. There are two kinds and they differ on disk.
4. A lightweight tag is just a ref file pointing straight at a commit. No extra object exists.
5. An annotated tag points at a tag object, and that tag object points at the commit. It carries a tagger, a date, a message and, optionally, a signature.
6. refs/remotes/ holds remote-tracking refs, such as origin/main. These are your local cached memory of what a remote looked like the last time you spoke to it.
7. refs/notes/ holds notes, a way of attaching text to a commit without changing the commit.
8. refs/stash holds stashed work. There are a few other special names too.

PLAIN38.8.2 a picture in your head#

1. Think of the shelf of books once more, and the sticky notes.
2. A branch note is one you move forward every day.
3. A lightweight tag is a note you stick on and never move.
4. An annotated tag is not a note at all. It is a small printed card in a sleeve, with a date, a signature and a sentence explaining why this book matters, and the card names the book.
5. A remote-tracking note is a note in a different colour saying “when I last visited the other library, their note called main was on this book”.
6. It does not move when their note moves. It moves when you next visit.

Where this comparison breaks:

7. On a shelf you could check the other library instantly by looking across the room. In git, checking costs a network round trip, which is exactly why the cached note exists and exactly why it goes stale.

PLAIN38.8.3 a worked example#

1. The complete ref tree of the sandbox repository, exactly as git reports it:

$ git for-each-ref --format='%(objectname:short) %(objecttype) %(refname)'
7c7506f commit refs/heads/feature
70a5d62 commit refs/heads/fix
39bf0c8 commit refs/heads/main
cc1849e commit refs/notes/commits
b53a9f4 commit refs/remotes/origin/main
b53a9f4 commit refs/tags/v1-light
d54ba18 tag    refs/tags/v1.0

2. Look at the last two rows. Both are tags, both point at the same release, but the object types differ.
3. refs/tags/v1-light has type commit. The ref points at the commit directly. That is a lightweight tag.
4. refs/tags/v1.0 has type tag. The ref points at a tag object, which in turn points at the commit. That is an annotated tag.
5. On disk, both are 41-byte files with 40 hex characters. The difference is what those characters name:

.git/refs/tags/v1-light -> b53a9f46...  (a commit)
.git/refs/tags/v1.0     -> d54ba182...  (a tag object)

6. And the tag object itself, which the lightweight tag simply does not have:

$ git cat-file -p v1.0
object b53a9f46d432aa53e687328841c6dd2b89963809
type commit
tag v1.0
tagger KedByte Reader <reader@example.com> 1767598200 +0530

First release of the demo

PLAIN38.8.4 what is really happening inside#

1. Remote-tracking refs are updated by exactly two things: git fetch and git push. Nothing else touches them.
2. So origin/main is not a live reading of the server. It is a note of what the server said last time. This matters enormously and section 38.12 returns to it.
3. git ls-remote is the command that asks the server right now, over the network, without changing anything locally:

$ git ls-remote origin
b53a9f46d432aa53e687328841c6dd2b89963809    HEAD
b53a9f46d432aa53e687328841c6dd2b89963809    refs/heads/main
b53a9f46d432aa53e687328841c6dd2b89963809    refs/tags/v1-light
d54ba1826a59225bc14a2b55e832536cb7a8e4ac    refs/tags/v1.0
b53a9f46d432aa53e687328841c6dd2b89963809    refs/tags/v1.0^{}

4. Notice the last line. The suffix ^{} means “the object this tag finally resolves to after following it through”. Servers advertise both, so clients need not fetch the tag object just to learn what it points at.
5. The packed-refs file records the same fact with a caret line:

d54ba1826a59225bc14a2b55e832536cb7a8e4ac refs/tags/v1.0
^b53a9f46d432aa53e687328841c6dd2b89963809

6. A note is stored as a whole commit and tree of its own, under refs/notes/commits. The tree maps a commit’s hash, used as a file name, to a blob holding the note text.
7. That is why notes can be added and changed without altering the commit they describe. The commit does not know about them.
8. A stash entry is a merge commit with two or three parents: the commit you were on, a commit holding the index state, and if needed one holding untracked files.

$ git cat-file -p refs/stash
tree a5419a1446acbb442a6da16df29e43c605ebd018
parent 39bf0c8354d383aaf21ff05eaff4304acc24f346
parent 96a1b317649d3354ec613c5f61bcc7ba7eba40c1

9. So a stash is not a special storage area. It is ordinary commits with a ref pointing at them.

TECHNICAL38.8.5 the engineer’s version#

1. The ref namespace, with the parts you will actually meet:

2. Beyond those: refs/stash, refs/bisect/* during a bisect, refs/replace/* for object replacement, and on GitHub the server-side refs/pull/<n>/head and refs/pull/<n>/merge, which are a hosting convention rather than part of git.
3. Lightweight versus annotated is a real operational difference, not a style choice:

4. git describe ignores lightweight tags unless you pass --tags. Release engineering therefore uses annotated tags almost universally.
5. The default fetch refspec, written into .git/config by git clone, is +refs/heads/*:refs/remotes/origin/*. The leading plus means non-fast-forward updates are allowed for these refs, which is why a force-push upstream quietly rewrites your origin/* refs.
6. git for-each-ref is the scriptable interface. git show-ref is the older one. Both read loose and packed refs correctly, which hand-written scripts that cat ref files do not.

WORDS38.8.6 remember these#

1. Lightweight tag — a fixed pointer to a commit — a ref under refs/tags/ whose value is a commit id.
2. Annotated tag — a tag with a message and author — a ref pointing at a tag object carrying tagger, date, message and optional signature.
3. Remote-tracking ref — your cached memory of a remote branch — a ref under refs/remotes/, updated only by fetch and push.
4. Refspec — the rule mapping their refs to yours — for example +refs/heads/*:refs/remotes/origin/*.
5. Peeled ref — the object a tag finally points to — shown as <tag>^{} and recorded with a caret line in packed-refs.
6. Note — text attached to a commit after the fact — a blob in a tree under refs/notes/, keyed by the commit’s object id.

38.9 The reflog#

PLAIN38.9.1 in simple words#

1. Every time a ref changes value, git writes a line in a log saying where it was and where it went.
2. That log is the reflog. There is one for HEAD and one for each branch.
3. It records the old hash, the new hash, who did it, when, and which command caused it.
4. This log is the reason “I destroyed my work with git” is almost always false.
5. A hard reset, a bad rebase, a deleted branch, an amended commit: in every case the old commit object is still on disk and the reflog still knows its hash.
6. The reflog is local and private. It is never pushed, never fetched, and never shared. Cloning a repository does not bring its reflog.
7. Entries do not last forever. By default reachable entries survive 90 days and unreachable ones 30 days.

PLAIN38.9.2 a picture in your head#

1. Think of a security camera pointed at the sticky notes on the shelf.
2. It does not record the books. It records every time a note moved, and to where.
3. If you come back and find a note in the wrong place, you rewind the tape, read off which book it used to be on, and put it back.
4. The books never went anywhere. Only the note moved. That is the whole trick.

Where this comparison breaks:

5. A camera records what everyone did. The reflog records only what happened in your copy, on your machine. Your colleague’s reflog is a different tape.
6. And the tape is wiped on a schedule. After about 30 days, a commit that nothing points at can genuinely be collected and deleted.

PLAIN38.9.3 a worked example#

1. Here is a real disaster and a real recovery, done in the sandbox. The starting point is a repository whose main is at a merge commit.

$ git log --oneline -3
7fc6cf8 Merge feature into main
25478c1 Fifth commit: extend README on main
7c7506f Fourth commit: add docs/feature.md on feature

2. Now the mistake. A hard reset two commits back, which throws away the merge and everything after it:

$ git reset --hard HEAD~2
HEAD is now at b53a9f4 Third commit: add src/util.c
$ git log --oneline -3
b53a9f4 Third commit: add src/util.c
fb17f98 Second commit: edit docs/intro.md
d9cbb74 First commit: readme, source and docs
$ ls
README.md  docs  src

3. The merge is gone from the log. Files that only existed after it are gone from the working directory. This is the moment people panic.
4. It is not gone. The commit object is still there and still readable:

$ git cat-file -t 7fc6cf8
commit
$ git cat-file -p 7fc6cf8 | head -3
tree b704d3e37ec1cb98ac5712b56206dbe28b5aac6e
parent 25478c16e00d681e80a0db23a6c7bca204e07404
parent 7c7506f6a3b659163ac6f5d9f155a1a9b89f298e

5. And even without knowing that hash, the reflog tells you it:

$ git reflog
b53a9f4 HEAD@{0}: reset: moving to HEAD~2
7fc6cf8 HEAD@{1}: checkout: moving from 3a0a3d6... to main
3a0a3d6 HEAD@{2}: commit: Work done while detached

6. HEAD@{1} is where HEAD was one move ago: 7fc6cf8. That is the merge.
7. The branch has its own reflog, which is usually the cleaner one to read:

$ git reflog show main
7fc6cf8 main@{0}: merge feature: Merge made by the 'ort' strategy
25478c1 main@{1}: commit: Fifth commit: extend README on main
b53a9f4 main@{2}: commit: Third commit: add src/util.c

8. The recovery is one command:

$ git rev-parse main@{1}
7fc6cf85594a850b727418867d4054af04412e88
$ git reset --hard main@{1}
HEAD is now at 7fc6cf8 Merge feature into main
$ git log --oneline -3
7fc6cf8 Merge feature into main
25478c1 Fifth commit: extend README on main
7c7506f Fourth commit: add docs/feature.md on feature

9. Everything is back. Total elapsed effort: one lookup and one command.

PLAIN38.9.4 what is really happening inside#

1. The reflog is a plain text file per ref. HEAD’s log is .git/logs/HEAD. A branch’s log is .git/logs/refs/heads/<name>.
2. Each line has five fields, tab-separated at the end: old object id, new object id, the person, the timestamp with offset, then the reason.
3. Here is one real line, wrapped across two lines to fit this page:

fb17f98cd37f546dfedb2f91e1b7a9762bf45d17 b53a9f46d432aa53e6873
28841c6dd2b89963809 KedByte Reader <reader@example.com>
1767594600 +0530    commit: Third commit: add src/util.c

4. The very first line of a branch’s log has all zeros as the old id, because the branch did not exist before:

0000000000000000000000000000000000000000 d9cbb74f1fb9bce240d20d
b0c8e545caf68c802b ... commit (initial): First commit ...

5. The @{n} syntax counts entries in that file. main@{0} is the current value, main@{1} is the value before the most recent change, and so on.
6. There is also a time form. main@{2026-01-06} asks what main pointed at on that date. If the log does not go back that far, git says so:

warning: log for 'main' only goes back to Thu, 13 Aug 2026 03:12:55

7. Notice @{n} and ~n are completely different questions. HEAD~1 means “the parent commit in the history”. HEAD@{1} means “wherever HEAD was pointing one move ago”. They frequently give different answers.
8. Reflogs are only kept if core.logAllRefUpdates is on, which it is by default for non-bare repositories. Bare repositories, the kind that sit on a server, do not keep reflogs by default. That is worth remembering before you rely on one.

TECHNICAL38.9.5 the engineer’s version#

1. Expiry defaults, from git’s own documentation:

2. “Unreachable” here means the reflog entry names a commit that is no longer reachable from any ref. Those are the entries you most need after a mistake, and they are the ones that expire soonest.
3. Reflog entries keep their objects alive. git gc treats reflog-referenced objects as reachable, which is why a commit lost by reset is not collected the same afternoon.
4. Proof from the sandbox. Immediately after abandoning a detached commit, git fsck --unreachable reported nothing, because the reflog still held it. Adding --no-reflogs revealed the truth:

$ git fsck --unreachable --no-reflogs
unreachable tree bcb2c24a564e8ff3d64369f03559123b68b04b03
unreachable blob 9a5abed11a92dda94398c590ec782181a789f4c5
unreachable commit 3a0a3d62188c8a51d4d914b5efd000ea8483695b

5. The recovery toolkit, in the order you should reach for it:
   1. git reflog or git reflog show <branch> — find the old hash.
   2. git reset --hard <sha> — move the branch back, discarding current work.
   3. git branch rescue <sha> — safer: give the lost commit a name without touching your current branch.
   4. git fsck --lost-found — last resort, when the reflog was pruned or the repository is bare. Writes dangling objects into .git/lost-found/.
6. git log -g prints the reflog as commits, with full messages, which is often easier to read than git reflog when you need to identify work by content.
7. Danger commands and what saves you from each:

8. What the reflog does not save: uncommitted changes in the working tree. git reset --hard, git checkout -- <file> and git clean -fd destroy work that was never turned into an object, and no log exists for it. Anything you staged with git add survives, as a dangling blob findable with git fsck --lost-found.

WORDS38.9.6 remember these#

1. Reflog — a local log of where each ref has been — per-ref append-only log in .git/logs/, recording old id, new id, actor, time and reason.
2. @{n} — the value a ref had n moves ago — reflog index selector, distinct from the ~n ancestry selector.
3. Unreachable object — an object no ref points at — a candidate for pruning once its reflog entry expires.
4. Dangling object — an unreachable object with nothing pointing at it at all — reported by git fsck and recoverable with --lost-found.
5. Prune — permanently delete unreachable objects — git prune, or the pruning step inside git gc, honouring gc.pruneExpire.

38.10 The commands that reveal the machinery#

PLAIN38.10.1 in simple words#

1. Git has two layers of commands, and it is worth knowing which is which.
2. Porcelain commands are the ones you use daily: add, commit, log, merge, status, push. They are built for people. Their output is meant to read nicely and is allowed to change between versions.
3. Plumbing commands are the low-level ones underneath: cat-file, hash-object, ls-tree, rev-parse, rev-list, update-ref. They are built for scripts. Their output format is stable on purpose.
4. The names are git’s own, from its documentation, and they come from the idea that you see the porcelain of a bathroom while the plumbing does the work.
5. Rule for scripts: never parse porcelain output. Use plumbing, or use the porcelain commands’ --porcelain flags, which are stable despite the confusing name.
6. This section is a tour of the plumbing, with real output for each command.

PLAIN38.10.2 a picture in your head#

1. A car has a dashboard and it has an engine.
2. The dashboard tells you the speed in a form you can act on. The designer may redesign it next year and nobody minds.
3. The engine has bolts of exact sizes that have not changed in a decade, because tools depend on them.
4. Porcelain is the dashboard. Plumbing is the bolts.
5. You drive with the dashboard. You diagnose with the bolts.

Where this comparison breaks:

6. In a car you rarely need the engine. In git the plumbing is genuinely useful during ordinary work, especially when something has gone wrong and you need an answer that is exactly true rather than nicely presented.

PLAIN38.10.3 a worked example#

1. git cat-file with its three most useful flags. -t gives the type, -s the size in bytes, -p the content:

$ git cat-file -t b53a9f4
commit
$ git cat-file -s b53a9f4
247
$ git cat-file -t b53a9f4^{tree}
tree

2. git rev-parse turns any way of naming a commit into the one true hash:

$ git rev-parse HEAD
39bf0c8354d383aaf21ff05eaff4304acc24f346
$ git rev-parse --short HEAD
39bf0c8
$ git rev-parse --short=12 HEAD
39bf0c8354d3
$ git rev-parse --abbrev-ref HEAD
main
$ git rev-parse --git-dir
.git
$ git rev-parse --show-toplevel
/tmp/kb38/demo

3. It also resolves paths inside a commit, which is how you get a blob’s hash without looking at a tree:

$ git rev-parse HEAD:docs/intro.md
bce6753328d0de8aaa42692aba92b99182ded5ce

4. git hash-object computes a name without storing anything, and with -w stores it:

$ printf 'Introduction.\n' | git hash-object --stdin
0e2a325c7de3369b164d48fcba823bca9c2cec4e

5. git ls-tree reads a tree. Plain, recursive, and with sizes:

$ git ls-tree HEAD
100644 blob 6af3a5ad4b99c155602b8f78298d89fc15b61209    README.md
040000 tree 8c33b40fe5eb264b8d43c66f01dc437cc4c90aa5    docs
040000 tree 069dcc578e4a416937da34a3dcc4d209c1048c63    src

$ git ls-tree -r HEAD
100644 blob 6af3a5ad... README.md
100644 blob 76c95066... docs/feature.md
100644 blob bce67533... docs/intro.md
100644 blob 61a98438... docs/mainnote.md
100644 blob 78f2de10... src/main.c
100644 blob df76ea63... src/util.c

6. git count-objects -v tells you the shape of the object store:

$ git count-objects -vH
count: 15
size: 60.00 KiB
in-pack: 31
packs: 1
size-pack: 4.60 KiB

7. git rev-list lists commit hashes, and counts them:

$ git rev-list --count HEAD
7
$ git rev-list --count --all
9

PLAIN38.10.4 what is really happening inside#

1. The revision syntax deserves its own explanation because two of its forms look alike and mean different things.
2. X~n means “go back n steps, always taking the first parent”. HEAD~2 is the grandparent.
3. X^n means “the n-th parent of this one commit”. HEAD^2 is the second parent, which only exists on a merge commit.
4. On the real merge commit in the sandbox:

$ git rev-parse HEAD      # the merge
7fc6cf85594a850b727418867d4054af04412e88
$ git rev-parse HEAD^1    # first parent, the branch merged into
25478c16e00d681e80a0db23a6c7bca204e07404
$ git rev-parse HEAD^2    # second parent, the branch merged in
7c7506f6a3b659163ac6f5d9f155a1a9b89f298e
$ git rev-parse HEAD~1    # same as ^1
25478c16e00d681e80a0db23a6c7bca204e07404
$ git rev-parse HEAD~2    # two first-parent steps back
b53a9f46d432aa53e687328841c6dd2b89963809

5. HEAD~1 and HEAD^1 gave the same answer. HEAD~2 and HEAD^2 gave completely different answers. That is the trap.
6. And branch@{1} is a third thing again: not ancestry at all, but the reflog. In the sandbox main@{1} was 7fc6cf8 while main~1 was a different commit entirely.
7. git show is porcelain that stitches several plumbing operations together. Given a commit it prints the metadata plus a computed diff. Given a blob it prints content. Given a tree it prints a listing.
8. git log --graph --oneline --all is the single most useful shape-revealing command. --all is the important part: without it you only see history reachable from HEAD.

TECHNICAL38.10.5 the engineer’s version#

1. Command classification, with the stability promise:

2. git cat-file --batch-check is the high-throughput form. It reads object names on standard input and writes name, type and size, avoiding one process per object:

$ git cat-file --batch-all-objects --batch-check
0e2a325c7de3369b164d48fcba823bca9c2cec4e blob 14
6af3a5ad4b99c155602b8f78298d89fc15b61209 blob 52
78f2de106c92b0d60772bd5aa6c1e6da7bf71005 blob 29

3. git verify-pack -v lists a packfile’s contents. Columns are: object id, type, size, size in the packfile, offset in the packfile, and for deltified objects the chain depth and the base object id:

5087b1a6efba7b591bda30d3badbd5056b4fab4b blob 111810 15280 30437
23841aba9acef22ead7d8505ba90653bc3544934 blob 78 93 45717 1 5087
6578910fb9573331428c33593424cb14b02cb49a blob 38 52 45810 2 2384

4. Read that carefully. The first blob’s real size is 111,810 bytes and it occupies 15,280 bytes in the pack. The second’s “size” column reads 78, but that is the size of the delta, not of the object. Depth 1 means one hop to its base.
5. git rev-list is the engine under git log. It walks the DAG and emits object ids. --objects also emits the trees and blobs reachable from each commit, which is exactly what git pack-objects consumes during a push.
6. git fsck verifies the whole store: every object’s hash against its content, every referenced object present, every ref resolvable. It is the command that turns content addressing into a usable integrity check.
7. Useful one-liners worth memorizing:

git cat-file -p HEAD^{tree}            # top directory of HEAD
git rev-list --count main ^origin/main # commits not pushed yet
git for-each-ref --sort=-committerdate refs/heads/
git log --graph --oneline --all --decorate
git rev-parse --verify --quiet <sha>   # test existence, no output

WORDS38.10.6 remember these#

1. Porcelain — the commands people use — the human-facing layer whose output format carries no stability promise.
2. Plumbing — the commands scripts use — the low-level layer with stable, machine-readable output.
3. rev-parse — turn a name into a hash — the revision-parsing plumbing that resolves every form of object naming.
4. ~ and ^ — steps back, versus which parent — ~n is n first-parent steps, ^n selects the n-th parent of one commit.
5. --batch-check — ask about many objects at once — git cat-file batch mode giving id, type and size without one process per object.

38.11 How git stores things efficiently#

PLAIN38.11.1 in simple words#

1. Everything so far has described whole snapshots. That sounds wasteful, and on its own it is.
2. Git solves this in two stages.
3. Stage one, loose objects. Each object gets its own file, squeezed with zlib compression. Simple, fast to write, wasteful of disk blocks.
4. Stage two, packfiles. Git gathers thousands of objects into one big file and, inside that file, stores similar objects as differences from one another.
5. This is the important sentence of the whole section: those differences are a storage trick only. Git’s idea of history is still whole snapshots. The deltas exist below that, and you never see them.
6. Git does the packing when you push, when you clone, and when housekeeping runs.
7. git gc is the command that does the housekeeping: pack loose objects, pack refs, trim reflogs, and delete objects nothing points at any more.

PLAIN38.11.2 a picture in your head#

1. Imagine keeping every draft of a long report in a filing cabinet, one folder per draft.
2. At first you photocopy the whole report each time. Two hundred drafts, two hundred thick folders. That is loose objects.
3. Then a clerk goes through the cabinet. She keeps one full copy of the report, and replaces the other 199 folders with slips of paper saying “same as the previous draft, but line 812 now reads this”.
4. The cabinet drawer that was full is now nearly empty.
5. Crucially, when you ask for draft 137, she reconstructs it and hands you a complete report. You never see the slips.

Where this comparison breaks:

6. A clerk works from the previous draft. Git compares each object against a window of candidates and picks whichever gives the smallest difference, which is often not the chronological predecessor.
7. And the clerk’s slips would be useless if the full copy were lost. Git guarantees every chain terminates at a complete object inside the same packfile.

PLAIN38.11.3 a worked example#

1. Here is a real repository built for this section: one text file of about 110 KB, edited and committed 200 times, one line changed each time.
2. Before any packing:

$ git count-objects -vH
count: 600
size: 6.25 MiB
in-pack: 0
packs: 0
$ du -sh .git/objects
7.2M    .git/objects

3. Six hundred loose objects: 200 blobs, 200 trees, 200 commits.
4. The total content, uncompressed, is much larger than what is on disk already:

$ git cat-file --batch-all-objects --batch-check='%(objectsize)'
  (summed)
22463780 bytes = 21.42 MiB

5. So zlib alone took 21.42 MiB of content down to about 4.6 MiB of file bytes, occupying 7.2 MB of disk blocks.
6. Now run housekeeping:

$ git gc
$ git count-objects -vH
count: 0
size: 0 bytes
in-pack: 600
packs: 1
size-pack: 100.87 KiB
$ du -sh .git/objects
140K    .git/objects

7. Read those numbers again. 21.42 MiB of content is now in one packfile of 85,423 bytes, with a 17,872-byte index beside it.

8. That is a reduction of about 263 to 1 against the raw content, and about 54 to 1 against the already-compressed loose form.
9. Where did it come from? Look inside the pack:

5087b1a6... blob 111810 15280 30437
23841aba... blob 78 93 45717 1 5087b1a6...
6578910f... blob 38 52 45810 2 23841aba...
2048ca7c... blob 44 58 45862 3 6578910f...

10. The first blob is stored whole: 111,810 bytes, compressed to 15,280. Every following version is stored as a delta of 38 to 78 bytes.
11. Of the 200 blobs, 198 are stored as deltas and only 2 are stored whole.

PLAIN38.11.4 what is really happening inside#

1. A packfile begins with the four ASCII bytes PACK, then a 4-byte version number, then a 4-byte object count. Here is the real header of the pack above:

00000000  50 41 43 4b 00 00 00 02  00 00 02 58 93 0e 78 9c

2. 50 41 43 4b is PACK. 00 00 00 02 is version 2. 00 00 02 58 is 600 in hexadecimal, exactly the object count git reported.
3. The last 20 bytes of the file are a SHA-1 checksum over everything before them, and that checksum is what the file is named after. The pack above is pack-700da35f...b936f4f9ea0477, and the file’s final bytes are b9 36 f4 f9 ea 04 77.
4. Beside it sits a .idx file starting with the magic bytes ff 74 4f 63 and version 2. It is a sorted table letting git find any object’s offset in the pack without scanning it.
5. Delta selection works like this. Git sorts objects by type, then by a path hint, then by size descending. It then slides a window over that order and, for each object, tries to express it as a difference against the others in the window.
6. Two limits control the result. The window size, default 10, is how many candidates are considered. The depth, default 50, is how many hops a chain may have.
7. The sandbox pack hit that limit exactly:

chain length = 48: 14 objects
chain length = 49: 24 objects
chain length = 50: 27 objects

8. Nothing has depth 51, because 50 is the ceiling. Deeper chains would compress better but cost more to read, because reading one object means applying every delta along the chain.
9. Note the direction. Git usually stores newer objects as deltas against older ones or the reverse, depending on size; the ordering heuristic prefers larger objects as bases. There is no rule that the base is the previous version in time.
10. git gc runs several jobs: repack loose objects into a pack, pack the refs into packed-refs, expire old reflog entries, prune unreachable objects older than the prune window, and update helper indexes such as the commit-graph.

TECHNICAL38.11.5 the engineer’s version#

1. Packfile format version 2 has been the on-disk format since 2005 and is stable. It is also the wire format: what travels during clone, fetch and push is a packfile.
2. Two delta encodings exist inside a pack. OBJ_OFS_DELTA refers to its base by a relative offset within the same pack, and OBJ_REF_DELTA refers to it by full object id. Offsets are smaller, so modern git prefers them locally and uses reference deltas for thin packs on the wire.
3. A thin pack omits bases the receiver is known to have. The receiving side completes it with git index-pack --fix-thin. This is why a push of one small change is measured in kilobytes even in a large repository.
4. Measured configuration and results from the sandbox:

5. Delta compression is completely orthogonal to git’s data model. Every object in a pack still has the same object id it had when loose, still hashes to the same value once reconstructed, and git cat-file -p gives identical output. You cannot tell from the model whether an object is deltified.
6. Consequences that do show through, and they matter in real work:
   1. Binary files that do not delta well, such as compressed images, video and compiled artefacts, cost their full size in every version forever.
   2. core.bigFileThreshold, default 512 MiB, makes git skip delta attempts and store such files whole.
   3. Repacking a very large repository is memory hungry. pack.windowMemory and pack.threads exist to control that.
7. git gc --aggressive re-computes deltas with a much larger window, by default 250. It can take hours on a large repository and typically yields a modest further reduction. It is rarely the right answer.
8. Modern maintenance uses git maintenance run with incremental repacking rather than periodic full git gc. That is a change of practice within the last several years, not a change to the format.
9. Historical note: packfiles were not in the first version of git. Linus Torvalds began the project on 3 April 2005 and committed the first version on 7 April 2005, with the message Initial revision of "git", the information manager from hell. Loose objects came first, and packing was added later in 2005 when the loose-object-only design proved too slow for the Linux kernel history. Junio Hamano took over as maintainer in July 2005 and released version 1.0 on 21 December 2005.

WORDS38.11.6 remember these#

1. Loose object — one object in one file — zlib-compressed file under .git/objects/xx/.
2. Packfile — many objects in one file — the .pack format, version 2, with a .idx lookup table beside it.
3. Delta — an object stored as a difference from another — OBJ_OFS_DELTA or OBJ_REF_DELTA inside a pack, purely a storage encoding.
4. Delta chain depth — how many hops to reach a whole object — controlled by pack.depth, default 50.
5. Thin pack — a pack that assumes you already have the bases — used on the wire, completed by git index-pack --fix-thin.
6. Garbage collection — tidying the object store — git gc, which repacks, packs refs, expires reflogs and prunes unreachable objects.

38.12 Reading the reader’s own situation#

PLAIN38.12.1 in simple words#

1. During the network outage described in this book, the reader’s machine reported origin/main as 0a95cc8.
2. Three separate facts are hidden in that one short line, and telling them apart is the whole skill.
3. First, 0a95cc8 is an abbreviated object id. It is the first 7 of the 40 hexadecimal characters of a commit’s hash.
4. Second, origin/main is a remote-tracking ref. It is a small local file holding that hash.
5. Third, and this is the part people get wrong, that file was written the last time the reader successfully talked to GitHub. It was not read from GitHub at the moment it was displayed.
6. During an outage, nothing on your machine can tell you what the server currently holds. Your machine can only tell you what it remembers.
7. The reader also rebased a local branch onto a moved main, and the branch’s hash changed afterwards. Given everything in this chapter, that was unavoidable.

PLAIN38.12.2 a picture in your head#

1. Imagine you write a friend’s address in your notebook after visiting them.
2. Months later somebody asks where your friend lives. You read out the notebook.
3. You are not reporting where they live. You are reporting where they lived when you last checked.
4. If the phone lines are down, you cannot check. The notebook is all you have, and it is still the same notebook whether they moved yesterday or not.
5. origin/main is that notebook page. git ls-remote is the phone call.

Where this comparison breaks:

6. A person might tell you they moved. A git server never volunteers anything. It answers only when asked, and only over the network.
7. And your notebook page can be updated by two events, not one: a fetch, and also a successful push, because after a push you know what you just put there.

PLAIN38.12.3 a worked example#

1. Start with the string itself. Seven characters, 0a95cc8. What can you learn?

$ git cat-file -t 0a95cc8
commit
$ git cat-file -s 0a95cc8
$ git cat-file -p 0a95cc8

2. Those three commands tell you the type, the size and the full contents: the tree, the parent, the author, the committer and the message.
3. If the object is not present locally, git says so plainly. Run against the sandbox repository, which has never seen that commit:

$ git cat-file -t 0a95cc8
fatal: Not a valid object name 0a95cc8
$ git rev-parse 0a95cc8
fatal: ambiguous argument '0a95cc8': unknown revision or path
not in the working tree.

4. That message is worth reading carefully. Git cannot tell the difference between “no such commit” and “a commit I have not fetched”. Both look identical from here.
5. To expand the abbreviation into the full 40 characters:

$ git rev-parse 0a95cc8
$ git rev-parse --short=12 origin/main

6. To see what the remote-tracking ref currently says, and where that value physically lives:

$ git rev-parse origin/main
b53a9f46d432aa53e687328841c6dd2b89963809
$ cat .git/refs/remotes/origin/main
b53a9f46d432aa53e687328841c6dd2b89963809

7. And to ask the server, right now, over the network:

$ git ls-remote origin refs/heads/main
b53a9f46d432aa53e687328841c6dd2b89963809    refs/heads/main

8. Compare those two values. If they differ, your cached ref is stale. If git ls-remote fails, you have learned about the network and nothing at all about the server’s contents.

PLAIN38.12.4 what is really happening inside#

1. Why a rebase changes the hash. Recall the rule from section 38.3: a commit’s name is the hash of its bytes, and its bytes include the tree, the parent, both people, both timestamps and the message.
2. A rebase does not move a commit. It cannot. It builds a new commit with the same message and the same changes but a different parent.
3. Here is exactly that, done in the sandbox. A branch fix was created off main, then main moved on, then fix was rebased.
4. Before the rebase:

tree 8376d5522743401a5e02cb99e72435cb6930e5a6
parent 7fc6cf85594a850b727418867d4054af04412e88
author KedByte Reader <reader@example.com> 1767760200 +0530
committer KedByte Reader <reader@example.com> 1767760200 +0530

Fix: add docs/fix.md

Its hash was 26c500778e51ea3a240b916e1ca97c33f2c37f55.

5. After the rebase:

tree 353596612537fa8a5e94b31dfa28bd33108ad139
parent 39bf0c8354d383aaf21ff05eaff4304acc24f346
author KedByte Reader <reader@example.com> 1767760200 +0530
committer KedByte Reader <reader@example.com> 1767767400 +0530

Fix: add docs/fix.md

Its hash is now 70a5d6220cc7944c8b3824701b4952027bc539af.

6. Compare the two blocks line by line. The message is identical. The author line is identical, including the original author timestamp.
7. Three things changed: the parent, because it is now on top of main; the tree, because the snapshot now also contains main’s newer file; and the committer timestamp, because the commit was recorded again just now.
8. Any one of those three would have been enough to change the hash. All three changed.
9. The old commit still exists. Nothing was destroyed:

$ git reflog fix
70a5d62 fix@{0}: rebase (finish): refs/heads/fix onto 39bf0c8
26c5007 fix@{1}: commit: Fix: add docs/fix.md
7fc6cf8 fix@{2}: branch: Created from main
$ git cat-file -t 26c5007
commit

10. So “the SHA changed after a rebase” is not really true. A different commit now exists, and the branch file was rewritten to point at it. Both commits are on disk.

TECHNICAL38.12.5 the engineer’s version#

1. Abbreviation length. Git’s default is core.abbrev=auto, which picks a length long enough that a collision among the objects present is unlikely, with a floor of 7 characters. Small repositories therefore show 7.
2. A 7-character prefix is 28 bits, about 268 million values. By the birthday bound, ambiguity becomes likely once a repository holds a few tens of thousands of objects, which is why large repositories abbreviate to 10, 12 or more.
3. An abbreviation is only unique within one repository at one moment. Adding objects can make a previously unique prefix ambiguous. Never store an abbreviated hash as a durable identifier; store all 40 characters.
4. What actually updates refs/remotes/origin/*:

5. The session evidence, applied. During the outage the reader had cached origin/main as 0a95cc8. That was a true statement about the past and carried no information about the present.
6. Two pushes in that session failed with send-pack: unexpected disconnect while reading sideband packet. The write may or may not have landed; a disconnect while reading the response tells you nothing about the request. The correct next step is not to guess but to re-query with git ls-remote origin refs/heads/<branch> and compare hashes.
7. This is why verifying by hash beats verifying by name. A branch name is a pointer that anyone can move. A commit hash is the content itself. In the same session, continuous integration was checked by run identifier against the exact head SHA rather than by trusting a coloured tick next to a branch name, and that is the stronger check for exactly this reason.
8. Practical checklist for “is my local view of the remote correct”:

git rev-parse origin/main              # what I remember
git ls-remote origin refs/heads/main   # what is true now
git fetch origin && git rev-parse origin/main
git log --oneline origin/main..main    # mine, not theirs
git log --oneline main..origin/main    # theirs, not mine

9. And after a rebase, the correct push is git push --force-with-lease, not --force. The lease form refuses if the remote moved since your last fetch, which converts a silent overwrite of someone else’s work into an error you can read.

WORDS38.12.6 remember these#

1. Abbreviated object id — the first few characters of a hash — a unique prefix within one repository, length chosen by core.abbrev.
2. Remote-tracking ref — your cached copy of a remote branch pointer — a ref under refs/remotes/, updated only by fetch and push.
3. ls-remote — ask the server what it holds right now — a network query that lists the remote’s refs without changing anything locally.
4. Rebase — replay commits onto a new base — creates new commit objects with new parents, therefore new object ids.
5. --force-with-lease — force push, but only if nothing moved — a compare-and- swap on the remote ref against your last known value.

38.13 Why this design is good, and what it costs#

PLAIN38.13.1 in simple words#

1. The benefits all fall out of one decision: name things by their content.
2. Deduplication across all of history. The same file content, in any commit, on any branch, from any author, is stored once.
3. Integrity for free. Every read can be checked against the name. Damage and tampering are both detectable without any extra machinery.
4. Cheap branching. A branch is 41 bytes, so having twenty of them costs less than one photograph.
5. Distributed operation with no coordination. Two people can create commits at the same time, offline, with no server involved, and the results will never clash, because names are computed from content and not assigned by an authority.
6. That last point is why the reader could keep committing all through a network outage. Nothing in git commit needs a server.
7. The costs are real too.
8. Large binary files are handled badly. Rewriting history is confusing. And the whole model takes genuine effort to learn, which is why this chapter exists.

PLAIN38.13.2 a picture in your head#

1. Think of two libraries in two cities that never phone each other.
2. Each catalogues books by a code derived from the book’s exact text.
3. When they finally exchange catalogues, matching codes mean identical books, with no argument and no committee.
4. Different codes mean different books, even if the titles match.
5. Neither library needed permission from the other to catalogue anything.

Where this comparison breaks:

6. Real libraries have to decide which copy is the official one. Git also has to decide, and it does not decide by content. A human decides, by choosing what main points at on the server.
7. Content addressing removes the need to coordinate naming. It does not remove the need to agree on what is authoritative.

PLAIN38.13.3 a worked example#

1. Deduplication, measured. In the sandbox, four files with three identical contents produced two blobs, not four.
2. Integrity, measured. A single bit was flipped in one loose object file:

$ git fsck
error: inflate: data stream error (incorrect data check)
error: unable to unpack header of .git/objects/6f/e3e873...
error: 6fe3e873...: object corrupt or missing
missing blob 6fe3e87371433ed417646e3cb2354cfb9cff08d3

3. That one was caught by zlib’s own checksum. So a more careful attack was tried: a perfectly well-formed object, holding different content, written under the old name.

$ git cat-file -p 6fe3e87
The EVIL three words.

$ git fsck
error: 8d71e45d309cb1328988a64591163f5c4f8f43f1: hash-path
mismatch, found at: .git/objects/6f/e3e873...
missing blob 6fe3e87371433ed417646e3cb2354cfb9cff08d3

4. Read that carefully, because it contains both the strength and an important limitation.
5. The strength: git fsck computed the content’s true name, 8d71e45..., found it stored under 6fe3e87..., and reported the mismatch. The tamper is caught with certainty.
6. The limitation, and it is honest: git cat-file -p handed back the evil content without complaint. Git does not re-verify every loose object on every ordinary read. Verification happens on transfer, when configured, and when you ask with fsck.
7. Cheap branching, measured: 1,000 branches created in 2.28 seconds, using 41,000 bytes of content.
8. Distributed operation: two commits created independently, offline, will have different hashes unless they are byte-for-byte identical, in which case they are the same commit and it does not matter.

PLAIN38.13.4 what is really happening inside#

1. Now the costs, honestly.
2. Large binary files. Git stores whole content per version. A 50 MB video edited ten times is ten blobs. Compressed formats do not delta, so the packfile saves almost nothing.
3. Worse, a clone fetches all of history by default. The 500 MB stays in every clone forever, even after the file is deleted, because the old commits still reference it.
4. The workarounds are all partial: git lfs, which stores a pointer file in git and the bytes elsewhere; shallow clones with --depth; partial clones with --filter=blob:none; and rewriting history with git filter-repo, which changes every hash from the rewrite point onwards.
5. History rewriting is confusing precisely because it is not rewriting. It is creating parallel objects and moving pointers. The old objects linger, other people still have the old hashes, and any branch built on the old ones must itself be rebuilt.
6. The model takes effort. Most tools let you learn the commands without the data model. Git punishes that. Almost every confusing git error makes immediate sense once you know that commits are immutable snapshots and branches are movable pointers, and almost none of them make sense otherwise.
7. There are smaller costs too. Git tracks content, not files, so it has no record of a rename; it infers renames when displaying a diff, and the inference can be wrong. It records only the executable bit, so full permissions and file ownership need a separate tool. And empty directories cannot be represented at all, because a tree with no entries in it is simply not referenced.

TECHNICAL38.13.5 the engineer’s version#

1. Benefits and their mechanisms, stated precisely:

2. The distributed property is worth stating exactly. Git needs no central allocator of identifiers because the identifier space is a hash of content, and independent parties producing identical content produce identical ids by construction. This is the same property that lets content-addressed systems such as IPFS and container image registries work.
3. Costs, with the mitigations and their prices:

4. core.bigFileThreshold, default 512 MiB, controls when git stops attempting delta compression. Below it, git still tries, and for compressed media it still fails to save anything.
5. Rename detection is a display-time heuristic controlled by diff.renameLimit and a similarity threshold, by default 50 percent. It is not stored, it is recomputed, and on very large changesets git may give up entirely and report the rename as a delete plus an add.
6. Where experts disagree: whether git’s monorepo behaviour at very large scale is a flaw in the model or an implementation gap. Facebook chose Mercurial in 2013 partly over this, and Microsoft built a virtual filesystem layer for the Windows repository, later replaced by scalar and partial clone in git itself. Both camps agree the model is sound; they disagree about how far it scales without special tooling.
7. Established fact: the object model has not changed since 2005, other than the addition of the optional SHA-256 format. Active work: the SHA-256 transition and the reftable ref back-end. Marketing claim territory: any statement that one hosting provider makes git itself faster. Hosting affects the network path, not the object model.

WORDS38.13.6 remember these#

1. Monorepo — one repository holding many projects — a scaling pattern that stresses git’s whole-history clone model.
2. Git LFS — a way to keep big files out of git — Large File Storage, which commits a small pointer file and stores the bytes on a separate server.
3. Partial clone — a clone that skips file contents until needed — enabled with --filter=blob:none, fetching blobs on demand.
4. Shallow clone — a clone with truncated history — --depth <n>, which omits older commits and their objects.
5. Rename detection — noticing a file moved — a similarity heuristic computed at diff time, never stored in any object.

38.98 Common wrong ideas#

1. Wrong: a commit stores a diff, the changes you made. Right: a commit names one tree, which is a complete snapshot of every file. The second version of docs/intro.md in this chapter is stored as all 38 bytes of the new file, not as “add one line”.
2. Wrong: a branch contains commits. Right: a branch is a 41-byte file holding one commit’s hash. The commits you see are the ones reachable by walking parent links backwards from there, and they may be reachable from twenty other branches too.
3. Wrong: git compresses by storing diffs between versions of a file. Right: git compresses in two independent ways. zlib compresses each object separately, and packfiles store some objects as deltas against any similar object, chosen by a size and window heuristic, not by version order. Neither is part of the history model.
4. Wrong: deleting a branch deletes its commits. Right: it deletes one small file. The commits stay until nothing references them, their reflog entries expire, and garbage collection runs, which by default is at least 30 days.
5. Wrong: git commit --amend edits the last commit. Right: it builds a new commit object with a new hash and moves the branch to it. The original is still on disk and is listed in the reflog.
6. Wrong: detached HEAD is an error state. Right: it is a normal state meaning HEAD names a commit instead of a branch. Every CI checkout runs this way. The only risk is losing sight of commits you make there.
7. Wrong: origin/main shows what is on the server. Right: it shows what was on the server the last time you fetched or pushed. Only git ls-remote or a fresh git fetch asks the server.
8. Wrong: rebasing moves commits onto a new base. Right: it creates new commit objects with new parents, new trees and new committer timestamps, therefore new hashes. The originals remain until they are collected.
9. Wrong: because git uses SHA-1, and SHA-1 is broken, git repositories can be forged. Right: SHA-1 collision resistance is broken in general, but since Git 2.13 in May 2017 git uses a collision-detecting SHA-1 that refuses known attack patterns, and a useful attack must also survive review and transport. The risk is real but it is not “anyone can forge a commit”.
10. Wrong: a tag is just a branch that does not move. Right: a lightweight tag is close to that, but an annotated tag is a separate stored object with its own hash, tagger, date, message and optional signature, and git describe treats the two differently.

38.99 Chapter summary in 20 lines#

1. Git names every stored thing by hashing the thing itself. That single decision explains almost everything else.
2. The name is SHA-1 of <type> <size>\0<content>, printed as 40 hexadecimal characters, and you can reproduce it with printf and sha1sum.
3. SHA-1’s collision resistance was publicly broken by CWI Amsterdam and Google on 23 February 2017, at about 2^63.1 evaluations.
4. Git responded in version 2.13, May 2017, by making the collision-detecting SHA-1 of Marc Stevens and Dan Shumow the default; a SHA-256 object format exists but is still not the default in 2026.
5. There are four object types. A blob is file content with no name. A tree is one directory listing with modes. A commit names one tree and zero or more parents. A tag object is an annotated pointer.
6. Tree entries store the mode in ASCII, the name, a zero byte, and 20 raw bytes of hash. Only five modes are legal, including 120000 for symlinks and 160000 for submodules.
7. Changing one file creates one new blob, one new tree per directory level up to the root, and one new commit. Everything else is pointed at again.
8. In the worked repository, editing docs/intro.md created 4 new objects and reused 4 existing ones.
9. A commit’s hash covers its parent’s hash, so one tip hash seals the entire history behind it. History is immutable because names are derived from content.
10. History is a directed acyclic graph, not a line. Merge commits have two or more parents, and ^1 and ^2 select between them.
11. A branch is a file of 40 hexadecimal characters and a newline. Creating one writes 41 bytes; 1,000 branches took 2.28 seconds and 41,000 bytes.
12. Refs may be packed into .git/packed-refs, in which case the individual files vanish. A loose file always wins over the packed entry.
13. HEAD normally holds ref: refs/heads/main. When it holds a raw hash you are detached, which is normal, not an error, and recoverable.
14. Other refs live under refs/tags, refs/remotes and refs/notes, plus pseudo-refs such as ORIG_HEAD and FETCH_HEAD.
15. The reflog records every ref movement locally, keeps reachable entries 90 days and unreachable ones 30, and turns a catastrophic git reset --hard into a one-line recovery with git reset --hard main@{1}.
16. Plumbing commands show the machinery: cat-file, rev-parse, hash-object, ls-tree, rev-list, count-objects, verify-pack. Porcelain commands are for people and their output may change.
17. Storage is loose objects first, then packfiles. In the worked example 21.42 MiB of object content became one packfile of 85,423 bytes, with 198 of 200 blobs stored as deltas up to depth 50.
18. Deltas are storage only. Git’s model is still whole snapshots, and the object ids are identical either way.
19. The reader’s origin/main of 0a95cc8 was a cached local memory, not a live fact; only git ls-remote or a fresh fetch asks the server.
20. The rebased branch had to get a new hash, because its parent, its tree and its committer timestamp all changed, and all three go into the hash.