37.0 What this chapter gives you#
- You will be able to say what version control is for, and name the six real problems it solves, rather than repeating that it “saves your work”.
- You will be able to state the difference between a centralized and a distributed version control system in one sentence, and defend it.
- You will be able to explain why the reader could keep committing to branches for the whole length of their network outage, and push everything later.
- You will be able to open a
.git directory, name every entry inside it, and say what each one holds.
- You will be able to say what “the whole history is local” really means, with real numbers: a 316 MiB
.git next to a 60 MiB working tree.
- You will be able to explain that
origin is a nickname for a URL in a text file, that it has no authority, and that a repository may have none.
- You will be able to explain that
origin/main is a memory, not a live reading, and say exactly which commands refresh it and which never do.
- You will be able to classify every common git command as local or remote, and explain why the split falls exactly where it does.
- You will be able to state the honest costs of the distributed model, and say why large game studios and hardware firms still buy Perforce.
- You will be able to say plainly that git and GitHub are different things, and point at the line between them.
37.1 What version control is for#
PLAIN37.1.1 in simple words#
- You are writing something over many days. Code, a report, a design.
- It changes. Sometimes it gets better. Sometimes you break it.
- A version control system is a program that remembers every state your files have ever been in, and lets you go back to any of them.
- That one sentence hides six separate problems. Here they are.
- History. What did this file look like three weeks ago, before the change that broke everything?
- Undo. Put the whole project back exactly as it was on Tuesday, including files you deleted and files you renamed.
- Blame. Who wrote this strange line, when, and what else did they change at the same moment, and why did they say they did it?
- Parallel work. You want to try a risky idea without disturbing the working version, and abandon it cheaply if it fails.
- Backup. Your laptop is stolen. The work is not gone, because a copy exists somewhere else.
- Collaboration. Four people edit the same project at once and the system combines their work instead of one person silently erasing another.
- Notice that only two of those six need another computer. Backup needs one. Collaboration needs one. The other four are questions about your own files.
- Hold on to that. It is the whole answer to why the reader could keep working with no network.
PLAIN37.1.2 a picture in your head#
- Before version control existed, people did the obvious thing. They copied the folder.
report, then report-final, then report-final-2, then report-final-USE-THIS-ONE, then report-final-v2-jan14-shikhar.
- It works for about a week. Then it stops working, for reasons that are worth naming one at a time.
- You cannot tell what changed between two copies without opening both and reading them side by side, line by line.
- You cannot tell why anything changed. The folder name holds a date at best. It never holds a reason.
- Disk use grows with the number of copies, not with the amount of change. Ten copies of a 200 MB project is 2 GB, even if you only edited one line.
- Two people cannot work at once. If both copy the folder and both edit, you have two folders and no way to combine them except by hand.
- And the names lie.
final is never final. Everyone has met a final-v3-really-final.
Where this comparison breaks:
- Dated folders are not useless. For a one-person, one-week job with no branching, they are genuinely fine, and pretending otherwise is dishonest.
- Version control’s real win is not that it saves copies. It is that it saves the reasons and the relationships between copies. A folder cannot say “this version came from that version, and here is why”.
- Also, git does not actually store a full copy per version, which is why ten versions of a big project cost far less than ten folders. That is 37.5.
PLAIN37.1.3 a worked example#
- Here is a real bug hunt, done with folders, then done with version control.
- The situation: a program worked on 1 April and is broken on 13 August. There were 400 changes in between.
- With dated folders you have perhaps twelve snapshots. You open them one at a time until you find the first broken one. Then you diff two folders that are three weeks apart and stare at 6,000 changed lines.
- With version control you run one command that does a binary search across all 400 changes.
- Here it is running for real, in a sandbox, on the git project’s own source repository, which holds 85,263 commits.
$ git bisect start
$ git bisect bad
$ git bisect good <first commit>
Bisecting: 2 revisions left to test after this (roughly 1 step)
[c35f063d2751d7340c2cdd4fe0046a137c9164c4] Offline commit 2
- It halves the range every step. 400 changes needs about 9 tests, because 2 to the power 9 is 512.
- At the end you do not get “somewhere in three weeks”. You get one commit, one author, one timestamp, one message, and the exact lines it touched.
- And every single step of that ran with no network at all. That matters later.
PLAIN37.1.4 what is really happening inside#
- A version control system stores two different kinds of thing.
- First, content: the bytes of your files at various moments.
- Second, metadata: who, when, why, and which earlier state this one came from.
- The second kind is what turns a pile of backups into a history. Backups are a set. A history is a chain.
- Systems differ in how they store the content. There are three families.
- Full snapshots: keep every version whole. Simple, fast to read, wasteful on disk unless you compress hard afterwards. This is git’s model.
- Forward deltas: keep the oldest version whole plus a list of edits to move forward. Cheap to store, slow to get the newest version, because you must replay every edit.
- Reverse deltas: keep the newest version whole plus edits to walk backward. Fast for the common case, because you usually want the newest. This is what RCS chose in 1982.
- Git looks like family one and behaves like family two after packing. It stores each version as a whole object, then later rewrites groups of objects into a compressed pack with deltas inside it. You never see this happen.
TECHNICAL37.1.5 the engineer’s version#
- The lineage of version control systems, with checkable dates.
| SCCS |
1972 |
Rochkind, Bell Labs |
| RCS |
1982 |
Tichy, Purdue |
| CVS |
1986 / 1990 |
Grune, then Berliner |
| Subversion |
2000 |
CollabNet |
| BitKeeper |
2000 |
McVoy, BitMover |
| Git |
2005 |
Torvalds |
- SCCS, the Source Code Control System, was written by Marc Rochkind at Bell Labs in late 1972, first for an IBM System/370 running OS/360.
- He rewrote it in C for Unix on a PDP-11 in 1973. The first publicly available version, SCCS version 4, shipped on 18 February 1977 with the Programmer’s Workbench edition of Unix.
- SCCS stored interleaved deltas: all versions of a file woven into one file, with markers saying which lines belong to which revision.
- RCS, the Revision Control System, came from Walter F. Tichy at Purdue University, first published in 1982.
- RCS switched to reverse deltas. Tichy’s argument was that recent revisions are read far more often than old ones, so the newest should be the cheap one to fetch.
- Both SCCS and RCS versioned one file at a time. There was no concept of a project-wide change. A change touching four files was four unrelated events.
- CVS, the Concurrent Versions System, began as shell scripts by Dick Grune in 1986. Brian Berliner’s C rewrite started in April 1989, and CVS version 1.0 was submitted to the Free Software Foundation on 19 November 1990.
- CVS was a front end over RCS. It added project-wide operation and a client or server split, but it still had no atomic commit: an interrupted commit could leave half the files updated.
- Subversion was founded by CollabNet in 2000, explicitly to be “a better CVS”. Version 1.0 was released in February 2004. It gave the world atomic commits, real directory versioning, and proper renames.
- Subversion became an Apache Incubator project in November 2009 and a top-level Apache project on 17 February 2010.
- Subversion is still centralized. That is the hinge of this chapter, and 37.2 takes it apart.
- Tools to observe history in git:
git log, git log -p, git log --stat, git blame, git bisect, git show, git diff, git reflog.
WORDS37.1.6 remember these#
- Version control — a program that remembers every state of your files — a system that records content plus metadata and the parent relationships between revisions.
- Commit — one saved point in the history — an immutable object naming a tree, zero or more parents, an author, a committer and a message.
- Delta — the difference between two versions — a stored edit script that reconstructs one revision from another.
- Snapshot — a whole copy of everything at one moment — a tree object recording the complete state of the project at one commit.
- Blame — finding who last changed each line — per-line attribution computed by walking history backward through renames and copies.
- Bisect — halving the search to find a breaking change — automated binary search over the commit graph with
git bisect.
37.2 Centralized versus distributed#
PLAIN37.2.1 in simple words#
- This is the most important section in the chapter. Everything else follows from it.
- There are two ways to build a version control system, and they differ in one question: where does the history live?
- In a centralized system, the history lives on a server. Exactly one copy exists, in one place.
- What is on your machine is a working copy. It is the files as they exist at one chosen revision, plus a little bookkeeping. It is not the history.
- So when you save a change, that change has to travel to the server, because the server is the only thing that can record it.
- Which means: a commit is a network operation. No network, no commit. The server is down, you cannot save your work as a version. You can only edit files and hope.
- In a distributed system, every copy is a complete repository. Not a working copy. A repository, with the entire history in it.
- Your machine has every commit, every version of every file, every branch, right there on your own disk.
- So when you save a change, nothing has to travel anywhere. The change is recorded in a directory on your laptop.
- Which means: a commit is a local file operation. No network needed. No server needed. No permission needed.
- And that is the whole answer to the reader’s question. During their outage they were writing to their own disk. There was nothing for the network to break.
PLAIN37.2.2 a picture in your head#
- Centralized version control is a library with one copy of each book.
- To read a book you go to the library. To add a chapter you go to the library, because the book is there and nowhere else.
- If the library is shut, or the road is flooded, you cannot add a chapter. You can write on loose paper at home, but that writing is not in the book yet, and nothing keeps track of it.
- Distributed version control is everyone owning a full photocopy of every book, including all the old editions.
- You write a new chapter into your own copy immediately. It is a real chapter in a real book, dated and numbered, the moment you write it.
- Later, when you next meet, you and a friend compare copies and exchange the chapters each of you is missing.
- There is no library in this picture. There can be one by agreement, and there usually is, but the system does not require it.
Where this comparison breaks:
- Photocopying every book sounds absurdly wasteful, and for paper it would be. Text compresses extraordinarily well, so in practice it is cheap. Real figures are in 37.5.
- The picture also suggests that merging two copies is easy. It is not always. If you and your friend both rewrote page 40, someone has to decide. Git can combine most changes automatically and refuses when it genuinely cannot.
- And the honest version: a distributed system does not remove the need for a shared place. Teams still nominate one. The difference is that the shared place is a convention, not a structural requirement.
PLAIN37.2.3 a worked example#
- Two engineers, same task, same broken link. One uses Subversion. One uses git. Here is what each can do.
| Edit a file |
works |
works |
| See old version |
needs server |
works |
| Commit a change |
needs server |
works |
| Create a branch |
needs server |
works |
| See full log |
needs server |
works |
| Merge two branches |
local, mostly |
works |
| Get others’ new work |
needs server |
needs server |
| Publish your work |
needs server |
needs server |
- In Subversion, five of those eight become impossible the moment the link dies. In git, six of the eight keep working.
- The two that stop are exactly the two that involve another computer: receiving other people’s work, and sending yours.
- That is not a coincidence or a clever optimization. It is the direct, inevitable consequence of where the history is stored.
- Here is the real demonstration, run in a sandbox with the remote deliberately made unreachable.
$ git commit -m "Committed during the outage"
COMMIT SUCCEEDED with no network
$ git fetch
fatal: Could not read from remote repository.
$ git ls-remote
fatal: Could not read from remote repository.
- Same repository, same second, same broken remote. The local command succeeded. The two remote commands failed instantly and cleanly.
PLAIN37.2.4 what is really happening inside#
- Let us be exact about what each system keeps on your disk.
- A Subversion working copy has a
.svn directory. Historically one per folder, and since Subversion 1.7 in 2011, a single one at the top.
- Inside it is a record of which revision you checked out, and a pristine copy of each file at that revision, so that
svn diff and svn revert can work without the server.
- That is a genuine and often-forgotten point: Subversion is not fully online. Local diff and local revert work offline, because of those pristine copies.
- What it does not have is any other revision. It cannot show you revision 900 if you checked out revision 1000. It has two states: yours and the one you checked out.
- A git repository has a
.git directory containing every object ever created in that project’s history, and the pointers that organize them.
- So
git log is a walk over local files. git diff HEAD~50 reconstructs a fifty-commit-old state from local files. git checkout of a two-year-old tag reads local files.
- Here is the structural difference, drawn.
CENTRALIZED (Subversion, CVS, Perforce)
[ server ] <- the history lives here, one copy
|
| network required to commit
|
[ laptop ] working copy = one revision + pristine files
DISTRIBUTED (Git, Mercurial)
[ server ] a repository, by agreement only
|
| network required only to sync
|
[ laptop ] a repository = ALL commits, ALL versions
- In the top picture, cutting the line stops work. In the bottom picture, cutting the line stops sharing.
TECHNICAL37.2.5 the engineer’s version#
- The formal distinction: in a centralized VCS the repository is a single authoritative store, usually addressed by revision numbers that are globally ordered. In a distributed VCS every clone holds the full object database, and revisions are named by content hash because no global ordering exists.
- That last clause is the deep reason git uses SHA-1 hashes rather than numbers like
r4213. With no central authority, nobody can hand out sequential numbers, so identity must be computed from content instead.
- Subversion can number commits 1, 2, 3 because exactly one process assigns them. Git cannot, so it names a commit
e83c5163316f89bfbde7d9ab23ca2e25604af290.
| History location |
server only |
every clone |
| Commit needs network |
yes |
no |
| Revision names |
integers |
content hashes |
| Branch cost |
server-side copy |
one 41-byte file |
| Offline log or diff |
no |
yes |
| Per-file locking |
supported |
not supported |
- Real timings, measured in a sandbox on the git project’s own repository, 85,263 commits and 4,845 tracked files, git 2.43.0 on Linux.
| git rev-parse HEAD |
local |
4 ms |
| git branch -a |
local |
3 ms |
| git log –oneline -1000 |
local |
19 ms |
| git status |
local |
228 ms |
| git ls-remote origin |
remote |
751 ms |
| git fetch origin (no-op) |
remote |
456 ms |
- Read that table carefully. The local commands are not merely faster. They are in a different class, because their cost has no round-trip term in it.
- And the sandbox had an excellent link to github.com. The reader, on home broadband in India reaching a Microsoft edge in Delhi, would see far higher remote numbers on a good day, and unbounded numbers on the bad day this book keeps returning to.
- The distributed model is not new with git. GNU Arch appeared in 2001, Monotone in 2003, and Darcs in 2003. Git and Mercurial, both from April 2005, are the two that survived at scale.
- Perforce, founded by Christopher Seiwald in 1995 and now sold as P4, formerly Helix Core, is centralized by design and remains widely used. 37.10 explains why that is a reasoned choice and not a mistake.
WORDS37.2.6 remember these#
- Centralized VCS — the history lives on one server — a system with a single authoritative repository that all clients must contact to commit.
- Distributed VCS — everybody has the whole history — a system where every clone contains the complete object database and refs.
- Working copy — the files as they are right now — a checkout of one revision plus local bookkeeping, without the full history.
- Repository — the store of all history — the object database plus refs, in git the contents of the
.git directory.
- Clone — your own full copy — a complete repository created from another, including all reachable objects, plus a remote entry pointing back.
- Round trip — one there-and-back over the network — the request and response latency that every remote operation must pay at least once.
37.3 Why Git was made#
PLAIN37.3.1 in simple words#
- Git was not designed in a calm room over two years. It was written in a hurry, in April 2005, because something was taken away.
- From 2002 the Linux kernel was developed using BitKeeper, a commercial distributed version control system made by a company called BitMover, run by Larry McVoy.
- BitKeeper was not free software, but McVoy gave the kernel community free use of it. Some kernel developers objected loudly to depending on a proprietary tool. It was used anyway, and it worked well.
- In April 2005 that arrangement ended. BitMover announced it would stop providing the free version to the community. The official cutoff was 1 July
- So the largest and busiest software project in the world suddenly had no version control system, and about three months to find one.
- Nothing available was good enough. CVS and Subversion were centralized and far too slow for the kernel’s volume. The existing distributed tools could not handle the size.
- So Linus Torvalds wrote a new one. He started on 3 April 2005. He announced it on 6 April. On 7 April it was storing its own source code in itself.
- That first commit still exists and you can read it today.
- Then he handed it over. Junio Hamano became the maintainer on 26 July 2005, and has led the project ever since. Version 1.0 was released on 21 December
PLAIN37.3.2 a picture in your head#
- Imagine a factory of ten thousand workers that runs on one specialized machine, borrowed free from a neighbour.
- One morning the neighbour says the machine goes back in ninety days.
- Nothing on the market fits. The machines you can buy are built for factories of thirty people, and they jam when a thousand parts arrive at once.
- So the foreman goes into the workshop and builds a replacement himself, from the simplest possible parts, choosing crude and fast over clever and fragile, because the deadline is real.
- Four days later it is running. Not finished. Running, and holding its own blueprints.
- That is what happened, and it explains git’s personality. The parts are very simple. The joins are visible. The commands were designed for the machine first and the human second.
Where this comparison breaks:
- Git in 2026 is not the tool of April 2005. Thousands of contributors have worked on it for twenty-one years. The friendly commands,
git switch and git restore, arrived in version 2.23 on 16 August 2019.
- And the “ten days” story is often told as though one person wrote all of git in ten days. He did not. He wrote a working core in days and led it for under four months. What you type today is mostly other people’s work.
PLAIN37.3.3 a worked example#
- Here is the first commit of git, read out of a real clone of the git project’s own repository, in a sandbox, today.
$ git log --reverse --max-parents=0 --format='%H%n%an <%ae>%n%ad%n%s'
e83c5163316f89bfbde7d9ab23ca2e25604af290
Linus Torvalds <torvalds@ppc970.osdl.org>
Thu Apr 7 15:13:13 2005 -0700
Initial revision of "git", the information manager from hell
- Every fact in that block is checkable by anyone with git installed and a network connection.
- The date confirms the timeline: four days after he started, git was already the thing storing git.
- The email domain
osdl.org is the Open Source Development Labs, which employed Torvalds at the time and later merged into the Linux Foundation in
- The machine name
ppc970 is a PowerPC 970 processor, which is what was inside an Apple Power Mac G5 of that era.
- The commit message is a joke about how unfriendly the tool was. It was accurate.
PLAIN37.3.4 what is really happening inside#
- Torvalds stated his goals for the new system. They are worth taking one at a time, because each one shows up in the design you use today.
- Speed. He wanted patching to take no more than three seconds on the kernel. This is why git stores things in a way that is cheap to read and why almost everything is local.
- Simple design. The core is a content-addressed store: a key-value database where the key is the hash of the value. Chapter 38 covers it. The simplicity is why the format has survived twenty-one years unchanged.
- Strong support for non-linear development. Thousands of parallel branches. In git a branch is a file containing a hash. Creating one costs 41 bytes and no network. This is why git users branch casually.
- Fully distributed. Every clone complete. This is the property this whole chapter is about.
- Able to handle large projects efficiently, in both speed and data size. The kernel was already enormous in 2005.
- He added a further requirement that is often left out: very strong safeguards against corruption, accidental or malicious. This is why every object is named by the hash of its own content, and why a commit’s hash covers its parent’s hash, so tampering with old history changes every hash after it.
- And one more thing shaped it: the kernel has thousands of contributors and no central authority who may write to everyone’s tree. The design had to assume that trust is granted person to person, not by a server.
TECHNICAL37.3.5 the engineer’s version#
- Verified timeline of April to December 2005.
| 4 May 2000 |
BitKeeper first public release |
| 2002 |
Kernel adopts BitKeeper |
| April 2005 |
BitMover ends free version |
| 3 April 2005 |
Git development begins |
| 6 April 2005 |
Git announced on the list |
| 7 April 2005 |
Git self-hosting, commit e83c516 |
| 19 April 2005 |
Mercurial announced |
| 26 July 2005 |
Junio Hamano becomes maintainer |
| 21 December 2005 |
Git 1.0.0 released |
| 9 May 2016 |
BitKeeper open-sourced |
- The trigger was a dispute over reverse engineering. Larry McVoy alleged that Andrew Tridgell, the author of Samba and rsync, had reverse engineered the BitKeeper protocol to build a tool called SourcePuller. BitMover withdrew the free community licence in response.
- BitKeeper itself was eventually released as open source on 9 May 2016, under the Apache License version 2, eleven years too late to matter.
- Mercurial was announced by Olivia Mackall on 19 April 2005, for the same reason and in the same month. Both projects were candidates to replace BitKeeper for the kernel. Git won.
- Git’s version history, taken from tags in the real repository, gives useful anchors for what is old and what is new.
| v1.0.0 |
21 Dec 2005 |
first stable |
| v2.19.0 |
10 Sep 2018 |
partial clone |
| v2.23.0 |
16 Aug 2019 |
switch, restore |
| v2.29.0 |
19 Oct 2020 |
SHA-256 experimental |
| v2.43.0 |
20 Nov 2023 |
used in this chapter |
| v2.55.0 |
29 Jun 2026 |
current at writing |
- The object format has been stable since 2005. A repository created with git 1.0 in December 2005 can be read by git 2.55 in 2026 without conversion. That is a rare property and it was deliberate.
- Git remains a C project with a small core and a very large surface. The original design is described in the
Documentation/technical directory of the source, which ships with every clone.
- The first Linux kernel release managed entirely with git was 2.6.12 in June 2005, about two months after git’s first commit.
WORDS37.3.6 remember these#
- BitKeeper — the commercial tool the kernel used before git — a proprietary distributed VCS by BitMover whose free community licence ended in April 2005.
- Self-hosting — a tool storing its own source in itself — the point at which a system is complete enough to manage its own development.
- Non-linear development — many people working on many branches at once — a commit graph that is a directed acyclic graph rather than a straight line.
- Content-addressed store — you look things up by what they are, not where they are — a key-value store where the key is a cryptographic hash of the value.
- Maintainer — the person who decides what goes in — the individual holding the canonical tree and applying or rejecting contributions.
- Directed acyclic graph — a web of points with one-way arrows and no loops — the DAG formed by commits and their parent pointers.
37.4 What is actually inside a .git directory#
PLAIN37.4.1 in simple words#
- When you run
git init, git makes one hidden directory called .git and puts a handful of files in it.
- That directory is the repository. Everything else you see is just your files as they currently are.
- Delete
.git and you have a plain folder with no history. Copy .git somewhere else and you have carried the entire project’s past with you.
- There is no database server. No background process. No registry entry. No cloud account. Files in a directory.
- This is worth pausing on, because it is the reason for everything in this chapter. If a thing is a local file, reading it does not need a network.
- Here is what a real
.git looks like immediately after git init and one commit, listed in a sandbox with git 2.43.0.
$ ls -a .git
COMMIT_EDITMSG description index objects
HEAD hooks info refs
branches config logs
- Eleven entries. That is a complete version control system.
PLAIN37.4.2 a picture in your head#
- Think of
.git as a warehouse with a whiteboard by the door.
objects is the warehouse itself. Every crate ever received is in there, stored by a label computed from its contents. Nothing is ever moved or edited, only added.
refs is the whiteboard. It has a short list of names, and next to each name a crate label. “main: crate 9902ff97”. “v0.1: crate b398d1b0”.
HEAD is a sticky note on the whiteboard saying which of those names you are currently standing at.
index is the loading bay: things you have carried out of the aisles and set down, ready to be made into the next crate, but not yet crated.
config is the notice board with local rules on it, including the phone numbers of other warehouses you exchange crates with.
logs is the security camera footage of the whiteboard: every time a name was pointed at a different crate, and when.
hooks are the little printed cards saying “before accepting a delivery, check this”. By default they are all examples and none are active.
Where this comparison breaks:
- In a real warehouse, taking a crate out removes it. In
objects nothing is removed by reading, and almost nothing is ever removed at all until a garbage collection runs, which needs no network either.
- And a real warehouse can run out of aisles. Git periodically rewrites thousands of loose crates into one enormous compressed container, called a pack. The contents are identical, the storage is a different shape.
PLAIN37.4.3 a worked example#
- Here is a genuine
.git from a repository created in a sandbox, one file committed, printed in full. Nothing is edited or shortened.
$ cat .git/HEAD
ref: refs/heads/main
$ cat .git/refs/heads/main
9902ff97b5e8ac502e8ac752d950f15257c19335
$ cat .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[user]
name = Reader
email = reader@example.com
- Read
HEAD first. It contains the text ref: refs/heads/main. It does not contain a commit. It contains the name of a place that contains a commit.
- Now read that place.
.git/refs/heads/main contains 40 hexadecimal characters and a newline. 41 bytes. That is a branch.
- A branch in git is a 41-byte file whose contents are a commit’s name. That is the entire implementation. There is nothing else.
- This is why branching is instant and free, and why it needs no network. Creating a branch writes 41 bytes to your own disk.
- And
config is an INI-style text file you can open in any editor. There is no binary blob, no encrypted store, no service account.
- Here is what
git init prints, verbatim, so you can see how little happens.
$ git init initdemo
Initialized empty Git repository in /tmp/gitdemo/initdemo/.git/
- One directory made. No server contacted. It works on a plane.
PLAIN37.4.4 what is really happening inside#
- Now every entry, one at a time, with what it is for.
- objects/ — the object database. Every commit, every directory listing, every file version, stored under its own hash. Chapter 38 is entirely about this. It is where nearly all the bytes are.
- refs/ — human names for commits.
refs/heads/ holds your branches, refs/tags/ holds tags, refs/remotes/ holds remote-tracking refs, which are section 37.7 and the heart of the reader’s question.
- HEAD — where you are now. Normally a symbolic reference: text saying
ref: refs/heads/main. If it holds a raw hash instead, you are in the state git calls detached HEAD.
- config — settings for this repository only. Your identity, your remotes, and any per-repository preferences. Text, editable, version 0 format.
- index — the staging area. A binary file listing exactly what will go into the next commit, with each path, its mode, its blob hash and cached file metadata so
git status can skip unchanged files. Chapter 39 covers it.
- hooks/ — scripts git will run at certain moments. On a fresh repository every one is a
.sample file, and a .sample file never runs. Nothing is active until you rename one and make it executable.
- logs/ — the reflog. A line-per-change record of every value each ref has held, with who and when. This is your safety net after a bad reset.
- info/ — extra bits.
info/exclude is a private ignore list that is not committed and not shared, unlike .gitignore.
- description — used only by the old
gitweb viewer. On a normal repository it is dead weight and you can ignore it.
- COMMIT_EDITMSG — a scratch file holding the message you typed last. Not part of the history. Purely a convenience.
- branches/ — an obsolete shorthand directory from very early git. It is still created for compatibility and is always empty. Ignore it.
- packed-refs — appears once refs are compressed into one file. See below.
- The honest version: the exact set of files is an implementation detail, not a standard. It has changed over time and can differ with configuration. What is guaranteed is the on-disk object format and the general layout.
TECHNICAL37.4.5 the engineer’s version#
- Real
du output for the .git of a full clone of the git project itself, git 2.43.0, cloned in a sandbox on 13 August 2026.
| objects |
316 MiB |
all history |
| index |
452 KiB |
staged tree, 4845 files |
| packed-refs |
104 KiB |
2025 refs in one file |
| hooks |
68 KiB |
14 inactive samples |
| logs |
32 KiB |
reflogs |
| refs |
28 KiB |
loose refs |
| config |
4 KiB |
settings |
| HEAD |
4 KiB |
current position |
- The object store is 99.9 percent of it. Everything else is bookkeeping.
- packed-refs: once a repository has many refs, git writes them into one sorted text file rather than one file per ref, because thousands of tiny files are slow on every filesystem. Real head of that file:
# pack-refs with: peeled fully-peeled sorted
165e5ad3169d0fd26637da3383a4514f1a9d1e72 refs/remotes/origin/bisect
426ec014a6630fb78c5a7334461bc33568f07743 refs/remotes/origin/jch
e9019fcafe0040228b8631c30f97ae1adb61bcdc refs/remotes/origin/maint
745601a9a94110d74769ab605ccd4f61339758d2 refs/remotes/origin/master
- A loose ref file always wins over a
packed-refs entry for the same name. That is the lookup rule.
- index format: a binary file beginning with the four bytes
DIRC, short for “dircache”, then a version number and an entry count.
$ head -c 12 .git/index | od -c
0000000 D I R C \0 \0 \0 002 \0 \0 \0 002
- Those twelve bytes read: signature
DIRC, version 2, two entries. Index versions 2, 3 and 4 exist; version 4 adds path prefix compression.
git ls-files --stage decodes it for you:
100644 bd9f1115522d0cca0ac0e7b22f7d256e08287438 0 README.md
100644 31489c9ee8c455a4964beabcf6dd91ca1cea9ad4 0 src/main.c
- Columns: file mode, blob hash, merge stage number, path. Stage 0 means no conflict. Stages 1, 2 and 3 appear during a conflicted merge and are Chapter 39.
- The fourteen sample hooks in git 2.43.0 are
applypatch-msg, commit-msg, fsmonitor-watchman, post-update, pre-applypatch, pre-commit, pre-merge-commit, pre-push, pre-rebase, pre-receive, prepare-commit-msg, push-to-checkout, sendemail-validate, update.
- Useful inspection commands:
git rev-parse --git-dir, git count-objects -vH, git for-each-ref, git cat-file -p HEAD, git ls-files --stage, git symbolic-ref HEAD, git config --list --show-origin --show-scope.
- A bare repository, made with
git init --bare, has these same entries at the top level with no working tree and no index. That is what a server stores. It is the same data with the checkout removed.
WORDS37.4.6 remember these#
- Object database — the store of everything ever committed — the
.git/objects directory holding loose and packed objects keyed by hash.
- Ref — a name that points at a commit — a file under
.git/refs or an entry in packed-refs containing a 40-character object ID.
- HEAD — where you are standing now — a symbolic ref, usually
ref: refs/heads/<branch>, or a raw object ID when detached.
- Index — the list of what goes in the next commit — a binary
DIRC file of staged path, mode, blob ID and cached stat data.
- Reflog — the record of where each name used to point — per-ref append-only logs under
.git/logs, used to recover lost commits.
- Bare repository — a repository with no working files — a
.git layout at the top level, with core.bare true and no index or checkout.
- Hook — a script git runs at a set moment — an executable in
.git/hooks with an exact name, disabled by default and never cloned.
37.5 What “the whole history” really means#
PLAIN37.5.1 in simple words#
- When we say a clone contains the whole history, that phrase is doing a lot of work. Let us make it exact.
- It means every commit ever made on any branch that was fetched.
- It means every version of every file, including files that were deleted years ago and no longer exist in your folder.
- It means every commit message, every author name, every date, every parent link, and every tag.
- All of it, on your disk, right now, readable with no network.
- The obvious worry is size. If a project has 85,000 commits, and each one is a snapshot of the whole project, surely the disk fills up?
- It does not, and the reason is that source code is text, text is enormously repetitive, and git compresses it very hard.
- Two mechanisms do the work. First, identical content is stored exactly once, because content with the same bytes gets the same name. If a file did not change between two commits, both commits point at the same stored copy.
- Second, git periodically rewrites the store into a pack, which compresses everything and stores similar files as small differences from each other.
- The result is that the whole past often costs only a few times the size of the present.
PLAIN37.5.2 a picture in your head#
- Imagine photographing a whiteboard every day for five years. That is 1,800 photographs.
- Stored naively, 1,800 full-resolution photographs is a lot of disk.
- But most days almost nothing changes. So instead you keep one full photograph and, for each other day, a short note: “same as yesterday except the top-right box now says 42”.
- Now 1,800 days of history costs slightly more than one photograph plus 1,799 short notes.
- That is a pack file. Git finds files that resemble each other, picks one to store whole, and stores the others as instructions relative to it.
- And it compresses everything on top of that, the way a zip file does.
Where this comparison breaks:
- Git does not compare “yesterday to today” the way the photo story suggests. It compares any object to any similar object, regardless of date, using a heuristic based on file name and size.
- Also a chain of differences cannot be infinitely long, or reading the newest version would require replaying thousands of steps. Git limits chain depth, by default to 50, so reads stay fast.
- And binary files break the trick. A photograph, a compiled program or a video changes almost entirely with every edit and compresses badly. This is a real limitation and 37.10 returns to it.
PLAIN37.5.3 a worked example#
- These are real measurements from a full clone of the git project’s own source repository, made in a sandbox on 13 August 2026.
| Commits in history |
85,263 |
| Files in working tree |
4,845 |
| Working tree on disk |
60 MiB |
| .git directory |
316 MiB |
| Objects stored |
418,900 |
| Pack files |
1 |
- So twenty-one years and 85,263 commits cost about 5.3 times the size of the current checkout. That is the answer to “surely it fills the disk”.
- Now the compression, measured object by object with
git cat-file --batch-all-objects.
| blob (file data) |
165,095 |
5364 / 194 MiB |
| tree (directories) |
167,536 |
2453 / 74 MiB |
| commit |
85,263 |
59 / 33 MiB |
| tag |
1,006 |
0.7 / 0.6 MiB |
- Add them up: about 7,877 MiB of raw object content stored in about 302 MiB on disk. A ratio of roughly 26 to 1.
- File data compresses best, 5,364 MiB down to 194 MiB, about 27 to 1, because most versions of most files are nearly identical to another version.
- Commits compress worst, 59 MiB down to 33 MiB, about 1.8 to 1, because every commit contains different hashes, and hashes are random-looking text that does not compress.
- That last line is a nice piece of evidence: compression works on repetition, and hashes have none by design.
PLAIN37.5.4 what is really happening inside#
- Two storage shapes exist and both are always valid.
- Loose objects. One file per object, at
.git/objects/ab/cdef..., where ab is the first two hex characters of the hash and the rest is the file name. The content is compressed with zlib.
- The two-character directory split exists because filesystems slow down badly with hundreds of thousands of files in one directory. It spreads them over 256 directories.
- Packed objects. Many objects in one
.pack file, with a .idx file beside it giving the byte offset of each object so lookup stays fast.
- Inside a pack, an object may be stored whole, or as a delta against another object in the same pack. The delta is a tiny instruction list: copy these bytes from the base, then insert these new bytes.
- Packing happens automatically.
git gc runs when loose objects accumulate, and a fresh git clone always arrives as a pack because the sending side builds one for the transfer.
- That is why a freshly cloned repository shows
count: 0 loose objects and one large pack.
- Nothing about this needs a network. Packing, unpacking, delta resolution and garbage collection are all local CPU and disk work.
TECHNICAL37.5.5 the engineer’s version#
- Object naming: git 1.0 through the present names objects by SHA-1 of the header plus content. SHA-256 repositories became available as an experimental feature in git 2.29, released 19 October 2020, and interoperability with SHA-1 repositories is still incomplete as of git 2.55 in June 2026.
- Default packing parameters, from
git config documentation for git 2.43: pack.depth 50, pack.window 10, core.compression -1 meaning zlib default level 6, gc.auto 6700 loose objects.
- Loose object path scheme:
.git/objects/<first 2 hex>/<remaining 38 hex>. Real listing from a sandbox repository:
.git/objects/b1/a8becadd478624373132f066d206859ade455f
.git/objects/fb/560ef022402ab49df96a3b868aa7c540be1a57
.git/objects/e6/69826a4efde29351886aa6e636e14635a29827
.git/objects/c3/5f063d2751d7340c2cdd4fe0046a137c9164c4
- Shallow clone:
git clone --depth N fetches only the last N commits per branch. The repository records a .git/shallow file listing the commits whose parents are deliberately absent.
- Partial clone:
git clone --filter=blob:none fetches all commits and trees but no file contents, downloading blobs on demand. Introduced in git 2.19, 10 September 2018. It sets remote.origin.promisor = true and remote.origin.partialclonefilter in the config.
- Real measurements of the same repository, four ways, same sandbox, same day.
| full |
316 MiB |
34 s |
| blobless filter |
130 MiB |
38 s |
| treeless filter |
51 MiB |
25 s |
| shallow depth 1 |
14 MiB |
4.6 s |
- What you lose, stated precisely.
- Shallow:
git log shows one commit, not 85,263. git blame cannot see past the cut. git bisect cannot search history you do not have. Pushing from a shallow clone works but has restrictions, and deepening requires the network.
- Blobless: history commands work, but any command needing file content from an old commit, such as
git blame or git log -p, silently triggers a network fetch. Offline, those commands now fail.
- That is the sting. A partial clone quietly reintroduces the network into operations that were local. During an outage, a blobless clone behaves less like git and more like Subversion for exactly those commands.
- Real config written by a blobless clone:
[remote "origin"]
url = https://github.com/git/git.git
fetch = +refs/heads/*:refs/remotes/origin/*
promisor = true
partialclonefilter = blob:none
- Guidance: use shallow clones for continuous integration runners that build once and are destroyed. Use full clones for machines where people work, especially machines that go offline.
WORDS37.5.6 remember these#
- Blob — the stored contents of one file version — a git object holding raw file bytes with no name and no permissions.
- Pack file — one big compressed container of many objects — a
.pack with a companion .idx giving offsets, using delta chains internally.
- Delta — a small “difference from another object” — copy and insert instructions against a base object within the same pack.
- zlib — the common compression used inside git — DEFLATE compression, RFC 1950 and 1951, applied to every loose object and pack entry.
- Shallow clone — a copy with only the newest commits — a clone with truncated history recorded in
.git/shallow, created by --depth.
- Partial clone — a copy that fetches file contents on demand — a promisor clone created with
--filter, requiring the remote for missing objects.
- Garbage collection — tidying the store —
git gc, which packs loose objects and prunes unreachable ones after a grace period, default 14 days.
37.6 What “origin” actually is#
PLAIN37.6.1 in simple words#
- Almost every git user believes something slightly wrong about
origin, so let us be blunt.
origin is a nickname for a URL, stored in a text file on your own computer.
- That is the entire definition. There is nothing else to it.
- It is not the master copy. It is not the real repository. It is not the authority. It does not own your work.
- The name
origin is not special to git. It is the name git clone happens to use by default. You may rename it to anything.
- That default is a convention, not a standard. No specification requires it. It is a habit that became universal.
- A repository can have zero remotes, one remote, or twenty remotes. All of them are equal. None outranks another.
- Zero is not a broken state. A repository with no remotes is a completely normal, fully functional git repository. You can commit to it forever.
- And the URL a remote points at does not have to be on the internet. It can be another folder on the same disk, or a USB stick.
PLAIN37.6.2 a picture in your head#
origin is an entry in your phone’s contact list.
- The contact is called “Mum”. That name means nothing to the phone network. It is a label you chose, stored on your handset, mapping to a number.
- Rename the contact to “Mother” and nothing anywhere else changes. The phone number still works. Your mother does not notice.
- Delete the contact entirely and your phone still works perfectly. You simply have to type the number when you want to call.
- You may have twenty contacts. None of them is in charge of your phone.
- And your mother has her own contact list, in which you are an entry. Neither list is the true one.
- That is
origin exactly. A label on your machine mapping a short name to an address you would otherwise have to type in full.
Where this comparison breaks:
- A phone contact is only a convenience. A git remote also carries a refspec, a rule saying which of their branches map to which of your remote-tracking names. That is real behaviour, not just a label.
- And in a team,
origin usually does have authority, because the team agreed it does and the hosting service enforces that agreement. That authority is social and administrative. It is not in the git data model.
PLAIN37.6.3 a worked example#
- Here is a real
.git/config immediately after a clone, printed verbatim from a sandbox.
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[remote "origin"]
url = /tmp/gitdemo/server.git
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
remote = origin
merge = refs/heads/main
- Look at the
[remote "origin"] block. Two lines. A URL and a fetch rule. That is what origin is made of.
- Now rename it and watch what happens, and what does not.
$ git remote rename origin upstream
$ git config --get-regexp '^remote\.|^branch\.'
remote.upstream.url /tmp/gitdemo/server.git
remote.upstream.fetch +refs/heads/*:refs/remotes/upstream/*
branch.main.remote upstream
branch.feature/offline-work.remote upstream
- Git rewrote the config, renamed the ref directory from
refs/remotes/origin/ to refs/remotes/upstream/, and updated every branch that referred to it.
- No commit changed. No hash changed. The server was never contacted. It was a local text edit plus a directory rename.
- Now remove every remote and keep working.
$ git remote remove origin
$ git remote
(empty: no remotes at all)
$ git commit -m "Commit in a repo with no remotes at all"
$ git log --oneline -1
4e2462c Commit in a repo with no remotes at all
- Zero remotes. Commit succeeded. That is the proof that
origin is optional.
PLAIN37.6.4 what is really happening inside#
- A remote is three things stored together in
.git/config.
- A name. The text between the quotes in
[remote "origin"]. Arbitrary.
- A URL. Where to reach it. This may be
https://..., git@host:path for SSH, ssh://..., git://..., or a plain filesystem path.
- A refspec. The line
fetch = +refs/heads/*:refs/remotes/origin/* reads: take everything under refs/heads/ on their side, and store it under refs/remotes/origin/ on mine. The leading + means overwrite even if the change is not a fast-forward.
- Optionally a pushurl, if you fetch from one place and push to another.
- Separately, each branch may record an upstream: the
[branch "main"] block saying which remote and which branch it pairs with. This is what lets you type git push with no arguments.
- When you run a remote command, git looks up the name, finds the URL, chooses a transport based on the URL scheme, and connects. Chapter 40 covers the transports and their ports.
- Everything before the connect step is reading a text file.
- The honest version: people say “push to origin” as though pushing were sending to a superior. It is not. It is one repository asking a peer to update some of its refs, and the peer may refuse. Chapter 40 covers refusal.
TECHNICAL37.6.5 the engineer’s version#
- Config precedence, highest wins: command line, then
.git/config, then ~/.gitconfig or ~/.config/git/config, then /etc/gitconfig. Verify with git config --list --show-scope --show-origin.
system file:/etc/gitconfig filter.lfs.required=true
global file:/root/.gitconfig user.name=Shikhar
global file:/root/.gitconfig user.email=shikhar@example.com
local file:.git/config remote.origin.url=...
- Remote management commands, all of which are pure local config edits except the last two.
| git remote -v |
no |
| git remote add |
no |
| git remote rename |
no |
| git remote remove |
no |
| git remote set-url |
no |
| git remote show origin |
yes |
| git remote update |
yes |
git remote show origin contacts the server, which surprises people. Use git remote show -n origin to stay offline and read only cached data.
- A repository may hold many remotes and they are peers in the data model. The common three-remote fork workflow is:
$ git remote -v
mirror /tmp/gitdemo/mirror.git (fetch)
mirror /tmp/gitdemo/mirror.git (push)
origin /tmp/gitdemo/server.git (fetch)
origin /tmp/gitdemo/server.git (push)
- Naming conventions widely used but not required by git:
origin for the place you cloned from, upstream for the original project when origin is your fork, fork for a personal copy. All are convention.
- The default remote name is set by
clone.defaultRemoteName, added in git 2.30, January 2021. Setting it to upstream makes git clone name it that.
- A remote URL of a local path is fully supported and often useful: a USB drive, a network mount, or a bare repository elsewhere on the same disk. The sandbox examples in this chapter use exactly that.
- There is no field anywhere in git’s data model marking one repository as canonical. Authority is entirely external: server permissions, branch protection rules and team agreement.
WORDS37.6.6 remember these#
- Remote — a saved nickname for another repository’s address — a named config section holding a URL and one or more refspecs.
- origin — the default nickname
git clone gives the place you cloned from — a conventional remote name with no special meaning to git.
- Refspec — the rule mapping their branch names to yours — a
+src:dst pattern, typically +refs/heads/*:refs/remotes/origin/*.
- Upstream — the branch yours is paired with — the
branch.<name>.remote and branch.<name>.merge config pair, shown as @{upstream}.
- Bare repository — a repository with no checked-out files — what a server holds, so nobody’s working tree is disturbed by a push.
- Peer — an equal repository, not a superior — the correct mental model for every remote in a distributed VCS.
37.7 Remote-tracking refs and staleness#
PLAIN37.7.1 in simple words#
- This section is the reader’s own case, so we will be very careful.
- During the outage, their machine reported
origin/main as 0a95cc8.
- It is natural to read that as “the server’s main branch is at 0a95cc8 right now”. That reading is wrong.
- The correct reading is: “the last time this computer successfully talked to that server, its main branch was at 0a95cc8”.
origin/main is a memory, not a measurement. It is a note your machine wrote to itself, at some past moment, and has not revised since.
- It lives in a file on your disk. Nothing about it is live. Git will keep reporting that value cheerfully, forever, with the network cable cut.
- And git will not warn you. There is no “possibly stale” marker in the normal output. It looks exactly the same when it is one second old and when it is six weeks old.
- This is not a bug. It is the only thing a distributed system can do. Asking the server on every command would be exactly the centralized design git was built to escape.
- So git gives you a fast, honest, local answer to a slightly different question than the one you thought you asked.
PLAIN37.7.2 a picture in your head#
- You keep a note on your fridge that says: “Priya’s flight lands at 18:40.”
- You wrote it on Tuesday, after speaking to Priya.
- On Friday you look at the fridge and read 18:40. The note has not changed.
- But the note is not the flight. It is what you were told on Tuesday. The flight may have moved to 21:10 and nobody has told the fridge.
- The fridge cannot be wrong, because it is not claiming to know the flight. It is claiming to record what you wrote. It does that perfectly.
- To learn the real time you must call the airline. That is
git fetch.
- If the phone line is down, you can still read the note. You just cannot refresh it. And the note gives you no hint of its own age.
Where this comparison breaks:
- Git does keep a dated record of when the note changed, in the reflog, so you can find out how old it is. The fridge does not. Section 37.7.5 shows how.
- And unlike a flight time, the true value of the remote’s
main can only move forward under normal rules. A stale origin/main is usually an underestimate of what the server now has, not a random wrong value.
PLAIN37.7.3 a worked example#
- Here is the whole effect, produced in a sandbox by making the remote unreachable, exactly as an outage would.
- Before the outage: clone, one commit, pushed. Both sides agree.
$ git rev-parse main
01d088e5131d972ed904171845efe601d994fea5
$ git rev-parse origin/main
01d088e5131d972ed904171845efe601d994fea5
- Now the remote becomes unreachable, and three commits are made locally.
$ git rev-parse main
2c9e7c6d79ab36d4a7ca7b229c750fc0ce5dda53
$ git rev-parse origin/main
01d088e5131d972ed904171845efe601d994fea5
main moved. origin/main did not, and cannot, because moving it requires hearing from the server.
- Here is the file itself, which is the point of the whole section.
$ cat .git/refs/remotes/origin/main
01d088e5131d972ed904171845efe601d994fea5
- Forty-one bytes on your own disk. That is
origin/main. It is not a connection. It is not a query. It is a file.
- The reader’s
0a95cc8 was the first seven characters of exactly such a file’s contents, on their own machine, written the last time a fetch or push had succeeded.
- Git will happily use it offline:
$ git log --oneline -1 origin/main
01d088e Initial project layout
- That command ran with the remote unreachable and printed a confident answer.
PLAIN37.7.4 what is really happening inside#
- Now the exact rules. Which commands update a remote-tracking ref and which never do.
git fetch updates them. That is its entire job: connect, download new objects, and move refs/remotes/<remote>/* to match what the server said.
git pull updates them, because a pull is a fetch followed by a merge or rebase. The fetch part does the updating.
git push updates them, for the branches it pushed. If your push succeeds, git knows the server now holds what you sent, so it moves the matching remote-tracking ref locally. This is a small but important detail.
git remote update and git fetch --all update them for all remotes.
- Nothing else does. Not
git commit, not git status, not git log, not git branch -vv, not git merge, not git rebase, not git checkout.
- That last group includes every command that displays “ahead 3, behind 2”. Those numbers are computed by comparing two local refs. Both are local, so both can be stale.
- So “your branch is behind origin/main by 2 commits” means “behind the value I last heard, which was some time ago”. It is not a live comparison.
- Here is the sequence in a diagram.
your disk the server
--------------------- ---------------------
refs/heads/main refs/heads/main
moved by: commit, moved by: pushes
merge, rebase, reset from anyone
refs/remotes/origin/main
moved ONLY by:
fetch, pull, push
otherwise frozen <-- the reader's 0a95cc8
- The left column can change every minute while you work. The middle entry changes only when a network operation succeeds. During an outage it is frozen by definition.
TECHNICAL37.7.5 the engineer’s version#
- Storage: remote-tracking refs live at
.git/refs/remotes/<remote>/<branch> when loose, or as refs/remotes/<remote>/<branch> lines in .git/packed-refs once packed. Both are read the same way; loose wins on conflict.
- They are ordinary refs in a reserved namespace. Git forbids committing onto them, which is why
git checkout origin/main puts you in detached HEAD rather than on a branch.
- How to tell how stale it is. The reflog for a remote-tracking ref records every update with a timestamp. Real output:
$ git reflog show origin/main --date=iso
d39bbb7 refs/remotes/origin/main@{2026-08-13 03:12:21 +0000}: update by push
01d088e refs/remotes/origin/main@{2026-08-13 03:09:20 +0000}: update by push
- The top line’s timestamp is the last moment this machine had confirmed knowledge of that branch. Everything since is unverified.
- Note the reason strings:
update by push here, fetch or fetch origin after a fetch. So the reflog also tells you how you learned it.
- Other ways to measure staleness:
| git reflog show origin/main |
when it last moved |
| stat -f %Sm .git/refs/… |
file mtime, macOS |
| stat -c %y .git/refs/… |
file mtime, Linux |
| git ls-remote origin main |
the live value, needs net |
- The file modification time is a decent proxy but imperfect: a fetch that finds no change may not rewrite the file, and packing rewrites timestamps. Prefer the reflog.
git ls-remote origin main is the only entry in that table that answers the question you actually wanted. It contacts the server and prints the live value without downloading any objects. It is the correct tool for “is my origin/main current”, and it needs the network.
git status output such as ## main...origin/main [ahead 3] is computed by git rev-list --left-right --count main...origin/main, entirely locally.
git fetch --dry-run still contacts the server. It only skips writing.
- Configuration worth knowing:
fetch.prune or git fetch --prune deletes remote-tracking refs whose upstream branch no longer exists. Without it, refs/remotes/origin/ accumulates names of branches deleted months ago, which is another form of staleness.
- The honest version:
origin/main is sometimes described as “the remote branch”. It is not the remote branch. It is a local cache of the remote branch, with no expiry, no freshness metadata in the ref itself, and no invalidation. The remote branch is on the remote.
WORDS37.7.6 remember these#
- Remote-tracking ref — your machine’s note of where their branch was — a ref under
refs/remotes/<remote>/ updated only by fetch, pull or push.
- Stale — out of date without saying so — holding a cached value with no freshness check and no invalidation signal.
- Fetch — go and get their latest, without changing my work — download objects and update
refs/remotes/*, leaving refs/heads/* untouched.
- Ahead and behind — how many commits each side has that the other lacks — counts from
git rev-list --left-right --count A...B, computed locally.
- Prune — forget remote branches that no longer exist —
git fetch --prune, deleting stale refs/remotes entries.
- ls-remote — ask the server what it has right now — a network query printing the remote’s refs without transferring objects.
37.8 Which git operations need the network#
PLAIN37.8.1 in simple words#
- Here is the whole answer in one line: a git command needs the network if and only if it must read from, or write to, another repository.
- Everything else is reading and writing files inside
.git, and files do not need a network.
- That rule is not a rough guide. It is exact, and you can apply it to a command you have never seen.
- Ask yourself: does this command need to know something only another machine knows, or change something only another machine holds?
- If yes, it is a network command. If no, it is local.
git log reads commits. All commits are local. Local command.
git branch writes a 41-byte file. Local command.
git merge combines two commits you already have. Local command.
git fetch needs objects you do not have. Network command.
git push must change refs on another machine. Network command.
- There is no third category and no exceptions worth memorizing, apart from a couple of traps listed at the end.
PLAIN37.8.2 a picture in your head#
- Think of your repository as a personal notebook and the remote as a friend’s notebook in another city.
- Anything you can do by turning pages in your own notebook is instant and always available. Reading old pages. Copying a page. Rewriting a draft.
- Anything requiring your friend’s notebook needs a phone call. Learning what they wrote. Dictating your pages to them.
- Nothing about a broken phone stops you turning your own pages.
- And notice: dictating a page to your friend needs a call, but writing the page did not. The writing already happened, in your own notebook, days ago.
- That separation between writing and sending is the whole distributed idea.
Where this comparison breaks:
- A few git commands look local but quietly place a call.
git remote show origin does. git submodule update does. In a partial clone, git log -p does. These are the traps, and 37.8.5 lists them.
PLAIN37.8.3 a worked example#
- Every one of these ran successfully in a sandbox with the remote made unreachable, so the outage was real, not simulated by a flag.
git branch -a rc=0 * feature/offline-work
git tag v0.1 rc=0
git diff HEAD~1 --stat rc=0 README.md | 1 +
git log --oneline -3 rc=0 d39bbb7 Committed during the outage
git blame -L 1,2 file rc=0 ^01d088e (Reader 2026-08-13 ...)
git reflog -3 rc=0 d39bbb7 HEAD@{0}: commit: ...
git switch main rc=0
git merge feature/... rc=0
git bisect start/good rc=0 Bisecting: 2 revisions left to test
- Branching, tagging, diffing, logging, blaming, reflogging, switching, merging and bisecting. All with no reachable remote.
- And in the same session, at the same second, these failed:
$ git fetch
fatal: Could not read from remote repository.
$ git ls-remote
fatal: Could not read from remote repository.
- The failures are immediate and specific. Git did not hang or half-work. It attempted a connection, failed, and told you.
PLAIN37.8.4 what is really happening inside#
- The definitive classification. Local commands first.
| git init |
yes |
creates .git |
| git add |
yes |
index |
| git commit |
yes |
objects, refs, logs |
| git status |
yes |
index, working tree |
| git diff |
yes |
index, objects |
| git log |
yes |
objects |
| git branch |
yes |
refs |
| git checkout |
yes |
index, working tree |
| git switch |
yes |
index, working tree |
| git restore |
yes |
index, working tree |
| git merge |
yes |
objects, refs, index |
| git rebase |
yes |
objects, refs, index |
| git reset |
yes |
refs, index |
| git stash |
yes |
objects, refs |
| git tag |
yes |
refs, objects |
| git blame |
yes |
objects |
| git bisect |
yes |
objects, refs |
| git cherry-pick |
yes |
objects, refs |
| git reflog |
yes |
logs |
| git gc |
yes |
objects |
| git show |
yes |
objects |
| git worktree |
yes |
.git, filesystem |
- Now the network commands.
| git clone |
in |
copy a whole repository |
| git fetch |
in |
download new objects, refs |
| git pull |
in |
fetch then merge or rebase |
| git push |
out |
send objects, move their refs |
| git ls-remote |
in |
read their refs only |
| git remote show |
in |
query their state |
| git submodule update |
in |
clone or fetch sub-repos |
| git request-pull |
in |
reads the remote to compare |
| git archive –remote |
in |
ask server to build a tarball |
- Why the line falls exactly there. Local commands read and write four things: the object database, the refs, the index, and your working files. All four are directories on your disk.
- Network commands do exactly one extra thing: they move objects and ref values between two object databases over a transport.
- That is why the split has no fuzzy middle. A git command is a manipulation of a local file store, or it is a synchronization between two file stores.
- The commit itself is proof.
git commit writes some objects, moves one ref, appends to one log file, and stops. It has no idea a remote exists.
TECHNICAL37.8.5 the engineer’s version#
- The traps, stated precisely, because each one has caught working engineers.
git pull is not git fetch plus nothing. It is fetch plus merge, or fetch plus rebase if pull.rebase is true. Offline it fails at the fetch and does not perform the merge.
git remote show origin contacts the server. git remote show -n origin does not, and prints cached data only.
git fetch --dry-run contacts the server. Dry run means it does not write refs, not that it does not connect.
- In a partial clone, previously local commands become remote. With
--filter=blob:none, commands needing historical file content, such as git log -p, git blame on old commits, or git checkout of an old tag, fetch blobs on demand and fail offline.
- Submodules.
git submodule update --init clones each submodule. Offline it fails, even though the parent repository’s own history is complete.
- Git LFS. Large File Storage replaces big files with pointer text and fetches the real bytes on checkout. A repository using LFS is not fully offline-capable for those files. LFS was released by GitHub in April 2015.
- Credential helpers and signing. A commit signed with a hardware key or an SSH agent may need that device present, though not a network. A commit signed via a remote signing service does need one.
- Hooks. A
pre-commit hook that runs a linter which downloads packages turns a local command into a network one. That is your hook’s doing, not git’s.
- Measured cost difference, real numbers from the sandbox on the git project repository, so you can see why the split matters for speed as well as availability.
| git rev-parse HEAD |
local |
4 ms |
| git log –oneline (85k) |
local |
928 ms |
| git checkout HEAD~100 – . |
local |
119 ms |
| git ls-remote origin |
remote |
751 ms |
| git fetch origin (no change) |
remote |
456 ms |
- Walking 85,263 commits locally took 928 ms. Asking the server a single question over a fast link took 751 ms. On the reader’s link that day, it would never have completed at all.
- Exit code discipline: remote commands fail with a non-zero status and a
fatal: message on stderr. Scripts should check status rather than parse text, because the messages are not a stable interface.
WORDS37.8.6 remember these#
- Local operation — something git does entirely on your own disk — a command touching only the object database, refs, index and working tree.
- Remote operation — something needing another repository — a command opening a transport connection to synchronize objects or refs.
- Transport — the way two repositories talk — HTTPS, SSH, the git protocol on port 9418, or a local filesystem path.
- Dry run — do the checking but not the writing —
--dry-run, which for fetch and push still performs the network connection.
- Submodule — another repository nested inside this one — a gitlink entry recording a commit ID, with the sub-repository cloned separately.
- Git LFS — a system for keeping huge files out of history — Large File Storage, storing pointers in git and content on a separate server.
37.9 The reader’s outage, worked through#
PLAIN37.9.1 in simple words#
- Now we put the whole chapter onto the reader’s actual day.
- They were in a flat in India, on home broadband, on a Mac. Their router was at
192.168.0.1 and their DNS resolver was 1.1.1.1.
github.com resolved to 20.207.73.82, an address in a Microsoft-owned range, because GitHub has been owned by Microsoft since 2018 and fronts traffic through Microsoft’s network edge.
curl -v https://github.com printed Trying 20.207.73.82:443... and then timed out after 15 seconds with no response at all. No refusal, no error message from any router. Silence.
- The same site loaded instantly on mobile data on the same phone, which showed the site itself was fine and the path from that flat was not.
- So: GitHub was reachable from the world and unreachable from that sofa.
- And the reader kept working for the whole outage. They edited files, ran
git add, ran git commit, created branches, committed again.
- Every one of those commands succeeded. Not partially. Not queued for later. Fully succeeded, with real commits and real hashes.
- The reason is now, we hope, boring: none of those commands involved another computer. They wrote files into
.git on that Mac’s own disk.
- Later, connectivity returned. One
git push sent everything that had piled up, in a single operation.
PLAIN37.9.2 a picture in your head#
- Think of a writer in a cottage during a storm that has taken out the phone line.
- They cannot post anything. They cannot receive anything. The postbox at the end of the lane may as well not exist.
- But the desk still works. The pen still works. The filing cabinet still works. They write chapters four, five and six, date each one, and file them.
- When the line comes back they walk to the postbox once and post all three chapters together.
- Nothing was lost, nothing had to be redone, and nothing had to be remembered in their head rather than on paper.
- The one real cost is that during the storm they did not know what their co-author had been writing. They found that out afterwards, and might then have had to reconcile.
Where this comparison breaks:
- The writer knows the line is down. Git does not always make it obvious, because
git status and git log behave identically offline and online. The reader’s origin/main looked like a fact and was a memory.
- And posting three chapters at once is not always safe in git. If the co-author moved
main while you were offline, your push may be rejected, and you must fetch and reconcile. That is Chapter 40.
PLAIN37.9.3 a worked example#
- The full sequence, with the reader’s own values where known, reconstructed step by step.
TIME EVENT origin/main local main
----- --------------------------------- ----------- ----------
t0 last successful fetch or push 0a95cc8 0a95cc8
t1 network path to GitHub goes dark 0a95cc8 0a95cc8
t2 edit files, git add, git commit 0a95cc8 (new #1)
t3 git switch -c feature/... 0a95cc8 (new #1)
t4 more commits on the branch 0a95cc8 (new #1)
t5 git status: "ahead 1" <- cached 0a95cc8 (new #1)
t6 git fetch -> fatal, no route 0a95cc8 (new #1)
t7 connectivity returns 0a95cc8 (new #1)
t8 git push origin main (new #1) (new #1)
t9 git push -u origin feature/... (new #1) (new #1)
- The
origin/main column is frozen from t0 to t8. That column is a file on the reader’s disk, and only a successful network operation writes to it.
- Here is the equivalent moment reproduced in a sandbox, with real output.
before push: origin/main = 01d088e5131d972ed904171845efe601d994fea5
before push: main = d39bbb76dc18fd2151d333cbf82d9405449342f8
commits waiting to be sent: 5
$ git push origin main
To /tmp/gitdemo/server.git
01d088e..d39bbb7 main -> main
after push: origin/main = d39bbb76dc18fd2151d333cbf82d9405449342f8
- Five commits, accumulated with no network, sent in one operation, and only then did
origin/main move.
- And the second branch, created entirely offline, published in one go:
$ git push -u origin feature/offline-work
To /tmp/gitdemo/server.git
* [new branch] feature/offline-work -> feature/offline-work
branch 'feature/offline-work' set up to track
'origin/feature/offline-work'.
PLAIN37.9.4 what is really happening inside#
- Let us be precise about what each offline command wrote.
git add README.md read the file, computed its SHA-1, wrote a blob object into .git/objects if that content was new, and updated .git/index.
git commit -m "..." built a tree object from the index, wrote a commit object naming that tree and the previous commit as parent, moved .git/refs/heads/main to the new commit’s hash, and appended two reflog lines.
- Total external dependencies of that sequence: none. Total network packets: zero.
git switch -c feature/offline-work wrote one new 41-byte file at .git/refs/heads/feature/offline-work and rewrote .git/HEAD to point at it. Two small writes.
- Now what was not possible, stated honestly, because this matters as much as what was.
- They could not fetch anyone else’s new work. If a colleague pushed during those hours, the reader’s machine had no way to learn it and no way to include it.
- They could not know whether anyone had pushed at all.
origin/main said 0a95cc8 and would have said 0a95cc8 if fifty commits had landed.
- They could not open a pull request, review one, comment, merge one, or read an issue. All of those are GitHub features reached over the network, not git features. Section 37.11 draws that line.
- They could not see continuous integration results, because those live on GitHub’s servers.
- They could not verify that their branch would merge cleanly into the real current
main, only into the main they last saw.
- So the honest summary: authoring was fully available, and coordination was fully unavailable. Git splits those two cleanly, and that split is the single most useful thing to remember from this chapter.
TECHNICAL37.9.5 the engineer’s version#
- Evidence classification for that session, keeping proof separate from inference, as this book does everywhere.
- Proven by observation. DNS resolution succeeded, returning
20.207.73.82. A TCP connection to port 443 at that address received no response of any kind within 15 seconds. The same request over a different access network succeeded immediately.
- Also proven. Local git commands returned exit status 0 and produced new objects and refs during the same period.
- Suggested, not proven. That traffic was being silently discarded somewhere on the path between the reader’s ISP and Microsoft’s edge. The absence of any RST and any ICMP unreachable is consistent with a silent drop and inconsistent with the server being down or actively refusing.
- Not established. Which party dropped it, or why. Traceroute showed
* * * from hop 13 onward, but many routers suppress ICMP time-exceeded replies, so silence at the end of a trace proves nothing on its own.
- What the git behaviour adds as evidence: the failure was at the transport layer, not at git’s application layer.
git fetch failing with Could not read from remote repository after a long stall is what a TCP connect timeout looks like from inside git.
- Recovery pattern for this situation, in order:
| 1 |
git status |
see local state |
| 2 |
git log –oneline –graph |
see what accumulated |
| 3 |
git fetch –all |
refresh the caches |
| 4 |
git log origin/main..main |
what you still owe |
| 5 |
git log main..origin/main |
what you missed |
| 6 |
git push |
publish |
- Step 3 is the one to insist on. Until a fetch succeeds, every “ahead” and “behind” number on the screen is arithmetic on a stale cache.
- Note the asymmetry between steps 4 and 5.
origin/main..main lists commits you have that the remote lacked. main..origin/main lists the reverse. The two-dot syntax is not symmetric and the order matters.
- If the push had failed midway with
send-pack: unexpected disconnect while reading sideband packet, the correct response is not to guess. It is to re-query the server, because a disconnect while reading the response tells you nothing about whether the write landed. Chapter 40 covers that case in full.
- Long-outage tip:
git bundle create out.bundle main writes a single file containing commits, which can travel on a USB stick and be cloned or fetched from on another machine. It is the offline transport of last resort and it is built into git.
WORDS37.9.6 remember these#
- Outage — a period when the network path does not work — loss of reachability between two endpoints, distinct from either endpoint being down.
- Silent drop — packets vanish with no error returned — a discard with no RST and no ICMP unreachable, leaving the sender to time out.
- Accumulate — pile up locally until you can send — build a series of local commits with no intervening synchronization.
- Authoring versus coordination — making your own work versus agreeing with others — the local-capable and network-required halves of a git workflow.
- Bundle — a whole repository transfer in one file —
git bundle, a pack plus refs, usable as a remote for clone and fetch.
- Two-dot range — commits on one side and not the other —
A..B meaning reachable from B but not from A, and not symmetric.
37.10 The trade-offs of the distributed model#
PLAIN37.10.1 in simple words#
- This book does not sell things, so here are the costs of the distributed model as honestly as the benefits.
- The benefits first, briefly. You can work with no network. Operations are fast because they are local. Every clone is a backup of the whole history. No single machine’s failure loses the project.
- Now the costs.
- The mental model is harder. In Subversion there is one history and one revision number. In git there are as many histories as there are clones, and they may disagree, and you must reason about which is which.
- You cannot lock a file. In some teams, a designer needs to say “nobody else touch this 400 MB scene file while I work on it”. Git has no way to enforce that, because there is no central place to hold the lock.
- Files that cannot be merged are a real problem. Two people edit the same image, the same 3D model, the same compiled asset. Git will detect the conflict and then be unable to help. One person’s work must be thrown away.
- Whole-history clones are large. For source code that is fine. For a project with twenty years of large binary assets it can be hundreds of gigabytes, and every new team member downloads all of it.
- There is no single source of truth unless the team agrees on one. Git does not provide it, so the team, the server permissions and the process must.
PLAIN37.10.2 a picture in your head#
- Distributed version control is like everyone owning a car instead of sharing one bus.
- You go when you like. You do not wait for the timetable. If the bus company strikes, you still get to work.
- But now there are twelve cars to park, twelve to fuel, and no driver telling everyone which route is correct. If two of you must arrive together, you have to arrange it yourselves.
- And a bus is genuinely better for some journeys. If the whole team must move one enormous object, one vehicle with one driver is the right design.
- Nobody thinks the bus company is foolish for existing. They are solving a different problem.
Where this comparison breaks:
- Cars and buses cost about the same to run per person. Git and Perforce do not: git is free software and Perforce is licensed per user. Cost is part of why git spread, and it should be named.
- And unlike vehicles, git and Perforce can coexist. Some studios run Perforce for art assets and git for engine code in the same building.
PLAIN37.10.3 a worked example#
- Concretely, why a game studio chooses Perforce. Take a project with 800 GB of assets and 300 people.
| New joiner setup |
clone all history |
sync latest only |
| Lock a scene file |
not possible |
exclusive checkout |
| Merge a 2 GB binary |
impossible |
prevented by lock |
| Check out one folder |
awkward |
native, standard |
| Work offline |
full |
very limited |
| Cost |
free |
per-seat licence |
- Rows one to four favour Perforce for this workload. Row five favours git. Row six favours git.
- For a 300-person studio where everyone sits in one building with a fast local network, row five is worth little and rows one to four are worth a great deal.
- For a distributed open-source project with 4,000 contributors on five continents who have never met, the weighting inverts completely.
- Neither team is confused. They are optimizing different things, and the correct engineering answer is “it depends on the workload”, which is an unsatisfying sentence that happens to be true.
PLAIN37.10.4 what is really happening inside#
- The locking problem deserves precision, because it is the clearest structural limit of the distributed model.
- A lock is a statement that everyone must agree on, at the same time. “Only Priya may edit
level3.blend right now.”
- Agreement at the same time requires a single place that answers the question. That is exactly what a distributed system does not have.
- So the absence of file locking in git is not laziness or a missing feature. It is a direct consequence of the architecture. You cannot have both.
- Git LFS adds a lock command, and it works by contacting the LFS server. That is the same admission: to lock, you must centralize something.
- The binary merge problem is different and is about file formats. Git merges text line by line. A
.psd, a .fbx or a .docx has no lines, so there is nothing to merge.
- Git will happily store binary files and version them. It just cannot combine two divergent edits, and it cannot compress them well, so the repository grows roughly linearly with every binary change.
- The size problem compounds because git history is immutable. A 500 MB file committed once by mistake in 2019 is in every clone forever, unless someone rewrites history and every clone is re-made.
TECHNICAL37.10.5 the engineer’s version#
- Honest comparison across the main systems still in use in 2026.
| Git |
distributed |
source code, everywhere |
| Perforce P4 |
centralized |
games, chips, film |
| Subversion |
centralized |
legacy enterprise |
| Mercurial |
distributed |
some large monorepos |
| Plastic SCM |
centralized+ |
Unity game teams |
- Perforce, founded 1995 by Christopher Seiwald, remains the standard in game development, semiconductor design and visual effects. It received the 2019 CEDEC Award for Engineering, an award from the Japanese game industry, which tells you where its users are.
- Perforce’s exclusive checkout is implemented with the
+l file type modifier, which makes the server grant a write lock to one client at a time.
- Perforce’s other decisive feature for these teams is the client workspace view, a mapping that lets a user sync one subdirectory of a 5 TB depot. Git has partial workarounds, sparse-checkout and partial clone, but they are less complete and less familiar.
- Where experts disagree: some argue git plus LFS plus sparse-checkout now covers the game studio case and Perforce persists mainly through inertia and tooling lock-in. Others argue locking and workspace views are structural and git will never match them. Both positions are held by serious practitioners and the disagreement is genuine.
- Scaling work on the git side is real and ongoing. Microsoft’s VFS for Git, released in 2017 as GVFS, and later Scalar, merged into git itself in version 2.38 in October 2022, exist to make very large repositories usable.
- Mitigations for the size problem, all mainstream as of 2026.
| Git LFS |
clone size |
needs LFS server |
| partial clone |
clone size |
fetches on demand |
| shallow clone |
clone size |
loses history |
| sparse-checkout |
working tree |
not history |
| commit-graph file |
traversal time |
none, local |
| multi-pack index |
lookup time |
none, local |
- Note which column those mitigations sit in. The last two are free local speedups. The first three all trade offline capability or history for size. That trade is the theme of this section.
git maintenance start, added in git 2.31, March 2021, schedules background packing, commit-graph writing and prefetching, and is the modern replacement for manually running git gc.
WORDS37.10.6 remember these#
- File locking — one person at a time may edit a file — a server-granted exclusive write lock, unavailable in a purely distributed model.
- Binary file — a file with no lines to compare — content without a text structure, so it cannot be three-way merged or delta-compressed well.
- Monorepo — one repository holding many projects — a single history covering multiple products, common at very large companies.
- Sparse checkout — having only part of the project in your folder — a cone-mode pattern list restricting which paths are materialized.
- Source of truth — the copy everyone agrees is correct — an external convention plus server permissions, not a property of git itself.
- Depot — Perforce’s name for the server-side store — the centralized repository from which client workspaces are mapped and synced.
37.11 Git is not GitHub#
PLAIN37.11.1 in simple words#
- This confuses more beginners than anything else in the subject, so it goes in plainly.
- Git is a program on your computer. You install it. It makes and reads files in a
.git directory. It is free software, first released in 2005.
- GitHub is a company that runs servers. You make an account. It stores copies of repositories and adds features around them. It launched on 10 April 2008 and Microsoft bought it in 2018.
- Git does not need GitHub. GitHub needs git.
- You can use git for thirty years and never make a GitHub account.
- You can host repositories on GitLab, on Bitbucket, on a Raspberry Pi in a cupboard, on a USB stick, or on a company server. Git does not care.
- The names are similar because GitHub chose a name based on git. That naming choice has cost the world a great deal of confusion.
- Here is the sharpest possible test, and the reader lived it: during the outage, git kept working and GitHub did not.
- That is not a coincidence. It is the boundary made visible.
PLAIN37.11.2 a picture in your head#
- Git is a word processor. GitHub is a print shop and a noticeboard.
- The word processor runs on your laptop. It works on a train. It never asks anyone’s permission.
- The print shop takes your document, keeps a copy, lets others read it, lets people leave notes on it, and runs a checking machine over it.
- If the print shop closes, your word processor is unaffected and your document is still on your laptop.
- If your laptop dies, the print shop still holds the copy you dropped off.
- Neither one is the other. They cooperate through one narrow channel: you carrying documents in and out.
Where this comparison breaks:
- A print shop holds a dumb copy. A GitHub repository is a real git repository and could replace yours entirely, which is why it feels authoritative.
- And some things blur the line deliberately. GitHub Actions can commit to your repository. The GitHub web editor makes real commits. But those are GitHub running git on its own machines, which does not change the boundary.
PLAIN37.11.3 a worked example#
- The division of labour, feature by feature.
| commit, branch, merge |
yes |
no |
| full local history |
yes |
no |
| pull requests |
no |
yes |
| code review comments |
no |
yes |
| issues and projects |
no |
yes |
| CI via Actions |
no |
yes |
| access control, teams |
no |
yes |
| releases page |
no |
yes |
| hosting a remote |
partly |
yes |
- The
partly in the last row: git includes git daemon and can serve over SSH, so a plain server with git installed is a perfectly good remote. It simply has no web interface.
- During the reader’s outage, the left column all worked and the right column all failed, because every item in the right column is an HTTPS request to a server in that unreachable direction.
- The clearest evidence in that session: plain git operations behaved as described in 37.8, and GitHub API calls failed. Same laptop, same minute, different dependency.
PLAIN37.11.4 what is really happening inside#
- What git actually is: a set of programs, mostly C, plus some shell and Perl. On a typical install it is a
git binary and a directory of helper executables, around a hundred subcommands.
- It speaks two wire protocols for talking to other repositories: the smart HTTP protocol and the SSH protocol, plus the older
git:// daemon protocol on port 9418. Chapter 40 covers all three.
- What GitHub actually is: a large web application and API, backed by many servers that hold bare git repositories, wrapped in authentication, permissions, a review system, an issue tracker and a CI system.
- When you
git push to GitHub, git on your laptop opens a connection, runs the standard protocol, and a program on GitHub’s side receives the pack. That receiving program is git-receive-pack, part of git.
- So the push path is pure git at both ends. GitHub’s own features do not participate.
- When you open a pull request, none of that happens. Your browser or the
gh command makes an HTTPS API call to GitHub. There is no git protocol involved and nothing is written to your .git.
- That is why one worked offline and the other did not. They are different systems reached by different means.
- The honest version: people say “push to GitHub” and “my code is on GitHub” as if GitHub were where the code lives. Your code lives on your disk. GitHub has a copy, which is extremely useful and is not the original.
TECHNICAL37.11.5 the engineer’s version#
- Git the software: created 2005, maintained by Junio Hamano since 26 July 2005, licensed under GPL version 2, current release 2.55.0 dated 29 June
- Written mainly in C.
- GitHub the service: launched 10 April 2008 by Tom Preston-Werner, Chris Wanstrath, P. J. Hyett and Scott Chacon. Microsoft announced acquisition on 4 June 2018 for 7.5 billion US dollars in stock. The deal closed on 26 October 2018.
- Alternatives, all of which speak the same git protocols: GitLab from 2011, Bitbucket from 2008, Gitea, Forgejo, Sourcehut, AWS CodeCommit, Azure Repos, and a plain SSH account on any Linux box.
- Interface boundary in one table.
| git clone, fetch, push |
git over HTTPS/SSH |
needs net |
| git commit, log, merge |
none |
works |
| open a pull request |
GitHub REST/GraphQL |
needs net |
| gh pr list |
GitHub REST/GraphQL |
needs net |
| view Actions run |
GitHub web/API |
needs net |
- The
gh command line tool is a GitHub client, not a git client. It has nothing to do with the git binary and fails independently of it. During the reader’s outage this distinction was visible directly.
- GitHub-specific concepts that do not exist in git at all: pull request, fork relationship, issue, label, milestone, review, required status check, branch protection rule,
mergeStateStatus, Actions workflow, release, Gist.
- Git concepts GitHub merely displays: commit, tree, blob, tag, branch, ref, merge, rebase, cherry-pick, reflog. GitHub cannot show you a reflog, because a reflog is local to a working repository.
- Chapter 40 covers remotes, transports, authentication, tokens and scopes, and the exact division of labour when a push both moves refs and triggers a GitHub workflow.
WORDS37.11.6 remember these#
- Git — the version control program on your machine — a distributed VCS implemented as a suite of C programs, GPLv2, first released 2005.
- GitHub — a company hosting git repositories with extra features — a web service owned by Microsoft since 2018, offering pull requests, issues and CI.
- Forge — the general word for a code hosting service — GitHub, GitLab, Bitbucket, Gitea and similar, all built around bare git repositories.
- Pull request — a request to merge, with discussion — a forge feature, not a git feature, implemented over the forge’s own API.
- git-receive-pack — the program on the server that accepts a push — the receiving half of git’s push protocol, run by the hosting service.
- API — a way for programs to talk to a service — here, GitHub’s REST and GraphQL interfaces over HTTPS, entirely separate from the git protocol.
37.12 Getting started properly#
PLAIN37.12.1 in simple words#
- There are exactly two ways a git repository comes to exist on your machine.
git init makes a new empty one, here, now, from nothing. No network.
git clone <url> copies an existing one, including all its history, and sets up a remote called origin pointing back at where it came from.
- That is the complete list. Everything else is one of these two, possibly done by a tool on your behalf.
- Before your first commit, tell git who you are. It records a name and an email address in every commit, permanently.
- That email does not have to be real or reachable. Nothing sends mail to it. It is a label. But it is baked into the commit hash and cannot be quietly changed later without changing every hash after it.
- Then create a
.gitignore file listing things git should not track: build outputs, dependency folders, editor files, secrets.
- And learn one hard fact early, because it catches everybody:
.gitignore does not untrack a file that is already tracked.
PLAIN37.12.2 a picture in your head#
git init is starting a new notebook. Blank, yours, immediate.
git clone is photocopying someone’s finished notebook, cover to cover, including their crossings-out, and writing their address inside your front cover so you can find them again.
.gitignore is a note on the front of the notebook saying “do not copy receipts into this book”.
- And here is the catch that the note cannot fix: if you already glued a receipt onto page 4, the note does nothing. The receipt is on page 4. It will be on page 4 in every photocopy anyone ever makes.
- To stop tracking it you must deliberately remove it, and even then it stays in the old pages, because old pages are never edited.
Where this comparison breaks:
- A photocopy is a dead copy. A clone is fully alive: it can commit, branch and serve as the source for further clones.
- And unlike a notebook, you can start with
git init and later attach a remote, which is the normal path for a project that begins locally.
PLAIN37.12.3 a worked example#
- The full first-time setup, exactly as it should be typed.
git config --global user.name "Shikhar Singh"
git config --global user.email "you@example.com"
git config --global init.defaultBranch main
git config --global pull.rebase false
git config --global core.autocrlf input # false on Windows
- Then a new project from nothing:
mkdir myproject && cd myproject
git init
printf 'node_modules/\n*.log\n.env\n' > .gitignore
git add .
git commit -m "Initial commit"
- That commit exists now, fully, with no account and no network.
- Now the
.gitignore trap, demonstrated with real output from a sandbox. A secret file was committed by mistake, then added to .gitignore.
$ git ls-files
.gitignore
app.py
config.env <- still tracked
$ printf 'CHANGED\n' >> config.env
$ git status --short
M config.env <- git is still watching it
- The
.gitignore had no effect at all, because ignore rules apply only to files git is not already tracking.
- The fix, and its limit:
$ git rm --cached config.env
$ git commit -m "stop tracking config.env"
$ git ls-files
.gitignore
app.py <- now untracked
$ git show HEAD~2:config.env
secret-key <- STILL IN HISTORY, forever
- Read that last line carefully. The secret is out of the current commit and still recoverable from an old one. If it was a real credential, it is compromised and must be rotated. Removing it from history requires rewriting every commit after it, and every clone must be remade.
PLAIN37.12.4 what is really happening inside#
git init creates the .git directory, writes a default config, writes HEAD containing ref: refs/heads/main, and installs the sample hooks. It contacts nothing.
- Note that
refs/heads/main does not exist yet at that point. HEAD points at a branch that has no file. That is the “unborn branch” state, and it is why git status on a fresh repository says “No commits yet”.
git clone does more. It creates the directory, runs init, adds the [remote "origin"] config, fetches everything, writes the remote-tracking refs, creates a local branch matching the remote’s default, and checks it out.
- Real output, so you can see how little it says:
$ git clone /tmp/gitdemo/server.git clonedemo
Cloning into 'clonedemo'...
done.
$ git branch -a
* main
remotes/origin/HEAD -> origin/main
remotes/origin/feature/offline-work
remotes/origin/main
- Notice
remotes/origin/HEAD -> origin/main. That records which branch the server said was its default, so git checkout of a bare remote name works.
.gitignore mechanics: on git add and on git status, git checks each untracked path against the ignore rules. Tracked paths skip the check entirely. That is the whole reason for the trap in 37.12.3.
- Ignore rules come from several files, most specific first: the path’s nearest
.gitignore, then parent directories, then .git/info/exclude, then the file named by core.excludesFile.
.gitignore is committed and shared. .git/info/exclude is private and never leaves your machine. Use the second for editor droppings that only you produce.
TECHNICAL37.12.5 the engineer’s version#
- Settings worth changing on day one, with what each does.
| user.name |
your name |
commit author |
| user.email |
your address |
commit author |
| init.defaultBranch |
main |
branch on init |
| pull.rebase |
false or true |
pull behaviour |
| push.default |
simple |
push safety |
| core.autocrlf |
input or false |
line endings |
| fetch.prune |
true |
drop dead refs |
| rerere.enabled |
true |
reuse conflict fixes |
| diff.algorithm |
histogram |
better diffs |
init.defaultBranch was added in git 2.28, July 2020. Before that the default was master, hard-coded. Git still defaults to master if you do not set it, and prints a hint saying so.
push.default = simple has been the default since git 2.0 in 2014. It pushes the current branch to its upstream of the same name and refuses otherwise, which prevents accidental mass pushes.
core.autocrlf matters only across operating systems. On Windows, true converts to CRLF on checkout and back to LF on commit. On macOS and Linux, input or false. Better still, commit a .gitattributes with * text=auto and stop thinking about it.
rerere stands for “reuse recorded resolution”. With it on, git remembers how you resolved a conflict and applies the same resolution if it sees the same conflict again. It is invaluable during a long rebase.
- Verify what is set and from where:
$ git config --list --show-scope --show-origin
system file:/etc/gitconfig filter.lfs.required=true
global file:/root/.gitconfig user.name=Shikhar
global file:/root/.gitconfig user.email=shikhar@example.com
git check-ignore -v --no-index <path> tells you which rule in which file at which line number is ignoring a path. Without --no-index it deliberately ignores tracked files, which is itself a source of confusion.
$ git check-ignore -v --no-index config.env
.gitignore:1:config.env config.env
- For removing a secret from all history the current recommended tool is
git filter-repo, which replaced the much slower git filter-branch. The git project has discouraged filter-branch since version 2.24, November
- Either way, every existing clone becomes invalid and the credential must be rotated regardless.
.gitignore pattern rules follow gitignore(5): # is a comment, a trailing / matches directories only, a leading / anchors to the .gitignore’s own directory, ** matches across directories, and a leading ! negates an earlier rule. A negation cannot re-include a file if its parent directory is itself excluded, which is the most common pattern bug.
WORDS37.12.6 remember these#
- init — start a new empty repository here — create a
.git directory with default config, HEAD and hooks, contacting nothing.
- Clone — take a full copy of an existing repository — init plus a remote plus a full fetch plus a checkout of the default branch.
- .gitignore — a list of things git should not track — patterns matched against untracked paths only, per
gitignore(5), committed and shared.
- Untracked file — a file git knows nothing about — a working tree path with no index entry, eligible for ignore rules.
- Tracked file — a file git is watching — a path present in the index, immune to
.gitignore until removed with git rm --cached.
- Unborn branch — HEAD points at a branch with no commits yet — the state after
git init, before the first commit creates the ref.
- rerere — git remembering how you solved a conflict before — reuse recorded resolution, enabled with
rerere.enabled true.
37.98 Common wrong ideas#
- Wrong: git needs a server. Right: git needs a directory.
git init, git add, git commit, git branch and git merge all work on a machine that has never been connected to anything.
- Wrong:
origin is the real repository. Right: origin is a nickname for a URL stored in .git/config. It has no authority in git’s data model. Rename it, remove it, or have twenty of them.
- Wrong:
origin/main tells you where the remote’s main branch is. Right: it tells you where it was the last time a fetch, pull or push succeeded on this machine. It is a file on your disk and it never expires.
- Wrong: cloning downloads the current version of the project. Right: cloning downloads every commit, every version of every file, every branch and every tag, then checks out one of them into your folder.
- Wrong: git and GitHub are the same thing. Right: git is a program from 2005 that runs on your computer. GitHub is a company from 2008, owned by Microsoft since 2018, that hosts copies and adds pull requests, issues and CI.
- Wrong: a commit is saved to the cloud. Right: a commit writes objects and one ref into
.git on your own disk. Nothing leaves your machine until you push.
- Wrong:
git pull is the safe way to get up to date. Right: git pull is git fetch plus a merge or rebase that changes your branch. git fetch alone updates your knowledge and changes nothing of yours.
- Wrong: adding a file to
.gitignore removes it from the repository. Right: ignore rules apply only to untracked files. An already-tracked file keeps being tracked until git rm --cached, and stays in history forever.
- Wrong: “ahead 3, behind 2” is a live comparison with the server. Right: it is arithmetic between two local refs, one of which is a cache that may be weeks old. Run
git fetch before believing it.
- Wrong: a shallow clone is just a smaller clone. Right: a shallow clone has no old history, so
git log, git blame and git bisect cannot see past the cut, and deepening requires the network.
37.99 Chapter summary in 20 lines#
- Version control solves six problems: history, undo, blame, parallel work, backup and collaboration. Only the last two need another computer.
- Dated folders fail because they record copies without recording reasons or relationships, and because two people cannot use them at once.
- The lineage runs SCCS 1972, RCS 1982, CVS from 1986 and 1990, Subversion from 2000 with 1.0 in February 2004, then git in April 2005.
- In a centralized system the history lives on a server and your machine holds a working copy, so a commit is a network operation.
- In a distributed system every clone is a complete repository, so a commit is a local file write. This is why the reader kept committing during the outage.
- Git exists because BitMover ended the kernel’s free BitKeeper licence in April 2005. Torvalds started on 3 April and git was self-hosting on 7 April.
- His stated goals were speed, simple design, strong support for non-linear development, fully distributed operation, and handling the Linux kernel.
- A
.git directory holds objects, refs, HEAD, config, index, hooks, logs, info, description and sometimes packed-refs. That is the entire repository.
- A branch is a 41-byte file containing a commit hash. That is why branching is instant, free and offline.
- In a real clone of the git project,
.git was 316 MiB against a 60 MiB working tree, holding 418,900 objects and 85,263 commits.
- Compression is why: about 7,877 MiB of raw object content stored in about 302 MiB, a ratio near 26 to 1, via zlib and pack deltas.
- Shallow and partial clones trade history or content for size, and both reintroduce the network into operations that were previously local.
origin is a nickname for a URL in .git/config. It is a convention, not a standard, it carries no authority, and a repository may have zero remotes.
origin/main is a local file recording where you last saw the remote’s main. It is a memory, not a measurement, and it never expires by itself.
- Only
fetch, pull and push update remote-tracking refs. git reflog show origin/main --date=iso tells you how stale yours is.
- Local commands touch only objects, refs, index and working tree. Remote commands synchronize two object databases. The split has no fuzzy middle.
- During the outage, authoring was fully available and coordination was fully unavailable: no fetching others’ work, no pull requests, no CI results.
- The costs of the distributed model are a harder mental model, no file locking, no useful merging of binaries, large clones, and no built-in truth.
- That is why game studios, chip designers and visual effects houses still buy Perforce, founded 1995, for its exclusive checkout and workspace views.
- Git is a program on your computer. GitHub is a company that hosts copies. During the outage git worked and the GitHub API did not. That is the line.