Architecture

A push, end to end

The single most important flow in Ghost, from `git push` to the bytes in the bucket and everything that happens after.

This lesson is the map; the Git transport and Storage chapters zoom into each step. Read it once now and again after those chapters.

The steps

  1. Advertisement. git push first asks info/refs?service=git-receive-pack. apps/api/src/git/middleware/git-basic-auth/git-basic-auth.middleware.ts checks the Basic credentials (username plus API key) and authorizes write right here, so a read-only user is refused before uploading anything. The cache is materialized from the log and git receive-pack --advertise-refs answers.
  2. The upload. The POST passes the same auth, then apps/api/src/git/middleware/git-raw-body.middleware.ts spools the body to a temp file, gunzipping if needed. Nothing ever holds a packfile in memory (ADR 0015).
  3. The probe. A push over 1 MiB is preceded by a POST whose body is exactly 0000, sent so git can handle an auth challenge before streaming. It gets a 200 and nothing else (ADR 0014).
  4. Commands. readReceivePackHeader reads only the start of the body: one old new ref line per ref, then where the packfile begins.
  5. Refusals before the log. Writes to refs/pull/* are refused (they belong to Ghost, ADR 0033), unless the key belongs to a running import.
  6. Verification. apps/api/src/lib/git/protocol/verify-push.ts › withVerifiedPack indexes the pack into a quarantine directory and checks every new ref names a real object the push or the log holds (ADR 0034). Inside it, the push's size is reserved against the account's quota under a per-account lock (ADR 0035).
  7. The commit point. apps/api/src/services/git/wal/push-transaction.service.ts › commitPush uploads the entry, then compare-and-swaps the index. This is the moment the push becomes real (ADR 0005).
  8. The cache catches up. Only now does git receive-pack --stateless-rpc apply the push to the local cache and produce the report the client prints.
  9. Afterwards, without making the client wait: reindex code search, update contribution counts, close issues named by fixes #12 in the new commits, resync pull request refs, record the push on pull request timelines, publish push events for webhooks.

If any check before step 7 fails, the client gets a per-ref refusal (! [remote rejected] main (reason)) in git's own report format, and the log is untouched.

The orders that matter

  • Log before cache. Reversed, a failed commit would leave a ref visible locally for a push that never happened. The cache may lag the log, never lead it.
  • Checks before the log. receive-pack's own refusals happen after step 7, too late: a ref it refuses would already be in the log forever. So Ghost does git's checks itself first.
  • Side effects after the response. A push must not wait for a history walk. Each side effect tolerates being lost (a later push or read redoes it).

Try it

Read apps/api/src/git/git.service.ts › receivePack top to bottom with the diagram open. Then find commitToLog in the same file and match each line to steps 5 to 7.

Check yourself

git receive-pack refuses a ref (say, an unpacker error). Why must Ghost have caught that before commitPush?
The code index fails to update after a push. Is the push lost?

On this page