Bodies, streams and the probe

A push body can be gigabytes. Three real bugs taught the transport how to read one. Each is an ADR; together they are the best introduction to how subtle streaming is.

Bug 1: the missing commands (ADR 0013)

receivePack used to materialize the cache, slow work against S3, before reading the body. An HTTP request stream is already running; data arriving while the handler was busy elsewhere was lost. Git writes its command section in its own socket write, so the lost chunk was exactly the ref commands, and the parser reported "no ref update commands": accurate, and pointing nowhere near the cause. It reproduced only against real object storage, never in tests.

Rule: never leave an incoming stream unattended across an await. Read it, then do the slow work.

The fix moved capture into middleware, ahead of handlers: apps/api/src/git/middleware/git-raw-body.middleware.ts. It also unwraps Content-Encoding: gzip/deflate, which git uses at its discretion.

Bug 2: 2 GiB (ADR 0015)

Pushing the Next.js repository (a 2.25 GiB pack) crashed with RangeError: data is too long: Node's hash cannot update() more than INT_MAX bytes at once. Every place that held a pack in a Buffer was a landmine. Now:

• the body is spooled to a temp file, deleted when the response closes;
• it travels as a GitRequestBody: { size, open(start?) }, which can be read several times from any offset;
• hashing, uploading and replay all stream.

// lib/git/protocol/git-request-body.ts
export function fileBody(path: string, size: number): GitRequestBody  // production
export function bufferBody(buffer: Buffer): GitRequestBody            // tests

The push path needs the body three times: the command section from the front, the packfile alone from packOffset for the log, and the whole thing for receive-pack. A single-pass Readable cannot do that; holding the bytes is what broke. A spec streams 2.15 GiB through commitPush in about a second as the regression guard.

Bug 3: the probe (ADR 0014)

Pushes over 1 MiB failed with HTTP 400 … the remote end hung up unexpectedly. Above http.postBuffer, git streams with chunked encoding, which cannot be rewound to retry after a 401. So git first sends a probe: a POST whose body is exactly 0000. Ghost parsed it as a push with no commands and answered 400, killing the real push before it started.

Now apps/api/src/lib/git/protocol/receive-pack-request.ts › isProbeRequest answers a flush-only body with 200 and nothing else. The probe still passes authentication first: getting challenged is the reason it exists.

What the bugs have in common

Each passed every test and failed only at real size or real latency. When you work on the transport, test with a large repository and a remote bucket, not just a fixture.