The binary format
Why the index and entry headers are hand-rolled bytes, and the two fields that look like overhead and are not.
Both the index and entry headers are binary: big-endian, magic-prefixed, versioned. apps/api/src/lib/git/wal/wal-codec.ts encodes and decodes; apps/api/src/lib/git/wal/cursor.ts provides Cursor (writes) and Reader (reads).
Why not JSON
The index is read on every push and every materialization. Object ids dominate its size, and JSON forces them to 40 hex characters where 20 raw bytes do, plus quotes and key names. Binary halves the dominant cost and removes parsing from the hot path (ADR 0008).
Cursor and Reader exist because the alternative, offset arithmetic at every field, reliably produces an off-by-one that corrupts a log instead of throwing.
Layouts
index entry
───── ─────
magic u32 "GWAL" magic u32 "GENT"
version u8 version u8
flags u8 bit 0 = tombstone headerLen u32 ← the pack starts here
seq u64 ulid 16 bytes
compactedThroughSeq u64 createdAt u64
refCount u32 pushedBy u16 length + utf8
ref u16 length + utf8 transitionCount u32
oid 20 bytes ref u16 length + utf8
layerCount u32 oldOid 20 bytes
ulid 16 bytes newOid 20 bytes
packSha 32 bytes <packfile bytes>
size u64Different magics per format: an entry read as an index fails at byte 0 instead of passing the version check and decoding garbage.
headerLen up front buys a real capability: an entry's metadata is readable with a ranged GET of the first 64 KiB instead of downloading a 500 MB pack (WalStoreService.readEntryHeader).
Two fields that are not overhead
versionis checked on decode; an unknown value throwsWalCorruptError. A format that cannot be versioned cannot evolve.flagscarries the tombstone bit for the index, and space for the SHA-1 → SHA-256 object format switch. Retrofitting a hash-length change without a flag byte would mean rewriting every object of every repository.
This is an on-disk contract
Any change to either layout needs a version bump and a decoder that reads both versions. Logs already in buckets will not be rewritten for you.
Try it
Open apps/api/src/lib/git/wal/wal-codec.spec.ts. Add a test asserting that the first four bytes of an encoded entry header read GENT in ASCII. Run bunx vitest run src/lib/git/wal in apps/api.