Storage

Deletion, repair and what is not built

Tombstones, purges, the repair tool, and the storage work still ahead.

Deleting a repository

DELETE /api/repositories/:owner/:slug removes everything before it answers 204 (ADR 0020):

  1. Tombstone. The index is replaced, by compare-and-swap, with an empty one whose deleted flag is set. From here every read and push sees the repository as gone.
  2. Purge, in parallel. Every entry under repos/<id>/entries/, the local cache (after any replay settles), and the code search shards.
  3. Drop the row. Issues, pull requests and the rest cascade.

If any step fails, the request fails and the row stays, tombstoned and deletable again. Every step is idempotent.

Why a tombstone, not deleting the index? A push authorized a moment before the delete would find no index and create a fresh one under the dead id. Against a tombstone, it loses its CAS, reads the repository as deleted, and removes its own entry before failing. The tombstone is never removed; it holds nothing but a flag and a seq.

A repository that heads an open pull request into someone else's repository cannot be deleted (409). Closed and merged requests it opened keep their history in the base; a merged one even keeps its diff, because the merge copied the commits into the base log.

The purge covers this process's cache only. That is one reason the API runs as one replica.

Repairing logs broken before 0034

apps/api/src/scripts/repair-repositories.ts (run by hand on the API host: node dist/scripts/repair-repositories.js [--apply]) uses apps/api/src/services/git/repair/repository-repair.service.ts:

  1. Replay the log into an empty scratch repository, never the cache, which could hide the damage. An entry that will not index is rebuilt with the cache lent for bases, or its layer is zeroed so replay skips it. Sequence numbers never move.
  2. Check every ref; rescue missing objects from the cache, or drop refs that can never replay.
  3. Commit the repair through commitPush, like any push, so every cache reconciles.

Not built yet

From ADR 0011:

  • Checkpoints and compaction. Every push reads the whole index; replay from zero is O(history). The plan: immutable checkpoint/<id>/<seq> objects holding a ref map and live layers. compactedThroughSeq exists for this and is always 0. Compaction must account for pending pull-ref billing first (ADR 0033).
  • Garbage collection of orphaned entries. The trap: an entry being uploaded right now is legitimately unreferenced. The sweep must require "unreferenced and older than hours", using the ULID's timestamp.
  • Multipart entry uploads. An entry is one PutObject (5 GiB max); a failure mid-flight re-uploads everything. A throughput problem, not a correctness one.
  • More than one API replica, which needs caches told about deletions.

JGit's DFS backend is this design built at scale (git on blob storage with a CAS ref database). Read it before building any of the above.

Check yourself

Why does a deleted repository keep a tombstone index forever?
Why must a garbage collector not simply delete every unreferenced entry?

On this page