Keep media cleanup and duplicate within Convex limits (#204)

* Keep media cleanup and duplicate within Convex limits

- Media cleanup no longer reads a strategy's content. A new assetReferences
  table (one small row per content row and image it shows, kept in step by
  every content writer) answers "does anything still show this image" with
  one indexed read. Purges queue images in a new assetReclaimCandidates
  table in the same transaction that removes the content; a small scheduled
  worker (plus an hourly cron) decides them in bounded batches, and a failed
  run leaves its candidates queued.
- Content purges take a few rows per run (rows can be ~900 KB each).
- The agent summary reads only agent elements and lineup origins.
- Duplicate refuses a strategy over 12 MiB or 4,000 content rows with
  STRATEGY_TOO_LARGE_TO_DUPLICATE, before Convex's own limit; the client
  says "This strategy is too large to duplicate."
- maintenance:backfillAssetReferences gives existing content its reference
  rows (run once after deploy).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Gate reference-based cleanup on the backfill and bound it fully

Codex's four confirmed findings on #204:

- Every deletion that trusts reference rows waits for the reference
  backfill, recorded durably in a new completedBackfills row. Until then
  isAssetReferenced answers "referenced", the reclaim worker leaves its
  queue alone, and both purges wait. The backfill records itself complete
  after the last lineup page and starts the purge and reclaim it held.
  Runs of the two removed cleanup functions scheduled before deploy land
  in compatibility handlers that queue their images for reclaim.
- The duplicate budget counts stored bytes (getConvexSize), not JSON
  characters.
- The duplicate budgets its whole transaction: copied rows get their
  reference rows inserted without reads, the copy's agent summary comes
  from the rows already read, and each image the copy shows is charged
  the most its asset copy can read, which also keeps index reads bounded.
- Purges take at most 200 reference rows per run; a row's remaining
  reference rows are its persisted progress, so a lineup showing
  hundreds of images is purged across runs that each commit their part.

Adds maintenance:checkAssetReferences to verify the backfill, and fixes
images:deleteAssetRef forwarding clientProtocolVersion to an internal
query that rejects it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Bound the backfill and gate every deletion on both tables

Codex's recheck of 7d318d6 confirmed five more gaps:

- The backfill budgets reference writes (2,000 per run), not content
  rows. A run that spends its budget runs the same page again, and
  syncing is idempotent, so a row showing thousands of images is
  backfilled across runs.
- The two compatibility handlers queue 100 images per run and schedule
  the rest in the same transaction.
- The duplicate budget starts before pages are copied, and charges each
  page's bytes, its settings read, and its two writes.
- Each content table's backfill is recorded on its own, and the gate
  opens only when both are, in either order.
- Stored bytes are never removed before the gate opens: the physical
  sweep claims nothing until then, and a replaced legacy blob goes to the
  sweep as a deleted row instead of being deleted inline.

Adds maintenance:findShownUnavailableAssets, a read-only check that
reads live content and lists shown images with no active asset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
79d856a207Dara Adedeji committed on 9/27/2026, 4:03:40 AM· committed by GitHubparenta997481
22 files changedLine totals unavailable