feat(netflow): model capture coverage#95
Problem
ATLANTIS treated missing capture evidence as zero traffic. Native gaps produced synthetic metric rows, CSV holes could look observed, sparse API responses compressed time, and charts connected across intervals the pipeline had not proven.
Solution
This adds explicit complete, partial, and unknown capture coverage across ingestion, SQLite/D1, API timelines, and charts. Metrics remain separate: complete zero is numeric zero, unknown is null, and partial data remains visible when usable observations exist.
Native coverage is derived from physical member evidence. CSV files merge into one logical source-bucket unit, including overlapping files, and unsorted imports use disk-backed staging to stay bounded-memory. Successful runs print a five-minute coverage summary; --require-complete validates the requested native source/window after leaving an incomplete product inspectable.
Aggregate endpoints now return continuous TimeBucket<T> timelines. Unknown buckets retain their x-position and break lines. Partial points are hollow/dashed on the ordinary time-series charts. The dense spectrum chart keeps uniform points and relies on the coverage strip for that signal. Missing derived spectrum/structure results remain null without changing capture coverage.
The dashboard adds a compact, source-separated coverage strip through a coverage-only endpoint. It is last by default and participates in the existing chart ordering system. Complete buckets use a solid line, partial buckets use a dashed line, and unknown buckets leave the lane blank. Coverage details live on this strip; ordinary metric tooltips remain metric-only.
The strip uses the shared Chart.js registry, synchronized crosshair store, theme tokens, and native tooltip callbacks. An earlier DOM-rendered version inserted its tooltip into a spaced container and changed the card height by four pixels on hover. The canvas version keeps layout fixed and removes hundreds of interactive bucket elements.
Reviewer fast path
Setup: build a fresh product database with the new contract; prior-contract databases are intentionally incompatible.
- Native flow: exercise a multi-member source with one missing file, one decoded empty file, and then add the missing file. Expect unknown coverage without metrics, complete observed zero, and repaired base/coarse coverage on rerun.
- CSV flow: exercise selected-out rows, timestamped rejected rows, an internal hole, overlapping configured files, and
input_order: "unsorted". Expect complete zero, partial/null, unknown/null, one merged source-bucket unit, and chronological output. - Dashboard flow: request NetFlow, IP, protocol, characteristics, spectrum, structure, and coverage across a gap. Expect continuous timestamps, broken unknown spans, qualified partial points outside Spectrum, uniform Spectrum points, ordinary complete zeroes, and one thin coverage lane per selected source. Hover the coverage canvas and expect the native chart tooltip without any card movement.
- Ordering flow: drag the coverage card above another chart, reload, and expect the order to persist.
Important edge cases:
- Partial products are the default; strict mode is opt-in and scoped to the requested finite native window.
- Newly arrived native files repair missing evidence. Disappeared observed files stay preserved. Changed native content requires
--force. - Empty/header-only CSV files establish no bounds. Overlapping files do not inflate expected units.
- Capture coverage and derived-result availability are independent.
- Metric and coverage writes are transactional, and product identity is versioned for the new contract.
- Coverage endpoint source IDs are deduplicated, and request bounds must align to the selected bucket granularity.
Verification
bun run formatbun run lintbun run typecheckbun run test:web, 18 files and 79 testsbun run test:db, 117 unit, 4 comparison, and 1 MAAD conformance testbun run test:e2e, 2 Chromium flows, including stable coverage hover layout- Created and queried a one-day UOregon demonstration copy locally. It contains complete, partial, and unknown five-minute source coverage while leaving the source database and active rebuild untouched. This machine-local database is not committed.
Remaining manual work
- Migrate the full UOregon rebuild only after it finishes. The one-off migration will use its known five-minute
flows = 0signal on a stable copy, verify provenance, remove fabricated metrics, and preserve the source database before promotion. - A reviewer should visually spot-check the coverage strip and time-series partial point styling with a representative product database.
Addresses #94. Stacked after #87.
Implemented with GPT-5.6 Sol in T3 Code through the Codex harness, with GPT-5.6 Luna subagents for mechanical changes and adversarial review.