feat(netflow-db): add concentration controls for MAAD address sets#128

Problem

netflow-predictor found that packet- and byte-weighted MAAD measures of external address sets carry forecasting information about near-term campus traffic. Reviewers will ask whether ordinary heavy-hitter concentration explains that. The pipeline stores no single-scale concentration measure to test against.

Change

address_concentration_stats stores conventional concentration controls for every address set and measure that MAAD computes. It uses the same keys as address_maad_stats.

• hhi: sum of squared weight shares.
• top1_share, top10_share, top100_share: share of weight in the heaviest addresses.
• entropy_p8/p16/p24/p32 for IPv4 and entropy_p32/p48/p64/p128 for IPv6: Shannon entropy in bits after summing weights by fixed-length prefix.
• weight_total and entry_count, for auditing.

Concentration uses MAAD's per-address weights, zero-weight exclusion, internal-side skipping, and minimum count of two. Rollups recompute from merged per-address child maps and never average child statistics. The computation reuses MAAD's sorted entries, with no extra flow scan or full weight sort.

The schema and result configuration record the contract. verify, compare, merge-shards, analysis exports, and the D1 schema handle the new table. docs/code/pipeline-contract.md has a new "Concentration statistics" section.

Validation

• 235 Rust tests and 213 web tests pass.
• bun run format, format:check, lint, and typecheck pass.
• A local dual-stack pipeline run and verify produce 216 concentration rows.
• Optimized synthetic benchmarks show 8.7–10.5% extra computation relative to MAAD. Extra SQLite write time is unmeasured.

Rollout

This branch will be built on pinot and run across the cluster for 2026-01-01 through 2026-06-30, for the full campus product and both daily_active_sources subsets, before merging. The new products go to fresh paths and do not touch existing databases.

Code by GPT-6.1-Sol (Codex) and description by Claude Opus 5.5 (Claude Code), coordinated in T3 Code.

🤖 Generated with Claude Code