feat: add 10-minute granularity#107
Note
🤖 Claude Opus 5.5 on behalf of Oliver
ELI5
Adds a 10-minute zoom level next to the existing 5-minute, 30-minute, hourly and daily views.
Why
30 minutes is too coarse for multi-day investigations and 5 minutes is too noisy. This is layer 1 of the MAAD stack, which ends in one fresh reprocess.
What changes
- New
10mgranularity: half-open, local-time aligned, exactly two 5m children, and the same additive rollup, coverage and zero-fill rules as 30m/1h. Every stats table (including MAAD andbucket_coverage) gets 10m rows. - Rust
Granularity, CHECK constraints, rollup scheduling,verifyrollup parity,exportandcompareunderstand10m. Contract and table versions bump, so this needs a fresh product database. - Web: granularity parsing, pickers, auto-selection (ranges up to ~8 days default to 10 min), and hour-click drilldowns now open 10 min so they match drag selection.
- Removed dead helpers (
build_rollups,getBucketStartQuery,StatsTable::schema_version).
Cost (accepted)
A normal day goes from 361 to 505 stored buckets per scope (288 × 5m + 144 × 10m + 48 × 30m + 24 × 1h + 1 × 1d), which is +40%. Each extra bucket needs its own address unions and MAAD run. Pipeline runs were about 30–40% slower in wall-clock time on the same day and machine. That cost comes with the feature: review found no duplicated work. The owner accepted it.
How to review
Setup: copy datasets.json.example to datasets.json, point root_path at any nfcapd tree, and build the nfdump fork (docs/user/setup-pipeline.md).
./scripts/netflow-db.sh pipeline --nfdump target/nfdump/libexec/nfdump --dataset example --start-date <day> --end-date <day> --database-path /tmp/10m.sqlite. Expected:SELECT granularity, COUNT(DISTINCT bucket_start) FROM bucket_coverage GROUP BY 1reports 144 for10mon a normal day../scripts/netflow-db.sh verify /tmp/10m.sqlite --require-rollup-parity. Expected: OK.bun run dev:web, open the dataset, choose 10 min, then click an hour on an hourly chart. Expected: the drilldown opens at 10 min.
The top layer of the stack covers the real-day end-to-end check.
Edge cases and decisions
- DST follows the existing civil-time convention. The 5-minute iterator walks local wall-clock times, so the repeated fall-back hour is not revisited. Both transition days therefore keep the existing 5m counts: 276 on spring-forward and 288 on fall-back. 10m pairs those children, which gives 138 10m buckets on spring-forward and 144 on fall-back. Tests pin this (
ten_minute_bounds_pair_local_five_minute_buckets_across_dst_transitions). This PR does not change the convention. compareagainst an older reference database accepts candidate-only10mrows.
Verification
- Automated (CI and local):
bun run format,bun run lint,bun run typecheck,bun run test:web,bun run test:db(includingten_minute_rollups.rsand the DST test). - Manual: steps 1–3 above on any local capture day.
Made by Claude Opus 5.5 (with Opus 5.5 subagents) via Claude Code.
Checks
- SuccessfulPlaywright E2EGitHub
- SuccessfulRust Pipeline ChecksGitHub
- SuccessfulWeb ChecksGitHub