feat(web): add weighted MAAD dashboard view#120

merged4
feat/cluster-shard-merge
maad/08-weighted-dashboard
opened27d agobyOoliver
View on GitHubTwo-way sync enabledPR last imported 3d ago

Note

🤖 Claude Opus 5.5 on behalf of Oliver

Explain Like I'm Lost

MAAD (multifractal address-structure analysis) is computed per bucket for three measures: plain addresses, and packet- and byte-weighted addresses. The pipeline already stores all three, but the dashboard only charted the address measure. This PR lets the dashboard switch between them and charts the stored D0/D1/D2 dimensions over time.

Why

Before this PR, you could only see weighted MAAD on a single file's detail page, so you couldn't watch how packet or byte weighting changes the address structure over a range. The MAAD controls were also spread across individual cards.

Implementation

Stacked on #106.

What changes

  • MAAD group in Controls. A second toolbar row labelled MAAD: holds the measure (Addresses | Packets | Bytes) and the IP address family (IPv4 (/8–/24) | IPv6 (/23–/64)). Both are stored in the URL, as ?measure=addresses|packets|bytes and the existing ?ipVersion=4|6. Defaults (addresses, 4) are left out of generated links. The family applies to both MAAD cards, so this is the only place it appears. Reset View sets both back to their defaults. Datasets built without MAAD don't show the group.
  • Tidier Controls toolbar. Reset View moves into the card header. The granularity control keeps all five options on one line. All toolbar segmented controls are horizontal and the same size. When locality is on, Direction shares the second row with MAAD.
  • New "MAAD Dimensions" card. It has two compact selectors, Source | Destination and D0 | D1 | D2, and draws one line per selected source. The default is Source D1, because D1 responds to the weighting and D0 barely does. Each source keeps a fixed colour from a colourblind-checked palette, with separate light and dark shades. The y-axis fits the data with 10% padding. The tooltip shows three decimals. Day boundaries get x-axis labels, as in Traffic Overview. Click and range drilldown work like the other line charts.
  • Spectrum card. The source and side are chosen with segmented controls in one row above the chart. With Packets or Bytes selected, the card shows a compact placeholder ("The spectrum is only computed for the Addresses measure…") and sends no spectrum-stats request.
  • Card headers show the active measure and family, for example MAAD Dimensions Bytes · IPv4 (/8–/24). That keeps the context visible when a card sits far below Controls.
  • Datasets built without MAAD (--no-maad, so no maad_q_grid rows): both MAAD cards collapse to a short "MAAD was not computed for this dataset…" placeholder and send no MAAD requests.
  • Drilldown keeps the measure and family. A 5-minute click from any dashboard chart opens /netflow/files/<slug>?…&ipVersion=6&measure=bytes. On the file page, the family and measure controls read and write those URL params, and an invalid value returns 400. The Next File link carries dataset, direction, ipVersion and measure.
  • New APIs:
    • GET /api/netflow/dimension-stats takes the same params as structure-stats: routers, granularity, startDate and endDate (a half-open [start, end) window), direction, ipVersion and measure. It returns per-router coverage timelines with {saD0, saD1, saD2, daD0, daD1, daD2}, where null means too few addresses.
    • GET /api/netflow/maad-status returns { computed }.
  • The chart-order storage key moves to v6 because a card was added, so any saved custom order resets once.

How to review

Setup: a database built with MAAD. Either a real one or the Playwright fixture works (bun run test:e2e seeds fixture-router with one 5-minute bucket for all three measures).

  1. At about 1440px wide, open /datasets/<id>?startDate=…&endDate=…&groupBy=hour. Controls should show two tidy rows (dates, granularity and sources first, then MAAD), with no wrapped or stacked segmented controls.
  2. Scroll to MAAD Dimensions. You should see one line per source for Source D1, a y-axis fitted to the data, and day labels on the x-axis. Switch to Destination and D2: the chart redraws and the source colours stay the same.
  3. In the MAAD group, click Packets, then Bytes. The URL gains measure=…, the dimensions request carries measure=, and the card header updates. The spectrum card shows the "only computed for the Addresses measure" placeholder.
  4. Click IPv6 once. The URL gains ipVersion=6, and both MAAD cards refetch with ipVersion=6.
  5. Click Addresses. The spectrum chart and its source and side selectors come back.
  6. With measure=bytes, ipVersion=6 and groupBy=5min, click a point on any chart. The file page opens with Bytes and IPv6 selected. Click Next File: both stay selected.
  7. Open the dashboard with ?measure=flows: it falls back to Addresses. /netflow/files/<slug>?measure=flows returns 400.
  8. Open a dataset built with --no-maad. Controls has no MAAD group, and both MAAD cards show the short "MAAD was not computed" placeholder.
  9. Repeat step 2 in dark mode and at about 1024px wide. Sources wrap onto their own line and nothing overflows.

Decisions and edge cases

  • The measure and family live in the global Controls, not in the card headers. Both are URL state that every chart's drilldown carries, and both MAAD cards share them. The cards can also be reordered and end up far apart, so controls in the card headers would either appear twice or be stranded in one card. The card subtitles show the current selection instead.
  • The weighted view charts the stored scalar dimensions (D0/D1/D2), not tau(q) curves. They are the only MAAD result that is defined for every measure and can be summarised over time. Structure curves stay on the file page.
  • Sources past the eighth use a neutral grey instead of cycling hues. A source's colour depends on its position in the full source list, so turning a source off doesn't recolour the others.
  • The x-axis ticks on the IP, protocol, dimensions and spectrum cards now sit on bucket starts, so day boundaries get labels. When fewer than two buckets with data are in view (for example a single 5-minute bucket), they fall back to the default ticks.
  • "MAAD computed" is detected from maad_q_grid, which the pipeline writes only when MAAD runs. In a D1 deployment where one database is shared across datasets, this flag is database-wide, not per dataset.
  • An unknown measure in the dashboard URL falls back to Addresses, which matches how the other search params behave. The APIs and the file page reject it with 400.
  • Buckets whose dimensions are NULL (too few addresses) show as gaps, not zeros.
  • The e2e fixture's MAAD rows store full 33-value tau/tau_sd blobs, so they pass the blob validation added in #119.

Verification

  • Automated: bun run format, bun run lint, bun run typecheck, bun run test:web (181 tests), bun run test:db and bun run test:e2e (14 tests) all pass.
    • The web tests add tick-placement and dimension-key tests, plus the api-maad-dimensions, dataset page load, schema, file navigation and file page tests.
    • dashboard-maad-measure.spec.ts covers switching the measure, a single shared family control that refetches both cards, choosing a side and dimension, rejecting an invalid param, and drilling down to a file.
    • maad-ip-version.spec.ts now drives the shared family control, and checks that the file page's measure and family both survive Next File.
  • Manual: once the full reprocess lands, check the dimensions chart against a real reprocessed dataset. The screenshots come from a synthetic two-source database with hourly buckets.

UI Changes

Screenshots are pending upload. They'll be added here: a before image of the first iteration (before the redesign), and after images for Addresses, Bytes, a dataset without MAAD, dark mode and a 1024px width.


Made by Claude Opus 5.5 in Claude Code (T3 Code).

Checks

All checks passed

Added 4 commits

  • feat(web): switch dashboard MAAD views between addresses, packets and bytes5c4f726
  • refactor(web): redesign the dashboard MAAD controls and cards7ed07e3
  • docs: describe the MAAD controls group on the dashboarda2b210d
  • feat(web): serve stored MAAD dimensions over a dashboard ranged5b19d1