Sign up
View stargazers0Sign up
Commits

docs/setup.md

Setup

Prerequisites

  • Bun 1.2+
  • Python 3.x
  • nfdump available on your PATH (or via Nix — see shell.nix)
  • SSH access to the research host if using live ONRG data

For MAAD-Backed Address Structure Stats

Only needed if you want MAAD-backed address structure stats:

  • a C++ compiler for tools/netflow-db/maad_fast.cpp
  • or nix-shell, then use scripts/build_maad_fast.sh

Install

bash
1git clone https://github.com/flamboh/atlantis.git
2cd atlantis
3bun install

Configure Datasets

Copy the example config and edit paths for your machine:

bash
1cp datasets.json.example datasets.json

root_path should point at the directory containing router/source subdirectories.

See datasets-json.md for more.

Environment

Start from the template:

bash
1cp .env.example .env

At minimum, make sure DEFAULT_DATASET matches one of the dataset ids in datasets.json.

Optional overrides: DATASETS_CONFIG_PATH, MAX_WORKERS, AGGREGATE_MAAD_MAX_WORKERS.

Populate the Database

bash
1python tools/netflow-db/pipeline.py --dataset uoregon --start-date 2025-02-11

This discovers and processes NetFlow files into SQLite. See pipeline-usage.md for flags and scheduling patterns.

D1 Migrations

The web app's D1 schema is tracked in apps/web/drizzle. Generate migrations from the Drizzle schema:

bash
1bun run --cwd apps/web db:generate

Wrangler is configured to read the same directory:

bash
1bun run --cwd apps/web d1:migrations:list
2bun run --cwd apps/web d1:migrations:apply:local

Current stack policy is greenfield: before a shared D1 database has applied these files, rebaseline apps/web/drizzle when the schema changes. After a D1 database has applied a migration, add a new migration instead of editing the applied file.

Optional: Compile MAAD Helper

If you need MAAD-backed address structure stats for real captures, build the fast MAAD helper:

bash
1git submodule update --init --recursive
2./scripts/build_maad_fast.sh

This helper is not required for the no-data setup verification flow above.

Run the App

bash
1bun run dev

The web app starts at http://localhost:5173.

SSH Tunnel (remote host)

bash
1ssh -L 5173:localhost:5173 user@pinot

Then open http://localhost:5173 locally.