Tidy the Caddy watcher's ingress-role read, local-ingress filter and apply-result handling#2774

opened23d agobyNNick
GitHub writeback is off. Issues and comments you change here stay in GitCafe. Sync settings

Three refactors in the Caddy watcher and status code that #2681 added. A review raised them, and none changes behaviour. They are deferred so the verified head of #2681 stays unchanged.

1. Read the ingress role from the local record

watch_caddy (core/crates/ployzd/src/ingress.rs) reads the Machine's own accepts_ingress from its replicated row on every pass (replicated.machine(machine.id)). Elsewhere the daemon reads the local record: the stranded-ingress restart in docker/observe.rs uses sink.local.record().machine().accepts_ingress. The watcher also still holds the Machine snapshot from daemon start, for the id and IngressProjection.machine.

Today this fails safe. A row that lags or is missing counts as taking ingress, so the record claims nothing and clears on the next 5 s pass. Each pass also costs one extra Corrosion query.

Suggested:

  • Pass the watch::Receiver<Arc<LocalMachineRecord>> (records, in scope in daemon.rs where ingress::run is spawned) through ingress::run and caddy::run to watch_caddy.
  • Read the role from it.
  • Add records.changed() to wait_for_debounced_change.
  • Drop the snapshot argument.
  • Tests then set the record's role instead of answering SELECT info FROM machines in the fake store.

2. One summary of the local ingress containers

The filter "local and system ingress" exists in three places:

  • serves_nothing in ingress.rs
  • local_ingress/newest_local_ingress in ingress.rs
  • stranded_ingress in docker/observe.rs

Each watcher pass runs it twice. Replace these with one helper, for example LocalIngress { newest, at_rest, any }, used by both the watcher and stranded_ingress. That way "the newest" and "all of them" can't drift apart again.

3. Classify an apply result once

Whether Caddy serves this input, the earlier load, or something unknown after a pass is decided in three places:

  • apply_caddy maps its error to BeforeServing/AfterServing by whether the admin is present.
  • The last_input match pairs the result with admin.is_some().
  • The publish step tests through_admin against the result again.

Use one enum, for example Served { This, Earlier, Unknown }, built from (loaded, admin, result), and drive both last_input and the published record from it.

Also

Name the routes type, for example pub type ServiceRoutes = BTreeMap<QualifiedService, BTreeMap<ContainerId, ContainerAddress>> in ployz-core/src/rpc.rs. It is written out in full in four places: status_servers.rs stale_addresses, ingress.rs (twice) and rpc.rs.