Sign up
View stargazersSign up

Add Boat sandboxes as a verify-cluster provider#1876

merged10
boat/1-ssh-destination
boat/2-provider
openedOct 14, 2026byNNick
7 files changed
View on GitHubGitHub import enabledPR last imported 21d ago

scripts/verify-cluster up --provider boat --machines N runs each Machine as a small Boat sandbox instead of an Incus VM. Agents get disposable Machines that cost no RAM or disk on the shared host, and Boat runs skip /tmp/ployz-warm-move-vc.lock. Incus stays the default and is unchanged.

  • core/scripts/verify/boat.py drives the boat CLI for create, info, stop and delete, and uses muxed root SSH for exec and push.
  • A per-run registry at ~/.cache/ployz/verify/boat/<run>.json is the only thing ls, reap and down act on.
  • up refuses a run that would exceed the account's active-sandbox headroom or its minute, hour or day start budget, counting placement relaunches.
  • A sandbox placed on a node another Machine of the run already uses is relaunched, since peers sharing one on-link /80 never complete a WireGuard handshake over IPv6.
  • Clone restores are settled before ssh restarts, and stray sshd listeners from the clone are cleared.
  • The move suites take --provider. The verify-server skill documents Boat.

Proof: three-Machine up, doctor and down on real sandboxes, plus the warm-move suite on Boat. Results are in the trail .audit/boat-verify.tsv.

Checks

1 pending · 15 successful · 2 skipped
Checks in progress

In progress1

1be3b1e
CcoderabbitaiBotOct 14, 2026
Synced from GitHub

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ed08e9c8-1d9f-4432-8a4e-f2b4819c761c
📥 Commits

Reviewing files that changed from the base of the PR and between c6ae943654c0f156b7843010c249d52bd8fbb743 and 1be3b1e165220a23e5415a4bb387fab154d531dc.

📒 Files selected for processing: 7
  • .agents/skills/verify-server/SKILL.md
  • core/scripts/test-verify-volume-fences.py
  • core/scripts/test-verify-volume-move.py
  • core/scripts/test-verify-warm-move.py
  • core/scripts/verify-cluster
  • core/scripts/verify/boat.py
  • core/scripts/verify/test_boat.py
ascii
1 ____________________________________________
2< Rabbit in the repo, here to fix that bug 🎶 >
3 --------------------------------------------
4 \
5 \ \
6 \ /\
7 ( )
8 .( o ).
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Added 10 commits

  • Budget only first Boat launches against the start minute1be3b1e
  • Relaunch a Boat sandbox that lands on a node another Machine uses20a7fc5
  • Budget every Boat placement relaunch against the hourly and daily starts2fb849d
  • Keep a Boat run going when another provider, run or delete fails3253bcf
  • Wire the Boat provider into verify-cluster and the move suites36a58dc
  • Add a Boat sandbox provider for verify-cluster56aa2bd
  • Clear a cloned sandbox's stray sshd listener before ssh restarts64597b6
  • Wait for a resumed Boat clone's systemd jobs before restarting ssh886cdbd
  • Document --provider boat for verify-server and the volume scripts98866e8
  • Retry a Boat root key install whose command fails inside the sandboxd83bece
Cchatgpt-codex-connectorBotOct 14, 2026
Synced from GitHub

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

ReviewStatusCommitReview trigger
📝 Code Review✅ Completed 2026-10-14T06:55:06.966742Z1be3b1eDraft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Cchatgpt-codex-connectorBotreviewed · 1 inline threadOct 14, 2026
Synced from GitHub

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1be3b1e165

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

core/scripts/verify/boat.pylines 290–292Open thread
……@@ -0,0 +1,376 @@
289+
290+ def state(self, node):
291+ if node["name"] in self.run.get("powered_off", []):
292+ return "STOPPED"
Cchatgpt-codex-connectorBoton lines 290–292Oct 14, 2026

P2 Badge Reject exec while a Boat Machine is powered off

After Boat power off, the sandbox reboots with SSH still running, and this method merely reports STOPPED from the manifest flag. Because the exec action invokes backend.exec() before consulting state(), a command such as verify-cluster exec <manifest> machine-1 -- true succeeds while the Machine is declared powered off, allowing verification scripts to inspect or mutate a supposedly unavailable Machine and contradicting the documented Incus-compatible behavior. Gate user-facing execs for deliberately powered-off Boat Machines while retaining an internal path for power_on.

Useful? React with 👍 / 👎.