A Move's commit-and-start times out at 10 s while the target still settles health#4127
A Move's commit-and-start on the target is one SetVolumeCopy (writer, start: create) that Cloud sends through the SDK's volume_switch. Its deadline is TARGET_RPC_TIMEOUT, 10 s (core/crates/ployz/src/sdk.rs, volume_switch). On the Machine, the same request can wait much longer than that by design. When Cloud routes the copy In, route_created → ensure_container → route_in → settle_health waits until the Service is healthy, for up to settle_deadline(spec). For a configured healthcheck, that is start_period + retries × (interval + timeout) + 5 s (healthcheck_deadline), with Docker's 30 s interval when none is set. The first healthy result comes no sooner than one interval after the start.
So for any Service with a healthcheck whose first pass is more than a few seconds after start, Cloud's call times out on a healthy Machine every time (set_volume_copy on <server>: unavailable: target Machine RPC timed out). The Move still finishes, because settle polls until it sees the Service running. But the downtime it reports becomes an estimate (was stopped about …s (its start was retried)), and the polling adds to the real downtime.
Measured (3-Machine verify lab, Postgres with its healthcheck disabled, so the health wait is 1 s): the commit-and-start RPC took 2.7 to 6.5 s over four Moves on quiet Machines. In two noisier windows, 7 of 8 Moves on both main and the E4 branch timed out and retried. There, the target's promote alone took 3.5 to 4.3 s before Docker created the Container (0.5 s when quiet). A healthchecked Service adds at least one check interval on top.
Scores: F 5: every Move of a Service with a healthcheck (databases often have one), plus a slow disk for any Service. S 4: the Move completes, but the downtime is longer and only estimated. F×S = 20.
Fix direction: give the commit-and-start call a deadline that covers the Machine's own settle budget (settle_deadline(spec) + SETTLE_INSPECT_GRACE plus the promote), or answer once the Container starts and settle health separately.
Found while measuring Move downtime for #4118. Not changed there.