Sign up
View stargazersSign up

Cloud's worker log masks the redial refusal warning: message and Machine read [redacted]#3905

opened7d agobyNNick
GitHub writeback is off. Issues and comments you change here stay in GitCafe. Sync settings

Ployz Cloud's worker log masks the new redial-refusal warning from #3810, so the warning that should name the Machine reads [redacted] in both its message and its Machine field.

Build: frozen 0.3.17 candidate 99a623f800b349f71a37af96cb91dce2658b7a63 (manifest revision 99a623f8), local Cloud from dashboard/scripts/verify/up.sh, sdk: hit local 348922e9 (2bec842e426d).

Repro

  1. Boat --cloud run with 3 Servers; a Service with a slow healthcheck so a Deploy waits ~50 s.
  2. ployz deploy (Cloud runs it through ployz-sdk). Once the new replica is created, make the Deploy session's entry Server (MY-SERVER, machine-1) report another Cluster's Sealing Key on its next confirmation: stop ployz, put another Cluster's sealing.key in place, start ployz. (Lab fault: models an entry that now belongs to another Cluster, same Machine ID.)
  3. The Deploy stops within ~4 s, as #3810 says (good). Read the Cloud worker's stderr log.

Expected

#3810: "one warning names the Machine": the entry now belongs to another Cluster; the session ends with the Machine ID.

Actual

1{"at":"crates/ployz/src/cluster.rs:308","fields":{"machine_id":"[redacted]"},"level":"WARN","message":"[redacted]","target":"ployz::cluster",...}

ployz-store/src/stderr_log.rs (#3802, new since v0.3.16) keeps a message only if it is in MESSAGES and a field only if it is in TEXT. #3810's warning text is not in MESSAGES, and it logs machine_id, while TEXT lists machine. Both merged for this release, so only the at location survives.

The CLI's Deploy output still names the Machine on the ingress line (Couldn't check ingress: Machine … belongs to another Cluster (on Server machine-1)), so a user isn't blind. An operator reading Cloud's log is.

On v0.3.16

Not applicable: v0.3.16 has neither the stderr log (#3802) nor the redial check (#3810). The two new features don't line up.

F: 2 S: 2 F×S: 4

Blocks the 0.3.17 cut: no. It costs log diagnosability only, and the Deploy's own output names the Machine.