[Bug]: Remote environments stay switched off after a protocol mismatch, even once the host is updated#15051

opened25d agobyVvitalyiegorov

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

Not sure

Steps to reproduce

  1. Update the client to a release with a new orchestration protocol (here #2829, 0.0.46-nightly.20261003.2610). A saved T3 Connect host, switched on, is still on the old protocol. Its row shows Client not supported.
  2. Update the host to the same release.
  3. Look at Settings → Connections, with or without relaunching the client.

Expected behavior

The environment reconnects once its host is compatible. The user never switched it off.

Actual behavior

It stays Off (tooltip: Switched off), and its threads disappear until the user notices and turns it back on by hand.

Cause

setCompatibility in packages/client-runtime/src/connection/registry.ts writes an incompatible environment into the user's persisted switched-off list (registrations.setEnabled(id, false)). The reason (unsupportedReason) is kept only in memory. When a fresh check clears it, or after a relaunch, the entry is indistinguishable from one the user switched off, so nothing turns it back on.

Suggested fix

Treat compatibility as runtime state. Keep setting unsupportedReason and disconnecting, but stop changing enabled or the persisted list. Connect only when enabled && !unsupportedReason, and reconnect when the block clears. An environment the user switched off stays off. I can open the PR.

Impact

Major degradation or frequent failure

Version or commit

t3@0.0.46-nightly.20261003.2610 (main 8283b481a3)

Environment

macOS desktop client. Hosts: macOS (launchd) and Linux (systemd) services over T3 Connect.

Screenshots, recordings, or supporting files

Both hosts updated to the same nightly with relay tunnels registered. The client still shows them switched off:

Both saved environments stuck at T3 Connect · Off after their hosts were updated

Workaround

Turn each environment back on in Settings → Connections.

Related

  • #14979: same symptom, and its reporter confirmed every host was on the same build. Its triage scoped that issue to switch contrast (#14984), so this failure is untracked.
  • #11990 introduced persisting the rejection as switched off.

Note

Grok responding on behalf of Julius.

Triage

Thanks for the write-up, @vitalyiegorov. This is a real bug, and it's separate from #14979.

What's wrong: EnvironmentRegistry.setCompatibility records a protocol rejection as if the user had switched the environment off. When a saved environment is on, it sets enabled: false and calls registrations.setEnabled(id, false), which writes to the persisted switched-off list. unsupportedReason lives only in memory. When a later compatible health check calls setCompatibility(id, null), only the reason is cleared: enabled stays false and nothing reconnects. After a relaunch the reason is gone, so the row shows Off with the Switched off tooltip. That's the same thing @michft described on #14979 once both hosts were on the same nightly.

The merged #11990 added this to stop an incompatible server from sitting in a retry loop. Holding the connection off while the host is old makes sense, but it shouldn't become the saved state once the host is compatible again. The in-app update flow already recovers, because outdatedHostUpdate clears the block and then calls setEnabled(true). A host updated on its own (launchd, systemd, or a manual upgrade) never goes through that path, so its threads stay hidden until someone notices the switch.

Intended behavior: compatibility should be runtime state, not a saved setting.

  • While the protocol doesn't match, keep unsupportedReason, disconnect, and show Client not supported with the switch disabled. That should apply to both relay discovery and a socket preflight rejection.
  • Leave enabled and the persisted switched-off list untouched for that block, so an environment the user turned off stays off.
  • When a fresh check clears the block, reconnect if enabled is still true. Discovery should still ignore a replayed health result, so an older relay snapshot can't clear a newer socket rejection.
  • Only reconnect when the block is cleared, so a host that's still on an old build doesn't loop.

Fix scope: the packages/client-runtime registry, plus the registry tests that currently expect the opposite. One trap: setEnabled(true) returns early when enabled is already true. Once the block stops flipping enabled, setCompatibility(null) has to connect an enabled environment itself, or the in-app update flow stops resuming.

Rows that were already saved as switched off will stay off. They have no stored reason, so they can't be told apart from a real user toggle, and on those machines the workaround is still to switch each environment back on once.

#14979 covers the dark-mode switch contrast, which the open PR #14984 addresses. This issue covers environments that stay off after the host becomes compatible again. A maintainer will decide on the fix direction.

Synced from GitHub

Hit this today with fresh evidence, relay (T3 Connect) route, Android app 2.0.0.

  • 23:00: the phone connects fine to the server on 0.0.45 (orchestration protocol 1).
  • Overnight the app updates to 2.0.0.
  • 07:55: the phone mints a credential, gets a token and a WebSocket ticket, and re-fetches /.well-known/t3/environment. It then never opens /ws, and the environment is saved as switched off.
  • 08:06: the server is upgraded to 0.1.0 (protocol 2) by the systemd user service, outside the app's update flow.
  • 08:18: an app relaunch and pull-to-refresh produce successful relay health checks against the protocol-2 server, but the row stays Off and the phone never tries to connect.
  • The phone showed no "Client not supported" text at any point.

Relaunching wasn't enough on Android, because the switch on a saved row can't be tapped (#14958). That left no way to recover through the normal UI.

Posted from a t3 triage session (Claude Opus 5.5 in Claude Code).

Synced from GitHub

Additional observation from my Android 2.0 / macOS 0.1.0 Stable case (#20416): T3 Code notifications still arrive on the phone, including a notification that a thread needs input. When I open a notification, the app asks me to reconnect to the saved environment, but the reconnection does not complete, so I cannot open the notified thread. This describes the user-visible behavior; notification delivery alone does not establish that the app-to-host connection is healthy. I have not removed or re-paired the environment.

Synced from GitHub

Following up on my report above and #20416: this still persists with Android 2.0 and macOS 0.1.0 Stable, both updated. The direct environment descriptor reports 0.1.0 / protocol 2, while the saved T3 Connect entry still shows 0.0.45 and “Client not supported.” Is there a published build containing the recovery fix, or a specific diagnostic that would help with this already-stuck Android state? I see #15052 and #19251 are still open.

Synced from GitHub

Workaround for an Android app that is already stuck in this state: clear the T3 app's local storage (Android Settings → Apps → T3 Code → Storage → Clear storage), then sign in again. After that the environment came back and connected. Reconnect, toggling and "Refresh cloud environments" did nothing before.

Our case, matching the cause above (T3 Connect relay, Linux hosts):

  • 2026-10-17 13:03 UTC: Android app updates to 2.0.0 and tries a host still on server 0.0.45. The host's auth_sessions table gets a row (token issued) but no WebSocket connection follows. The environment shows Off with a toggle that does nothing.
  • 13:29 UTC: the host switches 0.0.45 → 0.1.0. Android never tries it again (no new auth_sessions rows), while a second host that was already on 0.1.0 connects fine from the same phone.
  • iOS showed the mirror image (app 1.4.1 against a 0.1.0 host). It recovered on its own after updating to 2.0.0, because that environment was never persisted as switched off on that device.

Clearing local storage presumably works because it wipes the persisted switched-off list that setCompatibility wrote to.