Antigravity ACP (Gemini 4 Argon): Streaming text gets stuck in running/streaming state & thoughts are never streamed#28172

opened12d agobyMmahmoodnizamani94-png

Summary

When using Antigravity ACP with Gemini 4 Argon (gemini-4-argon / gemini-4-argon-high via Google OAuth / CCPA), assistant text in the T3 Code UI frequently freezes mid-turn for 15–45 seconds or remains stuck in streaming: true (status: "running") for several minutes (6–14+ minutes observed in statev2.sqlite orchestration_v2_projection_turn_items) after the assistant has finished writing its response. Additionally, reasoning (agent_thought_chunk) items are never emitted (0 reasoning items across 100+ gemini-4-argon turns).

Observed Evidence (statev2.sqlite & Provider Event Logs)

From orchestration_v2_projection_turn_items in statev2.sqlite:

  • assistant_message items remain in status = 'running' (streaming: true) long after the final agent_message_chunk has been received:
    • Example 1: started_at = 2026-10-31T12:54:39.643Z, completed_at = 2026-10-31T13:08:51.339Z (stuck in streaming: true for 14m 12s until the next user prompt).
    • Example 2: started_at = 2026-10-31T14:13:17.603Z, completed_at = 2026-10-31T14:19:48.160Z (stuck in streaming: true for 6m 31s after the 13,813-char response completed at 14:13:18Z).
    • Even within normal multi-step turns, assistant_message items stay in streaming: true (status: "running") for 15–45 seconds after text generation finishes while the model silently thinks on the next step.
  • Across 100+ gemini-4-argon turns, SELECT count(*) FROM orchestration_v2_projection_turn_items WHERE kind = 'reasoning' is 0.

Root Causes (Interaction Between T3 Code ACP Adapter & Antigravity ACP Server)

1. T3 Code ACP Adapter (makeAcpAdapterV2) keeps context.assistant.current open in streaming: true

In makeAcpAdapterV2 (handleSessionUpdate / appendText / closeTextStream):

  • Receiving an agent_message_chunk calls appendText(context, "assistant", text, update.messageId), which opens context.assistant.current with streaming: true (status: "running") and throttles flushes via ACP_TEXT_FLUSH_INTERVAL (50ms).
  • closeTextStream(context, "assistant") is only called when:
    1. Another text stream (reasoning or user) starts,
    2. emitTool fires, or
    3. The entire session/prompt RPC resolves (turn.completed).
  • Because gemini-4-argon on OAuth/CCPA never emits agent_thought_chunk (reasoning) between assistant text and subsequent thinking phases (see #2 below), and because session/prompt remains open whenever the Antigravity trajectory is in STATE_WAITING_FOR_TASKS / STATE_WAITING_FOR_SUBAGENTS (see #4 below), context.assistant.current never receives closeTextStream(context, "assistant") and stays stuck in streaming: true (status: "running") in the UI.
  • Furthermore, in the web client projection (timelineVisibleTurnItemsAtom / quietThreadAtom), streaming assistant items (streaming: true) have their text and updatedAt stripped from the base item in quietThreadAtom and rely on assistantMessageTextAtom while the streaming indicator remains active.

2. Antigravity ACP (agy_acp_server.par + localharness_external) strips ThinkingConfig (includeThoughts: true) on OAuth/CCPA

  • In google3/cloud/developer_experience/antigravity_extensions/acp_server/server.py:
    • _build_mapped_model_target (used for API key / Vertex) populates endpoint.options = GeminiModelOptions(thinking_level=thinking_level).
    • _build_literal_model_target (server.py:4025, used for AUTH_METHOD_OAUTH and AUTH_METHOD_OAUTH_BUSINESS at lines 4572 & 4588) constructs ModelTarget without setting endpoint.options (options=None).
  • In localharness_external (google3/third_party/jetski_prod/localharness/geminimodelapi.(*Client).Generate):
    • BuildGenerateContentRequest initially sets genConfig.ThinkingConfig = &ThinkingConfig{IncludeThoughts: true, ...}.
    • Immediately afterward, geminimodelapi.(*Client).Generate checks if len(c.thinkingLevel) == 0 (populated from GeminiModelOptions.GetThinkingLevel()). When c.thinkingLevel == "", it overwrites genConfig.ThinkingConfig = nil!
  • As a result, requests to http://127.0.0.1:<port>/v1beta/models/gemini-4-argon-high:streamGenerateContent are sent to ccpa_connection/proxy_server.py without thinkingConfig: {"includeThoughts": true}. CCPA still runs Argon with a HIGH thinking budget server-side (taking 10–45s per step), but streams zero thought: true chunks back over SSE, making the UI appear completely frozen during thinking phases and preventing makeAcpAdapterV2 from closing the previous assistant text stream when a new thinking step starts.

3. Antigravity ACP _stream_live_step (server.py:5856-5867) drops final content_delta / thinking_delta on StepStatus.DONE

In server.py:

python
1if step.status == sdk_types_module.StepStatus.DONE:
2 for tool_call in step.tool_calls or []:
3 if tool_call.id:
4 await self._register_live_step_tool_call(session_id, tool_call)
5if step.status != sdk_types_module.StepStatus.ACTIVE:
6 return
7if step.thinking_delta:
8 ...
9if step.content_delta:
10 ...

In localharness_external ((*StepToEventTranslator).translateStepLocked), text_delta and thinking_delta are computed on every StepUpdate, including the transition to CORTEX_STEP_STATUS_DONE. Returning early on step.status != StepStatus.ACTIVE before flushing step.thinking_delta and step.content_delta silently drops any final text/thought delta delivered on the DONE event.

4. Antigravity ACP LocalHarnessEventProcessor (event_processor.py:958-982) hangs session/prompt when background/orphaned tasks exist (STATE_WAITING_FOR_TASKS)

In localharness_external ((*runningTrajectory).computeCurrentStateLocked), if any background task is active (hasActiveTasks()), the trajectory transitions to STATE_WAITING_FOR_TASKS (4) rather than STATE_FULLY_IDLE (2) when the planner finishes its turn. Because LocalHarnessEventProcessor.process_event only sets self.is_idle.set() on STATE_FULLY_IDLE and STATE_CANCELLED, session/prompt never returns, leaving the last assistant_message in T3 Code stuck in streaming: true (status: "running") for minutes until the user manually interrupts or sends another prompt.

Suggested Fixes

  1. In T3 Code (makeAcpAdapterV2):
    • Flush and close (or transition out of active streaming: true after an idle/step boundary) context.assistant.current when a planner step finishes or when tool_call / tool_call_update / plan updates arrive, rather than waiting exclusively for reasoning chunks or session/prompt completion.
  2. In Antigravity ACP (agy_acp_server.par):
    • Populate GeminiModelOptions(thinking_level=...) in _build_literal_model_target (or ensure thinkingConfig.includeThoughts = True in ccpa_connection/proxy_server.py for Gemini thinking models).
    • Flush step.thinking_delta and step.content_delta in _stream_live_step on both StepStatus.ACTIVE and StepStatus.DONE (before registering DONE tool calls).
Synced from GitHub

Nightly 0.1.9-nightly.20261110.4491 still shows this on a long Antigravity 1.4.5 (gemini-4-carbon) thread. The merged 15-minute inactivity timeout (#30377, for #30271) does not settle it.

What happens: the session log's newest step for the prompt is a finished planner reply (step_type 15, status 3), with no non-terminal step after it. session/prompt stays started. The assistant row stays streaming=1. The prompt later becomes failed / AcpProcessExitedError only when a follow-up user message interrupts the hung process (~15s later). The provider turn is interrupted, not the inactivity failure (sent nothing for 15 minutes is absent). The same orchestration run stays running because that follow-up is attempt N+1 of the same run.

Observed on thread 632a763d-d37a-4b7a-926c-a6f25f6195a6, session fe72880c-976e-4cd2-8fc1-6881ab137d5b (times Taipei, 2026-11-11):

user steplast stepplanner finishedT3 closed the turn
idx 427 at 11:00:19idx 449 type 15 status 311:03:4511:17:52 interrupted
idx 450 at 11:18:12idx 462 type 15 status 311:21:1811:33:06 interrupted
idx 463 at 11:33:38idx 501 type 15 status 311:41:1611:49:01 interrupted

A short control thread on the same nightly, same official 1.4.5 binary, and the same stdio wrapper does settle: bd15b956-ca58-48ce-9bbe-332e8a3c5879 turn 1 ends type 15 status 3 at 11:04:38, and session/prompt is succeeded in that same second, while its delegated child is still running. So a live child is not why prompt() stays open. v2 prompt() discards the RPC body and waits for state_update / idle (effect-acp completePrompt). finalizeTurn / closeTextStreams run only after that. watchAntigravityPromptOutcome runs only after prompt() returns, and only looks for a type 17 error step. Antigravity does not set deferFinalizeForBackgroundWork or followAgentWorkState.

#30377 fails a silent prompt after 15 minutes. These answers were already stored, and a person sent the next message before that deadline, so the turn was interrupted instead of completed.

Suggested settle, raced with the real idle so a prompt that does return is unchanged: if every step since the pre-prompt index is terminal (status 3 or 7) and the newest is type 15 status 3 for 5 unchanged polls, finalizeTurn(context, "completed"), then session/cancel, and if promptWireSettled is still pending after 5s, close the runtime. catchCause already returns when context.finalized is set, so the later process-exit error does not rewrite the completed turn.

I tested that predicate, not a patched nightly (patching the running app would restart live threads). Synthetic cases: a finished planner tail settles; an in-progress planner step, a pending command, a tail that is still a tool, and an error tail do not. Five identical polls settle; a new idx or an open tail resets the count. On a copy of the session log above, the three stuck tails match, and cutting off the final planner step makes each one not match. Both control-thread tails match too, which is what we want: idle already won in the same second, so the watcher would see context.finalized and stop.

One limit: if a background run_command is stored as a terminal step while STATE_WAITING_FOR_TASKS still holds session/prompt (#28898), this would complete that turn after 5s. This session had no non-terminal step for many minutes after the planner reply, and the control delegate still receives idle immediately.