Sign up
View stargazersSign up

fix(server): agents show trends and comparisons as inline charts again#25769

merged2
main
fix/proactive-inline-visuals
opened11d agobyTtheo
3 files changed
View on GitHubTwo-way sync enabledPR last imported 11d ago

Agents in T3 Code stopped showing data as inline charts. #24850 changed the visual rule from "when a chart would say more than prose, render it" to "when asked to show a visual", so agents now answer data questions in prose.

The rule now asks for a chart or diagram for trends, comparisons, and flows, even unasked, and html_render's description names its uses (chart, diagram, mockup) again.

Eval

Two new MCP tool-use eval cases, added here, paste data (latency per release, CI job times) and ask a question without asking for a visual. Pass means html_preview then html_render.

ModelPre-#24850 textmainThis PR
Claude Opus 5.50 of 88 of 8
GPT-6.1 Sol4 of 80 of 84 of 4, and all 66 visual and control samples pass
Claude Sonnet 5.50 of 80 of 80 of 4

All code-task controls, which forbid html_render, still pass on every model. Known cost, shipped as a v0: Sonnet failed two preview cases it passes on main (an unasked preview_snapshot after an edit, and leaving its own preview tab open). Its other visual and control cases pass.

Created with Claude Opus 5.5 in Claude Code.

🤖 Generated with Claude Code

Checks

22 successful · 15 skipped · 1 neutral
All checks passed
f63a9a7

Added 2 commits

  • fix(server): agents show trends and comparisons as inline charts again212e9c4
  • test(evals): data questions where a chart says more than prosef63a9a7
CcoderabbitaiBot11d ago
Synced from GitHub

Review in Change Stack →

No actionable comments were generated in the recent review.

External image Recent review info
External image Run configuration
External image Commits

Reviewing files that changed from the base of the PR and between 5d91ab1117ad51ef1288a94717d85c70e7af09fe and f63a9a7bc21a98131ac8cfd01b4f4ef298c935fe.

External image Files selected for processing: 3
  • apps/server/src/mcp/toolkits/html/tools.ts
  • evals/mcp-tool-use/cases/visual-preview.jsonl
  • packages/provider-core/src/server/orchestrationInstructions.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

The orchestration instructions now call for charts or diagrams for trends, comparisons, and flows, and use the HTML preview and render flow for visuals requested here. The render tool description and visual-preview evaluation cases also reflect this guidance.

Changes: Visual guidance

Layer / File(s)Summary
Visual output guidance
packages/provider-core/src/server/orchestrationInstructions.ts, apps/server/src/mcp/toolkits/html/tools.ts, evals/mcp-tool-use/cases/visual-preview.jsonl		The instructions call for charts or diagrams for trends, comparisons, and flows, including when not requested. The render tool description names charts, diagrams, and mockups. Two evaluation cases expect html_preview followed by html_render and prohibit preview browser navigation.

Priority: External image Normal

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Possibly related PRs

Suggested labels: size:S

Suggested reviewers: maria-rcks, juliusmarminge

Merge Risk: External image Minimal · up to f63a9

No actionable merge-blocking risk is identified.

Pre-merge checks: Passed 4
Check nameStatusExplanation
Title checkPassedThe title clearly summarizes the primary change: restoring inline charts for trends and comparisons.
Description checkPassedThe description explains the problem, the implementation, and focused evaluation results. It omits a dedicated Scope and approval section, but the overall description is mostly complete and directly r…
Linked Issues checkPassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes checkPassedCheck skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description explains the problem, the implementation, and focused evaluation results. It omits a dedicated Scope and approval section, but the overall description is mostly complete and directly related to the change.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

MmacroscopeappBot11d ago
Synced from GitHub

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This changes the default agent behavior so trends, comparisons, and flows may trigger inline chart rendering without an explicit request, affecting existing request paths and tool usage. The product-default change therefore warrants human review despite the small diff.

Notes:

  • No code objects were reviewed. Approvability was decided on eligibility alone.

You can add or adjust custom eligibility rules. Learn more.

Ggithub-actionsBot11d ago
Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire5.0 KiB5.0 KiB−7 B (−0.1%)6.8 KiB✅
CodexThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
CodexLive turn WebSocket wire1.2 KiB1.2 KiB−7 B (−0.6%)2.0 KiB✅
CodexLive turn WebSocket decoded20.9 KiB20.9 KiB0 B (0.0%)29.3 KiB✅
CodexLive turn messages220 (0.0%)8✅
ClaudeTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
ClaudeThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
ClaudeLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
ClaudeLive turn WebSocket decoded21.2 KiB21.2 KiB0 B (0.0%)29.3 KiB✅
ClaudeLive turn messages220 (0.0%)8✅

Baseline: 5d91ab1 · PR result: f63a9a7 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.