fix(server): thread search finds phrases inside quotes and their comments#22553
Problem
Thread search can't find a phrase inside a quote or its comment (https://git.cafe/pingdotgg/t3code/issues/22304). A quote is stored in the message text as an [Assistant quote](t3-citation://v1/…) link, and URLSearchParams encodes the quote and comment inside it: what are you talking about is stored as what+are+you+talking+about. Thread search matches that stored text, both through the message index and the fallback scan, so phrases with spaces never match, and accented, CJK and apostrophe words in a quote never match through the index. Single words do, including words from the quote's hidden prefix and suffix selection context, so a thread can show up for a word its reader never saw in it.
Change
- When
ProjectionStoreprojects a message, it now storessearch_textonorchestration_v2_projection_messages:assistantCitationsToPlainText(text), the helper the queued-message and stash previews already use. A quote becomes its text plusComment: …, without the link or its hidden context. The column isNULLwhen that equals the stored text, so plain messages aren't stored twice. Everymessage.updatedupsert rewrites it, so edits keep it current. - Migration 082 adds the column, recreates the message search triggers, and backfills existing history. The backfill pages through messages whose payload contains
t3-citation://, 500 at a time bymessage_id, and writessearch_textonly where the readable text differs; the update trigger re-indexes each message it changes. - The search triggers come from the shared
createMessageSearchTriggers, which now indexessearch_textonce the column exists. Migrations 069 and 074 are unchanged and, running before 082, keep indexing the stored text; a later migration that recreates the triggers through the helper keeps quote indexing instead of reverting it. Migration 082 recreates them through a newrecreateMessageSearchTriggers.rebuildMessageSearch(t3 doctor,migrate-dev-db) indexes the same text. - Thread search matches
COALESCE(search_text, text), the check for short words outside ASCII included, and builds the snippet from the same value, through onemessageSearchTexthelper, on the same reader-worker query. - Migration id: 082. Main ends at 080, and another open PR (#28751) also adds 081, so this one takes the next free id.
Scope and approval
Refs https://git.cafe/pingdotgg/t3code/issues/22304. This fixes its matching half. @EtanHey reported it, traced both halves and suggested a derived search-text column written at projection time; this builds that.
Snippets come from the matched text, so a quoted match also shows readable text instead of the encoded link.
Verification
Linux (Ubuntu 26.04), in apps/server:
- New
ThreadSearch.test.tscases project a quote with hidden prefix and suffix words, find its accented, CJK, apostrophe and short Cyrillic words through the index, and rebuild the index. Quote and comment phrases match and the hidden words don't. After a secondmessage.updateddrops the quote, the comment phrase stops matching. 082_ProjectionMessageSearchText.test.tsmigrates up to id 081 (main's set, which ends at 080), inserts a quoted message, a plain one, an invalid citation link, a non-string text, an accented quote and 501 quoted rows to cross a page, then runs only 082 and checkssearch_textand the index. A second case recreates the triggers throughrecreateMessageSearchTriggers, as a later migration would, and checks that a new quote is still indexed as it reads.- With main
f12bb6229c'sThreadSearch.ts, the two newThreadSearchcases fail (2 failed, 29 passed). On this branch,ThreadSearch, 082, 077, 069,Migrations,reconcileV2PreviewMigration,DatabaseMaintenanceandProjectionStorepass, 111 of 111. - With the trigger helper forced back to the stored text, both 082 cases and the index case in
ThreadSearchfail. vp run check:changed: format, lint, knip, typecheck and tests pass.
Real app, on every client: each run starts on main with an isolated state of test threads, where I selected a line of an agent reply (from the dev Replay provider), chose Cite, added the comment what are you talking about, moved the quote into a new thread with quiet silver lantern, and sent it. Then I stopped main and started this branch on the same state. It ran only the new search-text migration, which filled search_text for that quoted message. Searching for the comment finds nothing on main; after the upgrade it finds the quoting thread with a readable snippet.
Web (dev servers, Chromium):
A quote sent after the upgrade goes through the projection path and is found by its comment.
Desktop (Linux Electron under Xvfb, vp run dev:desktop of main and of this branch):
Mobile (T3 Code Dev client on an Android 16 emulator and an iOS 26.5 simulator, against main's and this branch's dev servers):
Not checked:
- Desktop on macOS and Windows. They run the same server and web UI.
- Quoted-phrase and hidden-context searches in the app: the Replay provider gives every thread the same reply, so those words also match the replies there. The unit tests cover both.
- Migration time on a large database. The backfill test crosses one 500-row page.
Prior PRs: https://git.cafe/pingdotgg/t3code/pulls/22391 fixed the snippet only and was closed as a duplicate of this PR. No open or closed PR changes the matching. https://git.cafe/pingdotgg/t3code/pulls/16858 (open) edits the same query for qualifiers and may need a small rebase.
Generated with Claude Opus 5.5 in T3 Code Nightly
Fixes pingdotgg/t3code#22304
Checks
Approvability
Verdict: Approved at 694931a
Macroscope's review found this PR approvable — This is a focused server bug fix that makes thread search use readable citation text while preserving existing behavior for ordinary messages. The additive migration, fallback logic, and targeted projection/replay tests keep the change bounded and backward-compatible.
You can add or adjust custom eligibility rules. Learn more.
Note
Reviews paused
It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.
- Last reviewed commit: 8e7ea08b318617730ea6c87437489dadd7c94fb9
- Latest commit, not reviewed: 2d999f791ee5ae1edb5ab02495db8be002d13e6b
Use the following commands to manage reviews:
@coderabbitai resumeto resume automatic reviews.@coderabbitai reviewto trigger a single review.
Use the checkboxes below for quick actions:
- Resume reviews
- Trigger review
No actionable comments were generated in the recent review.
Walkthrough
-
The change adds readable citation text to projected messages and search indexing. Thread search uses the shared message-text expression for scans, indexed queries, and substring filters. Migration 81 adds and backfills the search-text column and recreates search triggers.
Changes: Citation-aware thread search
Layer / File(s) Summary Add and index readable search text
apps/server/src/orchestration-v2/messageSearchIndex.ts,apps/server/src/persistence/Migrations/081_ProjectionMessageSearchText.ts,apps/server/src/persistence/Migrations/081_ProjectionMessageSearchText.test.ts,apps/server/src/persistence/Migrations.ts,apps/server/src/persistence/reconcileV2PreviewMigration.test.tsMigration 81 adds search_text, backfills readable text for eligible citation messages, and recreates search triggers. Indexing and rebuilds use the selected text, with a stored-text fallback when the column is absent. Migration registration and tests cover the backfill and index behavior.Maintain search text on projected messages
apps/server/src/orchestration-v2/ProjectionStore.tsMessage inserts and conflict updates write search_textderived from message text.Search readable citation text
apps/server/src/orchestration-v2/ThreadSearch.ts,apps/server/src/orchestration-v2/ThreadSearch.test.tsScanned and indexed queries and substring filters use messageSearchText(sql). Tests cover citation quote and comment matching, hidden citation context, apostrophe variants, accented and CJK text, index rebuilding, and message updates.Estimated code review effort: 3 (Moderate) | ~20 minutes
Change: Bug fix · Severity of issue fixed: Medium
Sequence Diagram(s)
mermaid1 sequenceDiagram 2 participant ProjectionStore 3 participant ProjectionMessages 4 participant MessageSearchIndex 5 participant ThreadSearch 6 ProjectionStore->>ProjectionMessages: Write payload and readable search_text 7 ProjectionMessages->>MessageSearchIndex: Trigger index update 8 ThreadSearch->>MessageSearchIndex: Resolve messageSearchText(sql) 9 MessageSearchIndex-->>ThreadSearch: Return schema-aware text expression 10 ThreadSearch->>ProjectionMessages: Query matching message text Possibly related PRs
- pingdotgg/t3code#24343: Adds the indexed message-search paths that this change extends to use readable citation text.
Suggested reviewers:
maria-rcks,juliusmarminge
Merge Risk: External image Minimal · up to 8e7ea
-
Thread search now matches quoted text and quote comments. No concrete merge-blocking issue was found. The migration backfill time on very large databases was not measured.
Architecture risk: External image Low · up to bb269
-
The change affects 1 system.
Changed systems:
apps/serverArchitecture concerns No architecture-level concerns identified.
Review details
-
Systems and components
- observed — apps/server (service) was modified; 7 changed files map to changed impact.
Before / after behavior
- observed — Modified behavior in apps/server/src/orchestration-v2/ProjectionStore.ts: Added the
assistantCitationsToPlainTextimport used to derive searchable message text. - observed — Modified behavior in apps/server/src/orchestration-v2/ProjectionStore.ts:
message.updatednow derives citation-stripped plain text from the message text for search indexing. - observed — Modified behavior in apps/server/src/orchestration-v2/ProjectionStore.ts: The projected messages insert now includes the
search_textcolumn. - observed — Modified behavior in apps/server/src/orchestration-v2/ProjectionStore.ts: Message inserts store normalized citation text, using
NULLwhen it matches the original message text.
-
Pre-merge checks: Passed 4
-
Check name Status Explanation Title check Passed The title clearly describes the main change: thread search can find phrases inside quotes and comments. Linked Issues check Passed Issue #22304requires thread search to match readable quote text and comments, and to show readable snippets.ProjectionStorewritessearch_textfromassistantCitationsToPlainTextand refreshes …Out of Scope Changes check Passed The changes support issue #22304. The migration, trigger updates, index rebuild logic, projection writes, and tests implement or maintain searchable readable quote text and snippets. No unrelated chan…Description check Passed The description includes the required Problem, Change, Scope and approval, and Verification sections. It links the issue, explains the implementation, reports focused tests and app checks, and lists c… Full details: Linked Issues check
-
Explanation
Issue
#22304requires thread search to match readable quote text and comments, and to show readable snippets.ProjectionStorewritessearch_textfromassistantCitationsToPlainTextand refreshes it on message upserts. Migration 081 adds and backfills the column, then recreates the search triggers.ThreadSearchusesCOALESCE(search_text, text)for indexed and fallback matching, including short-word checks. The added tests cover quote and comment phrases, hidden prefix and suffix context, non-ASCII words, index rebuilds, and edits that remove a quote. The PR summary reports readable snippets from the same search text.
Full details: Out of Scope Changes check
-
Explanation
The changes support issue
#22304. The migration, trigger updates, index rebuild logic, projection writes, and tests implement or maintain searchable readable quote text and snippets. No unrelated change is evident in the reviewed diff or change summary.
Full details: Description check
-
Explanation
The description includes the required Problem, Change, Scope and approval, and Verification sections. It links the issue, explains the implementation, reports focused tests and app checks, and lists checks that were not performed. The description says the migration is 082, while the supplied change summary identifies it as 081; align the migration number for accuracy.
-
Finishing Touches
-
Generate unit tests (beta)
-
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
External image Recent review info
-
External image Run configuration
-
- Source mode: Auto
- Repository: pingdotgg/t3code
- Branch: main
- File: .coderabbit.config.ts
- Commit: f12bb62
- Layers: Repository TS config
- Review profile: CHILL
- Plan: Advanced
- Run ID:
918cc1a3-f4f6-42c7-9005-15d2c5085a69
External image Commits
-
Reviewing files that changed from the base of the PR and between 7c15aaf30eafd1d7542a56c2bd07b2794ebbfadf and 8e7ea08b318617730ea6c87437489dadd7c94fb9.
External image Files selected for processing: 8
-
apps/server/src/orchestration-v2/ProjectionStore.tsapps/server/src/orchestration-v2/ThreadSearch.test.tsapps/server/src/orchestration-v2/ThreadSearch.tsapps/server/src/orchestration-v2/messageSearchIndex.tsapps/server/src/persistence/Migrations.tsapps/server/src/persistence/Migrations/081_ProjectionMessageSearchText.test.tsapps/server/src/persistence/Migrations/081_ProjectionMessageSearchText.tsapps/server/src/persistence/reconcileV2PreviewMigration.test.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.
External image Recent review info
-
External image Run configuration
-
- Source mode: Auto
- Repository: pingdotgg/t3code
- Branch: main
- File: .coderabbit.config.ts
- Commit: f12bb62
- Layers: Repository TS config
- Review profile: CHILL
- Plan: Advanced
- Run ID:
918cc1a3-f4f6-42c7-9005-15d2c5085a69
External image Commits
-
Reviewing files that changed from the base of the PR and between 7c15aaf30eafd1d7542a56c2bd07b2794ebbfadf and 8e7ea08b318617730ea6c87437489dadd7c94fb9.
External image Files selected for processing: 8
-
apps/server/src/orchestration-v2/ProjectionStore.tsapps/server/src/orchestration-v2/ThreadSearch.test.tsapps/server/src/orchestration-v2/ThreadSearch.tsapps/server/src/orchestration-v2/messageSearchIndex.tsapps/server/src/persistence/Migrations.tsapps/server/src/persistence/Migrations/081_ProjectionMessageSearchText.test.tsapps/server/src/persistence/Migrations/081_ProjectionMessageSearchText.tsapps/server/src/persistence/reconcileV2PreviewMigration.test.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.
-
Added 4 commits
Note
🤖 Opus 5.5 replying on behalf of Victor.
I merged main into this branch in 8e7ea08b31 to clear the conflicts in Migrations.ts, ProjectionStore.ts and the V2 preview migration test.
- Main now has migrations 077 to 080, so the search text migration is now 081. Its test migrates to 080 first and then runs only 081.
- The new check for short words outside ASCII (#30332) now reads the same readable text as the other search checks, so a short word inside a quote or its comment matches too. A new row in the quote test covers this.
- I dropped the test that replayed projections from events. Main no longer replays projections outside its test oracle.
I updated the description to match.
Added 1 commit
- fix(server): renumber the message search text migration to 0822d999f7
Note
🤖 Opus 5.5 replying on behalf of Victor.
The search text migration is now 082 instead of 081, because another open PR (#28751) also adds a migration 081. Main still ends at 080. Commit 2d999f791e renames the migration and its test and updates the manifest row, the expected migration history and the comments that name the number. The migration itself is unchanged. I updated the description to match.