| … | … | @@ -0,0 +1,1247 @@ | |
| 613 | + | } | |
| 614 | + | // Written together as typed: the snippet starts there. Apostrophes are | |
| 615 | + | // straight in the query, and straightening keeps offsets. | |
| 616 | + | const together = phrase?.exec(plain.replace(CURLY_APOSTROPHES, "'")); | |
🟡 Medium src/paletteSearch.ts:616
messageMatcher("cafe open")("Café open") receives only the proximity bonus (1.1), while the unaccented phrase receives the phrase bonus (1.25), so accented messages rank lower despite matching the same words and adjacency. messagePhrase is built from the unfurled query and executed against plain; fold both sides consistently before detecting the phrase.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
| 1 | In file @packages/shared/src/paletteSearch.ts around line 616: |
| 2 | |
| 3 | `messageMatcher("cafe open")("Café open")` receives only the proximity bonus (`1.1`), while the unaccented phrase receives the phrase bonus (`1.25`), so accented messages rank lower despite matching the same words and adjacency. `messagePhrase` is built from the unfurled query and executed against `plain`; fold both sides consistently before detecting the phrase. |
Note
🤖 Claude Opus 5.5 responding on behalf of Theo
Fixed in 2825166b62. When the plain phrase test fails and either the query or the message has non-ASCII text, messageMatcher now runs the phrase on the accent-folded text too. "Le Café open late" for cafe open and "The cafe open late" for café open both score 1.25 (new cases in paletteSearch.test.ts).
| … | … | @@ -26,45 +54,208 @@ export class ThreadSearchError extends Schema.TaggedError<ThreadSearchError>()( | |
| 219 | + | AND messages.message_id NOT LIKE 'async-answer:%' | |
| 220 | + | AND NOT EXISTS ( | |
| 221 | + | SELECT 1 FROM json_each(${substrings}) AS token | |
| 222 | + | WHERE instr(lower(json_extract(messages.payload_json, '$.text')), token.value) = 0 | |
🟡 Medium orchestration-v2/ThreadSearch.ts:222
A query like rust a drops messages containing rust à, so adding the short token a makes accent-insensitive message search miss valid results. The SQL prefilter uses SQLite’s ASCII-only lower() and literal instr() matching before messageMatcher can fold accents; normalize this prefilter the same way or leave the short-token check to the matcher.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
| 1 | In file @apps/server/src/orchestration-v2/ThreadSearch.ts around line 222: |
| 2 | |
| 3 | A query like `rust a` drops messages containing `rust à`, so adding the short token `a` makes accent-insensitive message search miss valid results. The SQL prefilter uses SQLite’s ASCII-only `lower()` and literal `instr()` matching before `messageMatcher` can fold accents; normalize this prefilter the same way or leave the short-token check to the matcher. |
Note
🤖 Claude Opus 5.5 responding on behalf of Theo
Not changing this. The short-token prefilter is a substring check, so a message passes when the letters appear anywhere in it. rust à is only dropped when the message has no plain a at all. The filter runs in SQL before the 2,000-hit limit on purpose: if messageMatcher did that check after the read, a common word plus a rare short word would use up the bounded read on hits that then get dropped.
| … | … | @@ -0,0 +1,1247 @@ | |
| 62 | + | return text | |
| 63 | + | .replace(/[\u0080-\uffff]/g, (character) => { | |
| 64 | + | const folded = foldCharacter(character); | |
| 65 | + | return folded.length === 1 ? folded : character; | |
🟡 Medium src/paletteSearch.ts:65
Decomposed accents in stored text never match their canonically equivalent queries: for example, résumé does not find re\u0301sume\u0301, and messageMatcher("résumé") returns null for that text. foldInPlace retains a combining mark when folding it produces an empty string, breaking the match; handle decomposed marks consistently while preserving source offsets for highlights.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
| 1 | In file @packages/shared/src/paletteSearch.ts around line 65: |
| 2 | |
| 3 | Decomposed accents in stored text never match their canonically equivalent queries: for example, `résumé` does not find `re\u0301sume\u0301`, and `messageMatcher("résumé")` returns `null` for that text. `foldInPlace` retains a combining mark when folding it produces an empty string, breaking the match; handle decomposed marks consistently while preserving source offsets for highlights. |
Note
🤖 Claude Opus 5.5 responding on behalf of Theo
Not changing this here. It is a known limit of the fold. Queries are normalized to NFC, and message text from keyboards and agents arrives composed. Matching decomposed text would mean dropping the combining marks, which changes string length. Snippet highlights and in-thread find depend on offsets into the original text, so this needs an offset map, not a change to foldInPlace.
| … | … | @@ -75,93 +266,220 @@ export const make = Effect.gen(function* () { | |
| 361 | + | const hitCount = (threadId: ThreadId) => hitCounts.get(threadId) ?? 1; | |
| 362 | + | const matchMessage = messageMatcher(input.query); | |
| 363 | + | for (const row of rows) { | |
| 364 | + | if (input.excludedThreadIds?.has(row.threadId)) continue; | |
🟡 Medium orchestration-v2/ThreadSearch.ts:364
searchThreads can return no message matches even when eligible message-only threads exist: messageHitsQuery limits results to 2,000 before excludedThreadIds discards title-matching threads here. Apply the exclusion in the SQL query before its hit limit so those threads cannot crowd out the reserved message results.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
| 1 | In file @apps/server/src/orchestration-v2/ThreadSearch.ts around line 364: |
| 2 | |
| 3 | `searchThreads` can return no message matches even when eligible message-only threads exist: `messageHitsQuery` limits results to 2,000 before `excludedThreadIds` discards title-matching threads here. Apply the exclusion in the SQL query before its hit limit so those threads cannot crowd out the reserved message results. |
Note
🤖 Claude Opus 5.5 responding on behalf of Theo
Fixed in 2825166b62. Listed threads are now removed in the eligible CTE (excludedThreadIds on the search request), so they leave before the hit limit, and the JS skip is gone. New test: "leaves titled threads out before the hit limit" (a titled thread with 2,000 newer hits). It fails on the previous head and passes now.