opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-10-31 02:43:21 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix: stem chat search terms with the english text search config (#28319)

Browse Source 
Chat message search used the `simple` text search config, which folds
case but does no stemming, so `refactor` did not match messages
containing `refactoring`. Message bodies now index and query with the
`english` config, which stems both sides. Titles, PR titles, and all
other filters are unchanged.

Migration `000585` adds a `chat_messages.search_tsv_config` enum column
recording which config produced each stored vector and stamps existing
vectors `'simple'` (a column-only update; vectors and indexes are
untouched, since rewriting `search_tsv` would block message writes on
large tables). The dbpurge sweep gains a bounded
`ReindexStaleChatMessagesSearchTsv` pass that rewrites stale vectors
newest first and stops for the process lifetime once drained. Until a
row is rewritten, `GetChats` matches it with the config that produced
it, so pre-migration exact-form searches keep working during the drain.
Vectors written by old binaries mid rolling upgrade cannot stamp the
config, stay pending, and self-heal on the next upgraded sweep.
Steady-state DB load is unchanged.

The down migration resets `english` vectors to `NULL` so the parent
version's existing sweep rewrites them with `simple`.

Closes
[CODAGT-867](https://linear.app/codercom/issue/CODAGT-867/backend-search-improvements-to-handle-partial-matches)

<details>
<summary>Decision log</summary>

- **`english` stemming over prefix matching.** On a 300k-row corpus a
stemmed lookup is a single GIN entry-tree probe, while `term:*` prefix
matching unions posting lists of every lexeme sharing the prefix (up to
a full scan for short prefixes), is asymmetric, and loses the
`websearch_to_tsquery` phrase/OR/negation UX. `pg_trgm` rejected as a
heavier index with different semantics.
- **Scope to message bodies only** (review). Titles and PR titles keep
their `simple` FTS expressions and indexes, identical to main.
- **Config-versioned queue over migration-time reset** (review). An
unqualified `UPDATE ... SET search_tsv = NULL` rewrites every indexed
row in the migration transaction, and an old replica could backfill
reset rows with `simple` vectors that permanently leave the `search_tsv
IS NULL` queue. The config column makes staleness explicit and both
problems disappear.
- **Enum over `text`** (review) for the config column.
- **Trade-offs.** English stopwords are dropped from message search
(stopword-only queries match no messages); stemming is English-specific;
titles do not stem; prefix matching remains out of scope.

</details>

---

🤖 This PR was generated by Coder Agents on behalf of @f0ssel.
This commit is contained in:
Garrett Delfosse
2026-08-25 09:08:52 -04:00
committed by GitHub
parent 6284964b5f
commit 378ac91b49
21 changed files with 703 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reference/api/chats.md
Generated
+4 -4
View File
@@ -27,10 +27,10 @@ Experimental: this endpoint is subject to change.
### Parameters
| Name | In | Type | Required | Description |
|---------|-------|--------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `q` | query | string | false | Search query. Supports `title:<substring>` (case-insensitive, quote multi-word values), `archived:bool`, `has_unread:bool`, `pr_status:<draft\|open\|merged\|closed>` as repeated or comma-separated values, `source:<created_by_me\|shared_with_me>`, `diff_url:<url>` (quote values containing colons), `pr:<number>` (exact PR number match), `repo:<owner/repo>` (case-insensitive substring match against git remote origin or URL), `pr_title:<text>` (case-insensitive PR title substring), `search:<text>` (full-text search across chat titles, PR titles, PR numbers, and message bodies; quote multi-word values; cannot be combined with title, pr_title, or pr; a value that tokenizes to no searchable words returns an empty list). Bare terms are not supported; use `title:<value>` or `search:<value>`. |
| `label` | query | string | false | Filter by label as key:value. Repeat for multiple (AND logic). |
| Name | In | Type | Required | Description |
|---------|-------|--------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `q` | query | string | false | Search query. Supports `title:<substring>` (case-insensitive, quote multi-word values), `archived:bool`, `has_unread:bool`, `pr_status:<draft\|open\|merged\|closed>` as repeated or comma-separated values, `source:<created_by_me\|shared_with_me>`, `diff_url:<url>` (quote values containing colons), `pr:<number>` (exact PR number match), `repo:<owner/repo>` (case-insensitive substring match against git remote origin or URL), `pr_title:<text>` (case-insensitive PR title substring), `search:<text>` (full-text search across chat titles, PR titles, PR numbers, and message bodies; message bodies match English word stems, e.g. `refactor` matches `refactoring`, and ignore English stopwords; titles and PR titles match whole words case-insensitively without stemming; quote multi-word values; cannot be combined with title, pr_title, or pr; a value that tokenizes to no searchable words, e.g. punctuation only, returns an empty list). Bare terms are not supported; use `title:<value>` or `search:<value>`. |
| `label` | query | string | false | Filter by label as key:value. Repeat for multiple (AND logic). |
### Example responses