mirror of
https://github.com/simstudioai/sim.git
synced 2026-10-16 06:39:56 +08:00
Waleed
4ca7651ea6
* improvement(knowledge): eliminate N+1 on tag definitions in bulk upload createDocumentRecords previously called processDocumentTags per-doc, each running a SELECT against knowledge_base_tag_definitions — N queries that all returned the same kbId-scoped rows. Worse, those reads used the global db pool while the tx held a FOR UPDATE lock on the KB row, risking pool contention on large bulk uploads. Split the helper into loadTagDefinitions (single query, accepts the tx as executor) and resolveDocumentTags (pure, takes the pre-loaded Map). The bulk path loads once inside the transaction; createSingleDocument loads once outside its tx. Same throw-on-validation-error semantics preserved. * improvement(knowledge): fold processDocumentAsync prefetches into one JOIN processDocumentAsync was issuing three separate SELECTs per processed document: knowledge_base (config), workspace (billing settings), and document (tag values). For a typical Trigger.dev fleet processing thousands of docs, that's thousands of redundant pool checkouts. Collapsed into a single JOIN at the top of processDocumentAsync that fetches kb config + billed account user + document tag values in one roundtrip. The post-embedding tag SELECT (which previously held tags through the full embedding-generation wait) is gone; tags from the initial prefetch are reused. Behavior: - Missing/archived/deleted document or KB → same 'failed' status outcome as before, single consolidated error message. - Missing billed account → preserves existing error. - All 208 KB tests pass (test mock extended for innerJoin/leftJoin). * improvement(knowledge): skip tag-definitions load when no doc carries tags Trim verbose comments in the same pass. * lint