mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
Waleed
7e975e7dae
* fix(confluence): index mirrored/included page content via rendered view format The KB connector fetched page bodies as body-format=storage, which only carries unexpanded macro references (Include Page / Excerpt Include). Those 'mirrored' articles were stripped to empty content by htmlToPlainText and never synced. Switch getDocument to body-format=view (supported on the v2 single-item page/blogpost GET) so built-in include/excerpt macros render inline and the included text is indexed. * fix(confluence): invalidate existing doc hashes on representation change The version-based contentHash meant already-synced mirrored documents (with stale empty content) classified as 'unchanged' and never re-hydrated with the new rendered view content. Embed a body-representation marker in the hash so a representation change invalidates every previously-synced Confluence document, forcing a one-time re-hydration that picks up the expanded include/excerpt text. * feat(connectors): full resync re-hydrates rendered content (transclusions) Version-based change detection can't see when a Confluence page's rendered view changes because an *included* page was edited (the container's version doesn't bump). Add a 'Full resync' path so that drift can be recovered: - ConnectorMeta.rehydrateOnFullSync flag (set for Confluence) - on fullSync, classifyExternalDoc promotes unchanged deferred docs to update and the hydration guard re-indexes unconditionally, so rendered content is refreshed - fullSync threaded through the manual sync contract (query param), route, and hook - 'Sync now' / 'Full resync' dropdown on the connector card Incremental syncs stay hash-gated and cheap; only the deliberate full resync pays the re-index cost. * refactor(connectors): decouple rehydrate from fullSync deletion semantics The transclusion refresh only needs re-hydration, but reusing the fullSync flag also activated its deletion-cleanup semantics — which bypass three previously unreachable safety guards (empty-listing wipe, listingCapped, and the >50% mass-deletion threshold). Since fullSync had no caller before this PR, the new 'Full resync' button would have exposed all three to any KB editor. Introduce a dedicated 'rehydrate' request that ONLY forces re-hydration + re-index of already-synced docs. Listing and deletion reconciliation are identical to a normal sync (all safety guards stay armed). fullSync's cleanup semantics remain dormant and untouched. * refactor(connectors): tidy rehydrate flag + gate Full resync to supported connectors Cleanup/simplify pass over the connector changes: - use shared booleanQueryFlagSchema for the rehydrate query param (typed boolean at the boundary instead of a hand-rolled 'true'/'false' string enum) - move rehydrateOnFullSync onto the client-safe ConnectorMeta so the UI can gate on it - only Confluence (rehydrateOnFullSync) shows the Sync now / Full resync dropdown; every other connector keeps its original one-click sync button (Full resync is a no-op for them, and this restores the pre-change one-click UX) - wrap the sync trigger in a span so its tooltip still shows while disabled (cooldown) * fix(connectors): forward rehydrate flag through the Trigger.dev worker executeConnectorSyncJob (the production async sync path) destructured only fullSync from the payload and forwarded only fullSync to executeSync, silently dropping rehydrate. A manual Full resync would therefore never re-hydrate on the default Trigger.dev path. Forward rehydrate too. * fix(connectors): rehydrate forces a full listing so containers aren't omitted A rehydrate request set forceRehydrate but left listing incremental. For a connector that is both incremental and rehydrateOnFullSync, an unchanged container page that transcludes a changed page would be omitted from the incremental listing and never re-hydrated. Force a full (non-incremental) listing on rehydrate so every document is seen; deletion-safety guards stay armed (unlike fullSync). No-op for Confluence, which is already non-incremental.