fix(kb): let the server say a connector sync is queued (#6968)

* fix(kb): let the server say a connector sync is queued

The connector chip inferred "a sync is coming" from `createdAt` inside a
2-minute window, because nothing on the row distinguished a queued sync from
an idle connector until a worker took the lock. The guess was wrong under
queue backlog and under client clock skew, and it forced a pile of client
state to stand in for it.

Adds `pending`, written as the sync is handed to the queue and cleared when a
worker takes the lock or the hand-off is found to have been lost. It is a
phase of the same lock `syncing` holds, so it opens the lease and takes an
ownership token the same way — the lease is what the scheduler ages a stranded
queue entry against (`updatedAt` cannot serve: a pending connector is still
editable, so any unrelated write would renew the recovery it should trigger),
and the token is what proves a late release belongs to this dispatch.

Deletes the 2-minute window, the in-flight id sets, the 5-minute cooldown
timers and the forced re-render they needed. The cooldown lived in a ref
inside a modal, so it evaporated whenever the modal closed; the disable now
comes from durable server state and is shared across tabs.

Also fixes, all found while tracing the lifecycle:
- An on-demand sync on a paused or disabled connector silently resumed it for
  good. Nothing could put the pause back: success writes `active`, a lost
  queue entry writes `error`, and the due-sweep keeps syncing that. Refused.
- A failed hand-off no longer advances the connector's auto-disable breaker. A
  queue outage would otherwise increment every connector in the fleet until
  they all disabled themselves for a fault that was never theirs.
- Manual sync on an established connector gave no feedback at all: the poll
  only ran while the predicate matched, which it never did.
- Four over-broad invalidations that refetched every cached chunk page and
  chunk search in a base when one connector document was excluded.
- The dead-process reporter re-sent a PATCH per stale document on every poll.

* fix(kb): refuse to start a queued run on a paused connector

The queue outlives the decision to sync. Pausing a connector after its run was
queued cleared the queue entry's token but left the task itself alive, and the
lock CAS accepted any row that was not already `syncing` — so the worker took
the paused row and wrote its own terminal `active` over the pause.

Moves the rule to the two points that can enforce it: an explicit
`LOCKABLE_CONNECTOR_STATUSES` allowlist on the lock acquisition, and the same
allowlist on `markSyncPending`, which closes the mirror race where a dispatch
already in flight rewrites a just-paused row back to `pending`. Queueing and
starting now agree on one rule, and a skipped hand-off is reported as its own
outcome rather than a concurrency conflict.

Also patches the connector detail cache alongside the list on an optimistic
status write, so an already-expanded card starts its own sync poll instead of
showing stale history behind the list's spinner.

* fix(kb): make a queued sync prove it is the run that was queued

`markSyncPending` minted an ownership token but only `releaseFailedDispatch`
checked it, so the worker could consume a queue entry that was not its own. A
task delayed past its lease is reclaimed and replaced; the status check alone
let that stale task take the replacement's entry and run superseded options —
a plain sync where the user had just asked for a full resync — while the
replacement was turned away as `sync_in_progress`.

Carries the token in the task payload and matches it at lock acquisition, the
same discipline `holdsSyncLockToken` already applies to the `syncing` phase,
extended to the phase before it. A superseded run is now reported as such
rather than as a concurrency conflict.

The payload field is optional for the rollout window only: tasks already in
the queue carry no token, and stranding them would be worse than letting them
fall back to the status check for one deploy.

* fix(kb): report a paused connector as paused, not superseded

Pausing a queued connector releases its token, so testing ownership before
status reported every pause-while-queued — the common case — as a superseded
dispatch. The mismatch is the symptom there; the status is the reason.

* fix(kb): stop a status update landing on a run that already started

The update's guards ran against a row read moments earlier and the write
carried no compare-and-set, so a worker taking the lock in between meant the
write landed on a `syncing` row — overwriting the run's status and, because
leaving `pending` also clears the lock columns, wiping the token its heartbeat
and terminal write match on. That stranded a sync that had already begun.

The write is now conditional on the status the request was authorized against,
and a lost race is reported as a conflict rather than "not found".

Also restores the in-flight guard on the pause control. The optimistic status
flip relabels it Pause -> Resume immediately, so a second click could send
`active` before the first pause settled and resume a connector the user meant
to pause. Read from the mutation's own pending state rather than the local id
set this PR removed — React Query already knows which row is in flight.
This commit is contained in:
Waleed
2026-08-21 20:53:16 -07:00
committed by GitHub
parent 8689237b4f
commit 8937bb3550
22 changed files with 1382 additions and 266 deletions
+4 -4
View File
@@ -3450,8 +3450,8 @@
},
"status": {
"type": "string",
"enum": ["active", "paused", "syncing", "error", "disabled"],
"description": "Current connector state."
"enum": ["active", "paused", "pending", "syncing", "error", "disabled"],
"description": "Current connector state. `pending` means a sync is queued but not yet running."
},
"lastSyncAt": {
"anyOf": [
@@ -3840,8 +3840,8 @@
},
"status": {
"type": "string",
"enum": ["active", "paused", "syncing", "error", "disabled"],
"description": "Current connector state."
"enum": ["active", "paused", "pending", "syncing", "error", "disabled"],
"description": "Current connector state. `pending` means a sync is queued but not yet running."
},
"lastSyncAt": {
"anyOf": [