Files
cline/apps/cli/src/connectors/thread-bindings.ts
T
3af23c1c4c fix(cli,core): stop duplicate connector launches (#12770)
* fix: prevent duplicate connector launches during doctor/connect

Mark connectors as starting before the hub daemon spawns so autostart
skips in-flight instances, and improve doctor process filtering with
container-aware namespace/cgroup checks plus detached log rotation.

* Update apps/cli/src/connectors/common.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* feat(hub): supervise connector processes

* feat(connectors): enable tools by default, and stop replaying the Slack greeting

Tools were on by default only for Telegram (via --no-tools); Slack, Discord,
Linear, Google Chat and WhatsApp all required an explicit --enable-tools. All
six now default to tools on and opt out with --no-tools.

--enable-tools still parses everywhere, including Telegram which never accepted
it, so deployed scripts, systemd units and persisted autostart arguments keep
working. Passing both resolves to the safer answer: --no-tools wins. This also
affects hub/webview starts, which never emitted a tools flag and so ran those
five connectors with tools off.

Slack no longer posts the "Connected to Cline." first-contact message. It was
gated on per-thread welcomeSentAt, so a connector restart or a cleared history
made the next user message look like first contact and replayed the greeting.
The host mechanism is unchanged and the other adapters still greet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(connectors): recover a thread whose session is wedged mid-run

A connector thread keeps a long-lived mapping to a hub session. When that
session's runtime still had a run in flight and no abort had been requested,
every message in the thread came back as "SessionRuntime.shutdown called while a
run is in progress" instead of an answer, and stayed that way until someone
cleared the binding by hand. Observed on the Cline Mom Slack bot after a stack
restart.

The connector host already recovers from a session the hub no longer knows
about: it forgets the mapping and replays the turn once against a fresh session.
This widens the trigger from "session not found" to "session cannot serve
another turn" via isUnusableSessionError, so a wedged runtime takes the same
path.

The shutdown error now carries a stable code (SessionRunInProgressError,
session_run_in_progress) so callers can recognise it structurally. The predicate
also matches on message, because an error reaching a connector has crossed the
hub's JSON boundary and arrives as a bare message - and because a host commonly
runs a hub and CLI of different versions. Ordinary run failures still propagate
untouched: replacing the session on those would hide real errors and drop the
conversation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(connectors): serialise turns that share a session

Answering "what happens if I message the bot in another thread while it is still
replying": channel threads were already independent, but DMs were not.

findBindingForThread deliberately reuses one binding — and therefore one runtime
session — for every message in a DM channel, so a DM stays one continuous
conversation. The turn queue, though, was keyed by thread id, and a DM thread id
carries the message timestamp. Two messages in flight in the same DM therefore
got two independent queues and ran concurrently against a single session, which
fails with "shutdown called while a run is in progress" or interleaves two
conversations in one session history.

The queue key now follows the same identity rule as the binding lookup, via
resolveThreadTurnQueueKey next to findBindingForThread so the two cannot drift.
DM messages queue behind each other on the shared session; channel threads keep
their own key and still run in parallel. Applied to all six adapters, which all
had the same mismatch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(core): abort an in-flight run before tearing its session down

Where the Slack bot's "plugin-sandbox process exited (code=null, signal=SIGTERM)"
came from, and its "shutdown called while a run is in progress" sibling: both are
one event, a session released while a run was still going.

stopSession aborts the agent first "so shutdown can proceed", but callers that
reach shutdownSession or releaseSessionRuntime another way did not - hub
dispose() on a restart being the one that hurt. Without an abort the runtime
refuses to shut down, that error is rethrown from the cleanup, and the plugin
sandbox is SIGTERMed while tool calls are still pending, so those calls reject
with "plugin-sandbox process exited". A connector turn awaiting the run reports
whichever surfaced first instead of answering.

Both paths now abort and let the run drain before shutting the agent, runtime and
sandbox down, guarded on session.aborting so callers that already aborted do not
abort twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(connectors): stop announcing "Steering current task."

Every follow-up sent while the bot was replying added an acknowledgement line to
the thread, and the wording overstated what happens: the host treats delivery
"steer" the same as "queue", enqueuing the prompt for the session rather than
injecting it into the loop already running. The follow-up is now handed over
silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(core): retire a dead supervised entry before replacing it

A start arriving while an instance sat in backoff left the old entry's
restart timer live. The timer closes over the old entry object, so when
it fired it spawned a second process for the same (channel, instanceId)
- untracked by the supervisor's map, so invisible to list() and
unreachable by stop() - two connectors holding one bot token, which is
the exact failure supervision exists to prevent. Its exit handler then
kept reaping the live instance's state and rescheduling restarts.

The same window exists before the timer is even scheduled: the
exit-cleanup chain runs first, and a replacement made mid-chain would be
followed by a restart scheduled for the retired entry.

start() now retires a dead existing entry explicitly - cancel its timer,
mark it stopped, drop its exit listener. Both the timer callback and the
cleanup chain already stand down on "stopped", so one mark covers both
phases.

* fix(core): serialise supervisor start/stop and wait for stopped processes to die

Found by exercising a hub restart against a live webhook connector: the
new hub's boot reconnect restarts the adopted survivor - which suspends
inside stop() on the CLI cleanup - while the user's `cline connect`
arrives as connector.start. With no per-instance serialisation the two
starts interleaved across that suspension and both spawned. The map
tracked one process while the other lived on untracked, holding the
connector's webhook port; the tracked chain crash-looped on EADDRINUSE
through all five attempts and ended state=failed, while the ghost kept
running with no way to reach it through list() or stop().

Two changes:

- start/stop (and the backoff-restart spawn) now run under a per-
  instance-key promise queue, so one instance has exactly one lifecycle
  operation in flight. The exit-cleanup chain also stands down when its
  entry is no longer the one in the map.
- stop() waits for the process to actually die after SIGTERM (bounded,
  then SIGKILL) instead of returning while it still holds its listen
  port - the race that turned the double-spawn into a crash loop, and
  that could burn a backoff cycle on any webhook connector restart.

process.kill is now injectable (killProcess), which also stops the test
suite from signalling arbitrary real pids like 600 on the host.

Verified live: the same kill-hub-then-reconnect sequence now converges
to one tracked running process, with the concurrent user start
correctly answered "already running under the hub".

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Saoud Rizwan <7799382+saoudrizwan@users.noreply.github.com>
2026-08-04 13:28:18 -07:00

570 lines
16 KiB
TypeScript

import type { SerializedThread, Thread } from "chat";
import { readJsonFile, writeJsonFile } from "./common";
export type ConnectorThreadState = {
sessionId?: string;
enableTools?: boolean;
autoApproveTools?: boolean;
cwd?: string;
workspaceRoot?: string;
systemPrompt?: string;
participantKey?: string;
participantLabel?: string;
welcomeSentAt?: string;
};
export type ConnectorThreadBinding<TState extends ConnectorThreadState> = {
kind?: "conversation" | "participant" | "thread" | "thread-participant-mute";
channelId: string;
isDM: boolean;
participantKey?: string;
participantLabel?: string;
threadMutedAt?: string;
mutedParticipantKey?: string;
mutedParticipantLabel?: string;
participantMutedAt?: string;
serializedThread: string;
sessionId?: string;
state?: TState;
updatedAt: string;
};
export type ConnectorBindingStore<TState extends ConnectorThreadState> = Record<
string,
ConnectorThreadBinding<TState>
>;
export type SerializableConnectorThread<TState extends ConnectorThreadState> =
Thread<TState> & {
toJSON(): SerializedThread;
};
export type ConnectorBindingThreadIdentity = Pick<
Thread<ConnectorThreadState>,
"id" | "channelId" | "isDM"
> & {
participantKey?: string;
};
export type ConnectorMuteTarget = {
participantKey: string;
participantLabel?: string;
};
function normalizeParticipantKey(
value: string | undefined,
): string | undefined {
const trimmed = value?.trim();
return trimmed ? trimmed : undefined;
}
function readSerializedThreadIdentity(
serializedThread: string | undefined,
): Partial<ConnectorBindingThreadIdentity> | undefined {
if (!serializedThread?.trim()) {
return undefined;
}
try {
const parsed = JSON.parse(
serializedThread,
) as Partial<ConnectorBindingThreadIdentity>;
return parsed && typeof parsed === "object" ? parsed : undefined;
} catch {
return undefined;
}
}
function resolveThreadControlKey(
thread: ConnectorBindingThreadIdentity,
): string {
return `thread:${thread.id}`;
}
function resolveParticipantMuteControlKey(
thread: ConnectorBindingThreadIdentity,
participantKey: string | undefined,
): string | undefined {
const normalized = normalizeParticipantKey(participantKey);
return normalized
? `thread:${thread.id}:participant:${normalized}`
: undefined;
}
function isControlBinding(
binding: ConnectorThreadBinding<ConnectorThreadState> | undefined,
): boolean {
return (
binding?.kind === "thread" || binding?.kind === "thread-participant-mute"
);
}
function clearSerializedThreadSessionId(serializedThread: string | undefined): {
serializedThread: string | undefined;
updated: boolean;
} {
if (!serializedThread?.trim()) {
return { serializedThread, updated: false };
}
try {
const parsed = JSON.parse(serializedThread) as unknown;
if (!parsed || typeof parsed !== "object") {
return { serializedThread, updated: false };
}
const record = parsed as Record<string, unknown>;
let updated = false;
if ("sessionId" in record) {
delete record.sessionId;
updated = true;
}
if (record.state && typeof record.state === "object") {
const state = record.state as Record<string, unknown>;
if ("sessionId" in state) {
delete state.sessionId;
updated = true;
}
}
return {
serializedThread: updated ? JSON.stringify(parsed) : serializedThread,
updated,
};
} catch {
return { serializedThread, updated: false };
}
}
export function resolveThreadBindingKey(
thread: ConnectorBindingThreadIdentity,
_state?: ConnectorThreadState | null,
): string {
return thread.id;
}
export function readBindings<TState extends ConnectorThreadState>(
path: string,
): ConnectorBindingStore<TState> {
const parsed = readJsonFile<ConnectorBindingStore<TState>>(path, {});
return parsed && typeof parsed === "object" ? parsed : {};
}
export function writeBindings<TState extends ConnectorThreadState>(
path: string,
bindings: ConnectorBindingStore<TState>,
): void {
writeJsonFile(path, bindings);
}
/**
* Key under which turns for `thread` must be serialised.
*
* This has to follow the same identity rule as {@link findBindingForThread},
* because whatever shares a session has to share a queue. A DM reuses one
* binding — and therefore one runtime session — for every message in the
* channel, so keying the queue by thread id would let two messages in the same
* DM run against that one session concurrently. That surfaces as
* "SessionRuntime.shutdown called while a run is in progress", or as two
* conversations interleaved in one session's history.
*
* Channel threads each own their binding, so they keep their own key and go on
* running independently of one another.
*/
export function resolveThreadTurnQueueKey(
thread: Pick<ConnectorBindingThreadIdentity, "id" | "channelId" | "isDM">,
): string {
return thread.isDM ? `dm:${thread.channelId}` : thread.id;
}
export function findBindingForThread<TState extends ConnectorThreadState>(
bindings: ConnectorBindingStore<TState>,
thread: ConnectorBindingThreadIdentity,
): { binding: ConnectorThreadBinding<TState>; key: string } | undefined {
const exact = bindings[thread.id];
if (exact && !isControlBinding(exact)) {
return { key: thread.id, binding: exact };
}
if (!thread.isDM) {
return undefined;
}
for (const [key, binding] of Object.entries(bindings)) {
if (isControlBinding(binding)) {
continue;
}
if (
binding.channelId === thread.channelId &&
binding.isDM === thread.isDM
) {
return { key, binding };
}
}
return undefined;
}
export function readBindingForThread<TState extends ConnectorThreadState>(
path: string,
thread: Thread<TState>,
errorLabel: string,
participantKey?: string,
): ConnectorThreadBinding<TState> | undefined {
const bindings = readBindings<TState>(path);
const threadIdentity: ConnectorBindingThreadIdentity = {
id: thread.id,
channelId: thread.channelId,
isDM: thread.isDM,
participantKey,
};
const match = findBindingForThread(bindings, threadIdentity);
if (!match) {
return undefined;
}
const targetKey = resolveThreadBindingKey(
threadIdentity,
match.binding.state,
);
const normalizedParticipantKey = normalizeParticipantKey(participantKey);
const storedThread = readSerializedThreadIdentity(
match.binding.serializedThread,
);
const storedParticipantKey = normalizeParticipantKey(
match.binding.participantKey ?? match.binding.state?.participantKey,
);
const needsRefresh =
match.key !== targetKey ||
storedThread?.id !== thread.id ||
storedThread?.channelId !== thread.channelId ||
storedThread?.isDM !== thread.isDM ||
storedParticipantKey !== normalizedParticipantKey;
if (needsRefresh) {
bindings[targetKey] = {
...match.binding,
channelId: thread.channelId,
isDM: thread.isDM,
participantKey: normalizedParticipantKey ?? match.binding.participantKey,
serializedThread: serializeThread(thread, errorLabel),
updatedAt: new Date().toISOString(),
};
if (match.key !== targetKey) {
delete bindings[match.key];
}
writeBindings(path, bindings);
}
return bindings[targetKey];
}
export function serializeThread<TState extends ConnectorThreadState>(
thread: Thread<TState>,
errorLabel: string,
): string {
const candidate = thread as Partial<SerializableConnectorThread<TState>>;
if (typeof candidate.toJSON !== "function") {
throw new Error(`${errorLabel} thread cannot be serialized`);
}
return JSON.stringify(candidate.toJSON.call(thread));
}
export function persistThreadBinding<TState extends ConnectorThreadState>(
path: string,
thread: Thread<TState>,
state: TState,
errorLabel: string,
): void {
const bindings = readBindings<TState>(path);
const participantKey = normalizeParticipantKey(state.participantKey);
const bindingKey = resolveThreadBindingKey(
thread as ConnectorBindingThreadIdentity,
state,
);
bindings[bindingKey] = {
kind: "conversation",
channelId: thread.channelId,
isDM: thread.isDM,
participantKey,
participantLabel: state.participantLabel?.trim() || undefined,
serializedThread: serializeThread(thread, errorLabel),
sessionId: state.sessionId,
state,
updatedAt: new Date().toISOString(),
};
writeBindings(path, bindings);
}
export function isThreadMuted<TState extends ConnectorThreadState>(
path: string,
thread: ConnectorBindingThreadIdentity,
): boolean {
const bindings = readBindings<TState>(path);
return isThreadMutedInBindings(bindings, thread);
}
export function isThreadMutedInBindings<TState extends ConnectorThreadState>(
bindings: ConnectorBindingStore<TState>,
thread: ConnectorBindingThreadIdentity,
): boolean {
const binding = bindings[resolveThreadControlKey(thread)];
return Boolean(binding?.threadMutedAt);
}
export function isParticipantMuted<TState extends ConnectorThreadState>(
path: string,
thread: ConnectorBindingThreadIdentity,
participantKey: string | undefined,
): boolean {
const key = resolveParticipantMuteControlKey(thread, participantKey);
if (!key) {
return false;
}
const bindings = readBindings<TState>(path);
return isParticipantMutedInBindings(bindings, thread, participantKey);
}
export function isParticipantMutedInBindings<
TState extends ConnectorThreadState,
>(
bindings: ConnectorBindingStore<TState>,
thread: ConnectorBindingThreadIdentity,
participantKey: string | undefined,
): boolean {
const key = resolveParticipantMuteControlKey(thread, participantKey);
if (!key) {
return false;
}
const binding = bindings[key];
return Boolean(binding?.participantMutedAt);
}
export function findMutedParticipantsForThread<
TState extends ConnectorThreadState,
>(
bindings: ConnectorBindingStore<TState>,
thread: ConnectorBindingThreadIdentity,
): ConnectorMuteTarget[] {
const prefix = `thread:${thread.id}:participant:`;
return Object.entries(bindings)
.filter(
([key, binding]) =>
key.startsWith(prefix) &&
binding.kind === "thread-participant-mute" &&
Boolean(binding.participantMutedAt) &&
Boolean(normalizeParticipantKey(binding.mutedParticipantKey)),
)
.map(([, binding]) => ({
participantKey:
normalizeParticipantKey(binding.mutedParticipantKey) ?? "",
participantLabel: binding.mutedParticipantLabel,
}))
.filter((target) => target.participantKey.length > 0);
}
export function setThreadMuted<TState extends ConnectorThreadState>(
path: string,
thread: Thread<TState>,
muted: boolean,
errorLabel: string,
): string | undefined {
const bindings = readBindings<TState>(path);
const key = resolveThreadControlKey(thread as ConnectorBindingThreadIdentity);
if (!muted) {
if (bindings[key]) {
delete bindings[key];
writeBindings(path, bindings);
}
return undefined;
}
const mutedAt = new Date().toISOString();
bindings[key] = {
kind: "thread",
channelId: thread.channelId,
isDM: thread.isDM,
threadMutedAt: mutedAt,
serializedThread: serializeThread(thread, errorLabel),
updatedAt: mutedAt,
};
writeBindings(path, bindings);
return mutedAt;
}
export function setParticipantMuted<TState extends ConnectorThreadState>(
path: string,
thread: Thread<TState>,
target: ConnectorMuteTarget,
muted: boolean,
errorLabel: string,
): string | undefined {
const normalized = normalizeParticipantKey(target.participantKey);
const key = resolveParticipantMuteControlKey(
thread as ConnectorBindingThreadIdentity,
normalized,
);
if (!normalized || !key) {
return undefined;
}
const bindings = readBindings<TState>(path);
if (!muted) {
if (bindings[key]) {
delete bindings[key];
writeBindings(path, bindings);
}
return undefined;
}
const mutedAt = new Date().toISOString();
bindings[key] = {
kind: "thread-participant-mute",
channelId: thread.channelId,
isDM: thread.isDM,
mutedParticipantKey: normalized,
mutedParticipantLabel: target.participantLabel?.trim() || undefined,
participantMutedAt: mutedAt,
serializedThread: serializeThread(thread, errorLabel),
updatedAt: mutedAt,
};
writeBindings(path, bindings);
return mutedAt;
}
export function mergeThreadState<TState extends ConnectorThreadState>(
threadState: TState | null | undefined,
bindingState: TState | undefined,
base: ConnectorThreadState,
): TState {
return {
...(threadState ?? bindingState ?? {}),
sessionId:
threadState?.sessionId?.trim() ||
bindingState?.sessionId?.trim() ||
undefined,
enableTools:
threadState?.enableTools ?? bindingState?.enableTools ?? base.enableTools,
autoApproveTools:
threadState?.autoApproveTools ??
bindingState?.autoApproveTools ??
base.autoApproveTools,
cwd: threadState?.cwd || bindingState?.cwd || base.cwd,
workspaceRoot:
threadState?.workspaceRoot ||
bindingState?.workspaceRoot ||
base.workspaceRoot,
systemPrompt:
threadState?.systemPrompt ||
bindingState?.systemPrompt ||
base.systemPrompt,
participantKey:
threadState?.participantKey ||
bindingState?.participantKey ||
base.participantKey,
participantLabel:
threadState?.participantLabel ||
bindingState?.participantLabel ||
base.participantLabel,
welcomeSentAt:
threadState?.welcomeSentAt ||
bindingState?.welcomeSentAt ||
base.welcomeSentAt,
} as TState;
}
export async function loadThreadState<TState extends ConnectorThreadState>(
thread: Thread<TState>,
bindingsPath: string,
base: ConnectorThreadState,
): Promise<TState> {
const threadState = await thread.state;
const binding = readBindingForThread<TState>(
bindingsPath,
thread,
"Connector",
threadState?.participantKey,
);
return mergeThreadState(threadState, binding?.state, base);
}
export function findBindingForParticipantKey<
TState extends ConnectorThreadState,
>(
bindings: ConnectorBindingStore<TState>,
participantKey: string | undefined,
): { binding: ConnectorThreadBinding<TState>; key: string } | undefined {
const normalized = normalizeParticipantKey(participantKey);
if (!normalized) {
return undefined;
}
const exact = bindings[normalized];
if (exact) {
return { key: normalized, binding: exact };
}
for (const [key, binding] of Object.entries(bindings)) {
const bindingParticipantKey = normalizeParticipantKey(
binding.participantKey ?? binding.state?.participantKey,
);
if (bindingParticipantKey === normalized) {
return { key, binding };
}
}
return undefined;
}
export function findBindingForDeliveryTarget<
TState extends ConnectorThreadState,
>(
bindings: ConnectorBindingStore<TState>,
input: {
bindingKey?: string;
threadId?: string;
participantKey?: string;
},
): { binding: ConnectorThreadBinding<TState>; key: string } | undefined {
const bindingKey = normalizeParticipantKey(input.bindingKey);
if (bindingKey) {
const exact = bindings[bindingKey];
if (exact && !isControlBinding(exact)) {
return { key: bindingKey, binding: exact };
}
const participantMatch = findBindingForParticipantKey(bindings, bindingKey);
if (participantMatch) {
return participantMatch;
}
}
const threadId = input.threadId?.trim();
if (threadId) {
const exact = bindings[threadId];
if (exact && !isControlBinding(exact)) {
return { key: threadId, binding: exact };
}
}
return findBindingForParticipantKey(bindings, input.participantKey);
}
export async function persistMergedThreadState<
TState extends ConnectorThreadState,
>(
thread: Thread<TState>,
bindingsPath: string,
nextState: TState,
errorLabel: string,
): Promise<void> {
await thread.setState(nextState, { replace: true });
persistThreadBinding(bindingsPath, thread, nextState, errorLabel);
}
export function clearBindingSessionIds<TState extends ConnectorThreadState>(
path: string,
): void {
const bindings = readBindings<TState>(path);
let updated = false;
for (const binding of Object.values(bindings)) {
if ("sessionId" in binding) {
delete binding.sessionId;
updated = true;
}
if (binding.state && "sessionId" in binding.state) {
delete binding.state.sessionId;
updated = true;
}
const serialized = clearSerializedThreadSessionId(binding.serializedThread);
if (serialized.updated) {
binding.serializedThread = serialized.serializedThread ?? "";
updated = true;
}
}
if (updated) {
writeBindings(path, bindings);
}
}