mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
fix(triggers): repair broken idempotency dedup across 4 webhook/polling triggers (#5524)
* fix(instantly): add idempotency dedup for webhook deliveries
Instantly webhook deliveries had no extractIdempotencyId, so retried
deliveries were never deduped. Instantly doesn't send a delivery-id
header, so key on email_id when present (unique per email/reply) and
fall back to a content-based key (event_type + campaign_id + lead_email
+ event timestamp) otherwise. Added instantly.test.ts covering auth,
event matching, formatInput/output alignment, idempotency, and the
subscription lifecycle.
* fix(gitlab): add delivery idempotency and fix reserved type-field collision risk
extractIdempotencyId was missing entirely, so GitLab's automatic webhook
retries (triggered after timeouts/5xx, up to 4 consecutive failures before
temporary disable) would re-execute the workflow as a brand-new event.
Added a content-derived fallback key (checkout_sha for push, object
id + updated_at otherwise) since GitLab's X-Gitlab-Event-UUID delivery
header isn't available to extractIdempotencyId (body-only).
Also renamed the Issue Hook payload's object_attributes.type (GitLab
17.2+ work item type) to work_item_type in formatInput and exposed it
in buildGitLabIssueOutputs, since 'type' collides with the TriggerOutput
meta-key. Added detailed_merge_status to the merge request outputs to
match the deprecation of merge_status.
Added apps/sim/lib/webhooks/providers/gitlab.test.ts covering verifyAuth,
matchEvent, formatInput, and extractIdempotencyId.
* fix(gmail): correct labels output type and add formatInput/schema alignment test
The gmail_poller trigger declared its `labels` output as type: 'string'
while the description and actual runtime value (email.labelIds) are a
string array. Changed to type: 'json' to match the shape actually
delivered, consistent with how other triggers type array-of-string
fields.
Added apps/sim/lib/webhooks/providers/gmail.test.ts covering
gmailHandler.formatInput passthrough behavior and a regression check
that every key formatInput can deliver on `email` matches a key
declared in the trigger's output schema.
* fix(linear): tighten replay window, dedupe HMAC verify, add idempotency fallback
- Reuse the shared createHmacVerifier helper for Linear's signature check
instead of hand-rolling header/secret validation.
- Tighten the webhookTimestamp replay-protection skew from 5 minutes to 60
seconds, matching Linear's documented recommendation.
- Add extractIdempotencyId as a content-based fallback (type:action:id:
updatedAt) for when the Linear-Delivery header is unavailable.
- Remove unused teamOutputs/stateOutputs dead code from triggers/linear/utils.ts.
* fix(gitlab): preserve raw object_attributes.type alongside work_item_type
formatInput previously spread the entire raw body, so object_attributes.type
(GitLab 17.2+ work item type) was already reachable via a manual expression
even though undocumented. The earlier fix renamed it to work_item_type in
the delivered data too, which would silently break any pre-existing
workflow referencing the raw path. Keep both keys in the delivered data —
this is plain passthrough data, not schema-constrained, so there's no
reserved-key collision risk in doing so; only the declared *schema* needs
the work_item_type name.
* fix(instantly): differentiate idempotency key by timestamp when email_id repeats
Instantly's is_first field ("whether this is the first event of this
type for the lead") only makes sense if the same event_type can fire
more than once for the same email_id: every open of the same email,
every click, and every reply in a thread all share one email_id
(Instantly's reply_to_uuid). Keying idempotency on email_id alone
collapsed all of those distinct, legitimate deliveries into a single
dedup slot, so every open/click/reply after the first was silently
dropped for the 7-day idempotency TTL. Append timestamp (fixed per
event occurrence, confirmed not to regenerate on retry) to keep the
key both retry-stable and unique per occurrence.
* fix(linear): revert replay-window tightening — retry timestamp semantics unverified
The prior pass tightened LINEAR_WEBHOOK_TIMESTAMP_SKEW_MS from 5 minutes to
Linear's suggested 60 seconds, justified by the claim that webhookTimestamp
is re-stamped fresh on every delivery/retry attempt. That claim is not
stated anywhere in Linear's webhook docs (verified against the raw docs
page directly, including the exact wording around retries and the
webhookTimestamp field) — the docs only say it is "the time when the
webhook was sent" with no mention of retry behavior. A third-party
implementation guide explicitly recommends a 5-minute window instead of
Linear's literal 60s suggestion, citing this same ambiguity.
If webhookTimestamp is actually fixed to the original event time (not
refreshed per attempt), a strict 60s window would silently and permanently
reject every one of Linear's own 1hr/6hr retry deliveries following any
transient outage on our side, since Linear gives up after 3 failed
attempts. Idempotency dedup (Linear-Delivery header / extractIdempotencyId
fallback) already prevents double-processing of replayed/retried
deliveries within a wider window, so the extra replay-protection from
matching the literal 60s suggestion is marginal next to the risk of
dropping real business events. Reverting to the 5-minute window pending
explicit confirmation from Linear on retry timestamp semantics.
* fix(gitlab): include ref in push idempotency key to avoid branch-deletion collisions
GitLab sets checkout_sha to null on branch/tag deletion, so the
extractIdempotencyId fallback (checkout_sha || after) degenerated to the
all-zeros SHA for every deletion. Without ref in the key, two unrelated
branch deletions in the same project produced the same idempotency key,
silently dropping the second (and any subsequent) legitimate trigger
execution as a false duplicate.
* fix(gmail): use 'array' not 'json' for poller labels output type
The prior pass changed the gmail_poller trigger's `labels` output from
'string' to 'json', which is directionally correct (labelIds is a
string array, not a string) but not the precise type. The codebase's
own PrimitiveValueType union has a dedicated 'array' variant, used by
sibling triggers with identical semantics (github issue/PR triggers,
Jira triggers all declare their `labels` field as type: 'array').
Concretely, 'json' vs 'array' diverges in generateMockValue
(apps/sim/lib/workflows/triggers/trigger-utils.ts), which backs the
auto-generated "Event Payload Example" sample payload shown in the
trigger config UI (apps/sim/triggers/index.ts, gated on
trigger.id.includes('poller') — gmail_poller qualifies). 'json' mocks
as a single generic object ({id, name, status}), actively misleading
for a field that is always an array. 'array' mocks as a list, matching
the true shape (Gmail API docs confirm labelIds is a string array on
the Message resource).
No behavioral difference in output-path validation/resolution
(isPathInSchema, generateOutputPaths) since the field declares no
items/properties either way — this is purely a type-declaration
correctness fix, verified against Gmail API docs and cross-checked
against every other output field in poller.ts (all match runtime
shape).
* docs(triggers): correct GitLab retry claim, convert inline comments to TSDoc
GitLab does not automatically retry failed webhook deliveries — a failed
request only counts toward auto-disabling the webhook (4 consecutive
failures = temporary disable, 40 = permanent), and re-delivery only
happens via a manual "Resend Request" (UI or API), which carries the
same webhook-id/Idempotency-Key/X-Gitlab-Event-UUID headers as the
original delivery. Those headers are already in the shared idempotency
service's allowlist and checked ahead of extractIdempotencyId, so the
body-based fallback matters for the rare case those headers are
stripped in transit — not for "automatic retries" as previously stated.
Verified directly against docs.gitlab.com.
Also replaced inline `//` comments across the gitlab/gmail/linear/
instantly provider and trigger files with TSDoc blocks on the relevant
declaration where the reasoning was non-obvious, or removed them
outright where they only restated adjacent code.
* fix(triggers): close two Greptile-flagged crash/collision gaps
instantly: the timestamp-present branch was fixed for the email_id
collision risk, but the no-timestamp fallback still keyed on bare
email_id — same collision risk, unfixed. Return null instead so dedup
is skipped rather than risking a false collision between repeat opens/
clicks/replies on the same email.
linear: extractIdempotencyId cast `body` straight to a Record without
checking it was actually an object first, so a JSON `null` body (no
Linear-Delivery header available) would throw on `b.type` and turn a
malformed payload into a webhook 500 instead of a clean skip.
* fix(linear): guard matchEvent and formatInput against a null body
Found during a final backwards-compat pass, same class of bug Greptile
just caught in extractIdempotencyId: both cast body straight to a
Record without checking it was actually an object, so a genuinely
null/malformed body would throw instead of degrading gracefully.
gitlab.ts (asRecord's `|| {}`) and instantly.ts (isRecordLike checks)
already guard this everywhere; linear.ts was the one inconsistent
provider. Uses the same isRecordLike helper as instantly.ts.
* fix(gitlab): distinguish pipeline lifecycle transitions in idempotency key
Pipeline Hook payloads have no updated_at field (confirmed against
docs.gitlab.com — only Issue/Merge Request/Note hooks reliably include
it), so every lifecycle transition of the same pipeline (pending ->
running -> success/failed) collapsed onto the identical fallback key
and later real transitions were skipped as duplicates. Falls back to
status + finished_at/created_at when updated_at is absent.
This commit is contained in:
Waleed
@@ -0,0 +1,229 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { NextRequest } from 'next/server'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { gitlabHandler } from '@/lib/webhooks/providers/gitlab'
|
||||
import { isGitLabEventMatch } from '@/triggers/gitlab/utils'
|
||||
|
||||
function reqWithHeaders(headers: Record<string, string>): NextRequest {
|
||||
return new NextRequest('http://localhost/test', { headers })
|
||||
}
|
||||
|
||||
describe('GitLab webhook provider', () => {
|
||||
it('verifyAuth rejects when webhookSecret is missing', async () => {
|
||||
const res = await gitlabHandler.verifyAuth!({
|
||||
request: reqWithHeaders({}),
|
||||
rawBody: '{}',
|
||||
requestId: 't1',
|
||||
providerConfig: {},
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
expect(res?.status).toBe(401)
|
||||
})
|
||||
|
||||
it('verifyAuth rejects when X-Gitlab-Token header is missing', async () => {
|
||||
const res = await gitlabHandler.verifyAuth!({
|
||||
request: reqWithHeaders({}),
|
||||
rawBody: '{}',
|
||||
requestId: 't2',
|
||||
providerConfig: { webhookSecret: 'my-secret' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
expect(res?.status).toBe(401)
|
||||
})
|
||||
|
||||
it('verifyAuth rejects when the token does not match', async () => {
|
||||
const res = await gitlabHandler.verifyAuth!({
|
||||
request: reqWithHeaders({ 'X-Gitlab-Token': 'wrong' }),
|
||||
rawBody: '{}',
|
||||
requestId: 't3',
|
||||
providerConfig: { webhookSecret: 'my-secret' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
expect(res?.status).toBe(401)
|
||||
})
|
||||
|
||||
it('verifyAuth accepts a matching X-Gitlab-Token', async () => {
|
||||
const res = await gitlabHandler.verifyAuth!({
|
||||
request: reqWithHeaders({ 'X-Gitlab-Token': 'my-secret' }),
|
||||
rawBody: '{}',
|
||||
requestId: 't4',
|
||||
providerConfig: { webhookSecret: 'my-secret' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
expect(res).toBeNull()
|
||||
})
|
||||
|
||||
it('isGitLabEventMatch matches the configured trigger to its object_kind', () => {
|
||||
expect(isGitLabEventMatch('gitlab_push', 'push')).toBe(true)
|
||||
expect(isGitLabEventMatch('gitlab_push', 'issue')).toBe(false)
|
||||
expect(isGitLabEventMatch('gitlab_comment', 'note')).toBe(true)
|
||||
expect(isGitLabEventMatch('gitlab_webhook', 'anything')).toBe(true)
|
||||
})
|
||||
|
||||
it('matchEvent passes through all events for the all-events trigger', async () => {
|
||||
const result = await gitlabHandler.matchEvent!({
|
||||
body: { object_kind: 'issue' },
|
||||
requestId: 't5',
|
||||
providerConfig: { triggerId: 'gitlab_webhook' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
request: reqWithHeaders({}),
|
||||
})
|
||||
expect(result).toBe(true)
|
||||
})
|
||||
|
||||
it('matchEvent filters events that do not match the configured trigger', async () => {
|
||||
const result = await gitlabHandler.matchEvent!({
|
||||
body: { object_kind: 'issue' },
|
||||
requestId: 't6',
|
||||
providerConfig: { triggerId: 'gitlab_push' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
request: reqWithHeaders({}),
|
||||
})
|
||||
expect(result).toBe(false)
|
||||
})
|
||||
|
||||
it('formatInput derives event_type and branch from the push payload', async () => {
|
||||
const { input } = await gitlabHandler.formatInput!({
|
||||
body: { object_kind: 'push', ref: 'refs/heads/main', checkout_sha: 'abc123' },
|
||||
headers: { 'x-gitlab-event': 'Push Hook' },
|
||||
requestId: 't7',
|
||||
webhook: {},
|
||||
workflow: { id: 'w', userId: 'u' },
|
||||
})
|
||||
const i = input as Record<string, unknown>
|
||||
expect(i.event_type).toBe('Push Hook')
|
||||
expect(i.branch).toBe('main')
|
||||
expect(i.checkout_sha).toBe('abc123')
|
||||
})
|
||||
|
||||
it('formatInput exposes object_attributes.type as work_item_type on issue payloads, keeping the raw type key too', async () => {
|
||||
const { input } = await gitlabHandler.formatInput!({
|
||||
body: {
|
||||
object_kind: 'issue',
|
||||
object_attributes: { id: 1, iid: 2, title: 'Bug', type: 'Issue' },
|
||||
},
|
||||
headers: { 'x-gitlab-event': 'Issue Hook' },
|
||||
requestId: 't8',
|
||||
webhook: {},
|
||||
workflow: { id: 'w', userId: 'u' },
|
||||
})
|
||||
const i = input as Record<string, unknown>
|
||||
const attrs = i.object_attributes as Record<string, unknown>
|
||||
expect(attrs.work_item_type).toBe('Issue')
|
||||
expect(attrs.type).toBe('Issue')
|
||||
expect(attrs.title).toBe('Bug')
|
||||
})
|
||||
|
||||
it('extractIdempotencyId derives a stable key for push events from checkout_sha', () => {
|
||||
const body = {
|
||||
object_kind: 'push',
|
||||
project: { id: 42 },
|
||||
ref: 'refs/heads/main',
|
||||
checkout_sha: 'abc123',
|
||||
}
|
||||
const first = gitlabHandler.extractIdempotencyId!(body)
|
||||
const second = gitlabHandler.extractIdempotencyId!({ ...body })
|
||||
expect(first).toBe(second)
|
||||
expect(first).toContain('abc123')
|
||||
expect(first).toContain('42')
|
||||
})
|
||||
|
||||
it('extractIdempotencyId does not collide across different branches deleted in the same project', () => {
|
||||
const deleteMain = {
|
||||
object_kind: 'push',
|
||||
project: { id: 42 },
|
||||
ref: 'refs/heads/main',
|
||||
checkout_sha: null,
|
||||
after: '0000000000000000000000000000000000000000',
|
||||
}
|
||||
const deleteFeature = {
|
||||
object_kind: 'push',
|
||||
project: { id: 42 },
|
||||
ref: 'refs/heads/feature',
|
||||
checkout_sha: null,
|
||||
after: '0000000000000000000000000000000000000000',
|
||||
}
|
||||
const first = gitlabHandler.extractIdempotencyId!(deleteMain)
|
||||
const second = gitlabHandler.extractIdempotencyId!(deleteFeature)
|
||||
expect(first).not.toBeNull()
|
||||
expect(second).not.toBeNull()
|
||||
expect(first).not.toBe(second)
|
||||
})
|
||||
|
||||
it('extractIdempotencyId is stable for a repeated delivery of the same branch deletion', () => {
|
||||
const body = {
|
||||
object_kind: 'push',
|
||||
project: { id: 42 },
|
||||
ref: 'refs/heads/main',
|
||||
checkout_sha: null,
|
||||
after: '0000000000000000000000000000000000000000',
|
||||
}
|
||||
const first = gitlabHandler.extractIdempotencyId!(body)
|
||||
const second = gitlabHandler.extractIdempotencyId!({ ...body })
|
||||
expect(first).toBe(second)
|
||||
})
|
||||
|
||||
it('extractIdempotencyId derives a stable key for issue events from object_attributes', () => {
|
||||
const body = {
|
||||
object_kind: 'issue',
|
||||
project: { id: 7 },
|
||||
object_attributes: { id: 99, updated_at: '2026-01-01T00:00:00.000Z' },
|
||||
}
|
||||
const first = gitlabHandler.extractIdempotencyId!(body)
|
||||
const second = gitlabHandler.extractIdempotencyId!({ ...body })
|
||||
expect(first).toBe(second)
|
||||
expect(first).toContain('99')
|
||||
expect(first).toContain('7')
|
||||
})
|
||||
|
||||
it('extractIdempotencyId distinguishes pipeline lifecycle transitions despite no updated_at', () => {
|
||||
const pending = gitlabHandler.extractIdempotencyId!({
|
||||
object_kind: 'pipeline',
|
||||
project: { id: 7 },
|
||||
object_attributes: { id: 31, status: 'pending', created_at: '2026-01-01T00:00:00Z' },
|
||||
})
|
||||
const running = gitlabHandler.extractIdempotencyId!({
|
||||
object_kind: 'pipeline',
|
||||
project: { id: 7 },
|
||||
object_attributes: { id: 31, status: 'running', created_at: '2026-01-01T00:00:00Z' },
|
||||
})
|
||||
const success = gitlabHandler.extractIdempotencyId!({
|
||||
object_kind: 'pipeline',
|
||||
project: { id: 7 },
|
||||
object_attributes: {
|
||||
id: 31,
|
||||
status: 'success',
|
||||
created_at: '2026-01-01T00:00:00Z',
|
||||
finished_at: '2026-01-01T00:03:00Z',
|
||||
},
|
||||
})
|
||||
expect(pending).not.toBeNull()
|
||||
expect(pending).not.toBe(running)
|
||||
expect(running).not.toBe(success)
|
||||
|
||||
const retryOfSuccess = gitlabHandler.extractIdempotencyId!({
|
||||
object_kind: 'pipeline',
|
||||
project: { id: 7 },
|
||||
object_attributes: {
|
||||
id: 31,
|
||||
status: 'success',
|
||||
created_at: '2026-01-01T00:00:00Z',
|
||||
finished_at: '2026-01-01T00:03:00Z',
|
||||
},
|
||||
})
|
||||
expect(success).toBe(retryOfSuccess)
|
||||
})
|
||||
|
||||
it('extractIdempotencyId returns null when there is no stable identifier', () => {
|
||||
expect(gitlabHandler.extractIdempotencyId!({ object_kind: 'push' })).toBeNull()
|
||||
expect(gitlabHandler.extractIdempotencyId!({ object_kind: 'issue' })).toBeNull()
|
||||
})
|
||||
})
|
||||
@@ -99,16 +99,87 @@ export const gitlabHandler: WebhookProviderHandler = {
|
||||
return true
|
||||
},
|
||||
|
||||
/**
|
||||
* GitLab 17.2+ adds a `type` field inside `object_attributes` on Issue Hook
|
||||
* payloads (e.g. "Issue", "Incident", "Task"). `type` is a reserved
|
||||
* TriggerOutput meta-key, so it can't be declared in the output schema
|
||||
* under that name — exposed there as `work_item_type` instead. The raw
|
||||
* `type` key is kept in the delivered data alongside it (this is plain
|
||||
* passthrough data, not schema-constrained) so a workflow already
|
||||
* referencing the undocumented raw path keeps working.
|
||||
*/
|
||||
async formatInput({ body, headers }: FormatInputContext): Promise<FormatInputResult> {
|
||||
const b = asRecord(body)
|
||||
const eventType = headers['x-gitlab-event'] || ''
|
||||
const ref = (b.ref as string) || ''
|
||||
const branch = ref.replace('refs/heads/', '')
|
||||
return {
|
||||
input: { ...b, event_type: eventType, branch },
|
||||
const objectAttributes = b.object_attributes
|
||||
let input: Record<string, unknown> = { ...b, event_type: eventType, branch }
|
||||
if (
|
||||
objectAttributes &&
|
||||
typeof objectAttributes === 'object' &&
|
||||
!Array.isArray(objectAttributes)
|
||||
) {
|
||||
const workItemType = (objectAttributes as Record<string, unknown>).type
|
||||
if (workItemType !== undefined) {
|
||||
input = {
|
||||
...input,
|
||||
object_attributes: { ...objectAttributes, work_item_type: workItemType },
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { input }
|
||||
},
|
||||
|
||||
/**
|
||||
* GitLab does not automatically retry a failed delivery — a failed request
|
||||
* only counts toward auto-disabling the webhook (4 consecutive failures
|
||||
* disables it temporarily, 40 permanently), and re-delivery only happens
|
||||
* via a manual "Resend Request" (UI or API), which carries the same
|
||||
* `webhook-id`/`Idempotency-Key`/`X-Gitlab-Event-UUID` headers as the
|
||||
* original. Those headers are already in the shared idempotency service's
|
||||
* allowlist and checked ahead of this method, which only receives the body.
|
||||
* This is a content-derived fallback for the rare case those headers are
|
||||
* stripped in transit (e.g. by an intermediary proxy). checkout_sha is
|
||||
* null on branch/tag deletion (after falls back to the all-zeros SHA), so
|
||||
* ref is included to keep unrelated deletions in one project from
|
||||
* colliding onto the same key. Pipeline Hook payloads have no updated_at
|
||||
* at all (confirmed against docs.gitlab.com — only Issue/Merge Request/
|
||||
* Note hooks reliably include it), so status + finished_at/created_at is
|
||||
* used there instead to keep each lifecycle transition of one pipeline
|
||||
* (pending/running/success/failed) from colliding onto the same key.
|
||||
*/
|
||||
extractIdempotencyId(body: unknown): string | null {
|
||||
const b = asRecord(body)
|
||||
const objectKind = (b.object_kind as string) || ''
|
||||
const project = asRecord(b.project)
|
||||
const projectId = project.id != null ? String(project.id) : ''
|
||||
|
||||
if (objectKind === 'push' || objectKind === 'tag_push') {
|
||||
const ref = (b.ref as string) || ''
|
||||
const checkoutSha = (b.checkout_sha as string) || (b.after as string) || ''
|
||||
if (!checkoutSha && !ref) return null
|
||||
return `gitlab:${objectKind}:${projectId}:${ref}:${checkoutSha}`
|
||||
}
|
||||
|
||||
const objectAttributes = asRecord(b.object_attributes)
|
||||
const id = objectAttributes.id != null ? String(objectAttributes.id) : ''
|
||||
if (!id) return null
|
||||
const version =
|
||||
(objectAttributes.updated_at as string) ||
|
||||
[objectAttributes.status, objectAttributes.finished_at || objectAttributes.created_at]
|
||||
.filter(Boolean)
|
||||
.join(':') ||
|
||||
''
|
||||
return `gitlab:${objectKind || 'event'}:${projectId}:${id}:${version}`
|
||||
},
|
||||
|
||||
/**
|
||||
* Validates the optional self-managed host up front so a structurally
|
||||
* unsafe value surfaces as a clear error instead of an unhandled
|
||||
* UnsafeGitLabHostError.
|
||||
*/
|
||||
async createSubscription(ctx: SubscriptionContext): Promise<SubscriptionResult | undefined> {
|
||||
const config = getProviderConfig(ctx.webhook)
|
||||
const accessToken = config.accessToken as string | undefined
|
||||
@@ -120,8 +191,6 @@ export const gitlabHandler: WebhookProviderHandler = {
|
||||
throw new Error('GitLab Personal Access Token is required to create the webhook.')
|
||||
if (!projectId) throw new Error('GitLab Project ID is required to create the webhook.')
|
||||
|
||||
// Validate the optional self-managed host up front so a structurally unsafe
|
||||
// value surfaces as a clear error instead of an unhandled UnsafeGitLabHostError.
|
||||
try {
|
||||
getGitLabApiBase(host)
|
||||
} catch (error) {
|
||||
@@ -163,8 +232,6 @@ export const gitlabHandler: WebhookProviderHandler = {
|
||||
|
||||
const created = (await res.json().catch(() => ({}))) as { id?: number | string }
|
||||
if (created.id === undefined || created.id === null) {
|
||||
// The hook was created but we can't read its id — delete it by URL so it
|
||||
// is not orphaned in GitLab.
|
||||
await cleanupGitLabHookByUrl(projectId, accessToken, getNotificationUrl(ctx.webhook), host)
|
||||
throw new Error('GitLab webhook created but no hook ID was returned.')
|
||||
}
|
||||
@@ -173,6 +240,10 @@ export const gitlabHandler: WebhookProviderHandler = {
|
||||
return { providerConfigUpdates: { externalId: String(created.id), webhookSecret: secretToken } }
|
||||
},
|
||||
|
||||
/**
|
||||
* A structurally unsafe host must not abort cleanup in non-strict mode —
|
||||
* mirrors the graceful skip used for missing credentials below.
|
||||
*/
|
||||
async deleteSubscription(ctx: DeleteSubscriptionContext): Promise<void> {
|
||||
const config = getProviderConfig(ctx.webhook)
|
||||
const accessToken = config.accessToken as string | undefined
|
||||
@@ -188,8 +259,6 @@ export const gitlabHandler: WebhookProviderHandler = {
|
||||
return
|
||||
}
|
||||
|
||||
// A structurally unsafe host must not abort cleanup in non-strict mode — mirror
|
||||
// the graceful skip used for missing credentials above.
|
||||
try {
|
||||
getGitLabApiBase(host)
|
||||
} catch (error) {
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { gmailHandler } from '@/lib/webhooks/providers/gmail'
|
||||
import { gmailPollingTrigger } from '@/triggers/gmail'
|
||||
|
||||
const sampleEmail = {
|
||||
id: 'msg-123',
|
||||
threadId: 'thread-456',
|
||||
subject: 'Test subject',
|
||||
from: 'sender@example.com',
|
||||
to: 'recipient@example.com',
|
||||
cc: '',
|
||||
date: '2026-07-08T00:00:00.000Z',
|
||||
bodyText: 'plain text body',
|
||||
bodyHtml: '<p>html body</p>',
|
||||
labels: ['INBOX', 'UNREAD'],
|
||||
hasAttachments: false,
|
||||
attachments: [],
|
||||
}
|
||||
|
||||
describe('Gmail webhook provider', () => {
|
||||
it('formatInput passes through the polled email and timestamp unchanged', async () => {
|
||||
const { input } = await gmailHandler.formatInput!({
|
||||
webhook: {},
|
||||
workflow: { id: 'wf', userId: 'u' },
|
||||
body: { email: sampleEmail, timestamp: '2026-07-08T00:00:05.000Z' },
|
||||
headers: {},
|
||||
requestId: 'test',
|
||||
})
|
||||
|
||||
expect(input).toEqual({
|
||||
email: sampleEmail,
|
||||
timestamp: '2026-07-08T00:00:05.000Z',
|
||||
})
|
||||
})
|
||||
|
||||
it('passes the raw body through when it has no email key', async () => {
|
||||
const { input } = await gmailHandler.formatInput!({
|
||||
webhook: {},
|
||||
workflow: { id: 'wf', userId: 'u' },
|
||||
body: { foo: 'bar' },
|
||||
headers: {},
|
||||
requestId: 'test',
|
||||
})
|
||||
|
||||
expect(input).toEqual({ foo: 'bar' })
|
||||
})
|
||||
|
||||
it('every key formatInput can deliver on `email` matches a declared trigger output key', async () => {
|
||||
const { input } = await gmailHandler.formatInput!({
|
||||
webhook: {},
|
||||
workflow: { id: 'wf', userId: 'u' },
|
||||
body: { email: sampleEmail, timestamp: '2026-07-08T00:00:05.000Z' },
|
||||
headers: {},
|
||||
requestId: 'test',
|
||||
})
|
||||
|
||||
const declaredEmailKeys = Object.keys(gmailPollingTrigger.outputs.email)
|
||||
const deliveredEmailKeys = Object.keys((input as { email: object }).email)
|
||||
|
||||
expect(deliveredEmailKeys.sort()).toEqual(declaredEmailKeys.sort())
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,405 @@
|
||||
import { NextRequest } from 'next/server'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { instantlyHandler } from '@/lib/webhooks/providers/instantly'
|
||||
|
||||
function reqWithHeaders(headers: Record<string, string>): NextRequest {
|
||||
return new NextRequest('http://localhost/test', { headers })
|
||||
}
|
||||
|
||||
describe('Instantly webhook provider', () => {
|
||||
it('verifyAuth rejects when secretToken is missing (fail-closed)', async () => {
|
||||
const res = await instantlyHandler.verifyAuth!({
|
||||
request: reqWithHeaders({}),
|
||||
rawBody: '{}',
|
||||
requestId: 't1',
|
||||
providerConfig: {},
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
expect(res?.status).toBe(401)
|
||||
})
|
||||
|
||||
it('verifyAuth rejects when the token header is missing', async () => {
|
||||
const res = await instantlyHandler.verifyAuth!({
|
||||
request: reqWithHeaders({}),
|
||||
rawBody: '{}',
|
||||
requestId: 't2',
|
||||
providerConfig: { secretToken: 'expected-token' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
expect(res?.status).toBe(401)
|
||||
})
|
||||
|
||||
it('verifyAuth rejects an incorrect token', async () => {
|
||||
const res = await instantlyHandler.verifyAuth!({
|
||||
request: reqWithHeaders({ 'x-sim-webhook-token': 'wrong-token' }),
|
||||
rawBody: '{}',
|
||||
requestId: 't3',
|
||||
providerConfig: { secretToken: 'expected-token' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
expect(res?.status).toBe(401)
|
||||
})
|
||||
|
||||
it('verifyAuth accepts a matching token', async () => {
|
||||
const res = await instantlyHandler.verifyAuth!({
|
||||
request: reqWithHeaders({ 'x-sim-webhook-token': 'expected-token' }),
|
||||
rawBody: '{}',
|
||||
requestId: 't4',
|
||||
providerConfig: { secretToken: 'expected-token' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
expect(res).toBeNull()
|
||||
})
|
||||
|
||||
it('matchEvent passes all events through for the generic webhook trigger', async () => {
|
||||
const matched = await instantlyHandler.matchEvent!({
|
||||
body: { event_type: 'lead_interested' },
|
||||
requestId: 't5',
|
||||
providerConfig: { triggerId: 'instantly_webhook' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
expect(matched).toBe(true)
|
||||
})
|
||||
|
||||
it('matchEvent filters events that do not match the configured trigger', async () => {
|
||||
const matched = await instantlyHandler.matchEvent!({
|
||||
body: { event_type: 'email_opened' },
|
||||
requestId: 't6',
|
||||
providerConfig: { triggerId: 'instantly_email_sent' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
expect(matched).toBe(false)
|
||||
})
|
||||
|
||||
it('matchEvent matches events for the configured trigger', async () => {
|
||||
const matched = await instantlyHandler.matchEvent!({
|
||||
body: { event_type: 'email_sent' },
|
||||
requestId: 't7',
|
||||
providerConfig: { triggerId: 'instantly_email_sent' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
expect(matched).toBe(true)
|
||||
})
|
||||
|
||||
it('matchEvent accepts both link-click event type spellings', async () => {
|
||||
const matchedA = await instantlyHandler.matchEvent!({
|
||||
body: { event_type: 'link_clicked' },
|
||||
requestId: 't8a',
|
||||
providerConfig: { triggerId: 'instantly_link_clicked' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
const matchedB = await instantlyHandler.matchEvent!({
|
||||
body: { event_type: 'email_link_clicked' },
|
||||
requestId: 't8b',
|
||||
providerConfig: { triggerId: 'instantly_link_clicked' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
expect(matchedA).toBe(true)
|
||||
expect(matchedB).toBe(true)
|
||||
})
|
||||
|
||||
it('formats input with keys matching the trigger outputs', async () => {
|
||||
const body = {
|
||||
timestamp: '2026-07-08T12:00:00.000Z',
|
||||
event_type: 'reply_received',
|
||||
workspace: 'ws-1',
|
||||
campaign_id: 'camp-1',
|
||||
campaign_name: 'Q3 Outreach',
|
||||
lead_email: 'lead@example.com',
|
||||
email_account: 'sender@example.com',
|
||||
unibox_url: 'https://app.instantly.ai/unibox/1',
|
||||
step: 2,
|
||||
variant: 1,
|
||||
is_first: true,
|
||||
email_id: 'email-1',
|
||||
reply_text_snippet: 'Sounds good',
|
||||
reply_subject: 'Re: Q3 Outreach',
|
||||
reply_text: 'Sounds good, thanks!',
|
||||
reply_html: '<p>Sounds good, thanks!</p>',
|
||||
}
|
||||
|
||||
const result = await instantlyHandler.formatInput!({
|
||||
body,
|
||||
webhook: {},
|
||||
workflow: { id: 'wf-1', userId: 'user-1' },
|
||||
headers: {},
|
||||
requestId: 't9',
|
||||
})
|
||||
|
||||
expect(result.input).toEqual({
|
||||
timestamp: '2026-07-08T12:00:00.000Z',
|
||||
eventType: 'reply_received',
|
||||
workspace: 'ws-1',
|
||||
campaignId: 'camp-1',
|
||||
campaignName: 'Q3 Outreach',
|
||||
leadEmail: 'lead@example.com',
|
||||
emailAccount: 'sender@example.com',
|
||||
uniboxUrl: 'https://app.instantly.ai/unibox/1',
|
||||
step: 2,
|
||||
variant: 1,
|
||||
isFirst: true,
|
||||
emailId: 'email-1',
|
||||
emailSubject: null,
|
||||
emailText: null,
|
||||
emailHtml: null,
|
||||
replyTextSnippet: 'Sounds good',
|
||||
replySubject: 'Re: Q3 Outreach',
|
||||
replyText: 'Sounds good, thanks!',
|
||||
replyHtml: '<p>Sounds good, thanks!</p>',
|
||||
payload: body,
|
||||
})
|
||||
})
|
||||
|
||||
describe('extractIdempotencyId', () => {
|
||||
it('prefers the email_id when present, qualified by timestamp', () => {
|
||||
const id = instantlyHandler.extractIdempotencyId!({
|
||||
event_type: 'email_sent',
|
||||
email_id: 'email-123',
|
||||
campaign_id: 'camp-1',
|
||||
lead_email: 'lead@example.com',
|
||||
timestamp: '2026-07-08T12:00:00.000Z',
|
||||
})
|
||||
expect(id).toBe('instantly:email_sent:email-123:2026-07-08T12:00:00.000Z')
|
||||
})
|
||||
|
||||
it('returns null when email_id is present but timestamp is missing, rather than risk a false collision', () => {
|
||||
const id = instantlyHandler.extractIdempotencyId!({
|
||||
event_type: 'email_sent',
|
||||
email_id: 'email-123',
|
||||
})
|
||||
expect(id).toBeNull()
|
||||
})
|
||||
|
||||
it('falls back to a content-based key without an email_id', () => {
|
||||
const id = instantlyHandler.extractIdempotencyId!({
|
||||
event_type: 'lead_interested',
|
||||
campaign_id: 'camp-1',
|
||||
lead_email: 'lead@example.com',
|
||||
timestamp: '2026-07-08T12:00:00.000Z',
|
||||
})
|
||||
expect(id).toBe('instantly:lead_interested:camp-1:lead@example.com:2026-07-08T12:00:00.000Z')
|
||||
})
|
||||
|
||||
it('is stable across retries of the same delivery', () => {
|
||||
const body = {
|
||||
event_type: 'lead_interested',
|
||||
campaign_id: 'camp-1',
|
||||
lead_email: 'lead@example.com',
|
||||
timestamp: '2026-07-08T12:00:00.000Z',
|
||||
}
|
||||
const first = instantlyHandler.extractIdempotencyId!(body)
|
||||
const second = instantlyHandler.extractIdempotencyId!({ ...body })
|
||||
expect(first).toBe(second)
|
||||
})
|
||||
|
||||
it('is stable across retries of the same email_id-keyed delivery', () => {
|
||||
const body = {
|
||||
event_type: 'email_opened',
|
||||
email_id: 'email-123',
|
||||
timestamp: '2026-07-08T12:00:00.000Z',
|
||||
}
|
||||
const first = instantlyHandler.extractIdempotencyId!(body)
|
||||
const second = instantlyHandler.extractIdempotencyId!({ ...body })
|
||||
expect(first).toBe(second)
|
||||
})
|
||||
|
||||
it('does not collide across distinct occurrences of the same email_id (e.g. repeat opens/clicks/replies)', () => {
|
||||
const firstOpen = instantlyHandler.extractIdempotencyId!({
|
||||
event_type: 'email_opened',
|
||||
email_id: 'email-123',
|
||||
timestamp: '2026-07-08T12:00:00.000Z',
|
||||
})
|
||||
const secondOpen = instantlyHandler.extractIdempotencyId!({
|
||||
event_type: 'email_opened',
|
||||
email_id: 'email-123',
|
||||
timestamp: '2026-07-08T13:30:00.000Z',
|
||||
})
|
||||
expect(firstOpen).not.toBe(secondOpen)
|
||||
})
|
||||
|
||||
it('returns null when there is not enough data to build a stable key', () => {
|
||||
expect(instantlyHandler.extractIdempotencyId!({ event_type: 'account_error' })).toBeNull()
|
||||
expect(instantlyHandler.extractIdempotencyId!({})).toBeNull()
|
||||
expect(instantlyHandler.extractIdempotencyId!('not-an-object')).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('createSubscription', () => {
|
||||
const fetchMock = vi.fn()
|
||||
|
||||
beforeEach(() => {
|
||||
vi.stubEnv('NEXT_PUBLIC_APP_URL', 'https://app.test')
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
fetchMock.mockReset()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals()
|
||||
vi.unstubAllEnvs()
|
||||
})
|
||||
|
||||
it('creates an Instantly webhook with the mapped event type', async () => {
|
||||
fetchMock.mockResolvedValue({
|
||||
ok: true,
|
||||
status: 200,
|
||||
json: async () => ({ id: 'wh-123' }),
|
||||
})
|
||||
|
||||
const result = await instantlyHandler.createSubscription!({
|
||||
webhook: {
|
||||
id: 'webhook-1',
|
||||
path: 'abc-path',
|
||||
providerConfig: { triggerApiKey: 'instantly-key', triggerId: 'instantly_email_sent' },
|
||||
},
|
||||
workflow: {},
|
||||
userId: 'user-1',
|
||||
requestId: 'req-create-1',
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1)
|
||||
const [url, init] = fetchMock.mock.calls[0]
|
||||
expect(url).toBe('https://api.instantly.ai/api/v2/webhooks')
|
||||
expect(init.method).toBe('POST')
|
||||
expect(init.headers.Authorization).toBe('Bearer instantly-key')
|
||||
const sent = JSON.parse(init.body)
|
||||
expect(sent.event_type).toBe('email_sent')
|
||||
expect(sent.target_hook_url).toContain('/api/webhooks/trigger/abc-path')
|
||||
expect(typeof sent.headers['X-Sim-Webhook-Token']).toBe('string')
|
||||
|
||||
expect(result?.providerConfigUpdates?.externalId).toBe('wh-123')
|
||||
})
|
||||
|
||||
it('maps the link-clicked trigger to the email_link_clicked subscription event', async () => {
|
||||
fetchMock.mockResolvedValue({ ok: true, status: 200, json: async () => ({ id: 'wh-456' }) })
|
||||
|
||||
await instantlyHandler.createSubscription!({
|
||||
webhook: {
|
||||
id: 'webhook-2',
|
||||
path: 'p2',
|
||||
providerConfig: { triggerApiKey: 'instantly-key', triggerId: 'instantly_link_clicked' },
|
||||
},
|
||||
workflow: {},
|
||||
userId: 'user-1',
|
||||
requestId: 'req-create-2',
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
|
||||
const sent = JSON.parse(fetchMock.mock.calls[0][1].body)
|
||||
expect(sent.event_type).toBe('email_link_clicked')
|
||||
})
|
||||
|
||||
it('throws a friendly error on 401', async () => {
|
||||
fetchMock.mockResolvedValue({
|
||||
ok: false,
|
||||
status: 401,
|
||||
json: async () => ({ message: 'unauthorized' }),
|
||||
})
|
||||
|
||||
await expect(
|
||||
instantlyHandler.createSubscription!({
|
||||
webhook: {
|
||||
id: 'webhook-3',
|
||||
path: 'p3',
|
||||
providerConfig: { triggerApiKey: 'bad-key', triggerId: 'instantly_email_sent' },
|
||||
},
|
||||
workflow: {},
|
||||
userId: 'user-1',
|
||||
requestId: 'req-create-3',
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
).rejects.toThrow(/Invalid Instantly API Key/)
|
||||
})
|
||||
|
||||
it('throws when the API key is missing', async () => {
|
||||
await expect(
|
||||
instantlyHandler.createSubscription!({
|
||||
webhook: {
|
||||
id: 'webhook-4',
|
||||
path: 'p4',
|
||||
providerConfig: { triggerId: 'instantly_email_sent' },
|
||||
},
|
||||
workflow: {},
|
||||
userId: 'user-1',
|
||||
requestId: 'req-create-4',
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
).rejects.toThrow(/Instantly API Key is required/)
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
describe('deleteSubscription', () => {
|
||||
const fetchMock = vi.fn()
|
||||
|
||||
beforeEach(() => {
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
fetchMock.mockReset()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals()
|
||||
})
|
||||
|
||||
it('deletes the webhook by externalId', async () => {
|
||||
fetchMock.mockResolvedValue({ ok: true, status: 200, body: null })
|
||||
|
||||
await instantlyHandler.deleteSubscription!({
|
||||
webhook: {
|
||||
id: 'webhook-1',
|
||||
providerConfig: { triggerApiKey: 'instantly-key', externalId: 'wh-123' },
|
||||
},
|
||||
workflow: {},
|
||||
requestId: 'req-del-1',
|
||||
})
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1)
|
||||
const [url, init] = fetchMock.mock.calls[0]
|
||||
expect(url).toBe('https://api.instantly.ai/api/v2/webhooks/wh-123')
|
||||
expect(init.method).toBe('DELETE')
|
||||
})
|
||||
|
||||
it('skips when apiKey or externalId is missing and does not throw', async () => {
|
||||
await instantlyHandler.deleteSubscription!({
|
||||
webhook: { id: 'webhook-2', providerConfig: { externalId: 'wh-123' } },
|
||||
workflow: {},
|
||||
requestId: 'req-del-2',
|
||||
})
|
||||
await instantlyHandler.deleteSubscription!({
|
||||
webhook: { id: 'webhook-3', providerConfig: { triggerApiKey: 'instantly-key' } },
|
||||
workflow: {},
|
||||
requestId: 'req-del-3',
|
||||
})
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('does not throw on a non-ok response in non-strict mode', async () => {
|
||||
fetchMock.mockResolvedValue({ ok: false, status: 500, json: async () => ({}) })
|
||||
|
||||
await expect(
|
||||
instantlyHandler.deleteSubscription!({
|
||||
webhook: {
|
||||
id: 'webhook-4',
|
||||
providerConfig: { triggerApiKey: 'instantly-key', externalId: 'wh-9' },
|
||||
},
|
||||
workflow: {},
|
||||
requestId: 'req-del-4',
|
||||
})
|
||||
).resolves.toBeUndefined()
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -57,6 +57,38 @@ export const instantlyHandler: WebhookProviderHandler = {
|
||||
return true
|
||||
},
|
||||
|
||||
/**
|
||||
* `email_id` (Instantly's `reply_to_uuid`) identifies the sent email, not
|
||||
* one occurrence of an event on that email — the same event_type can fire
|
||||
* more than once for one email_id (every open, every click, every reply in
|
||||
* a thread), so email_id alone would collapse those distinct, legitimate
|
||||
* deliveries into one idempotency slot. `timestamp` is fixed per event
|
||||
* occurrence (not regenerated on retry), so appending it keeps the key
|
||||
* both retry-stable and unique per occurrence — without it there's no way
|
||||
* to distinguish occurrences, so dedup is skipped rather than risking a
|
||||
* false collision.
|
||||
*/
|
||||
extractIdempotencyId(body: unknown): string | null {
|
||||
if (!isRecordLike(body)) return null
|
||||
|
||||
const eventType = typeof body.event_type === 'string' ? body.event_type : undefined
|
||||
if (!eventType) return null
|
||||
|
||||
const timestamp = typeof body.timestamp === 'string' ? body.timestamp : undefined
|
||||
const emailId = typeof body.email_id === 'string' ? body.email_id : undefined
|
||||
if (emailId) {
|
||||
return timestamp ? `instantly:${eventType}:${emailId}:${timestamp}` : null
|
||||
}
|
||||
|
||||
const campaignId = typeof body.campaign_id === 'string' ? body.campaign_id : undefined
|
||||
const leadEmail = typeof body.lead_email === 'string' ? body.lead_email : undefined
|
||||
if (campaignId && leadEmail && timestamp) {
|
||||
return `instantly:${eventType}:${campaignId}:${leadEmail}:${timestamp}`
|
||||
}
|
||||
|
||||
return null
|
||||
},
|
||||
|
||||
async formatInput({ body }: FormatInputContext): Promise<FormatInputResult> {
|
||||
const payload = isRecordLike(body) ? body : {}
|
||||
|
||||
|
||||
@@ -75,4 +75,102 @@ describe('Linear webhook provider', () => {
|
||||
|
||||
expect(res).toBeNull()
|
||||
})
|
||||
|
||||
it('accepts signed requests within the 5-minute skew window (wider than Linear’s literal 60s suggestion, to tolerate retries)', async () => {
|
||||
const secret = 'linear-secret'
|
||||
const rawBody = JSON.stringify({
|
||||
action: 'update',
|
||||
type: 'Issue',
|
||||
webhookTimestamp: Date.now() - 61_000,
|
||||
})
|
||||
|
||||
const res = await linearHandler.verifyAuth!({
|
||||
request: requestWithLinearSignature(secret, rawBody),
|
||||
rawBody,
|
||||
requestId: 'linear-t4',
|
||||
providerConfig: { webhookSecret: secret },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
|
||||
expect(res).toBeNull()
|
||||
})
|
||||
|
||||
it('skips verification entirely when no webhookSecret is configured', async () => {
|
||||
const rawBody = JSON.stringify({ action: 'create', type: 'Issue' })
|
||||
|
||||
const res = await linearHandler.verifyAuth!({
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
rawBody,
|
||||
requestId: 'linear-t5',
|
||||
providerConfig: {},
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
})
|
||||
|
||||
expect(res).toBeNull()
|
||||
})
|
||||
|
||||
describe('matchEvent', () => {
|
||||
it('returns null-body-safe result instead of throwing when body is null', async () => {
|
||||
const result = await linearHandler.matchEvent!({
|
||||
body: null,
|
||||
requestId: 'linear-t6',
|
||||
providerConfig: { triggerId: 'linear_issue_created' },
|
||||
webhook: {},
|
||||
workflow: {},
|
||||
request: new NextRequest('http://localhost/test'),
|
||||
})
|
||||
expect(result).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('formatInput', () => {
|
||||
it('does not throw when body is null', async () => {
|
||||
const { input } = await linearHandler.formatInput!({ body: null } as any)
|
||||
expect(input).toMatchObject({ action: '', type: '', actor: null })
|
||||
})
|
||||
})
|
||||
|
||||
describe('extractIdempotencyId', () => {
|
||||
it('builds a stable key from type, action, entity id, and updatedAt', () => {
|
||||
const key = linearHandler.extractIdempotencyId!({
|
||||
type: 'Issue',
|
||||
action: 'update',
|
||||
data: { id: 'issue-1', updatedAt: '2026-07-08T00:00:00.000Z' },
|
||||
})
|
||||
|
||||
expect(key).toBe('linear:Issue:update:issue-1:2026-07-08T00:00:00.000Z')
|
||||
})
|
||||
|
||||
it('falls back to createdAt when updatedAt is absent (create events)', () => {
|
||||
const key = linearHandler.extractIdempotencyId!({
|
||||
type: 'Comment',
|
||||
action: 'create',
|
||||
data: { id: 'comment-1', createdAt: '2026-07-08T00:00:00.000Z' },
|
||||
})
|
||||
|
||||
expect(key).toBe('linear:Comment:create:comment-1:2026-07-08T00:00:00.000Z')
|
||||
})
|
||||
|
||||
it('returns null when the entity id is missing', () => {
|
||||
const key = linearHandler.extractIdempotencyId!({ type: 'Issue', action: 'create' })
|
||||
expect(key).toBeNull()
|
||||
})
|
||||
|
||||
it('returns null when type is missing', () => {
|
||||
const key = linearHandler.extractIdempotencyId!({ action: 'create', data: { id: 'x' } })
|
||||
expect(key).toBeNull()
|
||||
})
|
||||
|
||||
it('returns null instead of throwing when the body is null', () => {
|
||||
expect(linearHandler.extractIdempotencyId!(null)).toBeNull()
|
||||
})
|
||||
|
||||
it('returns null instead of throwing when the body is a non-object', () => {
|
||||
expect(linearHandler.extractIdempotencyId!('not an object')).toBeNull()
|
||||
expect(linearHandler.extractIdempotencyId!(42)).toBeNull()
|
||||
expect(linearHandler.extractIdempotencyId!(['array'])).toBeNull()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -3,6 +3,7 @@ import { safeCompare } from '@sim/security/compare'
|
||||
import { hmacSha256Hex } from '@sim/security/hmac'
|
||||
import { toError } from '@sim/utils/errors'
|
||||
import { generateId } from '@sim/utils/id'
|
||||
import { isRecordLike } from '@sim/utils/object'
|
||||
import { NextResponse } from 'next/server'
|
||||
import { getNotificationUrl, getProviderConfig } from '@/lib/webhooks/provider-subscription-utils'
|
||||
import type {
|
||||
@@ -15,6 +16,7 @@ import type {
|
||||
SubscriptionResult,
|
||||
WebhookProviderHandler,
|
||||
} from '@/lib/webhooks/providers/types'
|
||||
import { createHmacVerifier } from '@/lib/webhooks/providers/utils'
|
||||
|
||||
const logger = createLogger('WebhookProvider:Linear')
|
||||
|
||||
@@ -43,30 +45,37 @@ function validateLinearSignature(secret: string, signature: string, body: string
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Linear's docs recommend a 60s window ("Reject any webhooks not within 60 seconds of the
|
||||
* current time to prevent replay attacks") but do NOT document whether `webhookTimestamp` is
|
||||
* re-stamped per delivery attempt or fixed to the original event time. Linear's own retry policy
|
||||
* resends failed deliveries after 1 minute, 1 hour, and 6 hours (@see
|
||||
* https://linear.app/developers/webhooks) — if the timestamp is fixed rather than refreshed per
|
||||
* attempt, a strict 60s window would silently and permanently drop every legitimate 1hr/6hr retry
|
||||
* following any transient outage on our side, since Linear gives up after 3 failed attempts.
|
||||
* We keep a wider 5-minute window: idempotency dedup (Linear-Delivery header / extractIdempotencyId
|
||||
* fallback below) already prevents double-processing of any replayed or retried delivery within
|
||||
* that window, so the incremental replay-protection benefit of matching Linear's 60s suggestion
|
||||
* literally is marginal compared to the risk of dropping real business events.
|
||||
*/
|
||||
const LINEAR_WEBHOOK_TIMESTAMP_SKEW_MS = 5 * 60 * 1000
|
||||
|
||||
const verifyLinearSignature = createHmacVerifier({
|
||||
configKey: 'webhookSecret',
|
||||
headerName: 'Linear-Signature',
|
||||
validateFn: validateLinearSignature,
|
||||
providerLabel: 'Linear',
|
||||
})
|
||||
|
||||
export const linearHandler: WebhookProviderHandler = {
|
||||
async verifyAuth({
|
||||
request,
|
||||
rawBody,
|
||||
requestId,
|
||||
providerConfig,
|
||||
}: AuthContext): Promise<NextResponse | null> {
|
||||
const secret = providerConfig.webhookSecret as string | undefined
|
||||
if (!secret) {
|
||||
async verifyAuth(ctx: AuthContext): Promise<NextResponse | null> {
|
||||
const { rawBody, requestId, providerConfig } = ctx
|
||||
if (!providerConfig.webhookSecret) {
|
||||
return null
|
||||
}
|
||||
|
||||
const signature = request.headers.get('Linear-Signature')
|
||||
if (!signature) {
|
||||
logger.warn(`[${requestId}] Linear webhook missing signature header`)
|
||||
return new NextResponse('Unauthorized - Missing Linear signature', { status: 401 })
|
||||
}
|
||||
|
||||
if (!validateLinearSignature(secret, signature, rawBody)) {
|
||||
logger.warn(`[${requestId}] Linear signature verification failed`)
|
||||
return new NextResponse('Unauthorized - Invalid Linear signature', { status: 401 })
|
||||
}
|
||||
const signatureError = await verifyLinearSignature(ctx)
|
||||
if (signatureError) return signatureError
|
||||
|
||||
try {
|
||||
const parsed = JSON.parse(rawBody) as Record<string, unknown>
|
||||
@@ -98,7 +107,7 @@ export const linearHandler: WebhookProviderHandler = {
|
||||
},
|
||||
|
||||
async formatInput({ body }: FormatInputContext): Promise<FormatInputResult> {
|
||||
const b = body as Record<string, unknown>
|
||||
const b = isRecordLike(body) ? body : {}
|
||||
const rawActor = b.actor
|
||||
let actor: unknown = null
|
||||
if (rawActor && typeof rawActor === 'object' && !Array.isArray(rawActor)) {
|
||||
@@ -130,9 +139,9 @@ export const linearHandler: WebhookProviderHandler = {
|
||||
const triggerId = providerConfig.triggerId as string | undefined
|
||||
if (triggerId && !triggerId.endsWith('_webhook') && !triggerId.endsWith('_webhook_v2')) {
|
||||
const { isLinearEventMatch } = await import('@/triggers/linear/utils')
|
||||
const obj = body as Record<string, unknown>
|
||||
const action = obj.action as string | undefined
|
||||
const type = obj.type as string | undefined
|
||||
const obj = isRecordLike(body) ? body : {}
|
||||
const action = typeof obj.action === 'string' ? obj.action : undefined
|
||||
const type = typeof obj.type === 'string' ? obj.type : undefined
|
||||
if (!isLinearEventMatch(triggerId, type || '', action)) {
|
||||
logger.debug(
|
||||
`[${requestId}] Linear event mismatch for trigger ${triggerId}. Type: ${type}, Action: ${action}. Skipping.`
|
||||
@@ -143,6 +152,26 @@ export const linearHandler: WebhookProviderHandler = {
|
||||
return true
|
||||
},
|
||||
|
||||
/**
|
||||
* Fallback for dedup when the `Linear-Delivery` header (already handled generically by the
|
||||
* idempotency service) is unavailable. Keys on the entity id plus its own updatedAt/createdAt,
|
||||
* not a request-time timestamp, so retried deliveries of the same event still collapse.
|
||||
*/
|
||||
extractIdempotencyId(body: unknown): string | null {
|
||||
if (!body || typeof body !== 'object' || Array.isArray(body)) return null
|
||||
|
||||
const b = body as Record<string, unknown>
|
||||
const type = typeof b.type === 'string' ? b.type : undefined
|
||||
const action = typeof b.action === 'string' ? b.action : undefined
|
||||
const data = b.data as Record<string, unknown> | undefined
|
||||
const id = typeof data?.id === 'string' ? data.id : undefined
|
||||
if (!type || !id) {
|
||||
return null
|
||||
}
|
||||
const version = data?.updatedAt || data?.createdAt || b.createdAt
|
||||
return [`linear:${type}`, action, id, version].filter(Boolean).join(':')
|
||||
},
|
||||
|
||||
async createSubscription(ctx: SubscriptionContext): Promise<SubscriptionResult | undefined> {
|
||||
const config = getProviderConfig(ctx.webhook)
|
||||
const triggerId = config.triggerId as string | undefined
|
||||
|
||||
@@ -38,9 +38,12 @@ const ALL_EVENT_FLAGS = {
|
||||
tag_push_events: true,
|
||||
} as const
|
||||
|
||||
// Tag pushes (object_kind 'tag_push') only flow through the all-events trigger;
|
||||
// there is no dedicated single-event trigger for them. A future "GitLab Tag Push"
|
||||
// trigger would need its own object_kind mapping in TRIGGER_OBJECT_KINDS above.
|
||||
/**
|
||||
* Tag pushes (object_kind 'tag_push') only flow through the all-events
|
||||
* trigger; there is no dedicated single-event trigger for them. A future
|
||||
* "GitLab Tag Push" trigger would need its own object_kind mapping in
|
||||
* TRIGGER_OBJECT_KINDS above.
|
||||
*/
|
||||
const TRIGGER_EVENT_FLAGS: Record<string, Record<string, boolean>> = {
|
||||
gitlab_push: { push_events: true },
|
||||
gitlab_merge_request: { merge_requests_events: true },
|
||||
@@ -160,7 +163,8 @@ export function buildGitLabMergeRequestOutputs(): Record<string, TriggerOutput>
|
||||
action: { type: 'string', description: 'Action (open, close, reopen, update, merge, etc.)' },
|
||||
source_branch: { type: 'string', description: 'Source branch' },
|
||||
target_branch: { type: 'string', description: 'Target branch' },
|
||||
merge_status: { type: 'string', description: 'Merge status' },
|
||||
merge_status: { type: 'string', description: 'Merge status (deprecated by GitLab)' },
|
||||
detailed_merge_status: { type: 'string', description: 'Detailed merge status' },
|
||||
draft: { type: 'boolean', description: 'Whether the merge request is a draft' },
|
||||
url: { type: 'string', description: 'Merge request URL' },
|
||||
},
|
||||
@@ -182,6 +186,10 @@ export function buildGitLabIssueOutputs(): Record<string, TriggerOutput> {
|
||||
description: { type: 'string', description: 'Issue description' },
|
||||
confidential: { type: 'boolean', description: 'Whether the issue is confidential' },
|
||||
url: { type: 'string', description: 'Issue URL' },
|
||||
work_item_type: {
|
||||
type: 'string',
|
||||
description: 'Work item type (e.g. Issue, Incident, Task); GitLab 17.2+ only',
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -184,7 +184,7 @@ Return ONLY the Gmail search query, no explanations or markdown.`,
|
||||
description: 'HTML email body',
|
||||
},
|
||||
labels: {
|
||||
type: 'string',
|
||||
type: 'array',
|
||||
description: 'Email labels array',
|
||||
},
|
||||
hasAttachments: {
|
||||
|
||||
@@ -217,62 +217,6 @@ export const userOutputs = {
|
||||
},
|
||||
} as const
|
||||
|
||||
/**
|
||||
* Shared team output schema
|
||||
*/
|
||||
export const teamOutputs = {
|
||||
id: {
|
||||
type: 'string',
|
||||
description: 'Team ID',
|
||||
},
|
||||
name: {
|
||||
type: 'string',
|
||||
description: 'Team name',
|
||||
},
|
||||
key: {
|
||||
type: 'string',
|
||||
description: 'Team key (used in issue identifiers)',
|
||||
},
|
||||
description: {
|
||||
type: 'string',
|
||||
description: 'Team description',
|
||||
},
|
||||
private: {
|
||||
type: 'boolean',
|
||||
description: 'Whether the team is private',
|
||||
},
|
||||
timezone: {
|
||||
type: 'string',
|
||||
description: 'Team timezone',
|
||||
},
|
||||
} as const
|
||||
|
||||
/**
|
||||
* Shared state output schema
|
||||
*/
|
||||
export const stateOutputs = {
|
||||
id: {
|
||||
type: 'string',
|
||||
description: 'State ID',
|
||||
},
|
||||
name: {
|
||||
type: 'string',
|
||||
description: 'State name',
|
||||
},
|
||||
type: {
|
||||
type: 'string',
|
||||
description: 'State type (backlog, unstarted, started, completed, canceled)',
|
||||
},
|
||||
color: {
|
||||
type: 'string',
|
||||
description: 'State color',
|
||||
},
|
||||
position: {
|
||||
type: 'number',
|
||||
description: 'State position in the workflow',
|
||||
},
|
||||
} as const
|
||||
|
||||
/**
|
||||
* Build output schema for issue events
|
||||
*/
|
||||
@@ -1089,12 +1033,10 @@ export function isLinearEventMatch(triggerId: string, eventType: string, action?
|
||||
return false
|
||||
}
|
||||
|
||||
// Check event type
|
||||
if (config.type !== eventType) {
|
||||
return false
|
||||
}
|
||||
|
||||
// Check action if specified
|
||||
if (config.actions && action && !config.actions.includes(action)) {
|
||||
return false
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user