feat(connectors): add 9 knowledge base connectors (#6699)

* feat(connectors): add 9 knowledge base connectors

Box, Zoho Desk, PagerDuty, Trello, Microsoft Excel, Google Slides, Google
Vault, Mintlify, and SFTP. Selected by intersecting the published connector
catalogs of Glean, Onyx, Dust, Vectara, Writer, Guru, Elastic, Microsoft 365
Copilot, Notion AI, Unstructured, and Airbyte against services that already
ship a Sim block, so OAuth providers, credentials, and icons are reused. Box
was the largest gap, appearing in 7-8 of ~10 catalogs.

Every connector was validated against live provider documentation twice, the
second pass treating the first pass's conclusions as unproven. Notable
correctness work that came out of that:

Listing truncation. The sync engine hard-deletes documents past a cap that is
not flagged with `listingCapped`, and five connectors had a path there — an
empty Mintlify discovery, Zoho Desk's exact-multiple default caps, Trello's
archived lists and 1000-card ceiling, a Google Vault cursor bailout, and a
PagerDuty stalled page. The engine also gained a backstop: an empty or
collapsed listing blocks deletion reconciliation until the same observation
repeats on a consecutive sync, reconstructed from existing sync-log counters
so no migration is needed.

API alignment. `desk.zoho.ca` does not resolve (Canada is
`desk.zohocloud.ca`, and Singapore and UAE were missing); `modifiedTime` is
absent from Zoho's ticket list projection, so every ticket re-embedded on
every sync; Trello's `dateLastActivity` is documented to miss some edits;
PagerDuty's 10,000-record ceiling bounds `offset + limit`, not offset; Excel
indexed dates as raw serial numbers while Google Sheets renders them; Google
Vault truncated at roughly 249 matters.

Security. SFTP followed symlinks in `getDocument` and composed unchecked
server-supplied filenames into paths; it now also supports optional host-key
fingerprint verification, which runs during key exchange before any password
is sent. Trello interpolated user-supplied board ids into URL paths. Google
Vault is narrowed to `ediscovery.readonly`. `getDataverseBaseUrl` accepted
any host while attaching a bearer token, and is pinned to Microsoft's
Dataverse domains — pre-existing shipped code, fixed here.

Also adds `ConnectorAuthConfig.optional` so a public source can be configured
without inventing an API key, and teaches the scope check that a granted
read-write scope satisfies a required `.readonly` sibling.

Microsoft Dataverse was built and then removed: its OAuth cannot complete
consent. Dataverse requires a per-environment resource URI, the provider
declares a static `https://dynamics.microsoft.com/user_impersonation` that is
not an Entra Application ID URI, and the environment URL is only collected
after the credential exists. That predates this change and also affects the
12 shipped Dataverse tools.

* fix(dataverse): strip the bearer token when a request redirects

The host allowlist added alongside the connector work only constrains the
initial destination. `secureFetchWithPinnedIP` follows redirects and keeps the
`Authorization` header unless a tool opts out, so a redirect away from an
allowed Dataverse origin would forward the caller's OAuth token to whatever
host answers. Dataverse redirects in normal operation — file downloads hand
back a signed storage URL, and environment hosts move between regional
origins — so this is reachable without a compromised environment URL.

Sets `stripAuthOnRedirect` on all 18 Dataverse tools, matching the existing
GitHub job-logs and Windchill precedent.

* fix(connectors): address review findings on listing and hashing

- microsoft-excel: `fetchWorksheets` read only the first Graph page and never
  followed `@odata.nextLink`. A workbook with more sheets than fit in one page
  dropped the remainder from the listing without setting `listingCapped`, so
  the sync engine reconciled those documents away as deleted. The walk now
  pages, bounded by MAX_WORKSHEETS, and only follows a nextLink that stays on
  the Graph origin, since the link is server-supplied and carries the token.

- google-slides: the listing `contentHash` covered only the file id and
  modified time, so toggling the speaker-notes option left every stored hash
  matching and no presentation was ever re-hydrated with the new scope. The
  setting is now part of the hash, in the single shared stub builder so the
  list and hydrate paths stay identical.

- mintlify: `pathPrefix` filtered with a bare `startsWith`, so a prefix of
  `/guides` also matched a sibling like `/guides-archive`. It now shares the
  `/`-boundary rule `withinBasePath` already used, extracted as `isUnderPath`.

* fix(connectors): list newest first in zoho desk, accept a trailing slash prefix

- zoho-desk: `sortBy: 'createdTime'` is ascending — Zoho denotes descending
  with a `-` prefix — so the default 500-record caps kept the oldest tickets
  and articles and recent ones were never listed. Because the cap sets
  listingCapped, that stale tail could not reconcile away either. Now sorts
  `-createdTime`. Still ordering on createdTime rather than modifiedTime, so
  rows do not reshuffle mid-walk.

- mintlify: `resolvePathPrefix` kept a trailing slash while `isUnderPath`
  accepts an exact match or `prefix + '/'`, so `/guides/` matched neither
  `/guides` nor `/guides/intro` and the source synced nothing. A regression
  from the previous round, which replaced a bare `startsWith`. The prefix is
  now normalized before comparison.

* fix(dataverse): strip the bearer token on the upload route's own redirect

`upload_file` posts to an internal route rather than calling Dataverse
directly, so the tool-level `stripAuthOnRedirect` added in 903c94e9 only
covers the same-origin hop into that route. The route's own outbound PATCH
carries the caller's OAuth token and left redirect stripping at its default,
so a redirect to a signed storage host — which is exactly how Dataverse
serves file operations — would have handed that host a reusable credential.

The other 17 tools build the Dataverse URL directly, so the tool-level flag
already covers them.
This commit is contained in:
Waleed
2026-08-14 14:14:43 -07:00
committed by GitHub
parent cf78946529
commit 5bb59f08ee
60 changed files with 7593 additions and 31 deletions
@@ -14,23 +14,24 @@ Connectors continuously sync documents from external services into your knowledg
<Image src="/static/connectors/connectors-sources.png" alt="Connect Source picker showing a searchable list of available connectors including Airtable, Asana, Confluence, Discord, Dropbox, Evernote, Fireflies, GitHub, and Gmail" width={800} height={500} />
Sim ships with 49 built-in connectors:
Sim ships with 61 built-in connectors:
| Category | Connectors |
|----------|-----------|
| **Productivity** | Notion, Confluence, Asana, Linear, Jira, Jira Service Management, Monday, Google Calendar, Google Sheets, Google Forms, Typeform |
| **Cloud Storage** | Google Drive, Dropbox, OneDrive, SharePoint, Amazon S3 |
| **Documents** | Google Docs, WordPress, Webflow, DocuSign |
| **Productivity** | Notion, Confluence, Asana, Linear, Jira, Jira Service Management, Monday, Trello, ClickUp, Google Calendar, Google Sheets, Google Forms, Microsoft Excel, Typeform |
| **Cloud Storage** | Google Drive, Dropbox, OneDrive, SharePoint, Box, Amazon S3, SFTP |
| **Documents** | Google Docs, Google Slides, Mintlify, WordPress, Webflow, DocuSign |
| **Development** | GitHub, GitLab, Azure DevOps, Sentry |
| **Communication** | Slack, Discord, Microsoft Teams, Reddit, YouTube |
| **Communication** | Slack, Discord, Microsoft Teams, Reddit, X, YouTube |
| **Email** | Gmail, Outlook |
| **CRM** | HubSpot, Salesforce |
| **Support** | Intercom, ServiceNow, Zendesk |
| **Incident Management** | incident.io, Rootly |
| **Support** | Intercom, ServiceNow, Zendesk, Zoho Desk |
| **Incident Management** | incident.io, Rootly, PagerDuty |
| **Data** | Airtable |
| **Note-taking** | Evernote, Obsidian |
| **Meetings** | Zoom, Gong, Grain, Granola, Fathom, Fireflies |
| **Meetings** | Zoom, Google Meet, Gong, Grain, Granola, Fathom, Fireflies |
| **Recruiting** | Greenhouse, Ashby |
| **Compliance** | Google Vault |
## Adding a Connector
@@ -55,6 +56,9 @@ Other connectors use **API keys** or **personal access tokens** instead. The set
| **YouTube** | YouTube Data API key from the Google Cloud Console |
| **Amazon S3** | Secret Access Key (the Access Key ID, region, and bucket are entered as config fields) |
| **Sentry** | Auth token with `project:read` and `event:read` scopes |
| **PagerDuty** | REST API key from Integrations → API Access Keys |
| **SFTP** | Password or unencrypted private key (host, port, username, and root path are entered as config fields) |
| **Mintlify** | API key — optional for public documentation sites, which sync from `llms.txt` |
<Callout type="info">
If you rotate an API key in the external service, update it in Sim as well — OAuth tokens refresh automatically, but API keys do not.