Waleed 3848f97b4c fix(grafana): validate against the API docs, add data source querying and contact-point CRUD (#6712)
* fix(azure-data-explorer): correct the tags ingestion-property example

The example rendered as tags="[''daily'']" — doubled single quotes from an
escaping slip, which is not valid Kusto. The reference writes a tags list
as tags='["TagA","TagB"]': single outer quotes with the JSON array's own
double quotes inside.

The clause builder already handled that form; only the example text was
wrong. A template literal avoids the escaping entirely, since the metadata
generator reads the source verbatim and would otherwise carry the
backslashes into the description the model sees.

Adds a test asserting the reference's exact multi-property clause
round-trips, including the comma inside the quoted array.

* fix(grafana): correct response contracts, required alert fields, and outbound request hardening

Validated against Grafana's HTTP API reference and, where the docs
contradict themselves, against the Go wire structs.

Response shapes the tools got wrong:
- update_annotation declared an `id` that was always 0; a patch returns only
  a message, so the request's annotation id is echoed and labelled as such
- delete_folder discarded the numeric id Grafana returns and presented an
  input-echoed uid as if it came from the API
- delete_dashboard fabricated `id: 0` / `title: ''` via `||` on absent fields
- the contact-point `provenance` description was inverted: "api" means
  API-managed, empty means it stayed UI-editable

Requests that could not succeed:
- create_alert_rule left noDataState and execErrState unset and invisible to
  the model, but Grafana's validator rejects an empty value outright, so every
  model-driven create failed. Both are now sent with Grafana's own defaults,
  and skipped for recording rules, which take a different validator
- get_data_source routed a numeric input at /api/datasources/:id, which exists
  only behind an off-by-default feature toggle. UID only now
- list_annotations did not trim the dashboard UID, so a padded value matched
  nothing

Outbound hardening on the three proxy routes:
- the service-account token was re-sent to redirect targets; the shared fetch
  only drops it when asked, so stripAuthOnRedirect is now set
- no timeout was passed, leaving two sequential hops at the 5-minute default
- upstream error bodies were interpolated whole into the tool result, putting
  up to 10MB of HTML into logs and traces; now truncated
- UID path segments are URL-encoded so they cannot re-target the request
- update_folder sent both `version` and `overwrite: true`, which Grafana treats
  as alternatives, making the freshly fetched version decorative and silently
  clobbering a concurrent rename
- replaced the `any` casts with narrowed types

Block surface:
- 25 outputs the tools emit were undeclared and so unreferenceable downstream;
  get_data_source had 13 of its 18 unreachable
- `version` was typed string though the dashboard, folder, and data-source
  producers all emit a number
- the dashboard title field was shown only for create, so a dashboard could
  never be renamed through Update Dashboard
- six list outputs were typed json rather than array

* fix(grafana): let the health check report ill-health, and disambiguate block outputs

The data source health check could only ever report health. Grafana answers an
unhealthy source with HTTP 400 carrying the same {status, message} payload as a
healthy one, and the tool framework converts any non-2xx into an opaque tool
error — so the diagnostic the caller actually wants was unreachable. The check
now goes through an internal route that reads the verdict off either status and
reports it as a successful check, while a failure carrying no verdict (missing
data source, bad token, plugin with no health endpoint) stays a real error. The
plugin's `details` payload is surfaced too.

Also on that route, matching the other three: an outbound timeout, redirect
auth stripping, a truncated upstream error, and a URL-encoded UID.

Block output descriptions: ten keys are emitted by several tools with different
meanings and were described for only one producer — `database` meant both a
data source name and a health status, `annotations` both an annotation list and
an alert rule's summary map. Eleven `json` outputs were opaque although the
tools already document their inner fields. All rewritten to name every producer.

Smaller alignment fixes:
- the same EmbeddedContactPoint.settings field was typed `object` in list and
  `json` in create
- list_contact_points mapped non-nullable uid/name/type through `?? null`;
  Grafana returns an empty string, which is what create already assumed
- create_alert_rule sent `orgID`, which Grafana overwrites from the
  authenticated context, and `Number()` on a non-numeric value put NaN -> null
  in the body
- the three update routes declared `output` as required though the auth
  short-circuit omits it, and did not declare the `details` they emit on a
  validation error

* feat(grafana): complete contact-point CRUD, and add folder move and rule-group read

Four operations the integration was missing, taking it to 29.

update_contact_point / delete_contact_point close a real gap: contact points
could be listed and created but never corrected or removed. Two things worth
recording, because the published docs get both wrong:

- both verbs answer 202 with only a message, not the object. The rendered docs
  claim delete returns 204; the current spec and handler both say 202. So the
  UID is echoed from the request, the way delete_folder and update_annotation
  already do
- update is a full replace with no PATCH counterpart, so name, type, and
  settings are all required and the description says so. Omitting
  disableResolveMessage resets it

X-Disable-Provenance is exposed on update only. Its polarity is the opposite of
the alert-rule case: omitting it always succeeds, while sending it against an
API-provisioned contact point is rejected — with 403, not the 409 rules use. It
is not exposed on delete at all, because that handler never reads stored
provenance and the endpoint takes no such parameter.

move_folder reuses get_folder's mapping verbatim — same DTO. It always sends
the parentUid key, since Grafana reads an empty value as "move to the root",
which a conditionally-omitted field could not express.

get_alert_rule_group surfaces the group evaluation interval, the one alerting
knob the per-rule operations cannot reach. It reuses the shared mapAlertRule for
the nested rules, and the interval is documented as an integer of seconds.

* feat(grafana): add data source querying, and ground the skill and templates in real tools

query_data_source closes the largest gap in the integration: 29 tools could
read dashboards, folders, and alert configuration, but none could read a metric
value. It posts to /api/ds/query and returns both the raw response and the
frames flattened into rows.

The flattening is derived from the documented layout rather than any data
source's field names: a frame carries schema.fields[] alongside data.values[],
where values[i] is the whole column for fields[i], so zipping them by position
works for Prometheus, SQL, or anything else with a backend.

A failed query is a 400 by Grafana's own status table, so it stays a tool
error — unlike the health check, where the failure status carries the answer.

That also lets four templates and the review-firing-alerts skill stop promising
things the integration could not do. Three templates assumed a metric-query
tool, which now exists. The fourth, and the skill, assumed live alert instance
state, which the provisioning API never returns — they now derive firing rules
from alert-state annotations, which are documented to carry newState and
prevState, and say so explicitly rather than implying a live snapshot.

Deliberately not added: a tool over /api/prometheus/grafana/api/v1/rules for
live instance state. That endpoint appears on no Grafana HTTP API doc page, its
response is only readable from Go internals and test assertions, and the
instance-level state casing differs from the rule level with no documented
contract. Not something to build an output schema on.

* fix(grafana): declare the two block outputs the earlier fixes introduced

Renaming update_annotation's phantom `id` to `annotationId` and adding
`details` to the health check both created outputs the block never declared, so
neither was referenceable downstream. Caught by re-running the output-coverage
check over both integrations; the block now covers all 64 keys the 30 tools emit.

* fix(grafana): make Update Contact Point actually usable from the block

The new replace operation could never succeed. contactPointType and
contactPointSettings were widened to cover it, but contactPointNameNew was
left create-only — and the update maps `name` from that field, so the required
parameter was never supplied.

disableResolveMessage had the same gap, and it matters more than it looks:
the update is a full replace, so a block-driven update was silently clearing
resolve suppression on every contact point it touched. Both fields are now
shown, and required where the API requires them.

Also states a reason on each intentionally-unconstrained response field —
Zod issue objects, alert query stages, notification settings, recording-rule
config, and data-source health detail are all genuinely opaque, but that was
left implicit.
2026-08-14 15:58:17 -07:00

Sim.ai Documentation Slack X

Ask DeepWiki Set Up with Cursor

Sim — Integrate, Context, Build, and Monitor AI agents

A workspace to build, deploy and manage AI agents and workflows.

Quickstart

Cloud-hosted: sim.ai

Open sim.ai

Self-hosted

git clone https://github.com/simstudioai/sim.git && cd sim
bun install
bun run setup

Open http://localhost:3000

The Sim platform — chat on the left, the visual workflow builder on the right

Capabilities

  • Connect 1,000+ integrations and every major LLM
  • Add Slack, Notion, HubSpot, Salesforce, databases, and more
  • Build agents visually, conversationally, or with code
  • Ingest files, knowledge bases, and structured table data
  • Monitor runs, logs, schedules, and workflow activity

One workspace, every surface

Chat and workflows are just the start — tables, files, and knowledge all live in the same workspace.

Tables in Sim — structured data your agents can query

Tables — a database, built in

Files in Sim — documents for your team and every agent

Files — one store for your team and every agent

Knowledge bases in Sim — synced docs your agents can search

Knowledge — your agents' memory

Self-hosting

Requirements: Bun and Docker.

bun run setup is an interactive wizard: it provisions the database, generates secrets, writes your .env files, connects a Chat API key, and starts Sim the way you choose:

  • Local dev — run from source to contribute or hack on Sim
  • Docker Compose — a self-contained instance for testing self-hosting
  • Kubernetes (Helm) — deploy to a local cluster

When it finishes, open http://localhost:3000.

Reconfigure an optional capability without rerunning the full wizard:

bun run setup status
bun run setup email
bun run setup storage
bun run setup sandbox
bun run setup jobs
bun run setup cache
bun run setup knowledge
bun run setup llm
bun run setup integration slack

bun run setup status detects the effective local-dev, Docker Compose, or current-context Helm configuration and reports configured, missing, or invalid capabilities and OAuth integrations without printing credential values. This is separate from bun run sim status, which reports whether installed services are running and healthy.

Manage your install with bun run sim:

bun run sim start | stop | restart   # bring your install up / down / cycle
bun run sim update                   # pull/rebuild and apply Compose images
bun run sim status                    # what's installed and healthy
bun run sim logs                      # follow logs
bun run sim doctor                    # diagnose configuration problems
bun run sim down                      # remove containers (data kept)
bun run sim reset                     # archive .env and wipe managed data

sim detects how you're running (Docker Compose, local dev, or Kubernetes) and acts accordingly.

Prefer a bare sim? Run bun link once — but note sim lands in ~/.bun/bin, which Homebrew's bun doesn't add to your PATH, so you may need export PATH="$HOME/.bun/bin:$PATH" in your shell profile.

Sim also supports local models via Ollama and vLLM. See the self-hosting docs for details.

Chat API Keys

Chat is a Sim-managed service. bun run setup connects a Chat API key for you — sign in when it opens your browser and the key is stored automatically. To view, create, or revoke keys later, go to sim.ai/selfhost/settings/chat-keys.

Environment Variables

See the environment variables reference for the full list, or apps/sim/.env.example for defaults.

Tech Stack

Next.js · Bun · PostgreSQL · Drizzle · Better Auth · Tailwind — and the rest of the stack

Contributing

We welcome contributions! Please see our Contributing Guide for details.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Built by the Sim team in San Francisco

Languages
TypeScript 77%
MDX 20.8%
JavaScript 1.9%
CSS 0.1%