* feat(integrations): add ClickHouse block and expand Dagster + Tinybird tools
* fix(tinybird): fail loudly on invalid query_pipe parameters JSON
parsePipeParameters previously returned {} on any JSON parse error, so a
mistyped 'parameters' input produced a successful pipe call with the dynamic
filters silently dropped. Throw a clear error for non-empty, non-object input
instead; an omitted/empty value still means 'no parameters'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(dagster): guard NaN numeric coercions and bound list_assets pagination
Address PR review:
- Route all block numeric coercions (list_runs limit/createdAfter/createdBefore,
get_run_logs logsLimit, list_assets assetsLimit) through a toFiniteNumber()
guard so invalid/wand-generated text becomes undefined instead of NaN.
- list_assets now applies a default page size (100) when no limit is given, so
paging stays bounded and hasMore is meaningful even when limit is omitted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(dagster): make list_assets hasMore exact via fetch N+1
Address PR review (hasMore true on exact page): request one extra row
(pageSize + 1), use its presence as the authoritative hasMore, slice it off,
and derive the returned cursor from the last RETURNED asset's key path
(JSON-serialized; Dagster normalizes JS/Python whitespace on the way in).
This removes the false-positive hasMore when the final page is exactly full.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(clickhouse): enforce read-only query operation and harden WHERE-clause guard
* fix(dagster): make list_runs hasMore exact via fetch N+1
Address PR review (list runs false hasMore): request one extra row
(pageSize + 1), use its presence as the authoritative hasMore, and slice it
off before mapping. Removes the false-positive hasMore (and misleading cursor)
when the final page is exactly `limit` runs long. Mirrors the list_assets fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(clickhouse): restrict DROP PARTITION to literal values to prevent SQL injection
* fix(clickhouse): reject chained statements in read-only query operation
* fix(clickhouse): force JSON output on query path and ignore comments when detecting chained statements
* fix(tinybird): encode datasource/pipe names in URL paths to prevent traversal
A user-or-llm datasource/pipe name interpolated raw into the URL path (e.g.
'real_ds/../../other') is normalized by the WHATWG URL parser and can target a
different endpoint. Wrap the path segment with encodeURIComponent in the
truncate, delete, and query_pipe URLs. Events/append pass the name via
URLSearchParams, which already encodes, so they were unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(clickhouse): block WITH-led writes/DDL in read-only query operation
* fix(clickhouse): validate column types structurally and normalize FORMAT around SETTINGS
* fix(clickhouse): balance-check ORDER BY/PARTITION BY and skip leading comments in read-only guard
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Block Documentation Generator
This directory contains scripts to automatically generate documentation for all blocks in the Sim platform.
Available Scripts
generate-docs.ts: Generates documentation for all blocks. Run viabun run generate-docsfromapps/sim, or directly withbun run scripts/generate-docs.tsfrom the repo root.
How It Works
The documentation generator:
- Scans the
apps/sim/blocks/blocks/directory for all block definition files - Extracts metadata from each block including:
- Name, description, and category
- Input and output specifications
- Configuration parameters
- Generates standardized Markdown documentation for each block
- Updates the navigation metadata in
meta.json
Running the Generator
# From the repo root
bun run scripts/generate-docs.ts
Dependencies are managed by Bun workspaces — bun install at the repo root installs everything needed.
CI Integration
The documentation generator runs automatically as part of the CI/CD pipeline whenever changes are pushed to the main branch. The updated documentation is committed back to the repository.
Adding Support for New Block Properties
If you add new properties to block definitions that should be included in the documentation, update the generateMarkdownForBlock function in scripts/generate-docs.ts.
Preserving Manual Content
The documentation generator now supports preserving manually added content when regenerating docs. This allows you to enhance the auto-generated documentation with custom examples, additional context, or any other content without losing your changes when the docs are regenerated.
How It Works
- The generator creates clean documentation without any placeholders or markers
- If you add manual content to a file using special comment markers, that content will be preserved during regeneration
- The manual content is intelligently inserted at the appropriate section when docs are regenerated
Using Manual Content Markers
To add custom content to any tool's documentation, insert MDX comment blocks with section markers:
{/_ MANUAL-CONTENT-START:sectionName _/}
Your custom content here (Markdown formatting supported)
{/_ MANUAL-CONTENT-END _/}
Replace sectionName with one of the supported section names:
intro- Content at the top of the document after the BlockInfoCardusage- Additional usage instructions and examplesconfiguration- Custom configuration detailsoutputs- Additional output information or examplesnotes- Extra notes at the end of the document
Example
To add custom examples to a tool doc:
{/_ MANUAL-CONTENT-START:usage _/}
## Examples
### Basic Usage
```json
{
"parameter": "value",
"anotherParameter": "anotherValue"
}
```
Advanced Configuration
Here's how to use this tool for a specific use case... {/_ MANUAL-CONTENT-END _/}
When the documentation is regenerated, your manual content will be preserved in the appropriate section automatically. The script will not add any placeholders or markers to files by default.