* fix(uploads): direct-to-S3 workspace files + shared transport
* chore(testing): centralize posthog and storage-service mocks
* fix(uploads): address PR review — abort propagation, orphan cleanup, error handling
- Throw immediately on AbortError in KB retry loop (no useless 14s backoff)
- Cleanup S3/Blob object on quota or size-cap rejection in registerUploadedWorkspaceFile
- Enforce MAX_WORKSPACE_FILE_SIZE at registration (defense vs presigned PUT lying about size)
- Handle non-OK / non-JSON responses in workspace-files upload paths
* fix(uploads): add Zod contracts for workspace presigned/register routes
* fix(uploads): correct BlobServiceClient type name in headBlobObject
* fix(uploads): address PR review — typo, complete-failure cleanup, double-increment
* fix(uploads): preserve fallback size and reuse existing display name on re-register
* fix(uploads): surface server error message and bypass quota for local-storage fallback
* fix(uploads): align register response schema with UserFile; skip presigned for KB large files
- registerWorkspaceFileResponseSchema now matches the UserFile shape the route actually returns; previous schema required workspace DB-row fields that were never populated, causing requestJson validation to reject successful uploads.
- KB batch presigned fetch now skips files >= LARGE_FILE_THRESHOLD since multipart bypasses the per-file presigned URL anyway.
* fix(uploads): idempotent register skips duplicate audit/posthog; add edge-case tests
- registerUploadedWorkspaceFile now returns { file, created } so the route can skip captureServerEvent and recordAudit on idempotent re-register (existing metadata reused). Previously a re-register fired duplicate analytics + audit log entries.
- Add tests covering: idempotent re-register skips audit/analytics, isNetworkError matches econnreset/timeout/etc keywords, multipart complete failure fires action=abort cleanup.
* fix(uploads): include 50MiB boundary in batch presigned fetch
* fix(uploads): trust HEAD size to prevent quota inflation
The head.size > 0 fallback let a client PUT 0 bytes and register
with an inflated size, debiting quota without storing data. HEAD
on an existing object always returns the true byte count, so trust
it directly — a genuine 0-byte file correctly contributes 0.
* fix(uploads): audit verified file size, not client-supplied
* fix(uploads): handle register retries and name-collision races
Two bugs in registerUploadedWorkspaceFile:
1. Register retry could orphan storage. When a successful response
was lost on the wire and the client retried, the quota check saw
the bytes already counted, failed, and cleanupOrphan deleted the
already-registered storage object — leaving the DB row pointing
to nothing. Fix: check getFileMetadataByKey before quota guard
and short-circuit on existing record.
2. Concurrent same-named uploads could lose data. allocateUniqueWorkspaceFileName
is best-effort; two racing uploads can pass it and both attempt
the same display name. The loser's insert hits 23505, the catch
block called cleanupOrphan, and successfully-uploaded bytes
were deleted. Fix: retry on 23505 with a fresh allocateUniqueWorkspaceFileName,
matching the pattern in uploadWorkspaceFile. Throw FileConflictError
after exhaustion.
* fix(uploads): retry transient DirectUploadErrors at outer KB level
The KB outer retry only triggered on isNetworkError, missing
transient 5xx from S3/Azure (DirectUploadError code
DIRECT_UPLOAD_ERROR or MULTIPART_ERROR). Adds isTransientUploadError
and retries on it, restoring resilience for small-file presigned
PUTs against flaky cloud storage.
* fix(uploads): only retry transient 5xx, not deterministic 4xx
DirectUploadError now carries the HTTP status. isTransientUploadError
gates on 5xx so callers don't loop on 400/403/404 (e.g., malformed
request, expired signature). Multipart per-part retry also short-circuits
on 4xx — same reasoning.
* refactor(uploads): collapse getFileContentType into resolveFileType
The two helpers differed only in whether application/octet-stream
falls back to the extension map. Add an option flag to resolveFileType
and keep getFileContentType as a thin wrapper for direct-PUT callers
that need to preserve the exact browser-reported content-type.
* chore(uploads): trim verbose comments
Drop inline comments that restate code ("Use the full storageKey as fileName"),
collapse a multi-line block comment into a tighter TSDoc on the existence
check, and prune verbose vitest file headers — describe blocks already
document what's tested.
* fix(uploads): regenerate fileId per insert retry; require cloud storage for register
* fix(uploads): cap formdata fallback at 100MB; drop unused size param
* fix(uploads): abort multipart on get-part-urls failure; retry register on transient errors
* fix(uploads): drop vestigial size field from register contract
* fix(uploads): abort multipart on complete-fetch throw
* fix(uploads): set kb presignedEndpoint fallback; race-safe blob HEAD
* fix(uploads): include ?type=knowledge-base on kb presigned fallback
* fix(uploads): remove abort listener on xhr timeout
* fix(uploads): add timeout/abort to kb api fallback upload
Block Documentation Generator
This directory contains scripts to automatically generate documentation for all blocks in the Sim platform.
Available Scripts
generate-docs.sh: Generates documentation for all blockssetup-doc-generator.sh: Installs dependencies required for the documentation generator
How It Works
The documentation generator:
- Scans the
apps/sim/blocks/blocks/directory for all block definition files - Extracts metadata from each block including:
- Name, description, and category
- Input and output specifications
- Configuration parameters
- Generates standardized Markdown documentation for each block
- Updates the navigation metadata in
meta.json
Running the Generator
To generate documentation manually:
# From the project root
./scripts/generate-docs.sh
Troubleshooting TypeScript Errors
If you encounter TypeScript errors when running the documentation generator, run the setup script to install the necessary dependencies:
./scripts/setup-doc-generator.sh
This will:
- Install TypeScript, ts-node, and necessary type definitions
- Create a proper tsconfig.json for the scripts directory
- Configure the scripts directory to use ES modules
Common Issues
- Missing Type Declarations: Run the setup script to install @types/node and @types/react
- JSX Errors in block-info-card.tsx: These don't affect functionality and can be ignored if you've run the setup script
- Module Resolution: The setup script configures proper ES module support
CI Integration
The documentation generator runs automatically as part of the CI/CD pipeline whenever changes are pushed to the main branch. The updated documentation is committed back to the repository.
Adding Support for New Block Properties
If you add new properties to block definitions that should be included in the documentation, update the generateMarkdownForBlock function in scripts/generate-docs.ts.
Preserving Manual Content
The documentation generator now supports preserving manually added content when regenerating docs. This allows you to enhance the auto-generated documentation with custom examples, additional context, or any other content without losing your changes when the docs are regenerated.
How It Works
- The generator creates clean documentation without any placeholders or markers
- If you add manual content to a file using special comment markers, that content will be preserved during regeneration
- The manual content is intelligently inserted at the appropriate section when docs are regenerated
Using Manual Content Markers
To add custom content to any tool's documentation, insert MDX comment blocks with section markers:
{/_ MANUAL-CONTENT-START:sectionName _/}
Your custom content here (Markdown formatting supported)
{/_ MANUAL-CONTENT-END _/}
Replace sectionName with one of the supported section names:
intro- Content at the top of the document after the BlockInfoCardusage- Additional usage instructions and examplesconfiguration- Custom configuration detailsoutputs- Additional output information or examplesnotes- Extra notes at the end of the document
Example
To add custom examples to a tool doc:
{/_ MANUAL-CONTENT-START:usage _/}
## Examples
### Basic Usage
```json
{
"parameter": "value",
"anotherParameter": "anotherValue"
}
```
Advanced Configuration
Here's how to use this tool for a specific use case... {/_ MANUAL-CONTENT-END _/}
When the documentation is regenerated, your manual content will be preserved in the appropriate section automatically. The script will not add any placeholders or markers to files by default.