diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 16aecf2983..6b76fc708e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,6 +15,7 @@ Thank you for your interest in contributing to Sim Studio! Our goal is to provid - [Commit Message Guidelines](#commit-message-guidelines) - [Local Development Setup](#local-development-setup) - [License](#license) +- [Adding New Blocks and Tools](#adding-new-blocks-and-tools) --- @@ -162,4 +163,152 @@ This project is licensed under the MIT License. By contributing, you agree that --- +## Adding New Blocks and Tools + +Sim Studio is built in a modular fashion where blocks and tools extend the platform's functionality. To maintain consistency and quality, please follow the guidelines below when adding a new block or tool. + +### Where to Add Your Code + +- **Blocks:** Create your new block file under the `/blocks/blocks` directory. +- **Tools:** Create your new tool file under the `/tools` directory. + +In addition, you will need to update the registries: + +- **Block Registry:** Update the blocks index (usually `/blocks/index.ts`) to include your new block. +- **Tool Registry:** Update the tools registry (`/tools/index.ts`) to add your new tool. + +### How to Create a New Block + +1. **Create a New File:** + Create a file for your block (e.g., `newBlock.ts`) in the `/blocks/blocks` directory. + +2. **Create a New Icon:** + Create a new icon for your block in the `/components/icons.tsx` file. + +3. **Define the Block Configuration:** + Your block should export a constant of type `BlockConfig`. For example: + + ```typescript:blocks/blocks/newBlock.ts + import { SomeIcon } from '@/components/icons' + import { BlockConfig } from '../types' + + // Define response type if needed + interface NewBlockResponse { + output: { + // Define expected output here + result: string + } + } + + export const NewBlock: BlockConfig = { + type: 'new', + name: 'New Block', + description: 'Description of the new block', + category: 'blocks', + bgColor: '#123456', + icon: SomeIcon, + inputs: { + // Define inputs here + exampleInput: { type: 'string', required: true }, + }, + outputs: { + response: { + type: { + result: 'string', + }, + }, + }, + } + ``` + +4. **Register Your Block:** + Import and add your block to the blocks registry (`blocks/index.ts`) in the appropriate index file so it appears in the workflow builder. + +5. **Test Your Block:** + Ensure that the block displays correctly in the UI and that its functionality works as expected. + +### How to Create a New Tool + +1. **Create a New File:** + Create a file for your tool (e.g., `newTool.ts`) in the `/tools` directory. + +2. **Define the Tool Configuration:** + Your tool should export a constant of type `ToolConfig`. For example: + + ```typescript:tools/newTool.ts + import { ToolConfig, ToolResponse } from './types' + + interface NewToolParams { + apiKey: string + query: string + } + + interface NewToolResponse extends ToolResponse { + output: { + result: string + } + } + + export const newTool: ToolConfig = { + id: 'new_tool', + name: 'New Tool', + description: 'Description for the new tool', + params: { + apiKey: { type: 'string', required: true }, + query: { type: 'string', required: true }, + }, + request: { + url: 'https://api.example.com/query', + method: 'POST', + headers: (params) => ({ + 'Content-Type': 'application/json', + Authorization: `Bearer ${params.apiKey}`, + }), + body: (params) => JSON.stringify({ query: params.query }), + }, + transformResponse: async (response: Response) => { + const data = await response.json() + return { + success: true, + output: { result: data.result }, + } + }, + transformError: (error) => { + return error.message || 'An error occurred while processing the tool request' + }, + } + ``` + +3. **Register Your Tool:** + Update the tools registry in `/tools/index.ts` to include your new tool. For example, add it to the exported `tools` object: + + ```typescript:tools/index.ts + import { newTool } from './newTool' + // ... other imports + + export const tools: Record = { + // ... existing tools + new_tool: newTool, + } + + export function getTool(toolId: string): ToolConfig | undefined { + return tools[toolId] + } + ``` + +4. **Test Your Tool:** + Ensure that your tool functions correctly by making test requests and verifying the responses. + +### Guidelines & Best Practices + +- **Code Style:** Follow the project's ESLint and Prettier configurations. Use meaningful variable names and small, focused functions. +- **Documentation:** Clearly document the purpose, inputs, outputs, and any special behavior for your block/tool. +- **Error Handling:** Implement robust error handling and provide user-friendly error messages. +- **Testing:** Add unit or integration tests to verify your changes when possible. +- **Commit Changes:** Update all related components and registries, and describe your changes in your pull request. + +Happy coding! + +--- + Thank you for taking the time to contribute to Sim Studio. We truly appreciate your efforts and look forward to collaborating with you! diff --git a/instructions.txt b/instructions.txt new file mode 100644 index 0000000000..8219882809 --- /dev/null +++ b/instructions.txt @@ -0,0 +1,239 @@ +Below is a high-level implementation guide for syncing our workflow state from localStorage (and our Zustand stores) to our Supabase/Postgres database using Drizzle ORM. We will combine three approaches: +1. Debounced Sync: +When the workflow state changes (e.g. blocks, edges, loops, etc.), we debounce the update so that many small changes are batched together into a single write. This minimizes rapid consecutive writes. +2. Periodic Sync (Auto-save): +We set up a timer (for example, every 30 seconds) to ensure that the state is synced—even if the debounce did not trigger (say, because changes stopped for a while). +3. Critical Events Sync (e.g. BeforeUnload): +We add an event listener (such as on beforeunload) to flush any unsaved changes when the user navigates away or closes the tab. +> Note on Security: +> To keep our implementation open source friendly and secure, we will not expose any database credentials or secrets on the client side. Instead, we’ll create an API endpoint (using Next.js API Routes) that will perform the actual database update using our Drizzle ORM connection. This pattern keeps our sensitive configuration on the server and ensures our client code only makes secure HTTP requests. We’re also careful to associate workflow records with authenticated users (via Better Auth) once we have the user flow in place. +--- + +Step-by-Step Plan +1. Define the Database Schema (if needed) +If you haven’t yet created a table to store workflows, you should create one via a migration. A possible table could look like: +Table Name: workflows +Columns: +id (text, primary key) – the workflow ID +user_id (text) – to associate with the current user (via Better Auth) +state (JSONB) – a JSON column that includes the parts of your state (e.g. blocks, edges, loops, lastSaved, etc.) +updated_at and created_at (timestamps) +> We’re using Supabase/Postgres, so ensure you create the migration and run it using Drizzle’s migration tools (or Supabase dashboard). + + +2. Create a DB Endpoint for Workflow Sync +Because the client must not talk directly to the database, create an API route (e.g. /api/workflows/sync) that accepts a workflow state update. This API route will: +Validate the incoming data. +Check the authenticated user (so that workflows are attached to a user). +Use Drizzle ORM to upsert (insert or update) the workflow row. +A simplified version might look like this: +``` +import { NextResponse } from 'next/server' +import { db } from '@/db' +import { workflow } from '@/db/schema' +import { z } from 'zod' + +// Define the schema for incoming data +const WorkflowSyncSchema = z.object({ + id: z.string(), + userId: z.string(), + state: z.any(), +}) + +export async function POST(request: Request) { + try { + const body = await request.json() + const { id, userId, state } = WorkflowSyncSchema.parse(body) + + // Upsert the workflow (using your preferred upsert method) + await db.insert(workflow).values({ id, userId, state, updatedAt: new Date() }) + .onConflictDoUpdate({ + target: [workflow.id], + set: { state, updatedAt: new Date() }, + }) + + return NextResponse.json({ success: true }) + } catch (error) { + console.error('Workflow sync error:', error) + return NextResponse.json({ error: 'Sync failed' }, { status: 500 }) + } +} +``` +> Security note: +> Ensure that authentication middleware (e.g. Better Auth) protects this endpoint. Do not trust client-sent user IDs without verification. + +3. Implement the Client-Side Sync Functions +a. Debounced Sync Hook +Create a custom hook (for example, useDebouncedWorkflowSync) that watches your workflow state (using Zustand selectors) and uses a debounce function (like the one from lodash.debounce) to call your API endpoint. +``` +import { useEffect } from 'react' +import debounce from 'lodash.debounce' +import { useWorkflowStore } from '@/stores/workflow/store' +import { useWorkflowRegistry } from '@/stores/workflow/registry/store' + +export function useDebouncedWorkflowSync() { + const workflowState = useWorkflowStore((state) => ({ + blocks: state.blocks, + edges: state.edges, + loops: state.loops, + lastSaved: state.lastSaved, + })) + const { activeWorkflowId } = useWorkflowRegistry() + + useEffect(() => { + if (!activeWorkflowId) return + + const syncWorkflow = async () => { + try { + await fetch('/api/workflows/sync', { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ + id: activeWorkflowId, + // Once you have authentication in place, set userId appropriately. + userId: 'current-authenticated-user-id', + state: workflowState, + }), + }) + } catch (err) { + console.error('Debounced sync error:', err) + } + } + + const debouncedSync = debounce(syncWorkflow, 2000) + debouncedSync() + + return () => debouncedSync.cancel() + }, [ + workflowState.blocks, + workflowState.edges, + workflowState.loops, + workflowState.lastSaved, + activeWorkflowId, + ]) +} +``` + + +b. Periodic Sync Hook +Set up another hook that, on an interval (say, every 30 seconds), calls the same API endpoint. This ensures that even if the user pauses making changes, the latest state is still pushed to the database. +``` +import { useEffect } from 'react' +import { useWorkflowStore } from '@/stores/workflow/store' +import { useWorkflowRegistry } from '@/stores/workflow/registry/store' + +export function usePeriodicWorkflowSync(intervalMs = 30000) { + const workflowState = useWorkflowStore((state) => ({ + blocks: state.blocks, + edges: state.edges, + loops: state.loops, + lastSaved: state.lastSaved, + })) + const { activeWorkflowId } = useWorkflowRegistry() + + useEffect(() => { + if (!activeWorkflowId) return + + const syncWorkflow = async () => { + try { + await fetch('/api/workflows/sync', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + id: activeWorkflowId, + userId: 'current-authenticated-user-id', + state: workflowState, + }), + }) + } catch (err) { + console.error('Periodic sync error:', err) + } + } + + const interval = setInterval(syncWorkflow, intervalMs) + + return () => clearInterval(interval) + }, [workflowState, activeWorkflowId, intervalMs]) +} +``` + +c. Critical Event Sync (On Unload) +Finally, add a hook that listens for the beforeunload event and immediately syncs any unsaved changes. +``` +import { useEffect } from 'react' +import { useWorkflowStore } from '@/stores/workflow/store' +import { useWorkflowRegistry } from '@/stores/workflow/registry/store' + +export function useSyncOnUnload() { + const workflowState = useWorkflowStore((state) => ({ + blocks: state.blocks, + edges: state.edges, + loops: state.loops, + lastSaved: state.lastSaved, + })) + const { activeWorkflowId } = useWorkflowRegistry() + + useEffect(() => { + const handleBeforeUnload = async () => { + if (!activeWorkflowId) return + + try { + await fetch('/api/workflows/sync', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + id: activeWorkflowId, + userId: 'current-authenticated-user-id', + state: workflowState, + }), + // Use the keepalive option (if supported) to try flushing even during unload. + keepalive: true, + }) + } catch (err) { + console.error('Sync on unload error:', err) + } + } + + window.addEventListener('beforeunload', handleBeforeUnload) + return () => window.removeEventListener('beforeunload', handleBeforeUnload) + }, [workflowState, activeWorkflowId]) +} +``` + + +> Fallback for Unsynced Changes: +> Since we are already persisting the state in localStorage (and, in-memory, via Zustand), even if one of these sync methods fails, nothing is lost. When the application reloads, we can rehydrate the workflow state from localStorage. +4. Integrate the Hooks into Your Application +Within your main workflow component (or a top-level component that deals with workflow state), call these hooks: +``` +import { useEffect } from 'react' +import { useDebouncedWorkflowSync } from '@/hooks/useDebouncedWorkflowSync' +import { usePeriodicWorkflowSync } from '@/hooks/usePeriodicWorkflowSync' +import { useSyncOnUnload } from '@/hooks/useSyncOnUnload' + +export default function WorkflowContent() { + // Your workflow component code... + + // Start the syncing hooks: + useDebouncedWorkflowSync() + usePeriodicWorkflowSync() + useSyncOnUnload() + + // ... +} +``` + +--- +Summary +Define / Create the DB Table: Create a workflow table on Supabase/Postgres and use Drizzle ORM (with proper migrations) to manage this schema. +API Endpoint: Build a secure API route (/api/workflows/sync) that uses Drizzle ORM to upsert workflow state. Secure access by verifying the authenticated user. +Client Sync Hooks: +Debounced Sync: Batches rapid changes. +Periodic Sync: Ensures regular saves. +BeforeUnload Sync: Catches any unsaved changes as the user leaves. +Local Fallback: Since our state is preserved in localStorage (and memory) the application is resilient even if a write fails. +This approach follows best practices, is secure (by keeping sensitive operations server-side), and aligns well with Next.js, Zustand, and your current codebase structure. +--- +Let me know if you’d like to move on to implementing one or more of these parts in code. \ No newline at end of file