mirror of
https://github.com/Canner/WrenAI.git
synced 2026-08-29 08:18:10 +08:00
feat(mcp-server): add Web UI for connection info management with read-only mode (#1447)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Jax Liu
@@ -73,16 +73,16 @@ The completed workspace will look like:
|
||||
```
|
||||
${PWD}/wren-workspace/
|
||||
├── wren_project.yml
|
||||
├── connection.yml
|
||||
├── models/
|
||||
│ └── *.yml
|
||||
├── relationships.yml
|
||||
├── views.yml
|
||||
└── target/
|
||||
├── mdl.json # Compiled MDL — loaded by the container
|
||||
└── connection.json # Connection info — loaded by the container
|
||||
└── mdl.json # Compiled MDL — loaded by the container
|
||||
```
|
||||
|
||||
> **Connection info** is configured via the MCP server Web UI (`http://localhost:9001`) — it is not stored in the workspace.
|
||||
|
||||
### Phase 2 — Start the Docker container
|
||||
|
||||
#### Check for a newer image
|
||||
@@ -128,24 +128,24 @@ docker run -d \
|
||||
--name wren-mcp \
|
||||
-p 8000:8000 \
|
||||
-p 9000:9000 \
|
||||
-p 9001:9001 \
|
||||
-e ENABLE_MCP_SERVER=true \
|
||||
-e MCP_TRANSPORT=streamable-http \
|
||||
-e MCP_HOST=0.0.0.0 \
|
||||
-e MCP_PORT=9000 \
|
||||
-e WREN_URL=localhost:8000 \
|
||||
-e CONNECTION_FILE_ROOT=/workspace \
|
||||
-e MDL_PATH=/workspace/target/mdl.json \
|
||||
-e CONNECTION_INFO_FILE=/workspace/target/connection.json \
|
||||
-v ~/wren-workspace:/workspace \
|
||||
ghcr.io/canner/wren-engine-ibis:latest
|
||||
```
|
||||
|
||||
Two services start inside the container:
|
||||
Three services start inside the container:
|
||||
|
||||
| Service | Port | Purpose |
|
||||
|---------|------|---------|
|
||||
| wren-ibis-server | 8000 | REST API for query execution and metadata |
|
||||
| MCP server | 9000 | MCP endpoint for AI clients |
|
||||
| Web UI | 9001 | Configuration UI (connection info, MDL editor, read-only mode) |
|
||||
|
||||
Verify it is running:
|
||||
|
||||
@@ -166,25 +166,21 @@ In Claude Code, run:
|
||||
|
||||
The skill will:
|
||||
|
||||
1. Ask for your data source type (PostgreSQL, BigQuery, Snowflake, etc.)
|
||||
2. Ask for connection credentials — **sensitive fields are never stored in the conversation**; they go directly into `connection.yml` for you to fill in
|
||||
3. Call wren-ibis-server to introspect your database schema (tables, columns, types, foreign keys)
|
||||
1. Run `health_check()` to verify the connection is configured
|
||||
2. Ask for your data source type (PostgreSQL, BigQuery, Snowflake, etc.) and optional schema filter
|
||||
3. Call `list_remote_tables()` and `list_remote_constraints()` via the MCP server to introspect your database schema
|
||||
4. Build the MDL JSON (models, columns, relationships)
|
||||
5. Validate the manifest with a dry-plan
|
||||
5. Validate the manifest with `deploy_manifest()` + `dry_run()`
|
||||
|
||||
> **Connection info** must be configured in the Web UI (`http://localhost:9001`) before running `/generate-mdl`. Use `/wren-connection-info` in Claude Code for field reference per data source.
|
||||
|
||||
Then save the MDL as a versioned YAML project:
|
||||
|
||||
```
|
||||
```text
|
||||
/wren-project
|
||||
```
|
||||
|
||||
This writes human-readable YAML files to your workspace and compiles `target/mdl.json` + `target/connection.json`.
|
||||
|
||||
> **Security note:** `connection.yml` and `target/connection.json` may contain credentials. Add them to `.gitignore` before committing:
|
||||
> ```
|
||||
> target/
|
||||
> connection.yml
|
||||
> ```
|
||||
This writes human-readable YAML files to your workspace and compiles `target/mdl.json`.
|
||||
|
||||
### Phase 4 — Register the MCP server
|
||||
|
||||
@@ -233,7 +229,8 @@ Once set up, use `/wren-usage` in Claude Code for ongoing tasks:
|
||||
| Task | Skill |
|
||||
|------|-------|
|
||||
| Write or debug SQL | `/wren-sql` |
|
||||
| Change database credentials | `/wren-connection-info` |
|
||||
| Look up connection field reference | `/wren-connection-info` |
|
||||
| Reconfigure connection via Web UI | `http://localhost:9001` |
|
||||
| Add a model or column to the MDL | `/wren-project` |
|
||||
| Regenerate MDL after schema changes | `/generate-mdl` |
|
||||
| Restart or reconfigure the MCP server | `/wren-mcp-setup` |
|
||||
|
||||
+12
-9
@@ -121,14 +121,13 @@ docker run -d \
|
||||
--name wren-mcp \
|
||||
-p 8000:8000 \
|
||||
-p 9000:9000 \
|
||||
-p 9001:9001 \
|
||||
-e ENABLE_MCP_SERVER=true \
|
||||
-e MCP_TRANSPORT=streamable-http \
|
||||
-e MCP_HOST=0.0.0.0 \
|
||||
-e MCP_PORT=9000 \
|
||||
-e WREN_URL=localhost:8000 \
|
||||
-e CONNECTION_FILE_ROOT=/workspace \
|
||||
-e MDL_PATH=/workspace/target/mdl.json \
|
||||
-e CONNECTION_INFO_FILE=/workspace/target/connection.json \
|
||||
-v ~/wren-workspace:/workspace \
|
||||
-v "$JAFFLE_SHOP_DIR":/data \
|
||||
ghcr.io/canner/wren-engine-ibis:latest
|
||||
@@ -151,19 +150,23 @@ In Claude Code, run each skill in sequence:
|
||||
/generate-mdl
|
||||
```
|
||||
|
||||
When prompted, enter:
|
||||
- Data source type: `duckdb`
|
||||
Before running `/generate-mdl`, configure the connection via the Web UI at `http://localhost:9001`:
|
||||
- Data source type: `DUCKDB`
|
||||
- Database folder path: `/data` (the folder containing `jaffle_shop.duckdb`)
|
||||
|
||||
Then run:
|
||||
|
||||
```text
|
||||
/generate-mdl
|
||||
```
|
||||
|
||||
Then save the MDL as a versioned YAML project:
|
||||
|
||||
```
|
||||
```text
|
||||
/wren-project
|
||||
```
|
||||
|
||||
This writes human-readable YAML files to `~/wren-workspace/` and compiles `target/mdl.json` + `target/connection.json`.
|
||||
|
||||
> **Security note:** `connection.yml` and `target/connection.json` may contain credentials. Add them to `.gitignore` before committing.
|
||||
This writes human-readable YAML files to `~/wren-workspace/` and compiles `target/mdl.json`.
|
||||
|
||||
#### Phase 3 — Register the MCP server
|
||||
|
||||
@@ -245,7 +248,7 @@ The container must be running first. Run `docker ps --filter name=wren-mcp` to c
|
||||
Start a new Claude Code session after running `claude mcp add`. MCP servers are loaded at session start only.
|
||||
|
||||
**`health_check()` returns an error:**
|
||||
Check container logs: `docker logs wren-mcp`. Confirm both ports (8000, 9000) are listening: `curl http://localhost:8000/health`.
|
||||
Check container logs: `docker logs wren-mcp`. Confirm ports are listening: `curl http://localhost:8000/health`. Check connection info in the Web UI: `http://localhost:9001`.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+25
-45
@@ -11,7 +11,6 @@ YAML files use **snake_case** field names for readability. The compiled `target/
|
||||
```text
|
||||
my_project/
|
||||
├── wren_project.yml # Project metadata (catalog, schema, data_source)
|
||||
├── connection.yml # Data source connection parameters
|
||||
├── models/
|
||||
│ ├── orders.yml # One file per model
|
||||
│ ├── customers.yml
|
||||
@@ -20,18 +19,15 @@ my_project/
|
||||
└── views.yml # All views
|
||||
```
|
||||
|
||||
After building, compiled files are written to:
|
||||
After building, the compiled file is written to:
|
||||
|
||||
```text
|
||||
my_project/
|
||||
└── target/
|
||||
├── mdl.json # Deployable MDL JSON (camelCase)
|
||||
└── connection.json # Connection info JSON (camelCase)
|
||||
└── mdl.json # Deployable MDL JSON (camelCase)
|
||||
```
|
||||
|
||||
:::caution Security note
|
||||
`connection.yml` and `target/connection.json` may contain credentials. Add both `target/` and `connection.yml` to `.gitignore` before committing.
|
||||
:::
|
||||
> **Connection info** is managed via the MCP server Web UI (`http://localhost:9001`) — it is not stored in the project directory.
|
||||
|
||||
---
|
||||
|
||||
@@ -47,7 +43,6 @@ version: "1.0"
|
||||
catalog: wren
|
||||
schema: public
|
||||
data_source: POSTGRES
|
||||
connection_mode: security # "security" (default) | "inline"
|
||||
```
|
||||
|
||||
| Field | Description |
|
||||
@@ -56,7 +51,6 @@ connection_mode: security # "security" (default) | "inline"
|
||||
| `catalog` | MDL catalog (matches the `catalog` in your MDL manifest) |
|
||||
| `schema` | MDL schema |
|
||||
| `data_source` | Data source type (e.g. `POSTGRES`, `BIGQUERY`, `SNOWFLAKE`) |
|
||||
| `connection_mode` | `security` — credentials are in files and never shown; `inline` — test/dev mode where credentials may be provided inline |
|
||||
|
||||
### `models/<model_name>.yml`
|
||||
|
||||
@@ -115,21 +109,6 @@ views:
|
||||
properties: {}
|
||||
```
|
||||
|
||||
### `connection.yml`
|
||||
|
||||
Connection parameters for the data source. Fields use **snake_case** and are converted to **camelCase** when compiled to `target/connection.json`.
|
||||
|
||||
Example for PostgreSQL:
|
||||
|
||||
```yaml
|
||||
connector: POSTGRES
|
||||
host: localhost
|
||||
port: 5432
|
||||
database: my_database
|
||||
user: my_user
|
||||
password: "" # TODO: fill in before building
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Field Mapping
|
||||
@@ -152,56 +131,57 @@ All other fields (`name`, `type`, `catalog`, `schema`, `table`, `condition`, `mo
|
||||
|
||||
## Building the Project
|
||||
|
||||
Building compiles the YAML project into two JSON files under `target/`:
|
||||
Building compiles the YAML project into `target/mdl.json`:
|
||||
|
||||
```bash
|
||||
# target/mdl.json — assembled MDL manifest
|
||||
# target/connection.json — connection info
|
||||
# target/mdl.json — assembled MDL manifest (camelCase)
|
||||
```
|
||||
|
||||
After building, you can use the compiled output in two ways:
|
||||
After building, deploy the MDL via the MCP server:
|
||||
|
||||
- **Wren Engine API directly** — pass `mdl_file_path` and `connectionFilePath` as parameters in ibis-server REST API calls. See the [Wren Engine API reference](/oss/wren_engine_api/) for details.
|
||||
- **Wren MCP Server** — mount the `target/` directory when starting the MCP server. The MCP server calls `deploy()` on startup to activate the MDL and handles subsequent API calls for you.
|
||||
```text
|
||||
deploy(mdl_file_path="/workspace/target/mdl.json")
|
||||
```
|
||||
|
||||
In both cases, `connectionFilePath` points to `target/connection.json` so ibis-server reads credentials from the file directly — they never appear in API payloads or LLM context.
|
||||
Or place it in the workspace before starting the container so it is auto-loaded via `MDL_PATH`.
|
||||
|
||||
Connection info is configured separately via the Web UI (`http://localhost:9001`) — it is not part of the build output.
|
||||
|
||||
---
|
||||
|
||||
## Typical Workflow
|
||||
|
||||
**1. Set up connection info**
|
||||
**1. Configure connection info**
|
||||
|
||||
Create `connection.yml` with host, port, database, and user. Leave sensitive fields (password, token) empty with a `# TODO` comment. Fill them in before building, then compile to `target/connection.json`.
|
||||
Open the Web UI at `http://localhost:9001`, select the data source type, and fill in connection credentials. Use `/wren-connection-info` in Claude Code for per-connector field reference.
|
||||
|
||||
**2. Generate MDL**
|
||||
|
||||
Use ibis-server metadata endpoints to introspect the database. Pass `connectionFilePath="<abs_path>/target/connection.json"` in all API calls.
|
||||
Run `/generate-mdl` in Claude Code. The skill uses MCP tools (`list_remote_tables`, `list_remote_constraints`) to introspect the database and build the MDL JSON.
|
||||
|
||||
**3. Save project**
|
||||
|
||||
Write `wren_project.yml`, `models/*.yml`, `relationships.yml`, and `views.yml` by converting the MDL JSON (camelCase) to snake_case YAML.
|
||||
|
||||
**4. Add to `.gitignore`**
|
||||
**4. Add `target/` to `.gitignore`**
|
||||
|
||||
```
|
||||
```text
|
||||
target/
|
||||
connection.yml
|
||||
```
|
||||
|
||||
**5. Commit to version control**v
|
||||
**5. Commit to version control**
|
||||
|
||||
Commit the model files without secrets — `connection.yml` and `target/` are excluded.
|
||||
Commit the model YAML files — `target/` is excluded.
|
||||
|
||||
**6. Build**
|
||||
|
||||
Read the YAML files, rename snake_case → camelCase, and write `target/mdl.json` and `target/connection.json`.
|
||||
Read the YAML files, rename snake_case → camelCase, and write `target/mdl.json`.
|
||||
|
||||
**7. Use the compiled output**
|
||||
**7. Deploy**
|
||||
|
||||
- **Option A — Wren Engine API directly**: Pass `mdl_file_path` and `connectionFilePath` as parameters when calling the ibis-server REST API. See the [Wren Engine API reference](/oss/wren_engine_api/) for details.
|
||||
|
||||
- **Option B — Wren MCP Server**: Start the MCP server with the project's `target/` directory mounted. The MCP server calls `deploy(mdl_file_path="./target/mdl.json")` on startup to activate the MDL, then handles API calls on your behalf. See the Wren MCP Server guide for setup details.
|
||||
```text
|
||||
deploy(mdl_file_path="/workspace/target/mdl.json")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -211,5 +191,5 @@ Storing MDL as a YAML project (rather than a single JSON blob) gives you:
|
||||
|
||||
- **Readable diffs** — model changes show up as clear line-level diffs in pull requests
|
||||
- **One file per model** — merge conflicts are isolated to the affected model file
|
||||
- **Separation of secrets** — `connection.yml` and `target/` are gitignored; everything else is safe to commit
|
||||
- **Separation of secrets** — connection info lives in the Web UI, not in the project; `target/` is gitignored
|
||||
- **Reproducible builds** — `target/mdl.json` is always regenerated from source, never committed
|
||||
|
||||
@@ -5,7 +5,7 @@ compatibility: Requires a running ibis-server (default port 8000). No local data
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: wren-engine
|
||||
version: "1.2"
|
||||
version: "1.3"
|
||||
---
|
||||
|
||||
# Generate Wren MDL
|
||||
@@ -13,10 +13,10 @@ metadata:
|
||||
## Version check
|
||||
|
||||
Silently fetch `https://raw.githubusercontent.com/Canner/wren-engine/main/skills/versions.json`.
|
||||
Compare the `generate-mdl` key with this skill's version (`1.2`).
|
||||
Compare the `generate-mdl` key with this skill's version (`1.3`).
|
||||
If the remote version is newer, notify the user before proceeding:
|
||||
|
||||
> A newer version of the **generate-mdl** skill is available (remote: X.Y, installed: 1.2).
|
||||
> A newer version of the **generate-mdl** skill is available (remote: X.Y, installed: 1.3).
|
||||
> Update with:
|
||||
> ```bash
|
||||
> curl -fsSL https://raw.githubusercontent.com/Canner/wren-engine/main/skills/install.sh | bash -s -- --force generate-mdl
|
||||
@@ -32,68 +32,47 @@ Generates a Wren MDL manifest by using ibis-server to introspect the database sc
|
||||
|
||||
Follow these steps in order. Do not skip steps or ask unnecessary questions between them.
|
||||
|
||||
### Step 1 — Set up data source and connection info
|
||||
### Step 1 — Verify connection and choose data source
|
||||
|
||||
Follow the **wren-connection-info** skill (`skills/wren-connection-info/SKILL.md`) to:
|
||||
1. Choose the data source type (e.g. `POSTGRES`, `BIGQUERY`, `SNOWFLAKE`, …)
|
||||
2. Choose connection mode (Mode A: secure file path, or Mode B: inline for testing)
|
||||
3. Gather credentials and produce either a `connectionFilePath` or inline `connectionInfo`
|
||||
Confirm the MCP server has a working connection before proceeding:
|
||||
|
||||
Also ask the user for a **schema filter** (optional) — if the database has many schemas, ask which schema(s) to include.
|
||||
```text
|
||||
health_check()
|
||||
```
|
||||
|
||||
If the health check fails, ask the user to configure the connection via the Web UI at `http://localhost:9001` before continuing.
|
||||
|
||||
Ask the user for:
|
||||
1. **Data source type** (e.g. `POSTGRES`, `BIGQUERY`, `SNOWFLAKE`, …) — needed to set `dataSource` in the MDL
|
||||
2. **Schema filter** (optional) — if the database has many schemas, ask which schema(s) to include
|
||||
|
||||
After this step you will have:
|
||||
- `data_source`: e.g. `"POSTGRES"`
|
||||
- Either `connectionFilePath` (Mode A) or `connectionInfo` dict (Mode B) — used in all subsequent API calls
|
||||
- Optional `schema_filter`: used to narrow down results in subsequent steps
|
||||
|
||||
### Step 2 — Fetch table schema
|
||||
|
||||
Call the ibis-server metadata endpoint directly, using the connection output from Step 1:
|
||||
|
||||
```
|
||||
POST http://localhost:8000/v3/connector/<data_source>/metadata/tables
|
||||
Content-Type: application/json
|
||||
|
||||
{ "connectionFilePath": "/abs/path/to/target/connection.json" }
|
||||
— or —
|
||||
{ "connectionInfo": { <credentials dict> } }
|
||||
```text
|
||||
list_remote_tables()
|
||||
```
|
||||
|
||||
ibis-server returns a list of tables with their column names and types. Each table entry has a `properties.schema` field — use it to filter to the user's target schema if specified.
|
||||
Returns a list of tables with their column names and types. Each table entry has a `properties.schema` field — use it to filter to the user's target schema if specified.
|
||||
|
||||
If this fails, report the error and ask the user to correct the credentials.
|
||||
If this fails:
|
||||
1. Check that read-only mode is **disabled** in the Web UI (`http://localhost:9001`) — `list_remote_tables()` will fail when read-only mode is on, even if the connection is healthy.
|
||||
2. Ask the user to verify connection info in the Web UI if read-only mode is already off.
|
||||
|
||||
### Step 3 — Fetch relationships
|
||||
|
||||
```
|
||||
POST http://localhost:8000/v3/connector/<data_source>/metadata/constraints
|
||||
Content-Type: application/json
|
||||
|
||||
{ "connectionFilePath": "/abs/path/to/target/connection.json" }
|
||||
— or —
|
||||
{ "connectionInfo": { <credentials dict> } }
|
||||
```text
|
||||
list_remote_constraints()
|
||||
```
|
||||
|
||||
Returns foreign key constraints. Use these to build `Relationship` entries in the MDL. If the response is empty (`[]`), infer relationships from column naming conventions (e.g. `order_id` → `orders.id`).
|
||||
|
||||
### Step 4 — Sample data (optional)
|
||||
If this fails, verify that read-only mode is disabled in the Web UI (`http://localhost:9001`).
|
||||
|
||||
For columns where purpose is unclear from the name and type alone, query a few rows using the raw table name with schema prefix:
|
||||
|
||||
```
|
||||
POST http://localhost:8000/v3/connector/<data_source>/query
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"sql": "SELECT * FROM <schema>.<table> LIMIT 3",
|
||||
"manifestStr": "",
|
||||
"connectionFilePath": "/abs/path/to/target/connection.json"
|
||||
}
|
||||
— or use "connectionInfo": { <credentials dict> } in Mode B
|
||||
```
|
||||
|
||||
Note: use the raw `schema.table` reference at this stage, since the MDL is not yet deployed.
|
||||
|
||||
### Step 5 — Build MDL JSON
|
||||
### Step 4 — Build MDL JSON
|
||||
|
||||
Construct the manifest following the [MDL structure](#mdl-structure) below.
|
||||
|
||||
@@ -108,51 +87,29 @@ Rules:
|
||||
- For FK columns, add a `Relationship` entry linking the two models
|
||||
- Omit calculated columns for now — they can be added later
|
||||
|
||||
### Step 6 — Validate
|
||||
### Step 5 — Validate
|
||||
|
||||
Validate the MDL by running a dry-plan against a simple query. Base64-encode the manifest first:
|
||||
Deploy the draft MDL and validate it with a dry run:
|
||||
|
||||
```python
|
||||
import json, base64
|
||||
manifest_b64 = base64.b64encode(json.dumps(mdl).encode()).decode()
|
||||
```text
|
||||
deploy_manifest(mdl=<manifest dict>)
|
||||
dry_run(sql="SELECT * FROM <any_model_name> LIMIT 1")
|
||||
```
|
||||
|
||||
Then call:
|
||||
If `dry_run` succeeds, the MDL is valid. If it fails, fix the reported errors, call `deploy_manifest` again with the corrected MDL, and retry.
|
||||
|
||||
```
|
||||
POST http://localhost:8000/v3/connector/<data_source>/dry-plan
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"manifestStr": "<base64-encoded manifest>",
|
||||
"sql": "SELECT * FROM <any_model_name> LIMIT 1"
|
||||
}
|
||||
```
|
||||
|
||||
If validation succeeds, the response is the planned SQL string. If it fails, fix the reported errors and validate again.
|
||||
|
||||
> **Note:** Use the `/v3/` endpoint, not `/v2/`. The v2 dry-plan requires a separate Wren Engine Java process (`WREN_ENGINE_ENDPOINT`) which is not part of the standard Docker setup.
|
||||
|
||||
### Step 7 — Save project (optional)
|
||||
### Step 6 — Save project (optional)
|
||||
|
||||
Ask the user if they want to save the MDL as a YAML project directory (useful for version control).
|
||||
|
||||
If yes, follow the **wren-project** skill (`skills/wren-project/SKILL.md`) to write the YAML files and build `target/mdl.json` + `target/connection.json`.
|
||||
If yes, follow the **wren-project** skill (`skills/wren-project/SKILL.md`) to write the YAML files and build `target/mdl.json`.
|
||||
|
||||
### Step 8 — Deploy
|
||||
|
||||
**If Wren MCP tools are available** (i.e., Claude Code has the `wren` MCP server registered):
|
||||
### Step 7 — Deploy final MDL
|
||||
|
||||
```
|
||||
deploy_manifest(mdl=<manifest dict>)
|
||||
```
|
||||
|
||||
**If MCP tools are not available**, deploy by writing the MDL to the workspace file that the container watches:
|
||||
|
||||
1. Build `target/mdl.json` from the YAML project (see wren-project skill)
|
||||
2. Ensure the container was started with `-e MDL_PATH=/workspace/target/mdl.json`
|
||||
3. Restart the container to reload — or call the `deploy` MCP tool after connecting
|
||||
|
||||
Confirm success to the user. The MDL is now active and queries can run.
|
||||
|
||||
---
|
||||
@@ -248,6 +205,8 @@ When in doubt, use `VARCHAR` as a safe fallback.
|
||||
|
||||
---
|
||||
|
||||
## Connection info format
|
||||
## Connection setup
|
||||
|
||||
See the **wren-connection-info** skill (`skills/wren-connection-info/SKILL.md`) for the full per-connector field reference, secrets policy, and Mode A / Mode B workflow.
|
||||
Connection info is configured via the MCP server Web UI at `http://localhost:9001`. See the **wren-mcp-setup** skill for Docker setup instructions.
|
||||
|
||||
> **Note:** If the Web UI is disabled (`WEB_UI_ENABLED=false`), connection info must be pre-configured in `~/.wren/connection_info.json` before starting the container. Use `/wren-connection-info` in Claude Code for the required fields per data source.
|
||||
|
||||
+7
-15
@@ -7,8 +7,8 @@
|
||||
"skills": [
|
||||
{
|
||||
"name": "wren-connection-info",
|
||||
"version": "1.2",
|
||||
"description": "Set up data source type and connection credentials for Wren Engine.",
|
||||
"version": "1.3",
|
||||
"description": "Reference guide for Wren Engine connection info — required fields, sensitive values, Docker host hints, and BigQuery credential encoding.",
|
||||
"tags": [
|
||||
"wren",
|
||||
"credentials",
|
||||
@@ -20,8 +20,8 @@
|
||||
},
|
||||
{
|
||||
"name": "generate-mdl",
|
||||
"version": "1.2",
|
||||
"description": "Generate a Wren MDL manifest from a live database using ibis-server introspection.",
|
||||
"version": "1.3",
|
||||
"description": "Generate a Wren MDL manifest from a live database using MCP server introspection tools.",
|
||||
"tags": [
|
||||
"wren",
|
||||
"mdl",
|
||||
@@ -34,14 +34,11 @@
|
||||
"clickhouse",
|
||||
"trino"
|
||||
],
|
||||
"dependencies": [
|
||||
"wren-connection-info"
|
||||
],
|
||||
"repository": "https://github.com/Canner/wren-engine/tree/main/skills/generate-mdl"
|
||||
},
|
||||
{
|
||||
"name": "wren-project",
|
||||
"version": "1.4",
|
||||
"version": "1.5",
|
||||
"description": "Save, load, and build Wren MDL manifests as YAML project directories for version control.",
|
||||
"tags": [
|
||||
"wren",
|
||||
@@ -51,9 +48,6 @@
|
||||
"project",
|
||||
"git"
|
||||
],
|
||||
"dependencies": [
|
||||
"wren-connection-info"
|
||||
],
|
||||
"repository": "https://github.com/Canner/wren-engine/tree/main/skills/wren-project"
|
||||
},
|
||||
{
|
||||
@@ -74,7 +68,7 @@
|
||||
},
|
||||
{
|
||||
"name": "wren-mcp-setup",
|
||||
"version": "1.2",
|
||||
"version": "1.3",
|
||||
"description": "Set up Wren Engine MCP server via Docker and register it with an AI agent.",
|
||||
"tags": [
|
||||
"wren",
|
||||
@@ -88,7 +82,7 @@
|
||||
},
|
||||
{
|
||||
"name": "wren-quickstart",
|
||||
"version": "1.1",
|
||||
"version": "1.2",
|
||||
"description": "End-to-end quickstart for Wren Engine — from zero to querying.",
|
||||
"tags": [
|
||||
"wren",
|
||||
@@ -98,7 +92,6 @@
|
||||
"docker"
|
||||
],
|
||||
"dependencies": [
|
||||
"wren-connection-info",
|
||||
"generate-mdl",
|
||||
"wren-project",
|
||||
"wren-mcp-setup"
|
||||
@@ -117,7 +110,6 @@
|
||||
"mcp"
|
||||
],
|
||||
"dependencies": [
|
||||
"wren-connection-info",
|
||||
"generate-mdl",
|
||||
"wren-project",
|
||||
"wren-sql",
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
{
|
||||
"generate-mdl": "1.2",
|
||||
"wren-connection-info": "1.2",
|
||||
"wren-project": "1.4",
|
||||
"generate-mdl": "1.3",
|
||||
"wren-connection-info": "1.3",
|
||||
"wren-project": "1.5",
|
||||
"wren-sql": "1.0",
|
||||
"wren-mcp-setup": "1.2",
|
||||
"wren-quickstart": "1.1",
|
||||
"wren-mcp-setup": "1.3",
|
||||
"wren-quickstart": "1.2",
|
||||
"wren-usage": "1.0"
|
||||
}
|
||||
|
||||
@@ -1,22 +1,21 @@
|
||||
---
|
||||
name: wren-connection-info
|
||||
description: Set up data source type and connection credentials for Wren Engine. Use at the start of any workflow that connects to a database — produces either a connectionFilePath (secure, default) or an inline connectionInfo dict (opt-in for testing). Trigger before generate-mdl, wren-project, or any ibis-server API call that needs credentials.
|
||||
description: Reference guide for Wren Engine connection info — explains required fields per data source, sensitive field handling, Docker host hints, and BigQuery credential encoding. Use when the user asks how to configure a data source connection or what fields to fill in.
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: wren-engine
|
||||
version: "1.2"
|
||||
version: "1.3"
|
||||
---
|
||||
|
||||
# Wren Connection Info
|
||||
# Wren Connection Info Reference
|
||||
|
||||
Sets up the data source type and credentials before any workflow that queries a database.
|
||||
Find the specification of the connection info format in the `model` section of [API reference](https://docs.getwren.ai/oss/wren_engine_api).
|
||||
This skill answers questions about how to configure connection info for each data source in Wren Engine. Use it to explain required fields, flag sensitive values, and guide the user through any data-source-specific setup steps.
|
||||
|
||||
The connection info is entered by the user — in the MCP server Web UI (`http://localhost:9001`) for normal use, or directly in API calls for advanced workflows.
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Choose data source
|
||||
|
||||
Ask the user for their **data source type**:
|
||||
## Data source types
|
||||
|
||||
| Value | Database |
|
||||
|-------|----------|
|
||||
@@ -32,230 +31,120 @@ Ask the user for their **data source type**:
|
||||
| `ORACLE` | Oracle |
|
||||
| `DATABRICKS` | Databricks |
|
||||
|
||||
> **Docker note**: If the database runs on the host machine and ibis-server runs inside Docker, replace `localhost` / `127.0.0.1` with `host.docker.internal` in the host field.
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — Choose connection mode
|
||||
|
||||
Two modes are supported. Ask the user which they prefer, or infer from context.
|
||||
|
||||
The chosen mode is recorded as `connection_mode` in `wren_project.yml` so every subsequent workflow knows how to handle credentials.
|
||||
|
||||
### Mode A — Secure (default, recommended for production)
|
||||
|
||||
`connection_mode: security` in `wren_project.yml`.
|
||||
|
||||
The LLM never handles sensitive values. ibis-server reads the connection file directly.
|
||||
|
||||
Use this mode by default unless the user explicitly says they are in a test/development environment and willing to share credentials.
|
||||
|
||||
**When `connection_mode: security` is in effect** (either set explicitly or because the field is absent):
|
||||
- **Never** read `connection.yml` or `target/connection.json` without first asking the user for permission.
|
||||
- **Never** display or echo the contents of those files.
|
||||
- If debugging requires connection info, ask the user to share only non-sensitive fields (e.g. `host`, `port`, `database`, `user`) — never passwords, tokens, or keys.
|
||||
|
||||
### Mode B — Inline (opt-in, testing only)
|
||||
|
||||
`connection_mode: inline` in `wren_project.yml`.
|
||||
|
||||
> **How to opt in**: The user must say something like "I'm just testing, you can use my credentials" or "it's a dev environment, here are my connection details". Do not assume this mode.
|
||||
|
||||
In this mode, ask for all fields including sensitive ones and assemble an inline `connectionInfo` dict.
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — Gather credentials
|
||||
|
||||
Ask for the fields required for the chosen data source. Sensitive fields (marked **secret**) must **never** be filled in by the LLM in Mode A — leave them as `# TODO` comments.
|
||||
## Required fields per data source
|
||||
|
||||
### PostgreSQL / MySQL / MSSQL / ClickHouse / Oracle
|
||||
|
||||
```
|
||||
host: <hostname or IP>
|
||||
port: <port>
|
||||
user: <username>
|
||||
password: <SECRET>
|
||||
database: <database name>
|
||||
```
|
||||
| Field | Description | Sensitive |
|
||||
|-------|-------------|-----------|
|
||||
| `host` | Hostname or IP | |
|
||||
| `port` | Port number | |
|
||||
| `user` | Username | |
|
||||
| `password` | Password | ✓ |
|
||||
| `database` | Database name | |
|
||||
|
||||
Default ports: PostgreSQL `5432`, MySQL `3306`, MSSQL `1433`, ClickHouse `8123`, Oracle `1521`
|
||||
|
||||
### Trino
|
||||
|
||||
```
|
||||
host: <hostname>
|
||||
port: <port, default 8080>
|
||||
user: <username>
|
||||
catalog: <catalog name>
|
||||
schema: <schema name>
|
||||
```
|
||||
| Field | Description | Sensitive |
|
||||
|-------|-------------|-----------|
|
||||
| `host` | Hostname | |
|
||||
| `port` | Port (default `8080`) | |
|
||||
| `user` | Username | |
|
||||
| `catalog` | Catalog name | |
|
||||
| `schema` | Schema name | |
|
||||
|
||||
### BigQuery
|
||||
|
||||
```
|
||||
project_id: <GCP project ID>
|
||||
dataset_id: <dataset name>
|
||||
credentials_json_string: <SECRET — base64-encoded service account JSON>
|
||||
```
|
||||
| Field | Description | Sensitive |
|
||||
|-------|-------------|-----------|
|
||||
| `project_id` | GCP project ID | |
|
||||
| `dataset_id` | Dataset name | |
|
||||
| `credentials_json_string` | Base64-encoded service account JSON | ✓ |
|
||||
|
||||
> **BigQuery credentials**: Wren requires the service account JSON as a **base64-encoded string**, not the raw file.
|
||||
> **BigQuery credentials encoding**: Wren requires the service account JSON as a **base64-encoded string**, not the raw file.
|
||||
> After downloading `credentials.json` from GCP, run:
|
||||
> ```bash
|
||||
> # macOS
|
||||
> base64 -i credentials.json | tr -d '\n'
|
||||
> # Linux
|
||||
> base64 -w 0 credentials.json
|
||||
> ```
|
||||
> Paste the output as the value of `credentials_json_string`.
|
||||
> On Linux: `base64 -w 0 credentials.json`
|
||||
> Paste the output into the `credentials_json_string` field.
|
||||
|
||||
### Snowflake
|
||||
|
||||
```
|
||||
user: <username>
|
||||
password: <SECRET>
|
||||
account: <account identifier>
|
||||
database: <database name>
|
||||
sf_schema: <schema name>
|
||||
```
|
||||
|
||||
### File based (S3, Minio, GCS, local file)
|
||||
For the object storage connectors (S3, Minio, GCS) and local file sources, use this format:
|
||||
```
|
||||
format: <FILE_FORMAT> # e.g. "csv", "parquet"
|
||||
url: <file path or bucket URL>
|
||||
```
|
||||
If credentials are needed (e.g. S3 access key and secret), include these fields as well:
|
||||
```
|
||||
access_key: <SECRET>
|
||||
secret_key: <SECRET>
|
||||
```
|
||||
| Field | Description | Sensitive |
|
||||
|-------|-------------|-----------|
|
||||
| `user` | Username | |
|
||||
| `password` | Password | ✓ |
|
||||
| `account` | Account identifier | |
|
||||
| `database` | Database name | |
|
||||
| `sf_schema` | Schema name | |
|
||||
|
||||
### DuckDB
|
||||
It leverages the same connection info format as file-based sources, but with `format: duckdb` to indicate that it's a DuckDB data source rather than a generic file source.
|
||||
```
|
||||
format: duckdb
|
||||
url: <path to folder containing .duckdb file>
|
||||
```
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `format` | Must be `"duckdb"` |
|
||||
| `url` | Path to the folder containing the `.duckdb` file |
|
||||
|
||||
> When running via Docker, the `.duckdb` file must be inside the mounted workspace (e.g. `/workspace/mydb.duckdb`).
|
||||
|
||||
### File-based (S3, Minio, GCS, local file)
|
||||
|
||||
| Field | Description | Sensitive |
|
||||
|-------|-------------|-----------|
|
||||
| `format` | File format: `"csv"`, `"parquet"`, etc. | |
|
||||
| `url` | File path or bucket URL | |
|
||||
| `access_key` | Access key (object storage) | ✓ |
|
||||
| `secret_key` | Secret key (object storage) | ✓ |
|
||||
|
||||
### Athena
|
||||
|
||||
```
|
||||
s3_staging_dir: <s3://bucket/prefix/>
|
||||
region: <AWS region>
|
||||
aws_access_key_id: <SECRET>
|
||||
aws_secret_access_key: <SECRET>
|
||||
```
|
||||
| Field | Description | Sensitive |
|
||||
|-------|-------------|-----------|
|
||||
| `s3_staging_dir` | S3 staging directory (`s3://bucket/prefix/`) | |
|
||||
| `region` | AWS region | |
|
||||
| `aws_access_key_id` | AWS access key ID | ✓ |
|
||||
| `aws_secret_access_key` | AWS secret access key | ✓ |
|
||||
|
||||
### Databricks
|
||||
|
||||
```
|
||||
host: <workspace hostname>
|
||||
http_path: <SQL warehouse HTTP path>
|
||||
access_token: <SECRET>
|
||||
```
|
||||
| Field | Description | Sensitive |
|
||||
|-------|-------------|-----------|
|
||||
| `host` | Workspace hostname | |
|
||||
| `http_path` | SQL warehouse HTTP path | |
|
||||
| `access_token` | Personal access token | ✓ |
|
||||
|
||||
**Sensitive fields by connector** — LLM must never populate these in Mode A:
|
||||
---
|
||||
|
||||
## Docker host hint
|
||||
|
||||
If the database runs on the **host machine** and Wren Engine runs inside Docker, `localhost` and `127.0.0.1` cannot reach the host. Use `host.docker.internal` instead:
|
||||
|
||||
| Original | Inside Docker |
|
||||
|----------|--------------|
|
||||
| `localhost` | `host.docker.internal` |
|
||||
| `127.0.0.1` | `host.docker.internal` |
|
||||
| Cloud/remote hostname | No change needed |
|
||||
|
||||
The MCP server Web UI (`http://localhost:9001`) shows a hint for this when relevant.
|
||||
|
||||
---
|
||||
|
||||
## Sensitive fields summary
|
||||
|
||||
Never log, display, or pass sensitive values through the AI agent unnecessarily.
|
||||
|
||||
| Connector | Sensitive fields |
|
||||
|-----------|-----------------|
|
||||
| Postgres / MySQL / MSSQL / ClickHouse / Trino / Oracle | `password` |
|
||||
| Postgres / MySQL / MSSQL / ClickHouse / Oracle | `password` |
|
||||
| BigQuery | `credentials_json_string` |
|
||||
| Snowflake | `password`, `private_key` |
|
||||
| Athena | `aws_access_key_id`, `aws_secret_access_key`, `aws_session_token`, `web_identity_token` |
|
||||
| Athena | `aws_access_key_id`, `aws_secret_access_key`, `aws_session_token` |
|
||||
| S3 / Minio / GCS file | `access_key`, `secret_key` |
|
||||
| Databricks | `access_token`, `client_secret` |
|
||||
| Canner | `pat` |
|
||||
|
||||
---
|
||||
|
||||
## Step 4 — Produce output
|
||||
|
||||
### Mode A output
|
||||
|
||||
Write `<project_dir>/connection.yml` with non-sensitive fields filled in and sensitive fields as `# TODO` comments:
|
||||
|
||||
```yaml
|
||||
# Example: PostgreSQL
|
||||
host: my-db.example.com
|
||||
port: 5432
|
||||
user: my_user
|
||||
password: # TODO: fill in your database password
|
||||
database: my_db
|
||||
```
|
||||
|
||||
Then instruct the user:
|
||||
> Please fill in the sensitive fields in `connection.yml`, then let me know when done.
|
||||
|
||||
Wait for confirmation, then build `target/connection.json`:
|
||||
|
||||
```bash
|
||||
python -c "
|
||||
import yaml, json, pathlib
|
||||
p = pathlib.Path('connection.yml')
|
||||
d = yaml.safe_load(p.read_text())
|
||||
pathlib.Path('target').mkdir(exist_ok=True)
|
||||
json.dump(d, open('target/connection.json', 'w'))
|
||||
"
|
||||
```
|
||||
|
||||
**Do NOT read or display the contents of `target/connection.json` after building.**
|
||||
|
||||
> **Server requirement:** ibis-server must have the `CONNECTION_FILE_ROOT` environment variable set to the directory containing `target/connection.json`. When running via Docker (the standard deployment), the workspace is mounted at `/workspace` and `CONNECTION_FILE_ROOT=/workspace` is set by default. For local dev, set `CONNECTION_FILE_ROOT` to the project root before starting ibis-server.
|
||||
|
||||
Provide to the calling workflow:
|
||||
- `connectionFilePath`: absolute path to `target/connection.json` — use the **container-internal** path (e.g. `/workspace/target/connection.json`) when ibis-server runs in Docker, or the host path for local dev
|
||||
- `data_source`: the data source type string (e.g. `"POSTGRES"`)
|
||||
- `connection_mode`: `"security"` — record this in `wren_project.yml`
|
||||
|
||||
### Mode B output
|
||||
|
||||
Assemble the inline dict directly. Provide to the calling workflow:
|
||||
- `connectionInfo`: camelCase JSON dict (see [Field mapping](#field-mapping) below)
|
||||
- `data_source`: the data source type string
|
||||
- `connection_mode`: `"inline"` — record this in `wren_project.yml`
|
||||
|
||||
---
|
||||
|
||||
## Field mapping
|
||||
|
||||
When converting `connection.yml` to `target/connection.json`, rename these snake_case keys to camelCase:
|
||||
|
||||
| YAML (snake_case) | JSON (camelCase) |
|
||||
|-------------------|-----------------|
|
||||
| `project_id` | `projectId` |
|
||||
| `dataset_id` | `datasetId` |
|
||||
| `credentials_json_string` | `credentialsJsonString` |
|
||||
| `sf_schema` | `sfSchema` |
|
||||
| `ssl_mode` | `sslMode` |
|
||||
| `ssl_ca` | `sslCA` |
|
||||
| `connection_url` | `connectionUrl` |
|
||||
| `http_path` | `httpPath` |
|
||||
| `access_token` | `accessToken` |
|
||||
| `s3_staging_dir` | `s3StagingDir` |
|
||||
| `aws_access_key_id` | `awsAccessKeyId` |
|
||||
| `aws_secret_access_key` | `awsSecretAccessKey` |
|
||||
|
||||
Fields without underscores (`host`, `port`, `user`, `password`, `database`, `account`, `url`, `catalog`, `schema`, `region`) remain unchanged. All other snake_case fields should be converted to camelCase for JSON.
|
||||
|
||||
---
|
||||
|
||||
## Using connection info in API calls
|
||||
|
||||
After this skill completes, use the output in ibis-server API calls:
|
||||
|
||||
**Mode A (file path):**
|
||||
```json
|
||||
{
|
||||
"connectionFilePath": "/abs/path/to/target/connection.json",
|
||||
"manifestStr": "...",
|
||||
"sql": "..."
|
||||
}
|
||||
```
|
||||
|
||||
**Mode B (inline):**
|
||||
```json
|
||||
{
|
||||
"connectionInfo": { "host": "...", "port": "5432", ... },
|
||||
"manifestStr": "...",
|
||||
"sql": "..."
|
||||
}
|
||||
```
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
---
|
||||
name: wren-mcp-setup
|
||||
description: Set up Wren Engine MCP server via Docker and register it with an AI agent. Covers pulling the Docker image, running the container with docker run, mounting a workspace, fixing localhost → host.docker.internal for connection info, registering the MCP server in Claude Code (or other MCP clients) using streamable-http transport, and starting a new session to interact with Wren MCP. Trigger when a user wants to run Wren MCP in Docker, configure Claude Code MCP, or connect an AI client to a Dockerized Wren Engine.
|
||||
description: Set up Wren Engine MCP server via Docker and register it with an AI agent. Covers pulling the Docker image, running the container with docker run, mounting a workspace, configuring connection info via the Web UI (with Docker host hint), registering the MCP server in Claude Code (or other MCP clients) using streamable-http transport, and starting a new session to interact with Wren MCP. Trigger when a user wants to run Wren MCP in Docker, configure Claude Code MCP, or connect an AI client to a Dockerized Wren Engine.
|
||||
compatibility: Requires Docker Desktop (or Docker Engine).
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: wren-engine
|
||||
version: "1.2"
|
||||
version: "1.3"
|
||||
---
|
||||
|
||||
# Set Up Wren MCP via Docker
|
||||
@@ -13,10 +13,10 @@ metadata:
|
||||
## Version check
|
||||
|
||||
Silently fetch `https://raw.githubusercontent.com/Canner/wren-engine/main/skills/versions.json`.
|
||||
Compare the `wren-mcp-setup` key with this skill's version (`1.2`).
|
||||
Compare the `wren-mcp-setup` key with this skill's version (`1.3`).
|
||||
If the remote version is newer, notify the user before proceeding:
|
||||
|
||||
> A newer version of the **wren-mcp-setup** skill is available (remote: X.Y, installed: 1.2).
|
||||
> A newer version of the **wren-mcp-setup** skill is available (remote: X.Y, installed: 1.3).
|
||||
> Update with:
|
||||
> ```bash
|
||||
> curl -fsSL https://raw.githubusercontent.com/Canner/wren-engine/main/skills/install.sh | bash -s -- --force wren-mcp-setup
|
||||
@@ -84,17 +84,18 @@ fi
|
||||
> docker rm -f wren-mcp
|
||||
> ```
|
||||
|
||||
The workspace directory is mounted at `/workspace` inside the container. The container auto-loads the MDL and connection info at startup if you provide `MDL_PATH` and `CONNECTION_INFO_FILE` pointing to files inside the workspace.
|
||||
The workspace directory is mounted at `/workspace` inside the container. Place your compiled MDL at `<WORKSPACE_PATH>/target/mdl.json` so the container can load it at startup via `MDL_PATH`.
|
||||
|
||||
**Recommended workspace layout:**
|
||||
|
||||
```
|
||||
<WORKSPACE_PATH>/
|
||||
└── target/
|
||||
├── mdl.json # Compiled MDL (from wren-project build)
|
||||
└── connection.json # Connection info JSON
|
||||
└── mdl.json # Compiled MDL (from wren-project build)
|
||||
```
|
||||
|
||||
> **Note:** Connection info is managed via the MCP server Web UI (see Step 3) and persisted to `~/.wren/connection_info.json`. You can also pre-configure this file before starting the container.
|
||||
|
||||
Run the following command, substituting `<WORKSPACE_PATH>` with the path from Step 1:
|
||||
|
||||
```bash
|
||||
@@ -102,14 +103,13 @@ docker run -d \
|
||||
--name wren-mcp \
|
||||
-p 8000:8000 \
|
||||
-p 9000:9000 \
|
||||
-p 9001:9001 \
|
||||
-e ENABLE_MCP_SERVER=true \
|
||||
-e MCP_TRANSPORT=streamable-http \
|
||||
-e MCP_HOST=0.0.0.0 \
|
||||
-e MCP_PORT=9000 \
|
||||
-e WREN_URL=localhost:8000 \
|
||||
-e CONNECTION_FILE_ROOT=/workspace \
|
||||
-e MDL_PATH=/workspace/target/mdl.json \
|
||||
-e CONNECTION_INFO_FILE=/workspace/target/connection.json \
|
||||
-v <WORKSPACE_PATH>:/workspace \
|
||||
ghcr.io/canner/wren-engine-ibis:latest
|
||||
```
|
||||
@@ -121,19 +121,18 @@ docker run -d \
|
||||
--name wren-mcp \
|
||||
-p 8000:8000 \
|
||||
-p 9000:9000 \
|
||||
-p 9001:9001 \
|
||||
-e ENABLE_MCP_SERVER=true \
|
||||
-e MCP_TRANSPORT=streamable-http \
|
||||
-e MCP_HOST=0.0.0.0 \
|
||||
-e MCP_PORT=9000 \
|
||||
-e WREN_URL=localhost:8000 \
|
||||
-e CONNECTION_FILE_ROOT=/workspace \
|
||||
-e MDL_PATH=/workspace/target/mdl.json \
|
||||
-e CONNECTION_INFO_FILE=/workspace/target/connection.json \
|
||||
-v /Users/me/my-mdl-files:/workspace \
|
||||
-v /Users/me/wren-workspace:/workspace \
|
||||
ghcr.io/canner/wren-engine-ibis:latest
|
||||
```
|
||||
|
||||
> If `MDL_PATH` or `CONNECTION_INFO_FILE` are not set (or the files don't exist yet), the container starts without a loaded MDL. You can deploy later using the `deploy` MCP tool.
|
||||
> If `MDL_PATH` is not set (or the file doesn't exist yet), the container starts without a loaded MDL. You can deploy later using the `deploy` MCP tool or the Web UI.
|
||||
|
||||
This starts the container using the image `ghcr.io/canner/wren-engine-ibis:latest` with:
|
||||
|
||||
@@ -141,6 +140,7 @@ This starts the container using the image `ghcr.io/canner/wren-engine-ibis:lates
|
||||
|---------|------|---------|
|
||||
| ibis-server | 8000 | REST API for query execution and metadata |
|
||||
| mcp-server (streamable-http) | 9000 | MCP endpoint for AI clients |
|
||||
| Web UI | 9001 | Configuration UI (connection info, MDL editor, read-only mode) |
|
||||
|
||||
The workspace directory is mounted at `/workspace` inside the container.
|
||||
|
||||
@@ -152,21 +152,30 @@ docker logs -f wren-mcp
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — Fix connection info: localhost → host.docker.internal
|
||||
## Step 3 — Configure connection info via Web UI
|
||||
|
||||
**Critical:** The container cannot reach the host's `localhost` directly.
|
||||
Open the Web UI at **http://localhost:9001**.
|
||||
|
||||
If the user's database connection info references `localhost` or `127.0.0.1` as the host, it must be changed to `host.docker.internal` so the container can reach a database running on the host machine.
|
||||
The Web UI provides:
|
||||
|
||||
**Examples:**
|
||||
- **Read-only Mode** — toggle to prevent the AI agent from calling `deploy`, `deploy_manifest`, `list_remote_tables`, or `list_remote_constraints`. Useful for read-only query access.
|
||||
- **MDL Status** — shows the currently deployed MDL (models, columns, data source).
|
||||
- **Connection Info** — select the data source type and fill in credentials. Click **Save Connection** to test and persist.
|
||||
- **MDL Editor** — view and edit the live MDL JSON inline. Click **Save & Deploy** to update.
|
||||
|
||||
### Docker host hint
|
||||
|
||||
**Critical:** The container cannot reach the host's `localhost` directly. If your database runs on the host machine, use `host.docker.internal` instead of `localhost` or `127.0.0.1` as the hostname.
|
||||
|
||||
The Web UI shows a hint for this when it detects you may be connecting to a local database:
|
||||
|
||||
| Original | Inside Docker |
|
||||
|----------|--------------|
|
||||
| `"host": "localhost"` | `"host": "host.docker.internal"` |
|
||||
| `"host": "127.0.0.1"` | `"host": "host.docker.internal"` |
|
||||
| `localhost` | `host.docker.internal` |
|
||||
| `127.0.0.1` | `host.docker.internal` |
|
||||
| Cloud/remote host (e.g. `mydb.us-east-1.rds.amazonaws.com`) | No change needed |
|
||||
|
||||
When the user provides connection credentials later (via `setup_connection`), check the `host` field and warn if it is `localhost` or `127.0.0.1`.
|
||||
Fill in the connection credentials in the Web UI form, apply the Docker host hint if needed, and click **Save Connection**.
|
||||
|
||||
---
|
||||
|
||||
@@ -174,7 +183,13 @@ When the user provides connection credentials later (via `setup_connection`), ch
|
||||
|
||||
Claude Code uses **streamable-http** transport to connect to the containerized MCP server.
|
||||
|
||||
Add to `~/.claude/settings.json` under `mcpServers`:
|
||||
**Via Claude Code CLI (recommended):**
|
||||
|
||||
```bash
|
||||
claude mcp add --transport http wren http://localhost:9000/mcp
|
||||
```
|
||||
|
||||
Or add manually to `~/.claude/settings.json` under `mcpServers`:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -187,12 +202,6 @@ Add to `~/.claude/settings.json` under `mcpServers`:
|
||||
}
|
||||
```
|
||||
|
||||
**Via Claude Code CLI (recommended):**
|
||||
|
||||
```bash
|
||||
claude mcp add --transport http wren http://localhost:9000/mcp
|
||||
```
|
||||
|
||||
After adding, **restart Claude Code** for the new MCP server to be loaded into the session. Confirm with:
|
||||
```bash
|
||||
claude mcp list
|
||||
@@ -252,7 +261,7 @@ Look for startup errors or crash loops. If the container exited, `logs` will sho
|
||||
|
||||
### 2. Port already in use
|
||||
|
||||
The container exposes ports **8000** (ibis-server) and **9000** (MCP). If either port is already bound by another process on the host, the `docker run` command will fail with a bind error.
|
||||
The container exposes ports **8000** (ibis-server), **9000** (MCP), and **9001** (Web UI). If any port is already bound by another process on the host, the `docker run` command will fail with a bind error.
|
||||
|
||||
**Check what is using the port:**
|
||||
|
||||
@@ -260,9 +269,7 @@ The container exposes ports **8000** (ibis-server) and **9000** (MCP). If either
|
||||
# macOS / Linux
|
||||
lsof -i :9000
|
||||
lsof -i :8000
|
||||
|
||||
# or with ss (Linux)
|
||||
ss -tlnp | grep -E '8000|9000'
|
||||
lsof -i :9001
|
||||
```
|
||||
|
||||
If another process is occupying the port you have two options:
|
||||
@@ -282,18 +289,22 @@ docker run -d \
|
||||
--name wren-mcp \
|
||||
-p 18000:8000 \
|
||||
-p 19000:9000 \
|
||||
-p 19001:9001 \
|
||||
-e ENABLE_MCP_SERVER=true \
|
||||
-e MCP_TRANSPORT=streamable-http \
|
||||
-e MCP_HOST=0.0.0.0 \
|
||||
-e MCP_PORT=9000 \
|
||||
-e WREN_URL=localhost:8000 \
|
||||
-e WEB_UI_PORT=9001 \
|
||||
-e MDL_PATH=/workspace/target/mdl.json \
|
||||
-v <WORKSPACE_PATH>:/workspace \
|
||||
ghcr.io/canner/wren-engine-ibis:latest
|
||||
```
|
||||
|
||||
Then update the MCP client URL to match:
|
||||
Then update the MCP client URL and Web UI address to match:
|
||||
```bash
|
||||
claude mcp add --transport http wren http://localhost:19000/mcp
|
||||
# Web UI: http://localhost:19001
|
||||
```
|
||||
|
||||
### 3. Container started but MCP endpoint returns an error
|
||||
@@ -307,7 +318,7 @@ claude mcp add --transport http wren http://localhost:19000/mcp
|
||||
|
||||
### 4. Database connection refused inside the container
|
||||
|
||||
If `health_check()` passes but queries fail with a connection error, the database host is likely still set to `localhost`. See **Step 3** above — change it to `host.docker.internal`.
|
||||
If `health_check()` passes but queries fail with a connection error, the database host is likely still set to `localhost`. Open the Web UI at `http://localhost:9001`, edit the connection info, and change the host to `host.docker.internal`.
|
||||
|
||||
---
|
||||
|
||||
@@ -315,14 +326,17 @@ If `health_check()` passes but queries fail with a connection error, the databas
|
||||
|
||||
**Option A — Auto-load at startup (recommended):**
|
||||
|
||||
Place your compiled `mdl.json` in `<WORKSPACE_PATH>/target/mdl.json` and `connection.json` in `<WORKSPACE_PATH>/target/connection.json` before starting the container. The container reads `MDL_PATH` and `CONNECTION_INFO_FILE` at startup and logs:
|
||||
Place your compiled `mdl.json` in `<WORKSPACE_PATH>/target/mdl.json` before starting the container. The container reads `MDL_PATH` at startup and logs:
|
||||
|
||||
```
|
||||
Loaded MDL /workspace/target/mdl.json (9 models, 47 columns)
|
||||
Loaded connection info /workspace/target/connection.json
|
||||
```
|
||||
|
||||
**Option B — Deploy via MCP tool (after container is running):**
|
||||
**Option B — Deploy via Web UI:**
|
||||
|
||||
Open `http://localhost:9001`, paste or edit your MDL JSON in the MDL Editor, and click **Save & Deploy**.
|
||||
|
||||
**Option C — Deploy via MCP tool (after container is running):**
|
||||
|
||||
In the AI client (after MCP is connected):
|
||||
|
||||
@@ -340,25 +354,6 @@ deploy_manifest(mdl=<manifest dict>)
|
||||
|
||||
---
|
||||
|
||||
## Reference: connection info by data source
|
||||
|
||||
When calling `setup_connection`, use these formats. **Replace `localhost` with `host.docker.internal`** if the database runs on your host machine.
|
||||
|
||||
```
|
||||
POSTGRES : {"host": "host.docker.internal", "port": "5432", "user": "...", "password": "...", "database": "..."}
|
||||
MYSQL : {"host": "host.docker.internal", "port": "3306", "user": "...", "password": "...", "database": "..."}
|
||||
MSSQL : {"host": "host.docker.internal", "port": "1433", "user": "...", "password": "...", "database": "..."}
|
||||
CLICKHOUSE : {"host": "host.docker.internal", "port": "8123", "user": "...", "password": "...", "database": "..."}
|
||||
TRINO : {"host": "host.docker.internal", "port": "8080", "user": "...", "catalog": "...", "schema": "..."}
|
||||
DUCKDB : {"path": "/workspace/<file>.duckdb"} ← file must be inside workspace
|
||||
BIGQUERY : {"project": "...", "dataset": "...", "credentials_base64": "..."}
|
||||
SNOWFLAKE : {"account": "...", "user": "...", "password": "...", "database": "...", "schema": "..."}
|
||||
```
|
||||
|
||||
For DuckDB: the database file must be placed in the mounted workspace directory so the container can access it at `/workspace/<file>.duckdb`.
|
||||
|
||||
---
|
||||
|
||||
## You're ready!
|
||||
|
||||
Start a **new Claude Code session** (or restart your MCP client). The Wren tools (`health_check`, `query`, `deploy`, `setup_connection`, etc.) are now available.
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
---
|
||||
name: wren-project
|
||||
description: Save, load, and build Wren MDL manifests as YAML project directories for version control. Use when a user wants to persist an MDL as human-readable YAML files, load a YAML project back into MDL JSON, or compile a YAML project to a deployable mdl.json file. Also manages connection info stored in connection.yml and compiled to target/connection.json.
|
||||
description: Save, load, and build Wren MDL manifests as YAML project directories for version control. Use when a user wants to persist an MDL as human-readable YAML files, load a YAML project back into MDL JSON, or compile a YAML project to a deployable mdl.json file.
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: wren-engine
|
||||
version: "1.4"
|
||||
version: "1.5"
|
||||
---
|
||||
|
||||
# MDL Project
|
||||
@@ -12,10 +12,10 @@ metadata:
|
||||
## Version check
|
||||
|
||||
Silently fetch `https://raw.githubusercontent.com/Canner/wren-engine/main/skills/versions.json`.
|
||||
Compare the `wren-project` key with this skill's version (`1.4`).
|
||||
Compare the `wren-project` key with this skill's version (`1.5`).
|
||||
If the remote version is newer, notify the user before proceeding:
|
||||
|
||||
> A newer version of the **wren-project** skill is available (remote: X.Y, installed: 1.4).
|
||||
> A newer version of the **wren-project** skill is available (remote: X.Y, installed: 1.5).
|
||||
> Update with:
|
||||
> ```bash
|
||||
> curl -fsSL https://raw.githubusercontent.com/Canner/wren-engine/main/skills/install.sh | bash -s -- --force wren-project
|
||||
@@ -36,7 +36,6 @@ YAML files use **snake_case** field names for readability. The compiled `target/
|
||||
```text
|
||||
my_project/
|
||||
├── wren_project.yml # Project metadata (catalog, schema, data_source)
|
||||
├── connection.yml # Data source connection parameters
|
||||
├── models/
|
||||
│ ├── orders.yml # One file per model
|
||||
│ ├── customers.yml
|
||||
@@ -45,19 +44,14 @@ my_project/
|
||||
└── views.yml # All views
|
||||
```
|
||||
|
||||
After building, compiled files are written to:
|
||||
```
|
||||
After building, the compiled file is written to:
|
||||
```text
|
||||
my_project/
|
||||
└── target/
|
||||
├── mdl.json # Deployable MDL JSON (camelCase)
|
||||
└── connection.json # Connection info JSON (camelCase)
|
||||
└── mdl.json # Deployable MDL JSON (camelCase)
|
||||
```
|
||||
|
||||
> **Security note**: `connection.yml` may contain credentials. Add `target/` and `connection.yml` to `.gitignore` before committing.
|
||||
>
|
||||
> **Secrets policy (default)**: When generating `connection.yml`, leave sensitive fields empty with a `# TODO` comment — never ask the user for passwords or credentials in this conversation. ibis-server reads `target/connection.json` directly via its `connectionFilePath` parameter, so secrets stay in the file and out of the LLM context.
|
||||
>
|
||||
> **Testing / inline mode (opt-in)**: If the user explicitly says they are in a test/development environment and willing to share their credentials, you may help construct the full `connection.yml` or `connectionInfo` dict with actual values inline. Always confirm this intent before proceeding — do not assume.
|
||||
> **Note**: Connection info is managed separately via the MCP server Web UI, not stored in the project directory.
|
||||
|
||||
---
|
||||
|
||||
@@ -73,23 +67,8 @@ version: "1.0"
|
||||
catalog: wren
|
||||
schema: public
|
||||
data_source: POSTGRES
|
||||
connection_mode: security # "security" | "inline" (default: "security")
|
||||
```
|
||||
|
||||
`connection_mode` records how the user has chosen to manage connection credentials:
|
||||
|
||||
| Value | Meaning |
|
||||
|-------|---------|
|
||||
| `security` | `connection.yml` / `target/connection.json` contain sensitive credentials. **Never read these files without explicit user confirmation.** |
|
||||
| `inline` | Credentials are provided inline (test/dev environment). Connection files may be read freely. |
|
||||
|
||||
**Security mode rules** — enforced whenever `connection_mode: security` (or the field is absent):
|
||||
|
||||
1. **Never** read `connection.yml` or `target/connection.json` without first asking the user for permission.
|
||||
2. **Never** display, log, or echo the contents of those files.
|
||||
3. If debugging requires connection info, ask the user to share only the non-sensitive fields (e.g. `host`, `port`, `database`, `user`) and leave out passwords, tokens, and keys.
|
||||
4. After building the project, do **not** read `target/connection.json` to verify — confirm success by other means (e.g. checking that the file exists).
|
||||
|
||||
### `models/<model_name>.yml`
|
||||
|
||||
One file per model. Example for `orders`:
|
||||
@@ -140,19 +119,13 @@ relationships:
|
||||
views: []
|
||||
```
|
||||
|
||||
### `connection.yml`
|
||||
|
||||
Connection parameters for the data source. Field names use **snake_case** in YAML and are converted to **camelCase** in `target/connection.json`.
|
||||
|
||||
Follow the **wren-connection-info** skill (`skills/wren-connection-info/SKILL.md`) for the per-connector field reference, secrets policy, and how to generate `connection.yml` with sensitive fields left as `# TODO` comments.
|
||||
|
||||
---
|
||||
|
||||
## Load YAML project → MDL JSON
|
||||
|
||||
To assemble a YAML project back into an MDL JSON dict:
|
||||
|
||||
1. Read `wren_project.yml` → extract `catalog`, `schema`, `data_source`, `connection_mode`
|
||||
1. Read `wren_project.yml` → extract `catalog`, `schema`, `data_source`
|
||||
2. Read every file in `models/*.yml` → collect into `models` list
|
||||
3. Read `relationships.yml` → extract `relationships` list
|
||||
4. Read `views.yml` → extract `views` list
|
||||
@@ -170,29 +143,17 @@ To assemble a YAML project back into an MDL JSON dict:
|
||||
}
|
||||
```
|
||||
|
||||
To load connection info:
|
||||
|
||||
1. Check `connection_mode` from `wren_project.yml`. If it is `security` (or absent), **ask the user for permission before reading `connection.yml`**.
|
||||
2. Read `connection.yml`
|
||||
3. Resolve any `${VAR_NAME}` placeholders from environment variables
|
||||
4. **Rename snake_case keys to camelCase** (see Field mapping section below)
|
||||
5. Result is a flat JSON object ready to pass as `connectionInfo` to ibis-server APIs
|
||||
|
||||
---
|
||||
|
||||
## Build YAML project → `target/`
|
||||
|
||||
Same as **Load** above, but write both compiled files:
|
||||
Same as **Load** above, but write the compiled file:
|
||||
|
||||
- `<project_dir>/target/mdl.json` — assembled MDL JSON (camelCase)
|
||||
- `<project_dir>/target/connection.json` — connection info JSON (camelCase)
|
||||
|
||||
**Important**: If `connection_mode` is `security` (or absent), do NOT read `connection.yml` without user confirmation, and do NOT read or display `target/connection.json` after building — it contains credentials.
|
||||
|
||||
After building:
|
||||
- Pass `mdl_file_path="<project_dir>/target/mdl.json"` to `deploy()` to activate the MDL
|
||||
- Pass `connectionFilePath="<absolute_path>/target/connection.json"` in API requests instead of inline `connectionInfo`
|
||||
— ibis-server reads the file directly, so secret values never enter the LLM context
|
||||
- Connection info is managed via the MCP server Web UI — no file needed
|
||||
|
||||
---
|
||||
|
||||
@@ -214,40 +175,26 @@ When converting between YAML (snake_case) and JSON (camelCase):
|
||||
|
||||
All other MDL fields (`name`, `type`, `catalog`, `schema`, `table`, `condition`, `models`, `columns`, `cached`, `properties`) are the same in both formats.
|
||||
|
||||
**Connection fields:**
|
||||
|
||||
See the **wren-connection-info** skill (`skills/wren-connection-info/SKILL.md`) for the full field mapping and secrets policy.
|
||||
|
||||
---
|
||||
|
||||
## Typical workflow
|
||||
|
||||
```
|
||||
1. Set up data source and connection info
|
||||
Follow the wren-connection-info skill to choose data source type, gather credentials,
|
||||
and produce connection.yml (sensitive fields as # TODO) + target/connection.json.
|
||||
Use connectionFilePath="<abs_path>/target/connection.json" in all subsequent API calls.
|
||||
1. Generate MDL
|
||||
Follow the generate-mdl skill to introspect the database and build the MDL JSON dict.
|
||||
|
||||
2. Generate MDL
|
||||
Follow the generate-mdl skill to introspect the database and build the MDL JSON dict,
|
||||
using the connectionFilePath from step 1 for all ibis-server calls.
|
||||
|
||||
3. Save project
|
||||
2. Save project
|
||||
Write wren_project.yml + models/*.yml + relationships.yml + views.yml
|
||||
(convert camelCase → snake_case from the MDL JSON).
|
||||
Set connection_mode in wren_project.yml based on the user's chosen mode (default: security).
|
||||
connection.yml was already written in step 1.
|
||||
|
||||
4. Add target/ and connection.yml to .gitignore
|
||||
5. Commit project directory to version control (without secrets)
|
||||
3. Add target/ to .gitignore
|
||||
4. Commit project directory to version control
|
||||
|
||||
6. Later — Build: read wren_project.yml first to check connection_mode.
|
||||
If security mode, ask user before reading connection.yml.
|
||||
Read remaining YAML files, rename snake_case → camelCase,
|
||||
write target/mdl.json and target/connection.json.
|
||||
Do NOT read or display target/connection.json (security mode).
|
||||
5. Later — Build: read wren_project.yml, then models/*.yml, relationships.yml, views.yml.
|
||||
Rename snake_case → camelCase, write target/mdl.json.
|
||||
|
||||
6. Connection info: configure via the MCP server Web UI
|
||||
(typically http://localhost:9001; use the Docker host hint when running in a container)
|
||||
|
||||
7. Deploy: deploy(mdl_file_path="./target/mdl.json")
|
||||
use connectionFilePath="<absolute_path>/target/connection.json" in API requests
|
||||
(ibis-server reads the file directly — secrets stay out of this conversation)
|
||||
```
|
||||
|
||||
@@ -5,7 +5,7 @@ compatibility: Requires Docker Desktop (or Docker Engine). No local database dri
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: wren-engine
|
||||
version: "1.1"
|
||||
version: "1.2"
|
||||
---
|
||||
|
||||
# Wren Quickstart
|
||||
@@ -13,10 +13,10 @@ metadata:
|
||||
## Version check
|
||||
|
||||
Silently fetch `https://raw.githubusercontent.com/Canner/wren-engine/main/skills/versions.json`.
|
||||
Compare the `wren-quickstart` key with this skill's version (`1.1`).
|
||||
Compare the `wren-quickstart` key with this skill's version (`1.2`).
|
||||
If the remote version is newer, notify the user before proceeding:
|
||||
|
||||
> A newer version of the **wren-quickstart** skill is available (remote: X.Y, installed: 1.1).
|
||||
> A newer version of the **wren-quickstart** skill is available (remote: X.Y, installed: 1.2).
|
||||
> Update with:
|
||||
> ```bash
|
||||
> curl -fsSL https://raw.githubusercontent.com/Canner/wren-engine/main/skills/install.sh | bash -s -- --force wren-quickstart
|
||||
@@ -68,10 +68,8 @@ Recommended workspace layout after the quickstart completes:
|
||||
│ └── *.yml
|
||||
├── relationships.yml
|
||||
├── views.yml
|
||||
├── connection.yml
|
||||
└── target/
|
||||
├── mdl.json # Compiled MDL — loaded by Docker container
|
||||
└── connection.json # Connection info — loaded by Docker container
|
||||
└── mdl.json # Compiled MDL — loaded by Docker container
|
||||
```
|
||||
|
||||
---
|
||||
@@ -110,14 +108,12 @@ Direct the skill to write the project files into `<WORKSPACE_PATH>`:
|
||||
- `<WORKSPACE_PATH>/models/*.yml`
|
||||
- `<WORKSPACE_PATH>/relationships.yml`
|
||||
- `<WORKSPACE_PATH>/views.yml`
|
||||
- `<WORKSPACE_PATH>/connection.yml`
|
||||
|
||||
Then build the compiled targets:
|
||||
Then build the compiled target:
|
||||
|
||||
- `<WORKSPACE_PATH>/target/mdl.json`
|
||||
- `<WORKSPACE_PATH>/target/connection.json`
|
||||
|
||||
The Docker container will auto-load these files at startup.
|
||||
The Docker container will auto-load this file at startup.
|
||||
|
||||
---
|
||||
|
||||
@@ -133,11 +129,23 @@ Pass `<WORKSPACE_PATH>` as the workspace mount path when the skill asks.
|
||||
|
||||
The wren-mcp-setup skill will:
|
||||
1. Start the container with `-v <WORKSPACE_PATH>:/workspace`
|
||||
2. Set `MDL_PATH=/workspace/target/mdl.json` and `CONNECTION_INFO_FILE=/workspace/target/connection.json`
|
||||
2. Set `MDL_PATH=/workspace/target/mdl.json`
|
||||
3. Register the MCP server with the AI client (`claude mcp add`)
|
||||
4. Verify the container is running
|
||||
|
||||
> If the MDL files already exist in `<WORKSPACE_PATH>/target/` before the container starts, they are loaded automatically at boot. No separate `deploy` call is needed.
|
||||
> If `<WORKSPACE_PATH>/target/mdl.json` already exists before the container starts, it is loaded automatically at boot. No separate `deploy` call is needed.
|
||||
|
||||
### 3b — Configure connection info via Web UI
|
||||
|
||||
Once the container is running, open the MCP server Web UI to configure connection info:
|
||||
|
||||
```text
|
||||
http://localhost:9001
|
||||
```
|
||||
|
||||
Enter the data source credentials (host, port, database, user, password, etc.) in the UI form and save. The MCP server stores and applies the connection info without exposing credentials to this conversation.
|
||||
|
||||
> **Tip:** If your database is running locally, use `host.docker.internal` instead of `localhost` as the host address.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user