Commit Graph
5 Commits
Author SHA1 Message Date
wizardchen d46fa25e41 feat(plugin): dynamic form options, OAuth connections and resource limits
Plugin config forms can now ask the plugin while they are filled in:

- x-options {name, dependsOn, search} loads a field's choices from
  /v1/options/{name}. POST /api/v1/plugins/:id/options merges the form's
  values with the stored secrets (system, tenant or instance scope) so a
  redacted token still works. Go and Python SDKs, OpenAPI and conformance
  cover the endpoint.
- x-oauth turns a string field into an account connection. WeKnora runs
  the authorization code flow (PKCE, client credentials from the plugin's
  system config), keeps the tokens sealed in plugin_oauth_connections and
  the field holds "oauth:<id>". The invoker swaps references for fresh
  access tokens at call time, refreshing near expiry. Callback:
  /api/v1/plugin-oauth/callback, one-time states in Redis or memory.
- SchemaForm gains an options widget (dependsOn reload, remote search)
  and an OAuth connect widget; connector, web search, workspace and
  platform config forms provide the plugin behind them.

Host plugins are capped per runtime.resources on Linux: RLIMIT_DATA for
memory (WEKNORA_PLUGIN_MEMORY_DEFAULT when undeclared) and, with a
delegated cgroup v2 (WEKNORA_PLUGIN_CGROUP), memory.max and cpu.max.
2026-09-26 11:51:35 +08:00
wizardchen e095ee7444 feat(plugin): event delivery and webhooks for plugins
Events:
- A plugin lists what it wants in permissions.events: knowledge.ingested,
  knowledge.failed, knowledge.deleted or chat.answered. The admin reviews
  the list at install; a plugin that only listens needs no other
  contribution.
- A publish enqueues one plugin:event task per subscribed plugin the
  workspace has on (maintenance queue, Redis or Lite).
- The task calls the plugin's new /v1/events at least once. Retryable
  failures retry up to 10 times, keeping the event ID; other failures drop
  the event.
- The plugin host refuses a process that subscribes without serving
  events.

Where events come from:
- A document reaches completed or failed through a dozen paths but a few
  repository writes, so a knowledge repository wrapper raises
  knowledge.ingested and knowledge.failed on the transition only:
  UpdateKnowledge, UpdateKnowledgeForTransfer, UpdateKnowledgeColumns,
  FinalizeSubtask, CompleteProcessingWithoutSubtasks.
- knowledge.deleted comes from the shared delete path.
- chat.answered comes from where RAG and agent answers are persisted. The
  stop button's early completion, which has no question, is skipped.

Webhooks:
- contributes.webhooks gives each workspace a secret URL per webhook:
  /api/v1/plugin-callbacks/{plugin}/{webhook}/{tenant}.{mac}. The MAC is
  keyed from SYSTEM_AES_KEY/JWT_SECRET.
- WeKnora answers 404 unless the URL is genuine, the webhook exists and the
  workspace has the plugin on. Calls are rate-limited per URL, bodies are
  capped at 1 MB and cookies are stripped.
- The call is relayed to the plugin's /v1/webhooks/{id} with that
  workspace's envelope; the plugin verifies the sender.
- Admins get the URLs from GET /plugins/:id/webhooks, shown with copy
  buttons in the plugin center's configure drawer. Plugins get them in
  context.webhooks when APP_EXTERNAL_URL is set.

SDKs:
- Go: OnEvent and Webhook.
- Python: @plugin.on_event and @plugin.webhook.
- Conformance, OpenAPI and READMEs are updated.

Also: examples/plugins/activity (Python) keeps a workspace activity feed
from events and a secret-checked note webhook, and shows it on a toolbox
page.
2026-09-26 11:21:55 +08:00
wizardchen 3dc1470299 feat(plugin): sandboxed plugin pages and the @weknora/plugin-ui bridge
Plugins can add pages to the app. Each is an HTML entry under ui/ in the
package:

| Point              | Shown as                      | Default minRole |
| ------------------ | ----------------------------- | --------------- |
| `pages`            | toolbox tabs                  | viewer          |
| `settingsSections` | a section under Plugins in settings | admin     |
| `kbTabs`           | a tab on each knowledge base (gets knowledgeBaseId) | viewer |

Declarative plugins may have pages too.

Isolation:
- A page runs in `<iframe sandbox="allow-scripts allow-forms allow-popups
  allow-popups-to-escape-sandbox">` without allow-same-origin, so it has an
  opaque origin and cannot read the app's storage.
- Its files come from /api/v1/plugin-ui/assets/{id}/{version}/ui/...
  (served without login, only files under ui/ of the loaded version).
- They carry a CSP with connect-src 'none', so the page cannot reach the
  network.

Bridge:
- The page talks to the app only over postMessage: `api.request`,
  `ui.toast`, `ui.confirm`, `ui.navigate`, `ui.resize`, `ui.close`.
- The app answers only its frame's window, rate-limited, and validates
  paths and sizes.
- It sends init (plugin, mount, locale, role, theme tokens, context) and
  theme/locale changes.
- api.request goes to POST /api/v1/plugins/:id/ui-request. The app checks
  the workspace has the plugin on and the caller meets the page's minRole
  (same rules as the route guards), then calls the plugin's new
  /v1/ui/request endpoint with the caller's role.

SDKs:
- Go: `p.UI` and `UIJSON`.
- Python: `@plugin.ui` and `UIResponse`.
- Both list "ui" in their manifest; conformance checks the endpoint; the
  OpenAPI spec documents it.
- packages/plugin-ui is the page side: a dependency-free ES module with
  types that connects, applies the theme as --wk-* CSS variables and keeps
  the frame sized.

Also:
- Fix the Python SDK reading Go timestamps with fewer than 6 fraction
  digits on Python < 3.11. Key-value entries tripped on it.
- examples/plugins/links: a Python plugin with all three kinds of page,
  storing links in the Host API key-value store.
2026-09-26 10:54:33 +08:00
wizardchen b0767453e4 feat(plugin): document parsers from code plugins
Plugins can contribute parser engines (contributes.parsers, with the
fileTypes they handle). Each registers under its qualified ID
(acme.ocr/ocr) next to the builtin engines and the docreader's, kept in a
separate store so the builtin plugin catalog never describes it as a
builtin. A knowledge base picks it in its parser rules like any engine;
uploads of its file types are accepted.

The engine calls POST /v1/parsers/{id}/parse with the file as base64 and
takes back Markdown plus the images it references, which the pipeline
stores and chunks as usual. A retryable plugin error (unavailable, rate
limited) fails the call so the task retries; any other error fails the
document for good. Other engines' tenant credentials are never passed to
a plugin. A plugin engine that is gone no longer falls through to the
Python docreader; the parse fails saying the plugin is not running. The
engine list honours the workspace's plugin switches and shows plugin
engines by their localized names.

SDK: p.Parser(id, ...), openapi.yaml and a conformance check.
examples/plugins/subtitles parses SRT and WebVTT into a transcript and
counts parsed files per workspace through the Host API key-value store.
2026-09-25 23:50:08 +08:00
wizardchen 39045543d1 feat(pluginsdk): extension protocol v1, Go SDK and conformance suite
A new module, github.com/Tencent/WeKnora/pluginsdk, so plugin authors import
the protocol without WeKnora's dependency tree:

- pluginapi: the protocol (envelope with context and the system / tenant /
  instance config scopes, error codes, NDJSON stream events, web search and
  connector messages, the host handshake line, HMAC request signing) plus
  openapi.yaml describing it for other languages.
- pluginsdk: build a plugin in Go. Register web search providers and
  connectors, then Serve: started by the plugin host it listens on the
  socket it is given, prints the handshake and accepts only the host token;
  run on its own it verifies signed requests. Envelope deadlines reach the
  handler's context, panics become internal errors, and fetches stream
  items and checkpoints.
- client: calls a plugin over a handshake socket or HTTPS, with typed
  transport errors; WeKnora's adapters and the conformance suite share it.
- conformance + cmd/weknora-plugin-conformance: checks a running plugin
  answers discovery, rejects unauthenticated and malformed calls, and
  answers every endpoint it claims with an output or a protocol error.

A test keeps openapi.yaml and the SDK routes in step. A new workflow builds
and tests the module on Linux, macOS and Windows; go-lint lints it.
2026-09-25 22:26:29 +08:00