* feat(agent): preview skill artifacts and verify installs without running them
Generated files open in the chat drawer with overlay/Esc returning to the
list, and install verification no longer executes guessed skill entry
points. A failed install can be retried from the saved archive.
* ui(chat): use a folder icon and corner count for skill artifacts
The eye and document glyphs read as preview or copy, and TDesign's
loading spinner sat beside the icon. A folder with a top-right count
matches the generated-files drawer without covering the toolbar action.
* feat(artifact): implement inline artifact references with previews
Add support for inline artifact references in Markdown, allowing generated files to be displayed as clickable cards. Implemented a new rendering mechanism for artifacts, enabling previews directly from the chat interface. Updated localization files for new inline preview hints and missing file messages. Introduced tests for artifact reference normalization and rendering logic.
* fix(client): document AgentResponseType constants for revive
Touching the const block made golangci-lint require comments on every
exported value. Match the rest of the SDK and describe artifacts_pending.
* fix(agent): wrap artifact-reference prompt lines for lll
Pre-push golangci-lint rejected the branch on three lines over 120
characters in the sandbox: reference guidance.
* fix(agent): scope HTML scripts and tighten install verification
Keep scripted HTML preview on skill artifacts only, recreate sandbox
workspace dirs after they are deleted, and treat nested skill files as
dependencies rather than first-party imports.
* refactor(artifact): enhance artifact reference handling and metadata
Updated artifact metadata to include a stable resource handle for better identification. Adjusted artifact reference resolution to support both handle and name forms, ensuring compatibility with existing references. Enhanced tests to validate the new handling logic and ensure proper rendering of artifacts in chat messages. Improved documentation for clarity on artifact reference formats and their usage in the system.
A leftover .weknora-patched marker made the build skip recopying even when
Cargo.toml was missing. Require that file before reusing the tree, and build
the archive before cargo audit so CI has the gitignored patch path.
* test(datasource): cover scoped sync deletions
- deleted connector items remove the matching KB knowledge, scoped per data
source via metadata datasource_id + external_id
- SyncDeletions=false neither deletes nor counts
- stream handler classifies deleted/failed items like the batch loop
* fix(datasource): perform real sync deletions scoped per data source
- applyFetchedItem now performs the KB deletion instead of only counting
- lookup is scoped to datasource_id + external_id so identical external IDs
from two data sources cannot collide or overwrite each other
- ingestItem update-path lookup uses the same scoped query
* fix(datasource): note failed deletions retry only on full sync
- SyncResult.DeletionFailed tracks deletion failures (subset of Failed)
- streaming partial message warns that failed deletions only retry on a full
sync, since the connector cursor is checkpointed past the deleted item
* fix(datasource): humanize deletion errors, keep raw detail in logs
- raw lookup/delete errors go to server logs only, with full context
- UI samples carry a humanized "see server logs" fallback
- add datasource.syncError.* i18n keys
Note: ko-KR / ru-RU syncError translations were generated without a
native-speaker review.
* fix(datasource): hard-delete synced rows and tighten deletion scoping
Sync-internal deletions now call HardDeleteKnowledge after the soft-delete
cascade so tombstones cannot block re-sync; subtree sweeps and URL metadata
attach failures are handled with datasource scoping and explicit errors.
* chore: add git hooks aligned with CI and PR checks
Install via scripts/install-git-hooks.sh (sets core.hooksPath). Pre-commit
runs whitespace/gofmt/golangci-lint on staged files; pre-push mirrors
app/frontend/cli workflows on the diff since origin/main.
* fix: gofmt test file; tighten pre-push to match CI app workflow
Pre-push now gofmts the full PR diff and runs go vet on all app packages
(like app.yml), not only Go files in the latest commit. Previous pushes
skipped checks via --no-verify and because hook-only commits had no .go files.
* fix(hooks): align pre-push diff with GitHub PR base via gh
Use three-dot diff against the open PR baseRefOid (same as CI) instead of
a two-dot merge-base range; run full go vet on every push with Go PR changes.
* style: gofumpt datasource_service_test.go
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* feat(sandbox): unify remote sandbox on E2B protocol with self-hosted support
- Add e2b.proxy_url gateway config and generic gateway transport layer
- Add envd compat layer for Basic auth and multipart /files uploads
- Support Dockerfile.sandbox runtime and e2b-backend build targets in CI
- Rename sandbox image non-root user to align with E2B template convention
- Add sandbox-protocol.md and extend cluster deployment / local repro docs
Enables connecting to any E2B-compatible control plane (E2B Cloud,
CubeSandbox, Agent-Sandbox, etc.) without per-backend client code.
* style(sandbox): gofmt tenant_resolver struct alignment
This update addresses the issue where wiki summary-page slugs (summary/<knowledgeID>) were being mangled during the citation compaction process, leading to dead links. The changes include:
- Reordering the message encoding process to ensure resource references are aliased before source references, preventing the UUID from being replaced in the summary slug.
- Implementing a new function to selectively replace document IDs while preserving the integrity of summary slugs.
- Adding unit tests to verify that summary slugs remain intact and are not altered during compaction.
This fix enhances the reliability of cross-links in the wiki system and prevents potential dead links from being generated.
* feat(compose): align env vars in docker-compose and .env.example with code
docker-compose.yml / docker-compose.dev.yml
- app.environment: remove deprecated vars Go app no longer reads
(CRYPTO_MASTER_KEY, CRYPTO_SALT, TENANT_AES_KEY, ENABLE_GRAPH_RAG);
add missing vars the code actually consumes (OSS/S3/TOS/COS completion,
OpenSearch, Tencent VectorDB, Milvus credentials, Redis TLS/namespace,
OIDC, audit/invitation/quota, LLM tuning, doc timeouts, etc.)
- docreader.environment: complete PDF tuning knobs, SSRF/LOG_LEVEL/gRPC
params, GRPC_MTLS_REQUIRE_CLIENT_CERT; fix DOCREADER_PDF_JPEG_QUALITY
default 90 -> 85 to match code (config.py)
- mcp.environment: add MCP_SERVER_AUTH_TOKEN (required by HTTP transport),
WEKNORA_CHAT_TIMEOUT, WEKNORA_VERIFY_SSL, MCP_ALLOWED_UPLOAD_DIRS
- dev compose docreader.environment kept symmetric with prod
.env.example
- reorganize into A-J sections (deploy / data / retrieval / models /
parsing / auth / agent / integrations / observability / security)
- remove the four deprecated vars, document why
- add missing vars, fix NEO4J_URI to bolt:// (matches compose + single-node),
fix DORIS_COMPAT_MODE typo, complete docreader PDF tuning defaults
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(helm): inject NEO4J_ENABLE and drop deprecated TENANT_AES_KEY
helm graph bug fix
- NEO4J_ENABLE is the sole graph switch the Go app reads (since v0.1.6,
ENABLE_GRAPH_RAG is no longer read). helm never injected NEO4J_ENABLE,
so GraphRAG was silently broken: setting neo4j.enabled=true gave the
app NEO4J_URI/USERNAME/PASSWORD but not the switch, and the UI rejected
Node Extractor config with "请正确配置环境变量NEO4J_ENABLE".
- app.yaml: inject NEO4J_ENABLE=true inside the neo4j.enabled block;
remove the dead ENABLE_GRAPH_RAG injection
- NOTES.txt: update GraphRAG instructions (just enable neo4j, no need
to set the deprecated ENABLE_GRAPH_RAG)
- values.yaml: drop ENABLE_GRAPH_RAG, fix neo4j section comment
TENANT_AES_KEY cleanup (deprecated since v0.4.0, encryption now uses
SYSTEM_AES_KEY exclusively; Go app no longer reads it)
- secrets.yaml: stop generating/looking-up/writing TENANT_AES_KEY,
fix stale comment about tenants.api_key rotation
- app.yaml: remove TENANT_AES_KEY secretKeyRef
- values.yaml: drop tenantAesKey, fix existingSecret key list + comments
- cloud-image/firstboot.sh: stop generating/writing/printing TENANT_AES
- cloud-image/README.md: drop TENANT_AES_KEY from credential list
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
- Added a new script `build_frontend_dist.sh` to handle the building of frontend static assets.
- Updated `docker-compose.yml` to include a note about running the build script before building the frontend service.
- Modified the GitHub Actions workflow to integrate the new build script for frontend assets.
- Adjusted the Dockerfile to copy built assets directly from the `dist` directory.
- Refined the `.dockerignore` to exclude unnecessary files while keeping the `dist` directory.
- Added a new build argument `COMMIT_ID_ARG` to pass the commit ID during the Docker build process.
- Updated the Dockerfile to set an environment variable `VITE_FRONTEND_COMMIT` with the commit ID.
- Enhanced the Vite configuration to resolve the frontend commit ID from the environment or Git.
- Modified the SystemInfo component to display the frontend commit ID in the UI.
- Implemented a version info preparation step in the GitHub Actions workflow to capture the commit ID.
This commit removes Jaeger tracing support from the project, including the associated Docker configurations, Go modules, and middleware. The changes simplify the codebase by eliminating unused tracing functionality, which was previously integrated for observability. Documentation and scripts have also been updated to reflect the removal of Jaeger references.
Introduce opendataloader and PaddleOCR-VL parser engines with tenant-level
settings UI, replace liteparse, and harden Excel/PPT/Markdown parsing.
Optional odl-hybrid sidecar stays local-build only and is excluded from
default dev-start and full profiles.
Large scanned PDFs (hundreds of pages) were slow and fragile end-to-end.
This change addresses the parse, transport, and task-scheduling layers:
docreader (parse + transport):
- Parallelize per-page scanned rendering across processes (forkserver/fork),
with serial fallback. ~4-7x faster on large scanned PDFs; pdfium is not
thread-safe so we fan out across processes. Configurable via
DOCREADER_PDF_RENDER_PARALLELISM.
- Add server-streaming ReadStream RPC: emit one meta frame then one frame per
image, so documents with many page images are no longer capped by the unary
gRPC message-size limit (a 874-page PDF produced ~193MiB of images, far over
the 50MB cap) and memory is bounded on both ends. Unary Read is kept for
backward compatibility; the Go production reader switches to ReadStream.
VLM:
- Make the VLM HTTP timeout configurable (VLM_HTTP_TIMEOUT_SECONDS) and raise
the default 90s -> 180s so dense scanned-page OCR does not time out with
"context deadline exceeded".
Async task queues:
- Isolate high-volume, model-heavy fan-out tasks into dedicated asynq queues so
a single large document cannot saturate the shared worker pool and block
user-facing document parsing:
image:multimodal -> "multimodal"
chunk:extract -> "graph"
question:generation -> "question"
- Register the new queues in the server weight map and the cancel inspector's
scanned-queue set (so cancelling a knowledge still purges its pending tasks).
Updated the Vite configuration to allow dynamic setting of the API proxy target based on environment variables. The default target is now configurable via VITE_DEV_PROXY_TARGET or FRONTEND_BACKEND_URL, enhancing flexibility for different development setups. Additionally, the development script logs the current API proxy target for better visibility during startup.
Bump version to v0.5.2 across VERSION, Helm chart, frontend package
files, and cloud-image script comment. Update CHANGELOG and all four
language READMEs with v0.5.2 highlights (Wiki Mode at scale, MCP
human-in-the-loop approval, new LLM/vector/storage/search backends,
adaptive 3-tier chunking, global command palette, CLI preview, etc.).
Mainland China cloud VMs (Tencent Lighthouse, Aliyun, etc.) frequently
cannot reach get.docker.com, github.com, or even community GitHub
mirrors like gh-proxy.com. The cloud-image bootstrap previously had no
escape hatch for this and failed at the very first curl.
This adds a new DOCKER_INSTALL_MIRROR env var to prepare.sh. When set,
it skips get.docker.com and installs docker-ce + compose-plugin from an
apt mirror of Docker's official repo (e.g. mirrors.tencent.com,
mirrors.aliyun.com).
README.md also gets:
- A GH_PROXY env var threaded through bootstrap methods A and B so the
initial script pull can route through gh-proxy / ghfast.
- An explicit recommendation to prefer method C (scp from local) on
mainland China VMs.
- A consolidated "三件套" table mapping WEKNORA_GH_PROXY /
DOCKER_INSTALL_MIRROR / DOCKER_REGISTRY_MIRROR to per-cloud
endpoints, so users hit one place to copy the full env.
- Updated cleanup.sh to avoid recreating .env during cleanup, preventing exposure of default passwords before firstboot.
- Modified firstboot.sh to create .env from .env.example only if it doesn't exist, ensuring no sensitive data is present before initialization.
- Added support for Docker Hub and GitHub tarball download acceleration via new environment variables WEKNORA_GH_PROXY and DOCKER_REGISTRY_MIRROR.
- Implemented a mechanism to prune old WeKnora images based on the current version, reducing image size and maintaining a clean environment.
- Enhanced README.md with instructions for using the new acceleration features and image pruning options.
Updated the cleanup script to replace the broad `docker system prune` command with more targeted commands. This change ensures that only unused containers, build cache, and dangling volumes are removed, preventing the accidental deletion of important images like `wechatopenai/weknora-*` that are pre-pulled for firstboot. The script now explicitly prunes containers, builders, and dangling volumes to maintain a clean environment without disrupting the intended image setup.
Address review feedback on PR #1249:
- prepare.sh: when WEKNORA_REF looks like a version tag (v*), write the
matching WEKNORA_VERSION into .env so docker compose pulls images that
match the compose YAML's git ref (previously stuck on :latest).
- prepare.sh: detect docker binary path via `command -v docker` and
template it into weknora.service (replacing hardcoded /usr/bin/docker
that fails when docker lives in /usr/local/bin).
- firstboot.sh: write a /opt/WeKnora/.firstboot.done marker immediately
after rewriting .env, before `docker compose up -d`. If compose fails
mid-run, the next boot is gated by ConditionPathExists=!marker so we
never regenerate DB_PASSWORD against an already-initialized postgres
volume (which previously bricked the database).
- firstboot.sh: stop deleting its own unit file / script while the
oneshot is still executing; rely on the marker + `systemctl disable`
instead, avoiding "job failed" markings from systemd.
- firstboot.sh: use detected docker path instead of /usr/bin/docker;
add note in credentials file that .env is the source of truth.
- weknora-firstboot.service: add ConditionPathExists=!.firstboot.done.
- cleanup.sh: scope docker volume deletion to compose project label
(com.docker.compose.project=<name>) instead of fuzzy substring match
that could nuke unrelated postgres/redis volumes.
- cleanup.sh: also remove .firstboot.done marker, firstboot log, and
any leftover /root/weknora-credentials.txt so the image is clean.
- README.md: clarify how to actually disable registration (edit the
`replace` call list in firstboot.sh, not run that command in shell).
Recommend `sudo -i` to avoid the classic `sudo cmd >> file` failure
where the shell redirection runs as the unprivileged user. Also document
the `sudo tee -a` workaround and add a scp option C.
Add scripts and docs for packaging WeKnora into cloud images (AMI,
custom images, snapshots) so users can distribute one-click deployable
templates on any cloud provider.
- scripts/cloud-image/: cloud-agnostic prepare/cleanup/firstboot scripts
plus systemd units. Downloads only the 4 runtime files needed by the
compose stack (~100KB) instead of cloning the full repo, and pins to
any git ref via WEKNORA_REF for reproducible builds.
- firstboot.sh randomizes DB/Redis/JWT/AES secrets on first boot,
writes credentials to /root/weknora-credentials.txt and self-removes.
- docs/cloud-image/: per-platform packaging guides. Includes a guide
for Tencent Cloud Lighthouse / CVM covering image creation, sharing,
and marketplace listing.
Default-on services match the unprofiled compose stack (frontend, app,
docreader, postgres, redis); optional services (qdrant, milvus,
neo4j, langfuse, etc.) remain opt-in via compose profiles to keep the
image size small.
- Added a new entrypoint script to manage ownership of bind-mounted directories and merge built-in skills into preloaded ones.
- Updated the Dockerfile to include the `gosu` package for privilege management and to set the entrypoint to the new script.
- Ensured built-in skills are preserved and accessible after bind-mounting user directories, improving the application's flexibility and usability.
These changes streamline the container initialization process and enhance the management of skills within the application.
- Added `db_version` field to the `SystemInfo` interface to expose the current database migration version.
- Updated the system information response to include the database version, reflecting its state during application runtime.
- Enhanced the UI in the SystemInfo component to display the database version with appropriate labels and descriptions in multiple languages.
This update improves transparency regarding the database state within the system information settings.
- Added support for customizable APT mirror in the Dockerfile for the docreader service, allowing users to specify a mirror via build arguments.
- Updated docker-compose.yml to pass the APT_MIRROR argument during the build process.
- Modified build_images.sh script to include the APT_MIRROR argument when building the docreader image.
- Updated .gitignore to exclude .cursor/ directory.
This update improves flexibility in package management during the image build process.
lyingbug