Commit Graph
2534 Commits
Author SHA1 Message Date
veaterandwizardchen 37837c00a9 fix(mcp-server): migrate to MCPServer high-level API (mcp 2.x) [v1.1.0]
MCP Python SDK 2.0 removed the low-level Server decorator API
(@app.list_tools() / @app.call_tool() / app.get_capabilities()), so the
server crashed at startup with:
  AttributeError: 'Server' object has no attribute 'list_tools'

The released PyPI package weknora-mcp==1.0.1 declared 'mcp>=1.0.0' with
no upper bound, so uvx resolved to the latest 2.0.0 and the plugin's
weknora MCP server could no longer connect.

Fix: rewrite the server on top of the mcp 2.x high-level MCPServer API
(formerly FastMCP), and cap the dependency at 'mcp>=2,<3'.

- weknora_mcp_server.py: replace @app.list_tools()/@app.call_tool() with
  28 @mcp.tool() function-signature tools (input schema auto-generated
  from type hints, descriptions from docstrings, plain return values
  serialized by the framework). Transports use run_stdio_async() /
  sse_app() / streamable_http_app(). WeKnoraClient business logic is
  unchanged.
- deps: mcp>=2,<3 in pyproject.toml / requirements.txt / setup.py;
  commit uv.lock pinning mcp 2.0.0.
- version bump to 1.1.0 (pyproject, setup, __init__, main --version,
  server_version).
- check_imports.py: validate MCPServer / ToolError / mcp_types imports.
- ci(.github/workflows/mcp-server.yml): publish on merge to main +
  manual dispatch, not only on tags; add PyPI version-dedup check so
  re-runs on an unchanged version stay green (absorbed from PR #2399).

Verified: stdio initialize + tools/list returns all 28 tools; no
list_tools AttributeError; tool dispatch + error wrapping confirmed.

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-07-30 15:52:28 +08:00
wizardchen cbe668c06f fix(ci): scope gofmt check to PR commits and format kb_access_test
Compare base...head for pull_request events instead of the merge commit
so unrelated main-branch files are not checked. Also gofmt
kb_access_test.go, which was unformatted on main.
2026-07-30 14:13:15 +08:00
wizardchen fafa925b26 perf(ci): speed up App workflow Go dependency caching
Use setup-go v6 default go.mod cache key instead of go.sum, warm the
module cache with an explicit go mod download step, and scope vet to the
same package set as tests.
2026-07-30 14:13:15 +08:00
wizardchen 4efc987abb fix(ci): make DocReader client tests runnable in PR checks
Start the DocReader gRPC server in the DocReader workflow, skip client
integration tests when the server is unavailable, exclude docreader
packages from the App workflow test scope, and trigger DocReader checks
on go.mod/go.sum changes.
2026-07-30 14:13:15 +08:00
wizardchen a52137d450 ci: add app frontend and docreader checks 2026-07-30 14:13:15 +08:00
wizardchen cc846d8e29 fix: harden shared agent source selector validation
Reject invalid agent_source_tenant_id instead of silently falling back, fix a resolveAgent nil panic when a shared selector fails, and add regression tests for wiki write-tool filtering and agent resolution.
2026-07-30 12:28:23 +08:00
wizardchen ab9f705843 fix: harden shared agent source permissions 2026-07-30 12:28:23 +08:00
wizardchen 57fb1de45d fix(knowledge): harden batch tag dialog submit and list refresh
Keep the batch tag dialog open until the API succeeds, reset pagination
after batch updates, and add structural tests for BatchTagDialog.
2026-07-30 12:04:38 +08:00
Binks Riverton de262ba6dd fix(knowledge): pre-select common tags in batch tag dialog 2026-07-30 12:04:38 +08:00
Binks Riverton 0f667e0fd8 feat(knowledge): add batch tag dialog for selected documents 2026-07-30 12:04:38 +08:00
Binks Riverton 3f94895d60 fix(organization): pass avatar when creating shared space
The create flow was missing the avatar field in both the store's create() function signature and the modal's create call, while the update flow correctly passed it. Added avatar parameter to orgStore.create() and included formData.avatar in the create invocation.
2026-07-30 11:57:53 +08:00
wizardchen db9bdc27cd fix(session): scope API runtime admin-console bypass to owned sessions
Replace the broad channel-runtime principal check with owner-scoped rules so
API callers can still read their own api_tenant_key and api_external_user
sessions without opening IM sessions matched via legacy empty user_id scope.
2026-07-30 11:56:05 +08:00
wizardchen 123f2d6349 fix(session): include external users in API session group 2026-07-30 11:56:05 +08:00
xiaowang f3cd3d9724 fix(knowledge): support local HTML uploads
English: Accept .html and .htm knowledge uploads across frontend, Go, and DocReader. Parse static HTML with charset-aware decoding, preserve document structure and local links, and keep scripts/network execution disabled. Add regression coverage for validation, engine registration, encoding, and parser behavior.

中文:打通知识库本地 .html 与 .htm 文件从前端、Go 服务到 DocReader 的完整上传链路。使用可识别字符编码的静态解析,保留文档结构和本地链接,不执行脚本或发起网络请求,并补充文件校验、引擎注册、编码及解析行为的回归测试。
2026-07-29 20:11:17 +08:00
wizardchen 55840b1a02 test(mcp-server): restore unittest coverage after workflow dedup
Convert test_module.py to unittest.TestCase so mcp-server.yml's
unittest discover picks up integration checks. Rename test_imports.py
to check_imports.py to keep manual diagnostics out of discovery.
2026-07-29 19:51:47 +08:00
veater 510f6d1dd2 ci(mcp-server): remove duplicate workflow.yml (superseded by mcp-server.yml)
Upstream introduced mcp-server.yml on merge — an improved version of this
file (unittest instead of pytest, python3 in the guard). Keeping both
would run duplicate CI. Drop workflow.yml, keep mcp-server.yml.
2026-07-29 19:51:47 +08:00
wizardchen f0b7618721 refactor(frontend): redesign upload confirmation dialog
Restructure the upload confirm modal into a three-column layout with file list, settings navigation, and KB-editor-aligned configuration panels. Add per-section status summaries, polish parser Excel option alignment, and hide graph settings when unavailable.
2026-07-29 19:39:13 +08:00
wizardchen 4f7a36c7e6 fix(mcp-server): pin MCP Python SDK to v1.x
WeKnora MCP Server still uses v1 low-level Server decorators such as
@app.list_tools(). MCP SDK 2.0 removed those APIs, so fresh installs
could fail at startup with AttributeError. Cap the dependency at <2 and
require >=1.28 for Streamable HTTP transport support.
2026-07-29 15:43:45 +08:00
wizardchen 7d2a6311cb fix(agent): exclude disabled FAQ entries 2026-07-29 14:40:54 +08:00
wizardchen c291c6a59f fix(kb): retry KB delete when engine resolution is deferred
ProcessKBDelete must return retryable engine-resolution failures instead
of logging and continuing, which could delete knowledge rows while leaving
embeddings behind. Align the handler with ProcessIndexDelete and cover
cancellation and ErrVectorStoreUnavailable in tests.
2026-07-29 12:25:18 +08:00
ochan.kwon c15c3e972f fix(retriever): rebuild a missing vector store engine on demand
The engine registry is per-process, so a store's engine is missing
whenever this process did not create it. That happens three ways:

  - Startup skips a store whose engine fails to construct and never
    retries it, so that instance cannot serve the store again -- not
    even after a restart, if the backend is slow to come up.
  - A store registered on one instance is absent on every other until
    they restart, because registration is not broadcast.
  - Registration can fail on the instance handling the create, leaving
    the store missing even there.

Each one surfaces as "vector store is currently unavailable" and is
cleared today only by restarting the process.

Add GetOrLoadByStoreID, which falls back to building the engine from the
store row when the lookup misses, and route the two bound-store lookups
through it. The method goes on RetrieveEngineRegistry rather than
StoreRegistry because that is the interface the factory functions
receive; the two declare GetByStoreID separately and neither embeds the
other.

Concurrent callers collapse onto one build. That build runs on a context
detached from the caller that started it: callers share the result, so
letting the first one's cancellation abort it would fail everyone else
waiting. A panic inside the build is contained, because singleflight
re-raises it on a goroutine the HTTP recovery middleware cannot reach
and the build calls third-party client constructors.

Publishing is guarded by a per-store generation sampled before the
build, so an engine that finishes after the entry was registered or
removed by someone else does not overwrite it and leave that engine
orphaned with its connections open. A failed build starts a cooldown:
collapsing only helps concurrent callers, so without it a backend that
stays down would cost a full build timeout on every sequential request.

Separate "the store is gone" from "its engine cannot be built right
now". Async delete handlers answer ErrVectorStoreNotFound with
asynq.SkipRetry, so reporting a database blip or a backend outage that
way would permanently discard knowledge-base and index delete tasks and
orphan the vector data they exist to remove. ErrVectorStoreUnavailable
covers the retryable cases; the repository already distinguishes them,
returning (nil, nil) for a missing row and an error for a failure.

For the same reason a caller's own cancellation is never folded into a
store sentinel. The ownership check runs first and queries the database
with that context, so on a shutdown it is usually where cancellation is
noticed.

A search resolves one engine per distinct store, sequentially, and this
server sets no HTTP read or write timeout, so the resolution loop gets a
budget above a single build. An exhausted budget still leaves the
engines warming, since the build is detached, and is reported as an
unavailable store because retrying is likely to succeed.

Build failures are logged where they happen, with the panic stack. The
cause cannot travel back to the caller -- factory errors embed endpoints
and credentials -- so without this the one thing an operator needs when
self-healing fails to heal would be discarded with the error.

Note a deliberate contract change: an ownership lookup failure used to
be reported as not-found, and a test asserted that on purpose. A
database failure says nothing about whether the store exists, and
treating it as permanent is what discards the task, so that assertion is
inverted rather than preserved.
2026-07-29 12:25:18 +08:00
wizardchen bf68db4653 fix(storage): allow image uploads to use file service fallback 2026-07-29 11:36:24 +08:00
wizardchen 5db186a0fc fix(knowledge): harden file-type scoped duplicate detection
Use case-insensitive file_type matching, scope the filename fallback by type, and document the dedup contract for datasource sweep handling.
2026-07-29 10:36:17 +08:00
koishi514-Z 18b95ebb3a fix(knowledge): distinguish duplicate files by type 2026-07-29 10:36:17 +08:00
wizardchen 2358564bde fix(knowledge): default custom metadata on create 2026-07-28 22:22:59 +08:00
wizardchen b581094c7b fix(web_fetch): address review follow-ups for structured fetch results
Restore HTTP-only 15s pipeline fetching, reject SPA shell pages when
browser rendering fails, surface per-URL failure fields in the UI, and
keep structured model output when all URLs fail.
2026-07-28 22:17:31 +08:00
langcaiye 75a2eabe20 fix(agent): restore safe rendered web fetch fallback 2026-07-28 22:17:31 +08:00
langcaiye a3da5cb57a fix(agent): preserve web evidence when page fetches fail 2026-07-28 22:17:31 +08:00
wizardchen c193b949aa fix(chunker): require space after English ? and ! in overlap boundaries
Align English question and exclamation marks with period handling so
URL query strings are not split during overlap. Add regression tests for
URL overlap and inline code protection.
2026-07-28 22:02:30 +08:00
ttommybot 21861f4b0f D:\xinchen.ju\Code\WeKnora 2026-07-28 22:02:30 +08:00
ttommybot 2dfc696740 fix: preserve semantic boundaries in chunk overlap 2026-07-28 22:02:30 +08:00
wizardchen 36206ae62f fix(docreader): align XLS header override behavior with XLSX
Normalize Excel file types when resolving parser rules and applying the
xlsx_first_row_as_header override for both xlsx and xls. Skip the override
for non-builtin engines, unify legacy XLS parsing with the XLSX default path,
and extend tests for case handling and explicit false overrides.
2026-07-28 22:00:36 +08:00
chenjunhong 595d19a806 fix(docreader): restore optional XLSX header context 2026-07-28 22:00:36 +08:00
koishi514-Z 0e1e5a63ba fix(chat): prevent duplicate terminal answer events 2026-07-28 21:33:33 +08:00
nullkey 4e3b512af6 fix(im): render resource:// images in IM channels (nginx /r/ + config guards)
IM channels (DingTalk/Feishu) showed broken images because resource://
image references resolve to WeKnora capability links that were neither
reachable nor renderable:

- nginx never proxied /r/, so <APP_EXTERNAL_URL>/r/<token> links fell into
  the SPA fallback and returned a blank page. Add a /r/ proxy block
  mirroring /files, using the ${APP_SCHEME}/${APP_HOST}/${APP_PORT} vars.
- rewriteStorageURLs treated any "dst != src" as success and emitted
  non-HTTP results (e.g. an internal storage:// path when APP_EXTERNAL_URL
  is unset) that IM clients cannot fetch. Only substitute http(s) URLs;
  otherwise leave the reference unchanged and log an actionable WARN.
- Warn once at startup when IM channels are active but APP_EXTERNAL_URL is
  unset — the config state that silently breaks resource:// images on the
  default MinIO/local deployment.
- Document that IM knowledge-base images need an IM-reachable http URL via
  one of two paths: a publicly reachable storage backend, or APP_EXTERNAL_URL
  routing resource:// through nginx /r/.
2026-07-28 21:25:12 +08:00
annopickandwizardchen 400bbbf717 fix(mcp-server): route diagnostics to stderr to fix stdio protocol crash (#2371)
* fix(mcp-server): route diagnostics to stderr to fix stdio protocol crash

Entry scripts (run_server.py, main.py, run.py) printed diagnostics to
stdout via bare print(). Under the stdio transport (the default and the
path used by all MCP clients per MCP_CONFIG.md: `uv run run_server.py`),
stdout is the JSON-RPC channel, so the plain-text lines emitted before
the initialize handshake corrupted the protocol stream. Strict MCP clients
(Claude Desktop / Cursor / ZCode / VS Code Copilot) treated this as a
fatal protocol error and reported the server as failed-to-start.

Redirect all diagnostic print() to sys.stderr (per MCP spec, only
JSON-RPC may use stdout). weknora_mcp_server.py already routes through
logging (stderr) and is unchanged.

Verified against a live backend (initialize -> tools/list (28 tools) ->
tools/call list_knowledge_bases); stdout first byte is now '{' with all
diagnostics on stderr.

* chore(mcp-server): publish package name as weknora-mcp

Rename the PyPI distribution name from weknora-mcp-server to weknora-mcp
(tools used: pyproject [project].name + setup.py name). The installed
console-script commands (weknora-mcp-server, weknora-server) and the
Python module (weknora_mcp_server) are unchanged for backward compatibility.

* fix(mcp-server): include upload_paths in wheel + bump to 1.0.1

The 1.0.0 wheel was missing upload_paths.py: it was absent from the
[tool.setuptools] py-modules list (and setup.py py_modules), so setuptools
never packed it. weknora_mcp_server.py line 26 does
`from upload_paths import resolve_upload_file_path, set_active_transport`,
so any path that fully imports the module — i.e. the MCP client initialize
handshake — crashed with ModuleNotFoundError. `--version` survived only
because argparse exits before that import runs.

Fix: add 'upload_paths' to py-modules (pyproject.toml + setup.py) and bump
to 1.0.1 (1.0.0 is immutable on PyPI). Also sync the user-facing version
strings (__version__, server_version, main --version) to 1.0.1.

* ci(mcp-server): add workflow.yml to build & publish to PyPI via OIDC

Adds .github/workflows/workflow.yml adapted from annopick/tuomin's ci.yml:
- test: matrix Python 3.10–3.13 (matches requires-python >=3.10), runs the
  flat test_*.py suite in mcp-server/
- build: on mcp-server-v* tags, builds sdist + wheel via uv build, and
  guards against the 1.0.0 regression by asserting upload_paths.py is in
  the wheel before publishing
- publish: PyPI Trusted Publishing (OIDC, id-token: write) to weknora-mcp,
  no API token needed

Publishing is gated on tags named mcp-server-v* to decouple from the Go
release workflow that already targets v* tags.

* fix(mcp-server): route diagnostics to stderr and fix packaging/CI

Prevent MCP stdio startup crashes by sending entry-script diagnostics to
stderr, package upload_paths in the wheel, and add unittest-based CI with
stdout purity regression tests.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-07-28 21:23:09 +08:00
Ochan Kwonandwizardchen c9a342d965 fix(docparser): resolve remote images referenced by HTML img tags (#2355)
Remote http(s) images written as <img src="..."> reached no resolver.
Markdown images went through ResolveRemoteImages, HTML data-URIs and HTML
relative paths each had their own pass, but nothing claimed the remaining
combination, so those images were dropped without a log line and the
multimodal stage was skipped for documents that rely on HTML for layout.

ResolveRemoteImages now scans HTML img tags with a quoted src as a second
pass and rewrites only the src attribute value, leaving the tag intact
because the surrounding markup is why these documents use HTML at all.
The per-image work (whitelist, SSRF, download, icon filter, upload) moves
into one helper so the two syntaxes cannot drift apart on policy. Each
pass carries its own image budget; sharing one would let HTML images eat
into the number of Markdown images a document already gets resolved.

An attribute value is trimmed, HTML-unescaped and its scheme lowercased
before use, because HTML permits padding, spells a query separator &amp;
and tolerates any scheme case, while every fetcher downstream compares the
scheme byte-for-byte. Markdown targets get none of that normalization, so
no Markdown reference resolves differently than it does today. A
whitelisted host keeps serving the image from where it already lives, and
its span is rewritten with the normalized URL rather than the raw attribute
text: later stages both locate the image by searching the document for its
serving URL and fetch that same string, so the two have to agree.

The src attribute must now be preceded by whitespace, so a hyphenated
attribute name is no longer mistaken for it. Among HTML tags this cuts two
ways: one carrying both a hyphenated *-src and a real src now resolves the
real src instead of the placeholder, while one carrying only a hyphenated
*-src is no longer resolved at all. Recovering the latter needs an
attribute parser rather than a regex, and resolving a lazy-loading
placeholder while the real image goes unvisited is the worse of the two
failures. Both halves are pinned by tests.

Image-derived text also has to survive into the answer, which is why this
touches internal/searchutil. The chat enricher and the image_info filters
matched Markdown links only, so an HTML-only passage looked image-free:
every entry was dropped and the model received bare markup with the
caption and OCR text stripped, even though the analysis had run. Both now
recognise HTML img tags through one shared regex, so a tag the parser
stores can always be matched back. Both syntaxes are located against the
original passage and spliced in a single pass; scanning one regex over the
other's output let a metadata block that itself quotes an image reference
be rescanned and duplicated. The chat variant still enriches only images
present in the passage, so orphan entries stay out.

Known limits:

  - An unquoted src and a srcset-only tag are not matched, nor is a tag
    whose attribute values hold a bare > or a quoted src= of their own.
  - imgHTMLDataURI still matches src without a boundary, so a data-src
    holding a base64 placeholder is consumed as if it were the src and the
    whole tag replaced. Worth the same fix, but it changes what an existing
    lazy-loading document resolves, so it belongs on its own.
  - A reference shown as a documentation example is rewritten like any
    other. Recognising code spans needs a Markdown parser; a hand-rolled
    scanner risks the far worse failure of treating a document's real
    screenshots as examples.
  - Rewriting a whitelisted src writes the unescaped URL back into an HTML
    attribute, so a query parameter whose name spells an entity renders
    differently than the author wrote it. Keeping the encoded form would
    mean unescaping on every matcher instead.
  - Image text reaches the model only when the whole reference falls inside
    the retrieved window; an img tag is several times longer than a
    Markdown link and straddles a chunk boundary more often.
  - A reference repeated in one document is fetched once per occurrence,
    which is what gives each chunk its own analysed copy.
  - Nothing bounds how long one document's fetches may take in total.
  - Parent-context pruning removes only Markdown images.

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-07-28 21:08:31 +08:00
xiaowang 82d3e0b271 fix(wiki): prevent finalizing stalls on enqueue failure 2026-07-28 20:54:48 +08:00
zhangzherui b4b7509e3a fix(frontend): clarify processing timeline status 2026-07-28 20:45:59 +08:00
zhangzherui 00614b0148 docs: clarify contributor validation workflow 2026-07-28 20:45:30 +08:00
zhangzherui eb447365cd fix(knowledge): clear stale errors on reparse 2026-07-28 20:44:34 +08:00
zhangzherui d59151d872 fix(im): recover after leader lock loss 2026-07-28 20:40:58 +08:00
FincherYuan 0f8232b7eb fix(mcp-server): correct SSE message route to prevent initialization timeout
The SSE transport's POST handler was mounted at /messages/, but the
SseServerTransport SDK prepends the Starlette root_path ("/sse") to
the endpoint parameter, instructing clients to POST to /sse/messages/.
This path mismatch caused MCP initialization requests to never reach
the handler, resulting in a 60s+ timeout on connect.

Fix: move the Mount to /sse/messages/ and place it before the /sse
catch-all route, since Starlette's Mount uses prefix matching and
/sse would otherwise intercept /sse/messages/ requests.
2026-07-28 20:34:56 +08:00
wizardchenandCursor c3b08d6c26 fix(ui): harden chunk editor feedback after code review
Surface index-sync warnings instead of false save success, sync generated-question metadata after inline edits, and reload chunks on 409 revision conflicts.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-28 20:23:48 +08:00
wizardchenandCursor 497e8312e3 fix(knowledge): harden summary refresh after review feedback
Discard stale async refreshes without clobbering newer summaries, queue manual refreshes instead of blocking HTTP, sort edited chunks by index for summaries, escape document metadata in chat headers, and distinguish lookup failures from true staleness.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-28 20:23:48 +08:00
wizardchen 76b3256497 fix(ui): refresh document summary state 2026-07-28 20:23:48 +08:00
wizardchen 0766c5671f fix(retrieval): restore image parent context 2026-07-28 20:23:48 +08:00
wizardchen 431d6ab269 fix(ui): refresh chunk history after edits 2026-07-28 20:23:48 +08:00
wizardchen 48b661134b fix(chunks): retain questions across content edits 2026-07-28 20:23:48 +08:00
wizardchen 6a439a4c3b fix(chunks): keep edited retrieval content consistent 2026-07-28 20:23:48 +08:00