qilifanandClaude Opus 4.8
9afb8b7b68
Lark and Feishu are the same product deployed on two isolated clouds (open.larksuite.com and open.feishu.cn). WeKnora touches both from two places — the IM bot and the Wiki data-source connector — and this brings Lark to parity across both. Neither API surface differs, so rather than duplicate either implementation, each is parametrized by a Region and registered twice. IM (internal/im/feishu): Every API call routes through a.api(), which prefixes the region's host, and the WebSocket client passes larkws.WithDomain — the SDK defaults to open.feishu.cn, so Lark long connections would otherwise fail to authenticate. Data source (internal/datasource/connector/feishu): Lark was half-supported: the client honoured a base_url override and a LarkBaseURL constant existed, but the connector registered only as "feishu" and the UI never exposed base_url, so Lark wikis could not be configured at all. Lark is now its own connector type. base_url is kept as an explicit override so data sources that pointed a "feishu" connector at open.larksuite.com keep working. Bugs found and fixed along the way: - Wiki resource links were hardcoded to https://feishu.cn/wiki/{token} in four places, so a Lark wiki linked into the wrong cloud entirely. - Bot identity is now platform-prefixed (feishu:cli_x vs lark:cli_x). BotIdentity carries a unique index, so without this a Lark channel would be rejected as a duplicate of an unrelated Feishu bot sharing an app_id. - The card image cache is now keyed by app_id. image_keys are issued per app on one cloud; a Feishu key handed to a Lark app fails the card update with code=200570. - The datasource doc claimed Lark "adapts automatically" — it could not, since the UI had no way to reach base_url. Permissions are where the clouds genuinely diverge. Identical APIs do not mean identical permission catalogues: aily:file:read, aily:file:write and corehr:file:download exist on Feishu but not on Lark, so pasting Feishu's scope JSON into Lark fails to import. That JSON is also over-broad (it covers products WeKnora never calls) and simultaneously incomplete (it omits the drive:*/docx:* scopes the connector needs). Lark therefore gets its own list, derived from the 17 endpoints the code actually calls and checked identifier-by-identifier against Lark's own API reference, plus a full Feishu→Lark mapping table marking each scope verified, unverified, or absent. Both features use tenant_access_token only — no user_access_ token exists anywhere in the repo — so the user dimension stays empty. Known limitation: Region.ThinkingText makes Lark's initial card placeholder English, but once streaming starts the think-block headers come from the shared MarkdownThinkStyle and are Chinese on every platform (Lark, Slack, Telegram, Mattermost alike). Fixing that means threading a locale through FormatIMDisplayContent, which has no platform context — a cross-platform change, not a Lark one. Documented as such. The channel-creation 400 message is now derived from validIMPlatforms so the list cannot drift as platforms are added. Tests: region invariants and per-region URL construction on both sides, the Feishu/Lark bot-identity split, platform-to-channel mapping, base_url override precedence, an end-to-end SendReply against a fake Open Platform (token fetch, reply, thread-unsupported fallback, hard-error surfacing), and a ListResources drive asserting wiki links never leak the other cloud's host. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
| English | 简体中文 | 日本語 | 한국어 |
Overview • Architecture • Key Features • Getting Started • API Reference • Developer Guide
💡 WeKnora — Turn Documents into Living Knowledge with RAG, Agents and Auto-Wiki
📌 Overview
WeKnora is an open-source, LLM-powered knowledge framework built for enterprise-grade document understanding, semantic retrieval, and autonomous reasoning.
It is organized around three core capabilities: RAG-based Quick Q&A for everyday lookups, a ReAct Agent that autonomously orchestrates retrieval, MCP tools and web search to handle complex multi-step tasks, and a brand-new Wiki Mode in which agents distill raw documents into a self-maintaining, interlinked markdown knowledge base with an interactive knowledge graph. Combined with multi-source ingestion (Feishu / Notion / Yuque / RSS, and growing), website embed widgets for publishing agents to external sites, 20+ LLM provider integrations, full Langfuse observability, enterprise-ready multi-workspace RBAC (4-tier role matrix + per-resource ownership + per-workspace audit log), and a fully self-hostable modular architecture, WeKnora turns scattered documents into a queryable, reasoning-capable, continuously evolving knowledge asset.
The framework supports auto-syncing knowledge from Feishu, Notion, and Yuque (more data sources coming soon), handles 10+ document formats including PDF, Word, images, and Excel, and can serve Q&A directly through IM channels like WeCom, Feishu, Slack, and Telegram. It is compatible with major LLM providers including OpenAI, DeepSeek, Qwen (Alibaba Cloud), Zhipu, Hunyuan, Gemini, MiniMax, NVIDIA, and Ollama. Its fully modular design allows swapping LLMs, vector databases, and storage backends, with support for local and private cloud deployment ensuring complete data sovereignty. WeKnora also integrates with Langfuse for comprehensive observability into agent reasoning, token usage, and pipeline tracing.
✨ Latest Updates
- v0.6.3 — Website embed widget & Integrations Center (secure-mode token exchange + rate limits); chat experience overhaul (citation popovers, RAG pipeline progress, streaming markdown); document multi-tag & batch reparse; Wiki folders & hierarchy navigation; RSS data source; MCP OAuth2; EPUB / MHTML parsing; agent model-readiness checks; model test debugger; session source filter; workspace deletion UI. See
CHANGELOG.md. - v0.6.2 — Per-upload process configuration with upload-confirm dialog; document reparse with
process_config;weknoraCLI v0.9 (bundled Agent Skills,session stop, auth/profile harmonization); KB marquee multi-select; HNSW index for 1024-dim pgvector embeddings; chat resources store refactor; Langfuse-only tracing (Jaeger removed). SeeCHANGELOG.md. - v0.6.1 — Document parsing trace timeline (Langfuse-style span tree with stage-by-stage progress + stop-parse); OpenSearch vector store driver; declarative built-in models via YAML; system admin & consolidated platform settings + audit log; new-user onboarding guide; settings UI redesign;
weknoraCLI v0.7 / v0.8 (agent-first wire contract, NDJSON,--dry-run); OpenDataLoader + PaddleOCR-VL parsers; MCP server multi-transport (stdio / SSE / HTTP); per-model thinking-mode config; Tencent LKEAP rerank + native Gemini embeddings + MiniMax-M3. SeeCHANGELOG.md. - v0.6.0 — Workspace RBAC (4-tier role matrix
Owner/Admin/Contributor/Viewer+ per-KB ownership + per-workspace audit log), workspace member management & multi-workspace UX, self-service workspaces;weknoraCLI v0.4 GA withmcp serve; KB retrieval fan-out across vector stores; AES-256-GCM credential encryption + docreader gRPC TLS + Token; Zhipu embedder + Huawei OBS; server-side user preferences; Go 1.26.0. Seedocs/RBAC说明.mdandCHANGELOG.md. - v0.5.2 — Wiki ingest scales to 40k-document KBs (task queue + DLQ); MCP human-in-the-loop tool approval; Anthropic / Apache Doris / Tencent VectorDB / KS3 / SearXNG backends; adaptive 3-tier chunking with live preview; global ⌘K command palette; Yuque connector + WeChat Mini Program;
weknoraCLI preview. - v0.5.1 — Knowledge-base batch management; workspace-wide IM channels overview; session search + user-scoped pinning; unified Model / Web Search / MCP settings cards; per-agent LLM timeout; desktop workspace switching.
- v0.5.0 — Wiki Mode GA — agents auto-generate structured, interlinked Markdown wiki pages with a knowledge graph; wiki browser + visual graph in the UI.
- v0.4.0 — WeKnora Cloud (hosted LLM + parsing); Chrome Extension; ClawHub Skill; WeChat IM; attachment processing; Azure OpenAI / Alibaba OSS; Notion connector; Baidu + Ollama web search; VectorStore management.
- v0.3.6 — ASR (audio); Feishu data-source auto-sync; OIDC; IM quote-reply context + thread-based sessions; document summarization; Tavily search; parallel tool calling; agent @mention scope restriction.
- v0.3.5 — Telegram / DingTalk / Mattermost IM; IM slash commands + QA queue; suggested questions; VLM auto-describe MCP tool images; Novita AI; channel tracking.
- v0.3.4 — WeCom / Feishu / Slack IM; multimodal image support; NVIDIA model API; Weaviate; AWS S3; AES-256-GCM API-key encryption; built-in MCP service; hybrid-search optimization;
final_answertool. - v0.3.3 — Parent-child chunking; KB pinning; fallback response; passage cleaning for rerank; storage auto-creation; Milvus.
- v0.3.2 — Knowledge Search entry; per-source parser & storage engine config; image rendering in local storage; document preview; Volcengine TOS; Mermaid rendering; batch session management; memory graph preview.
- v0.3.0 — Shared Space; Agent Skills + sandboxed execution; custom agents; Data Analyst agent; thinking mode; Bing / Google web search; API Key auth; Helm chart; Korean i18n; Qdrant.
- v0.2.0 — Agent Mode (ReACT); multi-type knowledge bases (FAQ + document); conversation strategy config; DuckDuckGo web search; MCP tool integration; new UI with agent mode switching; MQ async task management.
📱 Interface Showcase
💬 Intelligent Q&A Conversation |
|
📖 Wiki Browser |
🕸️ Wiki Knowledge Graph |
🤖 Agent Mode · Tool Call Process |
⚙️ Conversation Settings |
🔭 Observability · Langfuse Tracing |
|
🏗️ Architecture
Fully modular pipeline from document parsing, vectorization, and retrieval to LLM inference — every component is swappable and extensible. Supports local / private cloud deployment with full data sovereignty and a zero-barrier Web UI for quick onboarding.
🧩 Feature Overview
Intelligent Conversation
| Capability | Details |
|---|---|
| Intelligent Reasoning | ReACT progressive multi-step reasoning, autonomously orchestrating knowledge retrieval, MCP tools, and web search |
| Quick Q&A | RAG-based Q&A over knowledge bases for fast and accurate answers |
| Wiki Mode | Agent-driven auto-generation of structured, interlinked markdown Wiki pages from raw documents |
| Tool Calling | Built-in tools, MCP tools (incl. OAuth2 remote services), web search |
| Conversation Strategy | Online Prompt editing, retrieval threshold tuning, multi-turn context awareness |
| Suggested Questions | Auto-generated question suggestions based on knowledge base content |
| Citations & RAG Progress | Inline citation popovers, shared markdown rendering, and stage-by-stage RAG pipeline progress in chat |
| Session Management | Filter and group sidebar sessions by source (Web / IM / Embed) |
Knowledge Management
| Capability | Details |
|---|---|
| Knowledge Base Types | FAQ / Document / Wiki with folder import, URL import, multi-tag management, and online entry |
| Per-Upload Process Config | Override parser, chunking, multimodal (VLM / ASR), graph extraction, and question generation per upload batch via upload-confirm dialog or process_config API; reparse with new settings |
| Batch Reparse | Re-queue parsing for multiple documents at once with optional per-batch process_config |
| Data Source Import | Auto-sync from Feishu / Notion / Yuque / RSS feeds (more data sources coming soon); incremental and full sync |
| Document Formats | PDF / Word / Txt / Markdown / HTML / EPUB / MHTML / Images / CSV / Excel / PPT / JSON |
| Retrieval Strategies | BM25 sparse / Dense retrieval / GraphRAG / parent-child chunking / HNSW-accelerated pgvector (1024-dim) / multi-dimensional indexing |
| Batch Selection | Marquee drag-select multiple documents in the KB list for batch operations |
| E2E Testing | Full-pipeline visualization with recall hit rate, BLEU / ROUGE metric evaluation |
Integrations & Extensions
| Capability | Details |
|---|---|
| LLMs | OpenAI / Azure OpenAI / Anthropic (Claude) / DeepSeek / Qwen (Alibaba Cloud) / Zhipu / Hunyuan / Doubao (Volcengine) / Gemini / MiniMax / NVIDIA / Novita AI / SiliconFlow / OpenRouter / Ollama |
| Embeddings | Ollama / BGE / GTE / Zhipu / OpenAI-compatible APIs |
| Vector DBs | PostgreSQL (pgvector) / Elasticsearch / OpenSearch / Milvus / Weaviate / Qdrant / Apache Doris / Tencent VectorDB |
| Object Storage | Local / MinIO / AWS S3 / Volcengine TOS / Alibaba Cloud OSS / Kingsoft Cloud KS3 / Huawei Cloud OBS |
| IM Channels | WeCom / Feishu / Lark / Slack / Telegram / DingTalk / Mattermost / WeChat |
| Website Embed | Publish agents via embed widget with domain allowlists, rate limits, and secure-mode token exchange |
| Web Search | DuckDuckGo / Bing / Google / Tavily / Baidu / Ollama / SearXNG |
Platform
| Capability | Details |
|---|---|
| Deployment | Local / Docker / Kubernetes (Helm) with private and offline support |
| UI | Web UI / RESTful API / CLI (weknora) / Chrome Extension / Website Embed Widget / WeChat Mini Program |
| Access Control | Workspace RBAC with 4-tier role matrix (Owner / Admin / Contributor / Viewer), per-KB resource ownership, per-workspace audit log, invite-only workspaces, self-service workspace creation, cross-workspace superuser |
| Security | AES-256-GCM at-rest encryption for API keys and MCP / data-source credentials with graceful key rotation; gRPC TLS + Token between app and docreader; SSRF-safe HTTP client; sandbox isolation for agent skills |
| Observability | Integrated Langfuse (sole tracing backend) for ReAct loops, token tracking, tool calls, and pipeline tracing; built-in Langfuse-style document parsing trace timeline with stage-by-stage progress |
| Task Management | MQ async tasks, automatic database migration on version upgrade |
| Model Management | Centralized config, declarative built-in models via YAML, per-knowledge-base model selection, per-model thinking-mode and embedding-dimension overrides, interactive model test debugger, multi-workspace built-in model sharing, WeKnora Cloud hosted models and parsing |
🧩 Chrome Extension
WeKnora Chrome Extension lets you capture web content directly into your WeKnora knowledge base. Select text, images, or entire pages in the browser and save them as knowledge entries with one click — no copy-paste or file upload needed.
📱 WeChat Mini Program
The WeKnora Mini Program provides a lightweight mobile client for configuring WeKnora API access, selecting knowledge bases, importing URLs, and asking knowledge chat from WeChat.
🦞 ClawHub Skill
WeKnora ClawHub Skill is a WeKnora skill published on the ClawHub platform. Once installed, it enables document import (file / URL / Markdown), hybrid search (vector + keyword) across knowledge bases, and knowledge entry management — all through the WeKnora REST API.
- Document Import — Upload files, import web pages, or write Markdown knowledge via the agent
- Hybrid Search — Search within or across knowledge bases with vector + keyword retrieval
- Knowledge Management — List, browse, edit, and delete knowledge entries programmatically
⌨️ Command-Line Interface
weknora is the official CLI for driving the API from a terminal or an AI
agent. It is agent-first: every command emits a stable JSON envelope by
default (with typed error codes mapped to exit codes), and --format text
renders for humans. It also serves a curated MCP tool surface
(weknora mcp serve) and ships bundled Agent Skills.
weknora profile add prod --host https://kb.example.com --use
weknora auth login
weknora kb list
weknora link --kb my-knowledge-base # bind the current directory
weknora doc upload notes.md
weknora chat "summarise the design doc"
For headless / CI use, set WEKNORA_API_KEY + WEKNORA_HOST and skip
auth login entirely — no credentials written to disk.
See cli/README.md for install + 5-minute quickstart and
cli/AGENTS.md for the operational contract AI agents rely on.
🚀 Getting Started
🛠 Prerequisites
📦 Installation & Launch
git clone https://github.com/Tencent/WeKnora.git
cd WeKnora
cp .env.example .env # Edit .env as needed, see comments in the file
docker compose up -d # Start core services
Once started, visit http://localhost to get started.
To use a local Ollama model, run
ollama serve > /dev/null 2>&1 &first.
🔧 Optional Services (Docker Compose Profiles)
Add --profile flags to enable additional components. Multiple profiles can be combined:
| Profile | Description | Command |
|---|---|---|
| (default) | Core services | docker compose up -d |
full |
All features | docker compose --profile full up -d |
neo4j |
Knowledge Graph (Neo4j) | docker compose --profile neo4j up -d |
minio |
Object Storage (MinIO) | docker compose --profile minio up -d |
langfuse |
Tracing (Langfuse) | docker compose --profile langfuse up -d |
Combine profiles: docker compose --profile neo4j --profile minio up -d
Stop services: docker compose down
🌐 Service URLs
| Service | URL |
|---|---|
| Web UI | http://localhost |
| Backend API | http://localhost:8080 |
| Langfuse Tracing | http://localhost:3000 |
MCP Server
Please refer to the MCP Configuration Guide for the necessary setup.
🔌 Using WeChat Dialog Open Platform
WeKnora serves as the core technology framework for the WeChat Dialog Open Platform, providing a more convenient usage approach:
- Zero-code Deployment: Simply upload knowledge to quickly deploy intelligent Q&A services within the WeChat ecosystem, achieving an "ask and answer" experience
- Efficient Question Management: Support for categorized management of high-frequency questions, with rich data tools to ensure accurate, reliable, and easily maintainable answers
- WeChat Ecosystem Integration: Through the WeChat Dialog Open Platform, WeKnora's intelligent Q&A capabilities can be seamlessly integrated into WeChat Official Accounts, Mini Programs, and other WeChat scenarios, enhancing user interaction experiences
📘 API Reference
Troubleshooting FAQ: Troubleshooting FAQ
Detailed API documentation is available at: API Docs
Product plans and upcoming features: Roadmap
🧭 Developer Guide
⚡ Fast Development Mode (Recommended)
If you need to frequently modify code, you don't need to rebuild Docker images every time! Use fast development mode:
# Start infrastructure
make dev-start
# Start backend (new terminal)
make dev-app
# Start frontend (new terminal)
make dev-frontend
Development Advantages:
- ✅ Frontend modifications auto hot-reload (no restart needed)
- ✅ Backend modifications quick restart (5-10 seconds, supports Air hot-reload)
- ✅ No need to rebuild Docker images
- ✅ Support IDE breakpoint debugging
Detailed Documentation: Development Environment Quick Start
🤝 Contributing
Welcome to submit Issues or Pull Requests.
Process: Fork → Create branch → Commit changes → Open PR
Standards: Format code with gofmt, follow Conventional Commits (feat: / fix: / docs: / test: / refactor:)
🔒 Security Notice
Important: Starting from v0.1.3, WeKnora includes login authentication functionality to enhance system security. For production deployments, we strongly recommend:
- Deploy WeKnora services in internal/private network environments rather than public internet
- Avoid exposing the service directly to public networks to prevent potential information leakage
- Configure proper firewall rules and access controls for your deployment environment
- Regularly update to the latest version for security patches and improvements
👥 Contributors
Thanks to these excellent contributors:
📄 License
This project is licensed under the MIT License. You are free to use, modify, and distribute the code with proper attribution.