Commit Graph
54 Commits
Author SHA1 Message Date
wizardchen ef1047bf67 feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements
Introduce opendataloader and PaddleOCR-VL parser engines with tenant-level
settings UI, replace liteparse, and harden Excel/PPT/Markdown parsing.
Optional odl-hybrid sidecar stays local-build only and is excluded from
default dev-start and full profiles.
2026-06-03 12:29:13 +08:00
wizardchen 1110615300 feat(web-search): add SearXNG provider (#1166)
支持对接自建/公共 SearXNG 实例作为网络搜索引擎,缓解免费搜索引擎在国内
网络环境下访问受限的问题。

- types: 新增 WebSearchProviderTypeSearxng 与 BaseURL 参数字段;
  类型元数据新增 RequiresBaseURL,前端可动态渲染 Instance URL 表单。
- infrastructure/web_search/searxng.go: 调用 /search?format=json,强制
  utils.ValidateURLForSSRF 校验 base_url,可选 api_key 透传给反代鉴权。
- service: isValidProviderType 与参数校验接入 searxng。
- container: 注册 NewSearxngProvider 工厂。
- frontend: WebSearchSettings 表单根据 requires_base_url 渲染 Instance
  URL 输入框;编辑回填、free 判定同步更新。
- docker: 新增可选 searxng 服务(profile=searxng/full),附带最小化
  settings.yml(启用 JSON 格式、关闭 limiter、关闭遥测),
  docker-compose 默认 SSRF_WHITELIST 包含 searxng 容器名。
- .env.example: 补充 SEARXNG_PORT / SEARXNG_SECRET 说明。

Closes #1166
2026-05-11 16:53:47 +08:00
wizardchen d65e647f95 chore: bump Go to 1.26 and slim docreader dependencies
- Bump base image in docker/Dockerfile.app from golang:1.24 to golang:1.26
  to match `go 1.26` declared in go.mod (fixes CI build failure on
  `go mod download`).
- Drop unused docreader components and their dependencies:
  - Remove `docreader/ocr/` package (paddle/vlm/dummy backends are
    unreferenced by the main flow; OCR/VLM is handled by the Go App).
  - Remove `docreader/parser/storage.py` (dead code; image persistence
    happens in the Go App via inline ImageRef bytes).
  - Remove `docreader/scripts/download_deps.py` (PaddleOCR pre-download).
  - Drop deps: paddleocr, paddlepaddle, openai, ollama, minio,
    cos-python-sdk-v5, oss2, asyncio, pypdf2, markdown, mistletoe,
    goose3, markdownify, pdfplumber, antiword, urllib3.
- Re-lock uv.lock: 145 -> 79 packages.
- Update docreader/README.md to reflect that OCR/VLM/storage are no
  longer configured at the docreader level.
2026-05-09 13:32:40 +08:00
jw_mac faaf6124b2 add output that if install package of offline protoc exitst 2026-04-24 22:42:17 +08:00
Liwx da8191cd57 fix(docker): 修复镜像构建兼容性问题 2026-04-21 23:29:41 +08:00
DaWesen bfa3341d4a Fix: Dockerfile build error (duplicate libsqlite3-0 and ffmpeg installation) 2026-04-10 01:26:07 +08:00
DaWesen b80bdb1079 feat: add video multimodal functionality 2026-04-09 19:19:42 +08:00
wizardchen 90c32b5926 feat: enhance Docker setup with entrypoint script and skill management
- Added a new entrypoint script to manage ownership of bind-mounted directories and merge built-in skills into preloaded ones.
- Updated the Dockerfile to include the `gosu` package for privilege management and to set the entrypoint to the new script.
- Ensured built-in skills are preserved and accessible after bind-mounting user directories, improving the application's flexibility and usability.

These changes streamline the container initialization process and enhance the management of skills within the application.
2026-03-05 18:14:14 +08:00
wizardchen 393ebf9f92 feat: standardize storage configuration and enhance backward compatibility
- Updated the `KnowledgeBase` struct to rename `cos_config` to `storage_config` for consistency across the application.
- Implemented a custom `UnmarshalJSON` method to maintain backward compatibility with legacy responses that still use `cos_config`.
- Modified various frontend components to reflect the new `storage_config` naming, ensuring seamless integration with the updated backend structure.
- Enhanced Dockerfiles and service configurations to support the new storage engine setup.

This update improves the clarity of storage configurations and ensures compatibility with existing data structures.
2026-03-02 21:21:49 +08:00
wizardchen 6d88619869 feat: enhance Dockerfile and build scripts for customizable APT mirror
- Added support for customizable APT mirror in the Dockerfile for the docreader service, allowing users to specify a mirror via build arguments.
- Updated docker-compose.yml to pass the APT_MIRROR argument during the build process.
- Modified build_images.sh script to include the APT_MIRROR argument when building the docreader image.
- Updated .gitignore to exclude .cursor/ directory.

This update improves flexibility in package management during the image build process.
2026-03-02 21:21:49 +08:00
wizardchen 397689d2f3 feat: introduce WeKnora Lite edition with lightweight configuration and deployment
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.

This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
2026-03-02 21:21:49 +08:00
begoniezhao bce968833e feat: Add async knowledge re-parse API and examples 2026-02-11 12:27:55 +08:00
nullkey 43a2c647ea feat: optimize security and deployment of agent skills 2026-02-04 20:08:49 +08:00
nullkey 3f21345546 feat: adapt preloaded skills to smart-reasoning agent 2026-02-04 20:08:49 +08:00
begoniezhao 1b0edccfc4 feat: add download_spatial target and DuckDB spatial extension handling 2026-01-05 15:01:50 +08:00
begoniezhao 7c302bea1f fix(docker): remove default value for APK_MIRROR_ARG in Dockerfile 2025-12-29 21:08:26 +08:00
begoniezhao b4f1846a84 fix(docker): streamline package installation in Dockerfile by removing upgrade step 2025-12-29 20:43:49 +08:00
begoniezhao 907e9a5522 feat: Add DataSchema tool for retrieving schema information from CSV and Excel files 2025-12-29 20:03:51 +08:00
wizardchen 7aed2415e6 feat: 更新Dockerfile,调整uvx权限,升级node版本 2025-12-15 19:13:58 +08:00
wizardchen 4cb8079dce feat: 修复Docker构建中pip安装依赖时的系统包冲突问题,添加--break-system-packages参数 2025-12-12 20:24:29 +08:00
Munish Thakur 54707b71a0 Update Dockerfile.app 2025-12-12 10:31:39 +08:00
wizardchen 0dbb853860 Merge remote-tracking branch 'github_public/main' 2025-11-20 15:49:43 +08:00
begoniezhao 2d66abedf0 feat: 新增文档模型类,调整配置与解析逻辑,优化日志及导入
移除日志设置与冗余代码,优化导入、类型提示及OCR后端管理
统一调整各文件模块导入路径为绝对导入
调整导入路径,移除部分导入,优化日志及注释
升级文档解析器为 Docx2Parser,优化超时与图片处理逻辑
2025-11-18 22:37:01 +08:00
wizardchen 9b7128d69d feat: Update docker-compose and Dockerfile for frontend build configuration
- Added build context and Dockerfile specification for the frontend service in docker-compose.yml.
- Refactored Dockerfile.app to ensure non-root user creation is idempotent and streamlined the ownership setup for application directories.
2025-11-13 20:16:15 +08:00
wizardchen ffd5cd2881 feat: Update Dockerfile to support customizable APK mirror
- Added APK_MIRROR_ARG to allow dynamic configuration of the Alpine package mirror.
- Updated the repository URL in the Dockerfile to use the specified APK mirror if provided.
- Enhanced runtime dependencies installation by including additional packages and updating pip before installing uvx.
2025-11-13 20:01:04 +08:00
wizardchen ae58801baf Merge remote-tracking branch 'github_public/main' 2025-11-13 14:28:41 +08:00
wizardchen 05154c8d75 feat: Enhance MCP service configuration with stdio transport support
- Added support for stdio transport type in MCP service configuration, allowing command execution via standard input/output.
- Updated MCPService interface to include optional stdio_config and env_vars fields for managing command arguments and environment variables.
- Enhanced frontend components to handle stdio transport, including dynamic form fields for command and arguments.
- Implemented validation logic for stdio configuration in the service update process, ensuring proper handling of required fields.
- Updated backend logic to manage stdio transport connections and ensure proper client lifecycle management.
2025-11-12 11:32:59 +08:00
begoniezhao c1f731e026 chore(docreader): 重新组织模块文件 2025-11-05 12:07:39 +08:00
orbisai-security 6d547131b6 fix: semgrep_yaml.docker-compose.security.writable-filesystem-service.writable-filesystem-service_docker/docker-compose.minio.yml_4 2025-10-31 15:09:40 +08:00
begoniezhao 0324e83b2c chore: upspeed build docker 2025-09-19 12:29:03 +08:00
begoniezhao bfd4fffbe3 chore: upspeed build docker 2025-09-19 12:29:03 +08:00
begoniezhao 53215b1f90 chore(build): Optimize Docker build configuration and process 2025-09-18 17:34:14 +08:00
begoniezhao a5c3623a02 chore(build): Optimize Docker build configuration and process 2025-09-18 17:34:14 +08:00
begoniezhao a2a9e3aed9 chore(build): Optimize Docker build configuration and process, adjust task names 2025-09-18 17:34:14 +08:00
begoniezhao 8f723b38fb chore(build): Optimize Docker build configuration and process, adjust task names 2025-09-18 17:34:14 +08:00
begoniezhao d734b80cb5 chore(build): Optimize Makefile and Dockerfile.app build configuration 2025-09-18 14:46:10 +08:00
begoniezhao 7973128f4c chore(build): Optimize Makefile and Dockerfile.app build configuration 2025-09-18 14:46:10 +08:00
wizardchen 98e47e83d2 feat(ui): Support multi knowledgebases operation 2025-09-17 16:02:08 +08:00
wizardchen 91e65d6445 feat(ui): Support multi knowledgebases operation 2025-09-17 16:02:08 +08:00
wizardchen 092b30af3e fix(docreader): Download binary by target arch in docker 2025-09-12 20:21:30 +08:00
wizardchen 74c121f7fb feat: Adjust App & Docreader log output 2025-09-11 23:14:23 +08:00
wizardchen bff0e742fa fix: try fix ocr avx not support 2025-09-11 13:21:21 +08:00
wizardchen 6f6ca84dae feat(docreader): add health check 2025-09-10 20:22:14 +08:00
wizardchen 7cfae7e0d3 fix: pre fetch ocr models in docker container 2025-09-10 17:24:26 +08:00
Liwx1014 3aad892a62 fix:build docreader timeout; update ocr config;support pdf tables parsing 2025-09-08 14:58:37 +08:00
begoniezhao a5cdda3c4c build: Removed platform parameters, added multi-platform builds and simplified image tags 2025-08-22 12:27:26 +08:00
wizardchen 785261313f feat: make CONCURRENCY_POOL_SIZE configurable 2025-08-16 13:27:01 +08:00
begoniezhao a46d15e579 build: Upgrade Python image and LibreOffice, update dependency path and add Playwright dependency 2025-08-15 15:34:21 +08:00
wizardchen 57c6e04e54 fix: 调整构建镜像采用的镜像源 2025-08-14 12:16:08 +08:00
wizardchen 8b43931886 feat: support minio storage 2025-08-14 12:16:08 +08:00