mirror of
https://github.com/Tencent/WeKnora.git
synced 2026-09-19 10:28:49 +08:00
Claude
49adf0d73c
Three documentation passes around the adaptive chunking work: UI - Frontend ChunkOverlap default consolidated to 80 (was 100), matching chunker.DefaultChunkOverlap on the backend. Both DEFAULT_CHUNKING_PRESET and initFormData updated. The KB-load fallback also uses 80 when a loaded KB has no chunk_overlap stored. - All four locales (en-US, zh-CN, ko-KR, ru-RU) get rewritten chunking setting descriptions: each now states the validated range, the default, and the situations where you'd deviate (FAQ vs narrative, embedder token limits, language-specific corpora). Source code - splitter.go: DefaultChunkSize / DefaultChunkOverlap constants get a longer block-comment explaining the per-language token math and the use-case sweet spots, plus the migration note on what the old inconsistent defaults were. - KBChunkingSettings.vue: new comment block above ChunkingConfig documents the slider min/max for each setting, why those bounds exist, and the recommended TokenLimit values per embedding model. Repo docs - New docs/CHUNKING.md: end-to-end guide covering why chunking matters, the adaptive 3-tier architecture, per-setting reference with ranges and sweet spots, parent-child explanation, the token-limit table per embedder (OpenAI / Voyage / Cohere / BGE / MiniLM / Jina), 7 use-case presets, the debug panel workflow, the API surface, and known trade-offs (recursive strategy hidden from UI, no auto-reindex on strategy switch, OCR limitations). - CHANGELOG.md gets a new [Unreleased] section consolidating all the adaptive-chunking work shipped on this branch: 5 features, 8 improvements, 6 fixes, 1 docs entry. The entry references docs/CHUNKING.md for deeper explanation. https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k