- Added logic to suppress whitespace-only content emitted before the actual answer starts, addressing issues with spurious empty events interleaved with tool call events.
- Enhanced the event emission process to ensure that only meaningful content is routed to the final answer area, improving the clarity and relevance of streamed responses.
- Introduced new tests to ensure correct routing of reasoning and answer content during streaming, addressing UX issues where answers appeared under the Thinking phase before transitioning.
- Implemented a ThinkStreamSplitter to separate inline reasoning from answer text, allowing for real-time updates to the UI without duplication.
- Updated the event emission logic to prevent re-emission of answers that were already streamed live, ensuring a seamless user experience.
- Enhanced the ChatResponse struct to include flags for streamed answers, improving the handling of final answer events.
- Added comprehensive unit tests for the ThinkStreamSplitter to validate its functionality across various scenarios.
- Redesigned SVG assets for screenshots to improve visual consistency and clarity, including updates to gradients, titles, and layout.
- Added a new screenshot asset to showcase the Wiki Knowledge Base feature.
- Enhanced localization strings across multiple languages to reflect updated terminology and descriptions for the knowledge framework, including agentic reasoning and hybrid search strategies.
- Improved the user interface in the login view to include new tags for agent and wiki features, enhancing user awareness of available functionalities.
- Introduced `started_at` and `uptime_seconds` fields to the `SystemInfo` structure for tracking server boot time and uptime.
- Updated the `GetSystemInfo` handler to populate these new fields based on server runtime.
- Enhanced the frontend to display server startup time and uptime in the system information view, improving visibility for users.
- Added localization support for new labels and descriptions related to server uptime and startup time in multiple languages.
- Refactored the trace display for in-flight and failed parsing statuses to improve user interaction.
- Introduced clickable elements for trace links and added buttons with icons for better accessibility and visual clarity.
- Updated styles for trace elements to enhance layout and user experience, ensuring consistent behavior across different statuses.
- Improved hover effects and button interactions to provide clearer feedback to users.
- Updated the `resolveStoreGroups` and `buildRetrievalParams` functions to handle multiple knowledge bases more effectively.
- Changed the structure of `buckets` to store pointers to `KnowledgeBase` objects instead of just IDs, allowing for richer data handling.
- Enhanced the retrieval parameter construction to differentiate between FAQ and document-type knowledge bases, ensuring correct routing for vector and keyword retrieval.
- Improved comments for clarity on the logic behind retrieval parameter construction based on knowledge base types.
- Added a conditional rendering for the `ChatRequestInfoButton` to prevent duplication in the toolbar when a completed answer with content is displayed.
- Introduced a new computed property, `hasDoneAnswerContent`, to track whether a completed answer is rendered, improving the user interface and experience in the chat component.
- Updated the toolbar logic to ensure it only displays the request info button when appropriate, enhancing clarity and usability for users.
- Added detailed communication and final answer instructions to the runtime context block, guiding the model on user-facing text and tool usage.
- Updated the XML structure to reflect these changes, ensuring clarity on metadata and instructions for each turn.
- Improved the overall safety and clarity of the model's output by preventing the exposure of internal tool names and IDs.
- Introduced a new test file for the Langfuse wrapper, implementing tests for the `buildLangfuseGenerationOutput` and `buildLangfuseMessages` functions.
- Validated the output structure for various scenarios, ensuring correct handling of content, tool calls, finish reasons, and reasoning content.
- Enhanced test coverage for the Langfuse functionality, improving reliability and maintainability of the codebase.
- Updated the logic in `Input-field.vue` to ensure that user-selected chat models are preserved when navigating away and returning, preventing unintended resets to default models.
- Enhanced the watch functionality to better manage model ID changes, ensuring consistent behavior when shared agents are involved.
- Modified `index.vue` to update conversation models based on the first model ID, improving the initialization process for chat sessions.
- Refactored the cancel parse logic in `knowledge-processing-timeline.vue` to remove the dialog dependency, directly handling cancellation confirmation within the function.
- Introduced a `t-popconfirm` component for confirming cancellation actions in both `DocumentListView.vue` and `DocumentBatchBar.vue`, improving user experience with clear confirmation prompts.
- Updated the `KnowledgeBase.vue` to utilize new confirmation methods for reparse and delete actions, ensuring consistent user interactions across the application.
- Enhanced localization strings for better clarity in confirmation messages, supporting a more intuitive user experience.
- Refactored the NewUserGuide component to utilize a new event for opening the guide, enhancing event handling.
- Updated UserMenu to include a button for reopening the guide, improving accessibility for users.
- Enhanced localization strings to provide clearer instructions on accessing the guide from the user menu.
- Introduced a new command in the Global Command Palette to open the product tour, streamlining user onboarding.
- Added styles for new guide-related elements to improve visual consistency and user interaction.
- Introduced a new `ChatRequestInfoButton` component to display detailed request information for chat sessions.
- Enhanced the `useStream` function to track the last stream request, including request ID, URL, method, and sanitized body.
- Updated localization files to include new strings for request information in both English and Chinese.
- Integrated the request info button into the chat interface, allowing users to easily access and copy request details.
- Improved handling of stream requests by attaching debug information to messages for better tracking and debugging.
- Introduced a constant `stopWatcherMaxDuration` to limit the lifetime of the stop watcher, preventing potential resource leaks in pathological stream scenarios.
- Updated the `startStopWatcher` function to use a context with a timeout, ensuring the watcher self-terminates after the specified duration.
- Refactored the handling of stop events to improve clarity and maintainability, ensuring the watcher exits appropriately on terminal stream events or user requests.
- Removed the `stopWatchCancel` from the `sseStreamContext` as the watcher now manages its own lifecycle more effectively.
- Updated the QA execution logic to differentiate between user-requested cancellations and actual errors, preventing unnecessary error events from being emitted when a stop is requested.
- Enhanced logging to provide clear feedback when a QA session is cancelled by the user, improving the user experience and reducing confusion.
- Added context cancellation checks to ensure proper handling of stopped generation scenarios, maintaining the integrity of the session flow.
Vector store create and raw connection-test accepted user-supplied
connection addresses without SSRF validation, while every other
admin-entered endpoint (e.g. model BaseURL) already runs
ValidateURLForSSRF. An admin could point an engine address at an
internal host and use the connection probe as an SSRF oracle.
Add validateConnectionAddrSSRF, applied at the two user-input
boundaries only:
- CreateStore: validated before any network dial.
- TestStoreRaw: a new TestRawConnection wrapper runs an engine-type
allowlist, required-field validation, and SSRF validation before
delegating to TestConnection. TestConnection stays validation-free
for trusted callers (env stores, stored configs already validated
at create time).
Validation covers every address-bearing field a driver dials
(Addr, qdrant Host+Port, weaviate Host and GrpcAddress) and
fails closed for unmapped engine types. The Elasticsearch probe no
longer follows redirects, closing a 302-to-internal bypass.
docker-compose whitelists the bundled vector-store service hostnames
(qdrant, milvus, weaviate, doris-fe) via SSRF_WHITELIST_EXTRA so the
out-of-box experience is preserved; postgres is intentionally not
whitelisted. .env.example and the OpenSearch dev guide document how
to whitelist external or local-dev addresses.
Cloning a knowledge base previously copied only the storage path strings
(knowledge.FilePath and chunk.ImageInfo.URL), so the source and the clone
shared the same physical objects in the storage backend. Once the original
file and extracted images are deleted on source removal, the clone is left
with dangling references and its document and images become unreadable —
data loss that occurs even for same-store clones.
Add a CopyFile primitive to the FileService interface and implement it in
every backend: server-side CopyObject on the object stores
(s3/obs/cos/oss/tos/ks3/minio), io.Copy on local, and a no-op on dummy.
Destinations use the knowledge-owned layout and reuse the existing
path/object-key guards; a sentinel ErrCrossBackendCopy is returned when the
source scheme does not match the backend.
Use CopyFile to deep-copy the document file in cloneKnowledge and the
extracted images in CloneChunk and cloneFAQKnowledgeBase via a shared
cloneChunkImageInfo helper that deduplicates identical image URLs per clone
and rewrites them to the new objects. Copied objects are cleaned up
best-effort if a clone fails partway through. A clone-time preflight rejects
cloning into a target bound to a different storage backend when the tenant
pins providers via StorageEngineConfig.
Adds unit tests for local CopyFile (independent copy survives source
deletion, traversal rejection, cross-backend rejection), cloneChunkImageInfo
(empty/multi/dedup/parse-failure/OriginalURL handling), and the storage
provider preflight.
- Introduced new components: AgentCreateContextualGuide, KbCreateContextualGuide, TenantModelsGuide, and SpotlightGuide to enhance user onboarding.
- Implemented dynamic step configurations for each guide, allowing tailored user experiences based on context.
- Enhanced the existing NewUserGuide component to utilize the new SpotlightGuide for improved visual guidance.
- Updated localization files to include new strings for contextual guides, ensuring a comprehensive user experience across languages.
- Refactored existing components to integrate with the new guide system, improving maintainability and user interaction.
- Added a new visual element for the guide's highlight area, improving user interaction.
- Refactored hole calculation logic to ensure consistent spacing around highlighted elements.
- Introduced a new computed property for dynamic styling of the highlight area.
- Updated styles for backdrop elements to enhance visual clarity and interaction.
- Improved accessibility by ensuring the guide's interactive elements are properly defined.
- Moved the _SEPARATOR_CELL regex definition to a more appropriate location in the Markdown parser.
- Implemented a fallback mechanism in the gRPC document reader to handle cases where the ReadStream RPC is unimplemented, ensuring compatibility with older versions.
- Added a readUnary method to maintain backward compatibility with the legacy unary Read RPC.
- Improved cancellation handling in the MinerUCloud and PaddleOCR-VL readers to prevent excessive API calls during context cancellation.
Introduce opendataloader and PaddleOCR-VL parser engines with tenant-level
settings UI, replace liteparse, and harden Excel/PPT/Markdown parsing.
Optional odl-hybrid sidecar stays local-build only and is excluded from
default dev-start and full profiles.
Large scanned PDFs (hundreds of pages) were slow and fragile end-to-end.
This change addresses the parse, transport, and task-scheduling layers:
docreader (parse + transport):
- Parallelize per-page scanned rendering across processes (forkserver/fork),
with serial fallback. ~4-7x faster on large scanned PDFs; pdfium is not
thread-safe so we fan out across processes. Configurable via
DOCREADER_PDF_RENDER_PARALLELISM.
- Add server-streaming ReadStream RPC: emit one meta frame then one frame per
image, so documents with many page images are no longer capped by the unary
gRPC message-size limit (a 874-page PDF produced ~193MiB of images, far over
the 50MB cap) and memory is bounded on both ends. Unary Read is kept for
backward compatibility; the Go production reader switches to ReadStream.
VLM:
- Make the VLM HTTP timeout configurable (VLM_HTTP_TIMEOUT_SECONDS) and raise
the default 90s -> 180s so dense scanned-page OCR does not time out with
"context deadline exceeded".
Async task queues:
- Isolate high-volume, model-heavy fan-out tasks into dedicated asynq queues so
a single large document cannot saturate the shared worker pool and block
user-facing document parsing:
image:multimodal -> "multimodal"
chunk:extract -> "graph"
question:generation -> "question"
- Register the new queues in the server weight map and the cancel inspector's
scanned-queue set (so cancelling a knowledge still purges its pending tasks).
- Introduced new localization strings for login hints and first-time user prompts in English, Korean, Russian, and Chinese.
- Updated the Login.vue component to enhance user guidance during the login process.
This change aims to improve the onboarding experience for new users by providing clear instructions on how to sign in or create an account.
- Added a NewUserGuide component to provide a guided tour for new users.
- Integrated the guide into the UserMenu for easy access, allowing users to reopen the guide at any time.
- Updated localization files to include new strings for the user guide steps and actions.
- Enhanced the menu item for the user button to include a data attribute for better accessibility.
This feature aims to improve user onboarding and enhance the overall user experience by providing contextual help throughout the application.
- Add MiniMax-M3 to the MiniMax provider description across i18n locales
- Remove deprecated MiniMax-M2.5 reference
- Retain MiniMax-M2.7 and MiniMax-M2.7-highspeed as supported models
MiniMax-M3 is the latest flagship model from MiniMax with extended context
and stronger reasoning capabilities. Users can now see it in the provider
selection UI when configuring a new Minimax model.
- Added a LocalImageResolver function to resolve local storage URLs to their byte data using the tenant's storage configuration.
- Integrated the resolver into the container setup to support reading local images in multimodal chat scenarios.
- Enhanced the handling of local storage paths to correctly parse tenant IDs and retrieve files based on the configured storage settings.
- Implemented a retry mechanism for DOC to DOCX conversion to handle concurrent `soffice` invocations, ensuring each attempt uses a dedicated user profile directory.
- Added logging for each conversion attempt, including success and failure messages, to improve visibility into the conversion process.
- Adjusted the handling of temporary directories for both conversion output and user profiles, enhancing robustness against conversion failures.
Updated the McpSettings, ModelSettings, VectorStoreSettings, and WebSearchSettings components to improve user interactions with service, model, store, and provider cards. Implemented click and keyboard event handling for better accessibility, allowing cards to be clickable based on user roles. Enhanced UI elements with appropriate roles and tabindex attributes, ensuring a consistent and user-friendly experience. Adjusted styles to reflect clickable states and improved focus visibility for better usability.
Updated event handling in KnowledgeBase and FAQEntryManager components to utilize the `@enter` directive for form submissions, enhancing clarity and consistency. Implemented custom keydown handling for the Escape key to improve user experience when editing tags and searching documents. This change simplifies the input event management across various components.
Modified event handling for form submissions in CreateTenantDialog and CredentialResource components to use the `@enter` directive for improved clarity. Removed console log from i18n initialization and updated localization files to include new section labels for the redesigned drawer across multiple languages. Enhanced the UI in Login and McpSettings views by adjusting input types and integrating test connection functionality into the drawer components. Streamlined the layout and interactions in ModelSettings, ParserEngineSettings, StorageEngineSettings, and VectorStoreSettings to ensure consistency and better user experience.
Added a connection test button in the ModelEditorDialog for remote sources, allowing users to verify API connectivity before saving. Improved the layout by restructuring the source selection section and updating the styling for better usability. Enhanced the drawer component with a customizable header and resizable width, improving the overall user experience. Updated localization files to reflect new UI elements and labels.
Removed redundant sections for file and manual types in the document content component. Consolidated download actions into a unified header actions area, enhancing the user interface for file downloads and timeline access. Updated styles for improved layout and responsiveness, ensuring a cleaner and more efficient document viewer experience.
Added functionality to allow users to resize the main drawer in the document content view. The new width is adjustable via a drag handle, and the selected width is saved in local storage for persistence across sessions. Updated styles to support the new resize handle and ensure smooth user experience during resizing. This enhancement improves the usability of the document viewer by providing a customizable layout.
Reconstructing full document content from chunks relied on position math
(offset = content length - (EndAt - lastEndAt)) that assumes
len([]rune(Content)) == EndAt - StartAt. Two upstream behaviors break that
invariant and caused the merged "全文" view to drop or duplicate text:
- The parent-child chunker prepends a synthetic table header to split
tables; that header is zero-width in position space (start == end), so
Content is longer than EndAt - StartAt.
- Chunk Content can retain HTML entities (", > ...), inflating the
rune count relative to the source span.
Replace position-based trimming with text-overlap matching: find where the
accumulated text's suffix reappears at the head of the next chunk and join
there. Positions are only used to size the search window. This naturally
skips the prepended header and is immune to entity length drift.
Extract the logic into searchutil.AppendWithOverlap / MergeTextChunks and
reuse it across the three backend stitching sites (wiki_ingest
reconstructContent, graph mergeChunkContents, chat_pipeline
mergeOverlappingChunks). Mirror the same algorithm in the frontend
doc-content merge. Also fix the markdown hr/heading styles (thin solid
divider, primary heading color).
The wiki ingest pipeline was passing raw locale codes (e.g. "zh-CN",
"en-US") directly into LLM prompts where a human-readable language name
is expected. The model would see "zh-CN" instead of "Chinese (Simplified)"
in its instructions, which could degrade output quality.
Two call sites were affected:
- SlugUpdate.Language (ProcessWikiIngest, retract path): consumed by the
reduce phase to tell the page editor model what language to write in.
- lang variable (mapOneDocument): consumed by entity/concept extraction,
chunk citation classification, and index page rebuild — all LLM prompts.
Wrap both with types.LanguageLocaleName() which maps locale codes to
descriptive names (e.g. "zh-CN" → "Chinese (Simplified)"). Unknown
locales fall through unchanged so no information is lost.
Introduced a new test file for the knowledge repository, implementing various scenarios to ensure the reliability of the finalizing subtask counter. Key tests include:
- Concurrent promotion of subtasks to verify atomic behavior.
- Handling of partial decrements to ensure the row remains in "finalizing" state.
- Safety checks to prevent underflow of the pending subtask count.
- Regression tests to confirm that updates do not overwrite the pending counter.
These tests address previous issues with the finalizing process and enhance overall stability.
The finalizing subtask counter (pending_subtasks_count) could get stuck at a
non-zero value, leaving knowledge permanently in "finalizing" and never
promoted to "completed". Root cause and related hardening:
- UpdateKnowledge does a full-row Save; pending_subtasks_count was NOT in the
omit list, so any concurrent enrichment subtask that loaded the row, did slow
work (LLM call), then saved an unrelated field wrote back the STALE counter,
clobbering decrements other subtasks performed in between. Add
PendingSubtasksCount to omitFieldsOnUpdate so only the atomic helpers
(SetFinalizing / FinalizeSubtask / explicit column writes) ever touch it.
- FinalizeSubtask no longer gates the finalizing->completed promote on a
separate re-read of the counter. Every caller unconditionally runs the
guarded promote UPDATE (WHERE pending_subtasks_count = 0), the single
authoritative atomic check, so a racy/stale read can't strand the row.
- Decrements run on a context detached from the caller's cancellation
(context.WithoutCancel + timeout) so graceful shutdown / preemption can't
silently skip a decrement.
- Reconcile seeded vs actually-enqueued subtask slots (summary/question/graph)
and release any shortfall so un-enqueued planned slots (e.g. graph with
NEO4J off) don't strand the row.
- Reparse paths reset pending_subtasks_count via an explicit column write since
full-row Save now omits it.
Question generation is also moved off the synchronous single-task path onto a
batched async fan-out (windows of text chunks), each batch independently
queued / retried / traced, grouped under a postprocess.question span.
When reopening a session whose last assistant message is still being
generated, handleMsgList marked every loaded message's agentEventStream /
_eventMap / _pendingToolCalls with markRaw for history-rendering
performance. But the last, not-yet-completed message is then resumed via
the continue-stream SSE endpoint, and handleAgentChunk keeps mutating
those exact structures. markRaw detaches them from Vue reactivity, so the
backend was streaming events but the UI never re-rendered — users only saw
the pre-refresh snapshot, and the full content appeared only after the run
finished and a later refresh rebuilt it from persisted agent_steps.
Skip markRaw for the in-progress (is_completed === false) message and keep
it reactive; completed history messages still use markRaw. Regression from
8f462615 ("improve history rendering stability").
The finalizing subtask counter (introduced when wiki ingest was counted)
could leak slots, leaving a fully-parsed doc stuck in "finalizing" until
the housekeeping sweep wrongly marked it "failed".
- wiki ingest: a doc skipped in map (knowledge deleted / no chunks /
insufficient text) produced no docResult and was not a failedOp, so
neither the success nor the dead-letter drain fired. Drain the slot on
that terminal skip path.
- summary & question: the drain was keyed on the span-error variable,
which assumes "err != nil => asynq will retry". Several branches set
that variable yet `return nil` (insufficient text content, KB/knowledge
fetch failures) - terminal, no retry - so the drain was skipped. Key
the drain on the value actually returned to asynq (named retErr)
instead, so terminal nil-returns drain and only retried errors wait
for the final attempt.
Also fix the trace panel header flashing "已完成" mid-wiki: the latest
attempt's root span closes while async post-pipeline subspans keep
running, so trace.status read terminal while the row was still
"finalizing". Prefer parse_status on the latest attempt while it is
non-terminal so the panel header, LIVE badge and doc card agree, and add
the "finalizing" status label to all locales.
Wiki ingest runs asynchronously after the parse pipeline and was not
counted in pending_subtasks_count, so a document flipped to "completed"
while wiki generation was still minutes away (30s debounce + batch +
retries). That hid the in-progress state and dropped the stop-parse
affordance before wiki actually finished.
Count wiki as a single enrichment subtask (when WikiEnabled and the doc
has text chunks, matching the enqueue condition) so the row stays in
"finalizing" until wiki is done. The batch worker drains the slot at the
op's terminal state: once on successful map, and once when an op is
dead-lettered after exhausting in-batch retries. Retract ops (deleted
knowledge) are skipped. FinalizeSubtask guards both the decrement and the
promote, so a wiki op enqueued before this accounting shipped is a
harmless no-op on an already-completed row. A wiki op that never drains is
bounded by the existing housekeeping finalizing sweep.
The trace drawer had no way to cancel an in-flight parse — users had to
go back to the card/list menu. Add a stop-parse control in the timeline
header, shown while parse_status is pending / processing / finalizing
(mirroring the backend CancelKnowledgeParse gate and the card/list menus).
It is a quiet icon-only button matching the other header controls
(refresh / close), revealing its destructive error tint only on hover,
and opens an in-app TDesign confirm dialog (not the native window.confirm)
before calling the existing cancel-parse API. After cancelling it refetches
the spans so the trace reflects the cancelled state. Reuses the existing
cancel-parse i18n keys, already present in all locales.
wizardchen