mirror of
https://github.com/Tencent/WeKnora.git
synced 2026-10-13 08:50:22 +08:00
lyingbug
6afb6ca9d5
* feat(knowledge): flag documents that stop making progress A document stuck in processing looked the same as one still working: a spinner that polled forever, no hint of where it stopped, and, hours later, a generic housekeeping failure. - The batch and spans endpoints return last_activity_at for in-flight rows: the later of updated_at and the newest span write. updated_at only moves at stage transitions; span writes keep moving while a long stage still works. - After 20 minutes with no progress, the list and card views show "may be stuck" with a tooltip. The timeline shows the stage it stopped at and a stop-parsing button, plus a "last progress" stat. Both pollers back off to 15s while everything in flight is stalled. - When housekeeping fails a row, the error names the stalled stage and the time of the last progress. It closes the row's open spans with TASK_STALLED (the running stage fails, the rest are cancelled), so the timeline shows where the run stopped instead of a spinner. * fix(knowledge): locate stalls in finalizing and tell backlog from stuck - Housekeeping reads the stall site from the latest attempt. It takes the running stages, or, when none runs (post-process closes its stage once enrichment is fanned out), the innermost running spans, mapped back to their stage. Those spans fail with TASK_STALLED and are named in the message; every stalled stage is listed in pipeline order. The other open spans are cancelled. Every closed span gets a duration, and the message leaves out the time when the heartbeat query failed. - The spans tree prefers a TASK_STALLED failure for last_error over a later subtask failure. current_stage falls back to the stage owning a running subspan. - Batch and spans responses flag rows quiet past the stall hint whose work is still queued (waiting_in_queue), using the sweep's own probes. At most 20 rows are probed per request, cached for a minute. The UI shows these as queued, with no stop button in the banner. * fix(knowledge): answer stall verdicts from one shared queue scan - Probing a quiet document for a backlog rescanned the whole queue once per document. When the document was absent, which is exactly the stuck case, the scan ran to the end, and it read a Redis error as "nothing queued". QueuedKnowledgeIDs now collects every referenced knowledge ID in one pass and returns list errors. HousekeepingService keeps that snapshot for a minute, shared across requests and single-flight. A failed scan is not cached. - With the per-request cap of 20 gone, every quiet row on a page gets a verdict. The response carries it as stall_state (queued or stalled), replacing waiting_in_queue. The UI marks a row as stuck only on a "stalled" verdict. A row with no verdict, because the probe failed, shows as ordinary processing instead of inviting a stop.
68 lines
3.0 KiB
Go
68 lines
3.0 KiB
Go
// Package errors: parse stage error codes.
|
|
//
|
|
// These constants are the stable wire format the frontend uses to map a
|
|
// failure to a localized "what to do about it" message. Adding a new code
|
|
// is non-breaking; renaming an existing one requires a coordinated
|
|
// frontend release because the i18n keys are looked up by code.
|
|
//
|
|
// Codes are SCREAMING_SNAKE so they survive JSON case transforms and look
|
|
// distinct from Go identifiers in logs/dashboards.
|
|
package errors
|
|
|
|
const (
|
|
// ErrCodeDocReaderTimeout — DocReader RPC exceeded
|
|
// WEKNORA_DOCREADER_CALL_TIMEOUT (default 30m). Suggest splitting
|
|
// large files or checking docreader load.
|
|
ErrCodeDocReaderTimeout = "DOCREADER_TIMEOUT"
|
|
|
|
// ErrCodeDocReaderUnavailable — no DocReader configured for the
|
|
// requested file type / engine, or the service refused connection.
|
|
ErrCodeDocReaderUnavailable = "DOCREADER_UNAVAILABLE"
|
|
|
|
// ErrCodeDocReaderParseFailed — DocReader returned an explicit parse
|
|
// error (encoding, corrupted file, OCR engine crash, ...).
|
|
ErrCodeDocReaderParseFailed = "DOCREADER_PARSE_FAILED"
|
|
|
|
// ErrCodeChunkingFailed — text chunking step itself failed (rare;
|
|
// usually only on extreme-size inputs).
|
|
ErrCodeChunkingFailed = "CHUNKING_FAILED"
|
|
|
|
// ErrCodeEmbeddingRateLimit — embedding provider returned 429 or
|
|
// equivalent. Suggest retrying later or scaling out.
|
|
ErrCodeEmbeddingRateLimit = "EMBEDDING_RATE_LIMIT"
|
|
|
|
// ErrCodeEmbeddingProviderFail — non-rate-limit embedding error
|
|
// (auth, model not found, bad input). Usually permanent without
|
|
// config changes.
|
|
ErrCodeEmbeddingProviderFail = "EMBEDDING_PROVIDER_FAIL"
|
|
|
|
// ErrCodeVectorStoreWriteFailed — chunks embedded fine but the
|
|
// vector DB write failed (quota, connectivity, schema mismatch).
|
|
ErrCodeVectorStoreWriteFailed = "VECTORSTORE_WRITE_FAILED"
|
|
|
|
// ErrCodeMultimodalVLMFailed — a single image's OCR or caption call
|
|
// failed. Note: per-image failures don't fail the whole parent;
|
|
// see image_multimodal.go finalize-on-last-attempt.
|
|
ErrCodeMultimodalVLMFailed = "MULTIMODAL_VLM_FAILED"
|
|
|
|
// ErrCodeMultimodalAllFailed — every image task hit dead-letter.
|
|
// The parent still completes (caption / OCR is optional content),
|
|
// but stage status is marked failed so the UI can warn.
|
|
ErrCodeMultimodalAllFailed = "MULTIMODAL_ALL_FAILED"
|
|
|
|
// ErrCodeTaskTimeout — asynq retry budget exhausted. Used by the
|
|
// dead-letter callback when promoting a task failure into a stage
|
|
// failure. Distinct from DocReaderTimeout: this is the asynq-level
|
|
// timeout (whole task), not the docreader-call-level timeout.
|
|
ErrCodeTaskTimeout = "TASK_TIMEOUT"
|
|
|
|
// ErrCodeTaskStalled — housekeeping found no progress for longer than
|
|
// the stale threshold with nothing left queued, and failed the row.
|
|
ErrCodeTaskStalled = "TASK_STALLED"
|
|
|
|
// ErrCodeUnknown — fallback when a wrapped error doesn't classify.
|
|
// The full message is still recorded in error_detail so operators
|
|
// can debug; the UI shows a generic "see admin" hint.
|
|
ErrCodeUnknown = "UNKNOWN"
|
|
)
|