mirror of
https://github.com/Tencent/WeKnora.git
synced 2026-11-04 23:32:11 +08:00
* cubesandbox support
* feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system
- Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read
- shell_exec: safely execute shell commands in session sandbox with timeout & output truncation
- sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination
- sandbox_read: read sandbox file content with offset-based chunked reading
- Add Artifacts system:
- ArtifactCollector: real-time collection of sandbox tool output files, attached to messages
- /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend
- ChatArtifactsDrawer.vue: frontend artifacts sidebar UI
- DB migration 000066_message_artifacts: add artifacts column to messages table
- Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access
- Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules
- Extend Agent Observe: include agent steps in Langfuse observability tracing
- Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries
- Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales
- Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc.
* feat(agent):add remote sandbox client abstract & client errors
* feat(sandbox): implement Redis binding store with distributed lifecycle lock;
* feat(sandbox): add remote session lifecycle with ownership context
Implement remoteSessionLifecycle to coordinate persistent remote sandboxes
across multi-instance deployments. Supports Resolve (lazy provision,
recovery, race-safe create) and Destroy with compare-delete semantics.
Add ownership context isolation to decouple remote resource cleanup from
request cancellation.
* feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient
Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient
implementation. Error normalization, timeout mapping, and metadata
support are fully integrated. SDK details remain private to the
adapter.
* refactor(sandbox): decouple SessionBoundManager from Cube
Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add
RemoteSandbox as provider-neutral Sandbox interface implementation.
Rewrite SessionBoundManager to depend on RemoteSandboxClient +
SessionSandboxBindingStore + SessionExistenceChecker, with all
lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed
lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry
with time.Time ModTime. Cleanup delegated to provider idle-timeout.
* refactor(agent): enhance session capability handling in sandbox manager
- Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution.
- Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends.
- Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types.
- Removed Cube-specific checks, promoting a more provider-neutral approach to session management.
* refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding
- Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation
covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove)
- Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager)
- Dependency: matiasinsaurralde/go-e2b v0.1.0
- Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go,
cube_defaults.go) and replace with direct *cubesandbox.Client usage
- Parse CubeProxy URL directly in adapter (parseProxyURL), removing the
old hand-rolled envd transport layer
- Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra
- Target Cube deployment v0.5.1
- Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess,
AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters
- Both adapters default to permissive (internet+public on) for backwards compat
- Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by
WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE
- Add repositorySessionExistenceChecker — checks session existence via DB
(gorm.ErrRecordNotFound → not exists, transient errors → preserve binding)
- newSandboxManager() now accepts redisClient + sessionRepo; deprecated
hardcoded NewMemorySessionSandboxBindingStore() in production path
- buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume
(Cube and E2B). Sessions survive brief idle periods; providers still
handle TTL-based reaping.
- buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized
by RemoteProvider (was hardcoded to Cube Config fields)
- NewSessionBoundManager validates provider and delegates defaults to
per-provider apply*Defaults helpers
- Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go,
cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines)
- Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal,
IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding
- Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds
- Dead ExecuteResult method: GetOutput
- Unused test: session_artifacts_integration_test.go
- Preloaded skill: sandbox-test-python
- Extract shared httpErrorKind() from per-adapter normalize functions
- Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore
advertisement (E2B returns true post-reconnect, Cube always true)
- Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future"
- Bump session sandbox TTL defaults: 604800s → 300s (.env.example)
- Update docs/agent-skills.md: full multi-instance deployment guide
Total: 29 files, +1188 / -3721
* feat(agent): bind collected skill artifacts to owning message in resource registry
When ArtifactCollector persists a skill-generated file it now registers a
resource:// binding (owner_type=message, owner_id=assistantMessageID,
relation=artifact) in addition to the messages.artifacts JSONB record. This
mirrors how knowledge uploads and chat attachments bind their resources, so
artifacts can be enumerated and garbage-collected through the resource
registry instead of only via the message JSONB column.
- ArtifactCollector: inject ResourceCatalog, add messageID to Collect/
maybePersist, and add best-effort bindArtifactResource (binding failures are
logged, never drop the artifact; only resource:// refs are bound).
- agent_stream_handler: pass assistantMessageID into Collect.
- Renumber message_artifacts migration 000066 -> 000077 to resolve the
version collision with existing migrations (upstream now uses 000075/000076).
- Tests: update collector signatures and add coverage for the binding path
and for graceful degradation when Bind fails.
Note: binding applies to newly generated artifacts only; existing rows are
not backfilled.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(sandbox): harden remote sandbox security and artifact collection
Disable Cube local fallback so skill scripts never run on the host when
the remote provider is unavailable. Require Redis for cube/e2b binding
unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev.
Collect artifacts with WithoutCancel so user stop does not drop files,
reject shell_exec work_dir outside /workspace, and refuse session-scoped
Execute when a local fallback would be engaged.
* fix(ci): gofmt sandbox files and resolve artifact drawer type error
Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer
props in AgentStreamDisplay so session-id is always a string.
* fix(sandbox): adapt to upstream main after rebase
Renumber the message_artifacts migration 000077 -> 000079: upstream now
uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata.
Declare empty model-handle policies for shell_exec / list_sandbox_files /
read_sandbox_file. Upstream added a coverage test requiring every
UI-exposed tool to declare one; sandbox output is arbitrary skill-authored
content, so it gets the same treatment as execute_skill_script.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant.
* feat(sandbox): add sandbox config entity, cordon lease and session pin field
* feat(sandbox): add sandbox config repository with tenant scoping and cordon lease
* feat(sandbox): stamp sandbox config ID onto sandbox metadata
* feat(sandbox): scope sandbox listing, release and reaping to a single config
* feat(sandbox): resolve managers per config and refuse cordoned or missing configs
* feat(sandbox): implement session pinning and config identity change detection
- Add SessionSandboxPinner with read/pin/clear to bind session to config
- Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config
- Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL)
- Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts
- Fix artifact collection for default backend and destroy for legacy empty-pin sessions
* feat(sandbox): add multi-tenant sandbox config management and HTTP APIs
Service layer:
- Add sandbox config business service supporting CRUD and inventory listing
- For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions
- For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side,
and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting
- Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config
- Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation
HTTP interfaces and routing:
- Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory)
- Mask sensitive credentials (API key, etc.) in all read responses
- Return 409 with structured inventory details on rejection scenarios
- sandbox-check probing supports specifying an existing config_id directly
- Remove old handler file, consolidate validation logic into service layer
- DI wiring and route registration for all six endpoints
* feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config
- Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults
- Validate config existence on create/update; return 400 if not found
- Inject sandboxConfigService into CustomAgentHandler
- Add unit tests covering field serialization and validation logic
* feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy
Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default.
* feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch
- Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios.
- Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned.
- Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace.
* feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities
- Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked
- Add named backend type validation, only cube/e2b can be stored as workspace named config
- Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default
- Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents
- Adapt artifact collector and other callers to fill missing resolve signature parameter
* fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube
Session teardown & deletion ordering
- Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins.
- Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions.
## Config update edge cases
- When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409.
- UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results.
Artifact collection for Cube skills
- Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`.
HTTP pool sharing for named Cube configs
- Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request.
- Add `CubeTransportPool` and `cubeSplitTransport`:
- Control plane traffic reuses the same `SharedTransport` as E2B.
- Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header.
- Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes.
* feat(sandbox): sandbox execution DefaultUser change from root to user
* fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults
- Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars
instead of DefaultConfig() hardcoded values, allowing empty fields to inherit
from .env rather than Go constants
- Centralize deployment baseline parsing in sandbox/defaults.go with
DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv
- Delegate container startup to shared helpers to avoid drift
- Propagate tenant env_vars through ResolveEffectiveConfig into
buildSessionCreateRequest for both Cube and E2B session sandboxes
* fix(multi-sandbox-config-per-tenant): fix some bugs frontend.
* fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant
Three defects found while reviewing the remote sandbox work:
- The Cube connectivity probe and the deployment-default Cube/E2B managers
built clients without a guarded transport, so an admin-supplied api_url or
proxy_url could reach link-local addresses via DNS rebinding even though
save-time validation passed. All three now dial through the same outbound
guard the per-tenant resolver enforces.
- Attachment staging was gated on the process-wide manager, so a workspace
pointing an agent at a named Cube/E2B config while the deployment default
stayed docker/local would run its tools remotely against an empty
/workspace/input. The gate now resolves the manager the session actually
runs on.
- Session to sandbox bindings were keyed by the tenant in context, which a
shared agent deliberately swaps for the agent owner's so models, KBs and
named configs resolve in the sharing workspace. Session deletion only ever
knows the session's own tenant, so teardown looked up a key nobody wrote
and abandoned a paused MicroVM that keeps billing. The binding tenant is
now carried separately and survives CloneContext.
* chore(sandbox): renumber migrations above upstream and de-collide test stub
Upstream main took 000079 (knowledge_folder_path) and 000080
(knowledge_base_auto_tag_config) while this branch was open, so the three
sandbox migrations move to 000081-000083. golang-migrate requires unique,
ordered versions, and the RAISE NOTICE labels move with the file names.
Main also added a handler test declaring stubSessionService in the same
package; the artifact download test's configurable variant is renamed rather
than merged, since the two stubs answer GetSession differently.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
735 lines
27 KiB
Go
735 lines
27 KiB
Go
package agent
|
|
|
|
import (
|
|
"context"
|
|
"encoding/base64"
|
|
"fmt"
|
|
"strings"
|
|
"time"
|
|
|
|
agentmemory "github.com/Tencent/WeKnora/internal/agent/memory"
|
|
"github.com/Tencent/WeKnora/internal/agent/skills"
|
|
agenttoken "github.com/Tencent/WeKnora/internal/agent/token"
|
|
agenttools "github.com/Tencent/WeKnora/internal/agent/tools"
|
|
"github.com/Tencent/WeKnora/internal/common"
|
|
appconfig "github.com/Tencent/WeKnora/internal/config"
|
|
"github.com/Tencent/WeKnora/internal/event"
|
|
"github.com/Tencent/WeKnora/internal/logger"
|
|
"github.com/Tencent/WeKnora/internal/modelcontext"
|
|
"github.com/Tencent/WeKnora/internal/models/chat"
|
|
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
|
"github.com/Tencent/WeKnora/internal/types"
|
|
)
|
|
|
|
// langfuseQueryPreview caps the query length we ship as the agent.execute
|
|
// span input — long quoted-context queries can be many KB of prose.
|
|
const langfuseQueryPreview = 2000
|
|
|
|
// AgentEngine is the core engine for running ReAct agents.
|
|
//
|
|
// History persistence note: the engine is stateless across turns. Conversation
|
|
// history is rebuilt from the DB once per turn by the caller
|
|
// (see service.LoadAgentHistory) and passed into Execute as llmContext. The
|
|
// engine therefore does not maintain its own cache, system-prompt store, or
|
|
// cross-turn buffer.
|
|
type AgentEngine struct {
|
|
config *types.AgentConfig
|
|
toolRegistry *agenttools.ToolRegistry
|
|
chatModel chat.Chat
|
|
eventBus *event.EventBus
|
|
knowledgeBasesInfo []*KnowledgeBaseInfo // Detailed knowledge base information for prompt
|
|
selectedDocs []*SelectedDocumentInfo // User-selected documents (via @ mention)
|
|
pinnedMCPServices []*PinnedMCPServiceInfo // User @mentioned MCP services for this turn
|
|
pinnedSkills []*PinnedSkillInfo // User @mentioned skills for this turn
|
|
sessionID string // Session ID for logging and event emission
|
|
systemPromptTemplate string // System prompt template (optional, uses default if empty)
|
|
skillsManager *skills.Manager // Skills manager for Progressive Disclosure (optional)
|
|
appConfig *appconfig.Config // Application config for prompt template resolution (optional)
|
|
imageDescriber ImageDescriberFunc // VLM function for describing images in tool results (optional)
|
|
tokenEstimator *agenttoken.Estimator // Token estimator for context window management
|
|
memoryConsolidator *agentmemory.Consolidator // Memory consolidator for LLM-powered summarization (optional)
|
|
lastUsage types.TokenUsage // Token usage from the most recent LLM call
|
|
lastSentMsgCount int // Number of messages sent in the most recent LLM call
|
|
modelContext *modelcontext.Registry // single request-local boundary for every model handle
|
|
}
|
|
|
|
// ImageDescriberFunc generates a text description of an image.
|
|
// Signature matches vlm.VLM.Predict so it can be injected without importing the vlm package.
|
|
type ImageDescriberFunc func(ctx context.Context, imgBytes []byte, prompt string) (string, error)
|
|
|
|
// NewAgentEngine creates a new agent engine
|
|
func NewAgentEngine(
|
|
config *types.AgentConfig,
|
|
chatModel chat.Chat,
|
|
toolRegistry *agenttools.ToolRegistry,
|
|
eventBus *event.EventBus,
|
|
knowledgeBasesInfo []*KnowledgeBaseInfo,
|
|
selectedDocs []*SelectedDocumentInfo,
|
|
sessionID string,
|
|
systemPromptTemplate string,
|
|
) *AgentEngine {
|
|
if eventBus == nil {
|
|
eventBus = event.NewEventBus()
|
|
}
|
|
tokenEst, err := agenttoken.NewEstimator()
|
|
if err != nil {
|
|
return nil
|
|
}
|
|
engine := &AgentEngine{
|
|
config: config,
|
|
toolRegistry: toolRegistry,
|
|
chatModel: chatModel,
|
|
eventBus: eventBus,
|
|
knowledgeBasesInfo: knowledgeBasesInfo,
|
|
selectedDocs: selectedDocs,
|
|
sessionID: sessionID,
|
|
systemPromptTemplate: systemPromptTemplate,
|
|
tokenEstimator: tokenEst,
|
|
modelContext: modelcontext.NewRegistry(config.CitationsEnabled()),
|
|
}
|
|
|
|
// Initialize memory consolidator if context window management is configured
|
|
if config.MaxContextTokens > 0 {
|
|
engine.memoryConsolidator = agentmemory.NewConsolidator(
|
|
chatModel, tokenEst, config.MaxContextTokens, 0,
|
|
)
|
|
}
|
|
|
|
return engine
|
|
}
|
|
|
|
// SetPinnedMentions sets per-turn @mention scope for MCP services and skills.
|
|
func (e *AgentEngine) SetPinnedMentions(mcpServices []*PinnedMCPServiceInfo, skills []*PinnedSkillInfo) {
|
|
e.pinnedMCPServices = mcpServices
|
|
e.pinnedSkills = skills
|
|
}
|
|
|
|
func (e *AgentEngine) systemPromptOptions(ctx context.Context) *BuildSystemPromptOptions {
|
|
opts := &BuildSystemPromptOptions{
|
|
Language: types.LanguageNameFromContext(ctx),
|
|
Config: e.appConfig,
|
|
}
|
|
if e.skillsManager != nil && e.skillsManager.IsEnabled() {
|
|
opts.SkillsMetadata = e.skillsManager.GetAllMetadata()
|
|
}
|
|
if e.toolRegistry != nil {
|
|
_, err := e.toolRegistry.GetTool(agenttools.ToolShellExec)
|
|
opts.ShellExecEnabled = err == nil
|
|
}
|
|
return opts
|
|
}
|
|
|
|
func (e *AgentEngine) buildSystemPrompt(ctx context.Context) string {
|
|
prompt := BuildSystemPromptWithOptions(
|
|
e.knowledgeBasesInfo,
|
|
e.config.WebSearchEnabled,
|
|
e.systemPromptOptions(ctx),
|
|
e.systemPromptTemplate,
|
|
)
|
|
return strings.TrimRight(prompt, " \t\r\n") + e.modelContext.ProtocolPrompt()
|
|
}
|
|
|
|
// NewAgentEngineWithSkills creates a new agent engine with skills support
|
|
func NewAgentEngineWithSkills(
|
|
config *types.AgentConfig,
|
|
chatModel chat.Chat,
|
|
toolRegistry *agenttools.ToolRegistry,
|
|
eventBus *event.EventBus,
|
|
knowledgeBasesInfo []*KnowledgeBaseInfo,
|
|
selectedDocs []*SelectedDocumentInfo,
|
|
sessionID string,
|
|
systemPromptTemplate string,
|
|
skillsManager *skills.Manager,
|
|
) *AgentEngine {
|
|
engine := NewAgentEngine(
|
|
config,
|
|
chatModel,
|
|
toolRegistry,
|
|
eventBus,
|
|
knowledgeBasesInfo,
|
|
selectedDocs,
|
|
sessionID,
|
|
systemPromptTemplate,
|
|
)
|
|
engine.skillsManager = skillsManager
|
|
return engine
|
|
}
|
|
|
|
// SetAppConfig sets the application config for prompt template resolution.
|
|
// This allows the engine to read default prompts from config/prompt_templates/ YAML files.
|
|
func (e *AgentEngine) SetAppConfig(cfg *appconfig.Config) {
|
|
e.appConfig = cfg
|
|
}
|
|
|
|
// SetImageDescriber sets the VLM function for generating text descriptions of images
|
|
// in tool results. When set, MCP tool result images are automatically analyzed and
|
|
// their descriptions are appended to the tool message content.
|
|
// This follows the same pattern as Handler.analyzeImageAttachments() in the handler layer.
|
|
func (e *AgentEngine) SetImageDescriber(fn ImageDescriberFunc) {
|
|
e.imageDescriber = fn
|
|
}
|
|
|
|
// SetSkillsManager sets the skills manager for the engine
|
|
func (e *AgentEngine) SetSkillsManager(manager *skills.Manager) {
|
|
e.skillsManager = manager
|
|
}
|
|
|
|
// GetSkillsManager returns the skills manager
|
|
func (e *AgentEngine) GetSkillsManager() *skills.Manager {
|
|
return e.skillsManager
|
|
}
|
|
|
|
// estimateCurrentTokens returns the best estimate of the current context token count.
|
|
// When API-reported usage from a previous round is available, it uses that as a
|
|
// baseline and only BPE-estimates the delta (newly appended messages). Otherwise it
|
|
// falls back to a full BPE estimation of all messages.
|
|
func (e *AgentEngine) estimateCurrentTokens(messages []chat.Message) int {
|
|
if e.lastUsage.TotalTokens > 0 && e.lastSentMsgCount > 0 && e.lastSentMsgCount < len(messages) {
|
|
delta := e.tokenEstimator.EstimateMessages(messages[e.lastSentMsgCount:])
|
|
return e.lastUsage.TotalTokens + delta
|
|
}
|
|
return e.tokenEstimator.EstimateMessages(messages)
|
|
}
|
|
|
|
// Execute executes the agent with conversation history and streaming output
|
|
// All events are emitted to EventBus and handled by subscribers (like Handler layer)
|
|
func (e *AgentEngine) Execute(
|
|
ctx context.Context,
|
|
sessionID, messageID, query string,
|
|
llmContext []chat.Message,
|
|
imageURLs ...[]string,
|
|
) (*types.AgentState, error) {
|
|
logger.Infof(ctx, "[Agent] Starting execution: session=%s, message=%s, query_len=%d, context_msgs=%d",
|
|
sessionID, messageID, len(query), len(llmContext))
|
|
// Ensure tools are cleaned up after execution
|
|
defer e.toolRegistry.Cleanup(ctx)
|
|
|
|
common.PipelineInfo(ctx, "Agent", "execute_start", map[string]interface{}{
|
|
"session_id": sessionID,
|
|
"message_id": messageID,
|
|
"query": query,
|
|
"context_msgs": len(llmContext),
|
|
})
|
|
|
|
// Open a top-level Langfuse span so the agent run — including every
|
|
// round's LLM call and every tool execution — groups under a single
|
|
// node in the Langfuse UI instead of being flat children of the HTTP
|
|
// trace. No-op when Langfuse is disabled.
|
|
imgCount := 0
|
|
if len(imageURLs) > 0 {
|
|
imgCount = len(imageURLs[0])
|
|
}
|
|
kbIDs := make([]string, 0, len(e.knowledgeBasesInfo))
|
|
for _, kb := range e.knowledgeBasesInfo {
|
|
if kb != nil {
|
|
kbIDs = append(kbIDs, kb.ID)
|
|
}
|
|
}
|
|
spanCtx, agentSpan := langfuse.GetManager().StartSpan(ctx, langfuse.SpanOptions{
|
|
Name: "agent.execute",
|
|
Input: map[string]interface{}{
|
|
"query": truncateRunes(query, langfuseQueryPreview),
|
|
"query_len": len(query),
|
|
"context_msgs": len(llmContext),
|
|
"image_count": imgCount,
|
|
},
|
|
Metadata: map[string]interface{}{
|
|
"session_id": sessionID,
|
|
"message_id": messageID,
|
|
"max_iterations": e.config.MaxIterations,
|
|
"parallel_tool_calls": e.config.ParallelToolCalls,
|
|
"web_search": e.config.WebSearchEnabled,
|
|
"multi_turn": e.config.MultiTurnEnabled,
|
|
"knowledge_base_ids": kbIDs,
|
|
"allowed_tools": e.config.AllowedTools,
|
|
},
|
|
})
|
|
ctx = spanCtx
|
|
|
|
// Initialize state
|
|
state := &types.AgentState{
|
|
RoundSteps: []types.AgentStep{},
|
|
KnowledgeRefs: []*types.SearchResult{},
|
|
IsComplete: false,
|
|
CurrentRound: 0,
|
|
}
|
|
|
|
// Build system prompt using progressive RAG prompt
|
|
// If skills are enabled, include skills metadata (Level 1 - Progressive Disclosure)
|
|
systemPrompt := e.buildSystemPrompt(ctx)
|
|
logger.Debugf(ctx, "[Agent] SystemPrompt: %d chars", len(systemPrompt))
|
|
|
|
// Initialize messages with history
|
|
var imgs []string
|
|
if len(imageURLs) > 0 {
|
|
imgs = imageURLs[0]
|
|
}
|
|
messages := e.buildMessagesWithLLMContext(systemPrompt, query, sessionID, llmContext, imgs)
|
|
|
|
// Get tool definitions for function calling
|
|
tools := e.buildToolsForLLM()
|
|
toolListStr := strings.Join(listToolNames(tools), ", ")
|
|
logger.Infof(ctx, "[Agent] Ready: %d messages, %d tools [%s], %d images",
|
|
len(messages), len(tools), toolListStr, len(imgs))
|
|
common.PipelineInfo(ctx, "Agent", "tools_ready", map[string]interface{}{
|
|
"session_id": sessionID,
|
|
"tool_count": len(tools),
|
|
"tools": toolListStr,
|
|
})
|
|
|
|
_, err := e.executeLoop(ctx, state, query, messages, tools, sessionID, messageID)
|
|
if err != nil {
|
|
logger.Errorf(ctx, "[Agent] Execution failed: %v", err)
|
|
e.eventBus.Emit(ctx, event.Event{
|
|
ID: generateEventID("error"),
|
|
Type: event.EventError,
|
|
SessionID: sessionID,
|
|
Data: event.ErrorData{
|
|
Error: err.Error(),
|
|
Stage: "agent_execution",
|
|
SessionID: sessionID,
|
|
},
|
|
})
|
|
finishAgentSpan(agentSpan, state, err)
|
|
return nil, err
|
|
}
|
|
|
|
logger.Infof(ctx, "[Agent] Completed: %d rounds, %d steps, complete=%v",
|
|
state.CurrentRound, len(state.RoundSteps), state.IsComplete)
|
|
common.PipelineInfo(ctx, "Agent", "execute_complete", map[string]interface{}{
|
|
"session_id": sessionID,
|
|
"rounds": state.CurrentRound,
|
|
"steps": len(state.RoundSteps),
|
|
"complete": state.IsComplete,
|
|
})
|
|
finishAgentSpan(agentSpan, state, nil)
|
|
return state, nil
|
|
}
|
|
|
|
// finishAgentSpan records the final outcome of an agent execution onto the
|
|
// top-level Langfuse span. Extracted so the same payload is used for both
|
|
// success and error return paths in Execute().
|
|
func finishAgentSpan(span *langfuse.Span, state *types.AgentState, err error) {
|
|
if span == nil {
|
|
return
|
|
}
|
|
totalToolCalls := 0
|
|
for _, step := range state.RoundSteps {
|
|
totalToolCalls += len(step.ToolCalls)
|
|
}
|
|
output := map[string]interface{}{
|
|
"rounds": state.CurrentRound,
|
|
"steps": len(state.RoundSteps),
|
|
"tool_calls": totalToolCalls,
|
|
"complete": state.IsComplete,
|
|
"final_answer_len": len(state.FinalAnswer),
|
|
"final_answer": truncateRunes(state.FinalAnswer, langfuseQueryPreview),
|
|
}
|
|
span.Finish(output, map[string]interface{}{
|
|
"rounds": state.CurrentRound,
|
|
"steps": len(state.RoundSteps),
|
|
"tool_calls": totalToolCalls,
|
|
"complete": state.IsComplete,
|
|
}, err)
|
|
}
|
|
|
|
// truncateRunes caps s to n runes and appends "…" when truncated. Identical
|
|
// in spirit to the helper in act.go, but kept locally so both files stay
|
|
// independent and the truncation budget can diverge if needed.
|
|
func truncateRunes(s string, n int) string {
|
|
if n <= 0 || s == "" {
|
|
return s
|
|
}
|
|
r := []rune(s)
|
|
if len(r) <= n {
|
|
return s
|
|
}
|
|
return string(r[:n]) + "…"
|
|
}
|
|
|
|
// executeLoop executes the main ReAct loop
|
|
// All events are emitted through EventBus with the given sessionID
|
|
func (e *AgentEngine) executeLoop(
|
|
ctx context.Context,
|
|
state *types.AgentState,
|
|
query string,
|
|
messages []chat.Message,
|
|
tools []chat.Tool,
|
|
sessionID string,
|
|
messageID string,
|
|
) (*types.AgentState, error) {
|
|
startTime := time.Now()
|
|
common.PipelineInfo(ctx, "Agent", "loop_start", map[string]interface{}{
|
|
"max_iterations": e.config.MaxIterations,
|
|
})
|
|
|
|
// Guarantee exactly-one EventAgentComplete emission on every exit path
|
|
// (normal finish, ctx cancel observed at the loop head, or iteration error
|
|
// bubbling up while ctx was cancelled). The emission triggers
|
|
// agent_stream_handler.handleComplete, which is where state.RoundSteps
|
|
// (thinking + tool_call history) gets written onto assistantMessage.AgentSteps
|
|
// so it can be persisted by the caller's defer. Use WithoutCancel so a
|
|
// cancelled ctx does not short-circuit the emit on the stop path.
|
|
completionEmitted := false
|
|
emitCompletion := func() {
|
|
if completionEmitted {
|
|
return
|
|
}
|
|
completionEmitted = true
|
|
e.emitCompletionEvent(context.WithoutCancel(ctx), state, sessionID, messageID, startTime)
|
|
}
|
|
defer emitCompletion()
|
|
|
|
emptyRetries := 0
|
|
consecutiveSameContent := 0
|
|
lastResponseContent := ""
|
|
loop:
|
|
for state.CurrentRound < e.config.MaxIterations {
|
|
// Check for context cancellation (request timeout, user cancel, etc.)
|
|
select {
|
|
case <-ctx.Done():
|
|
logger.Warnf(ctx, "[Agent] Context cancelled at round %d: %v",
|
|
state.CurrentRound+1, ctx.Err())
|
|
// Try to salvage existing results
|
|
if totalTC := countTotalToolCalls(state.RoundSteps); totalTC > 0 {
|
|
logger.Infof(ctx, "[Agent] Synthesizing final answer from %d existing tool results",
|
|
totalTC)
|
|
_ = e.streamFinalAnswerToEventBus(ctx, query, state, sessionID)
|
|
state.IsComplete = true
|
|
}
|
|
return state, ctx.Err()
|
|
default:
|
|
}
|
|
|
|
// Each iteration runs inside an "agent.round.<N>" Langfuse span.
|
|
// We execute the body in a closure so `defer span.Finish()` fires at
|
|
// every exit path (break/continue/next) without having to sprinkle
|
|
// manual finish calls throughout the many branches below.
|
|
outcome, iterErr := e.runReActIteration(ctx, state, &messages, tools,
|
|
sessionID, messageID, query, &emptyRetries, &consecutiveSameContent, &lastResponseContent)
|
|
if iterErr != nil {
|
|
return state, iterErr
|
|
}
|
|
switch outcome {
|
|
case iterOutcomeContinue:
|
|
continue loop
|
|
case iterOutcomeBreak:
|
|
break loop
|
|
case iterOutcomeNext:
|
|
state.CurrentRound++
|
|
}
|
|
}
|
|
|
|
// If loop finished without final answer, generate one — but skip this
|
|
// when the context was cancelled (user pressed stop). In that case the
|
|
// fallback LLM call would fail on the already-cancelled ctx and set
|
|
// state.FinalAnswer to the generic "Sorry, I was unable to generate a
|
|
// complete answer." message, which then leaks to the UI as the final
|
|
// answer for a conversation the user deliberately stopped.
|
|
if !state.IsComplete && ctx.Err() == nil {
|
|
e.handleMaxIterations(ctx, query, state, sessionID)
|
|
}
|
|
|
|
return state, nil
|
|
}
|
|
|
|
// iterOutcome directs executeLoop's control flow after one ReAct iteration.
|
|
// Using a sentinel (rather than bare return values from runReActIteration)
|
|
// keeps the loop's break/continue/next branches explicit in one place.
|
|
type iterOutcome int
|
|
|
|
const (
|
|
// iterOutcomeNext advances state.CurrentRound and loops again.
|
|
iterOutcomeNext iterOutcome = iota
|
|
// iterOutcomeContinue re-runs the loop without advancing the round
|
|
// counter. Used by the empty-content retry path.
|
|
iterOutcomeContinue
|
|
// iterOutcomeBreak exits the loop (final answer, stuck loop, or end).
|
|
iterOutcomeBreak
|
|
)
|
|
|
|
// runReActIteration executes one ReAct step: think → analyze → act → observe.
|
|
// Extracted from executeLoop so the whole iteration body can live inside a
|
|
// single `defer span.Finish()` scope — otherwise we'd need to sprinkle
|
|
// manual finish calls across every break/continue/return branch.
|
|
//
|
|
// The mutable loop state (messages, empty-retry counter, stuck-loop detector)
|
|
// is passed by pointer so iterations share progress.
|
|
func (e *AgentEngine) runReActIteration(
|
|
parentCtx context.Context,
|
|
state *types.AgentState,
|
|
messagesPtr *[]chat.Message,
|
|
tools []chat.Tool,
|
|
sessionID, assistantMessageID, query string,
|
|
emptyRetries, consecutiveSameContent *int,
|
|
lastResponseContent *string,
|
|
) (outcome iterOutcome, retErr error) {
|
|
roundStart := time.Now()
|
|
round := state.CurrentRound + 1
|
|
|
|
// Open the round-level Langfuse span. Any chat/tool calls made inside
|
|
// this iteration will attach under it via ctx, giving the UI a clean
|
|
// trace → agent.execute → agent.round.N → (chat + tools) structure.
|
|
ctx, roundSpan := langfuse.GetManager().StartSpan(parentCtx, langfuse.SpanOptions{
|
|
Name: fmt.Sprintf("agent.round.%d", round),
|
|
Input: map[string]interface{}{
|
|
"round": round,
|
|
"message_count": len(*messagesPtr),
|
|
"max_iterations": e.config.MaxIterations,
|
|
},
|
|
Metadata: map[string]interface{}{
|
|
"iteration": state.CurrentRound,
|
|
"round": round,
|
|
"session_id": sessionID,
|
|
},
|
|
})
|
|
|
|
var (
|
|
response *types.ChatResponse
|
|
toolCallCount int
|
|
)
|
|
defer func() {
|
|
if roundSpan == nil {
|
|
return
|
|
}
|
|
out := map[string]interface{}{
|
|
"round": round,
|
|
"outcome": outcome.String(),
|
|
"tool_calls": toolCallCount,
|
|
}
|
|
if response != nil {
|
|
out["has_tool_calls"] = len(response.ToolCalls) > 0
|
|
out["finish_reason"] = response.FinishReason
|
|
out["content_len"] = len(response.Content)
|
|
if response.Usage.TotalTokens > 0 {
|
|
out["prompt_tokens"] = response.Usage.PromptTokens
|
|
out["completion_tokens"] = response.Usage.CompletionTokens
|
|
out["total_tokens"] = response.Usage.TotalTokens
|
|
}
|
|
}
|
|
out["duration_ms"] = time.Since(roundStart).Milliseconds()
|
|
roundSpan.Finish(out, map[string]interface{}{
|
|
"round": round,
|
|
"tool_calls": toolCallCount,
|
|
"outcome": outcome.String(),
|
|
"duration_ms": time.Since(roundStart).Milliseconds(),
|
|
}, retErr)
|
|
}()
|
|
|
|
// Context window management: estimate current token count using
|
|
// the API-reported usage from the previous round plus a BPE delta
|
|
// for newly appended messages (assistant reply + tool results).
|
|
currentTokens := e.estimateCurrentTokens(*messagesPtr)
|
|
beforeLen := len(*messagesPtr)
|
|
*messagesPtr = e.manageContextWindow(ctx, *messagesPtr, round, currentTokens)
|
|
if len(*messagesPtr) < beforeLen {
|
|
currentTokens = e.tokenEstimator.EstimateMessages(*messagesPtr)
|
|
}
|
|
|
|
logger.Infof(ctx, "[Agent][Round-%d/%d] Starting: %d messages, %d tools, est_tokens=%d",
|
|
round, e.config.MaxIterations, len(*messagesPtr), len(tools), currentTokens)
|
|
common.PipelineInfo(ctx, "Agent", "round_start", map[string]interface{}{
|
|
"iteration": state.CurrentRound,
|
|
"round": round,
|
|
"message_count": len(*messagesPtr),
|
|
"pending_tools": len(tools),
|
|
"max_iterations": e.config.MaxIterations,
|
|
})
|
|
|
|
// 1. Think: Call LLM with function calling (includes retry + graceful degradation)
|
|
e.lastSentMsgCount = len(*messagesPtr)
|
|
resp, err := e.callLLMWithRetry(ctx, *messagesPtr, tools, state, query, state.CurrentRound, sessionID)
|
|
if err != nil {
|
|
retErr = err
|
|
return iterOutcomeNext, err
|
|
}
|
|
if resp == nil {
|
|
return iterOutcomeBreak, nil
|
|
}
|
|
response = resp
|
|
if response.Usage.TotalTokens > 0 {
|
|
e.lastUsage = response.Usage
|
|
logger.Debugf(ctx, "[Agent][Round-%d] Usage: prompt=%d, completion=%d, total=%d",
|
|
round, response.Usage.PromptTokens,
|
|
response.Usage.CompletionTokens, response.Usage.TotalTokens)
|
|
}
|
|
|
|
// Detect stuck loops: if the LLM keeps returning the same content
|
|
// without tool calls (e.g., an unhandled finish reason), break early.
|
|
if len(response.ToolCalls) == 0 && response.Content != "" {
|
|
if response.Content == *lastResponseContent {
|
|
*consecutiveSameContent++
|
|
} else {
|
|
*consecutiveSameContent = 0
|
|
}
|
|
*lastResponseContent = response.Content
|
|
if *consecutiveSameContent >= maxRepeatedResponseRounds {
|
|
logger.Warnf(ctx, "[Agent][Round-%d] Detected stuck loop: same content repeated %d times (finish=%s), stopping",
|
|
round, *consecutiveSameContent+1, response.FinishReason)
|
|
state.FinalAnswer = response.Content
|
|
state.IsComplete = true
|
|
return iterOutcomeBreak, nil
|
|
}
|
|
} else {
|
|
*consecutiveSameContent = 0
|
|
*lastResponseContent = ""
|
|
}
|
|
|
|
// Create agent step
|
|
step := types.AgentStep{
|
|
Iteration: state.CurrentRound,
|
|
Thought: response.Content,
|
|
ReasoningContent: response.ReasoningContent,
|
|
ToolCalls: make([]types.ToolCall, 0),
|
|
Timestamp: time.Now(),
|
|
}
|
|
|
|
// If the request was cancelled while the LLM was streaming (e.g. the
|
|
// user pressed "stop"), the stream driver still returns a usable
|
|
// response (partial content / finish_reason="stop" / no tool calls).
|
|
// Do NOT let analyzeResponse treat that partial thinking text as the
|
|
// final answer — it would pollute Message.Content with mid-stream
|
|
// thinking and show up as a duplicate card next to the intermediate-
|
|
// steps tree. Preserve the partial thinking as an AgentStep and break
|
|
// out of the loop without marking state.IsComplete. executeLoop's
|
|
// deferred emitCompletionEvent will persist the step onto
|
|
// Message.AgentSteps so it still appears in the tree on refresh.
|
|
if ctx.Err() != nil {
|
|
logger.Warnf(ctx, "[Agent][Round-%d] Context cancelled during LLM call; preserving partial step",
|
|
round)
|
|
if step.Thought != "" || len(step.ToolCalls) > 0 {
|
|
state.RoundSteps = append(state.RoundSteps, step)
|
|
}
|
|
return iterOutcomeBreak, nil
|
|
}
|
|
|
|
// 2. Analyze: Check for stop conditions (natural stop with no tool calls)
|
|
verdict := e.analyzeResponse(ctx, response, step, state.CurrentRound, sessionID, roundStart)
|
|
if verdict.isDone {
|
|
// Guard against empty content: when the LLM stops naturally with no
|
|
// content and no tool calls (e.g., thinking-only loop without KB),
|
|
// retry with a nudge message instead of accepting an empty answer.
|
|
if verdict.emptyContent {
|
|
*emptyRetries++
|
|
if *emptyRetries <= maxEmptyResponseRetries {
|
|
logger.Warnf(ctx, "[Agent][Round-%d] Empty content with stop - retrying (%d/%d)",
|
|
round, *emptyRetries, maxEmptyResponseRetries)
|
|
*messagesPtr = append(*messagesPtr, chat.Message{
|
|
Role: "user",
|
|
Content: "Please provide your complete answer now as plain text.",
|
|
})
|
|
return iterOutcomeContinue, nil
|
|
}
|
|
// Retries exhausted — use fallback message rather than empty answer
|
|
logger.Warnf(ctx, "[Agent][Round-%d] Empty content after %d retries - using fallback",
|
|
round, maxEmptyResponseRetries)
|
|
state.FinalAnswer = "I'm sorry, I was unable to generate a response. Please try again."
|
|
state.IsComplete = true
|
|
state.RoundSteps = append(state.RoundSteps, verdict.step)
|
|
return iterOutcomeBreak, nil
|
|
}
|
|
state.FinalAnswer = verdict.finalAnswer
|
|
state.IsComplete = true
|
|
state.RoundSteps = append(state.RoundSteps, verdict.step)
|
|
return iterOutcomeBreak, nil
|
|
}
|
|
|
|
// This round is non-terminal (it will execute tools and loop again). Any
|
|
// plain assistant text streamed live to the answer area this round was a
|
|
// preamble (e.g. "let me search the knowledge base…"), not the final
|
|
// answer. No explicit retraction signal is emitted: the agent only ends by
|
|
// stopping naturally with plain text and no tool calls, so the upcoming
|
|
// tool-call events are themselves the authoritative "that wasn't the final
|
|
// answer" marker. Both the stream handler and the UI treat any answer text
|
|
// preceding a tool call in the same stream as a preamble and relocate it
|
|
// into the steps tree. The preamble is still preserved as this round's
|
|
// Thought.
|
|
|
|
// 3. Act: Execute tool calls
|
|
e.executeToolCalls(ctx, response, &step, state.CurrentRound, sessionID, assistantMessageID)
|
|
toolCallCount = len(step.ToolCalls)
|
|
|
|
// 4. Observe: Add tool results to messages and write to context
|
|
state.RoundSteps = append(state.RoundSteps, step)
|
|
*messagesPtr = e.appendToolResults(*messagesPtr, step)
|
|
common.PipelineInfo(ctx, "Agent", "round_end", map[string]interface{}{
|
|
"iteration": state.CurrentRound,
|
|
"round": round,
|
|
"tool_calls": toolCallCount,
|
|
"thought_len": len(step.Thought),
|
|
})
|
|
|
|
return iterOutcomeNext, nil
|
|
}
|
|
|
|
// String returns a stable label for Langfuse output payloads.
|
|
func (o iterOutcome) String() string {
|
|
switch o {
|
|
case iterOutcomeNext:
|
|
return "next"
|
|
case iterOutcomeContinue:
|
|
return "continue"
|
|
case iterOutcomeBreak:
|
|
return "break"
|
|
default:
|
|
return "unknown"
|
|
}
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Tool result image VLM description helpers
|
|
// ---------------------------------------------------------------------------
|
|
|
|
const toolImageAnalysisPrompt = "Describe the content of this image in detail. " +
|
|
"If it contains text, extract all readable text. " +
|
|
"If it contains charts or diagrams, describe the data and structure."
|
|
|
|
// describeImages generates text descriptions for tool result images using the
|
|
// configured imageDescriber (VLM). Each image is decoded from a data URI and
|
|
// analyzed independently. Failures are logged and skipped gracefully.
|
|
// This follows the same pattern as Handler.analyzeImageAttachments().
|
|
func (e *AgentEngine) describeImages(ctx context.Context, imageDataURIs []string) []string {
|
|
if e.imageDescriber == nil {
|
|
return nil
|
|
}
|
|
var descriptions []string
|
|
for i, dataURI := range imageDataURIs {
|
|
if ctx.Err() != nil {
|
|
logger.Warnf(ctx, "[Agent] Context cancelled, skipping remaining %d tool result images", len(imageDataURIs)-i)
|
|
break
|
|
}
|
|
imgBytes, err := decodeDataURIBytes(dataURI)
|
|
if err != nil {
|
|
logger.Warnf(ctx, "[Agent] Failed to decode tool result image %d: %v", i, err)
|
|
continue
|
|
}
|
|
desc, err := e.imageDescriber(ctx, imgBytes, toolImageAnalysisPrompt)
|
|
if err != nil {
|
|
logger.Warnf(ctx, "[Agent] VLM analysis failed for tool result image %d: %v", i, err)
|
|
continue
|
|
}
|
|
descriptions = append(descriptions, strings.TrimSpace(desc))
|
|
}
|
|
return descriptions
|
|
}
|
|
|
|
// decodeDataURIBytes extracts raw bytes from a "data:mime;base64,..." URI.
|
|
// Retries with RawStdEncoding when standard base64 decoding fails (some MCP
|
|
// servers omit trailing '=' padding).
|
|
func decodeDataURIBytes(dataURI string) ([]byte, error) {
|
|
if !strings.HasPrefix(dataURI, "data:") {
|
|
return nil, fmt.Errorf("not a data URI")
|
|
}
|
|
idx := strings.Index(dataURI, ";base64,")
|
|
if idx < 0 {
|
|
return nil, fmt.Errorf("unsupported data URI encoding (expected base64)")
|
|
}
|
|
raw := dataURI[idx+8:]
|
|
decoded, err := base64.StdEncoding.DecodeString(raw)
|
|
if err != nil {
|
|
// Retry without padding — some MCP servers omit trailing '='
|
|
decoded, err = base64.RawStdEncoding.DecodeString(raw)
|
|
}
|
|
return decoded, err
|
|
}
|