opensource/dify
3
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-09-19 02:07:44 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(agent): make home snapshots optional and use backend defaults (#39702)

Browse Source 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
This commit is contained in:
盐粒 Yanli
2026-07-29 02:01:12 +00:00
committed by GitHub
co-authored by autofix-ci[bot]
parent 7eba562f7d
commit f6683c2c6b
47 changed files with 786 additions and 589 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dify-agent/docs/dify-agent/concepts/runtime-resources/index.md
+22 -16
View File
@@ -67,12 +67,17 @@ streams are observability state, not the Home/Workspace/Binding ledger.
## Creation and execution flow
Home Snapshot initialization uses `POST /home-snapshots/initialize`. Build Draft
Apply uses `POST /home-snapshots/from-binding`: Dify Agent acquires the exact
source Binding, snapshots its materialized Home through the backend-native
operation, releases the lease, and returns a new opaque snapshot ref. Dify API
then stores a new immutable `agent_home_snapshots` row and records its logical id
on the resulting config version. There is no replay or initialization fallback
Agent creation does not create a Home Snapshot. A config with no logical Home
Snapshot asks the selected backend to materialize its deployment-default Home
when the Binding is created. This default Home is mutable and private to the
Binding; it does not produce an `agent_home_snapshots` row or an implicit
snapshot ref.
Build Draft Apply uses `POST /home-snapshots/from-binding`: Dify Agent acquires
the exact source Binding, snapshots its materialized Home through the
backend-native operation, releases the lease, and returns a new opaque snapshot
ref. Dify API then stores a new immutable `agent_home_snapshots` row and records
its logical id on the resulting config version. There is no replay or fallback
when the source Binding is unavailable.
Before an Agent request, Dify API loads the specific product context. If it has
@@ -82,10 +87,12 @@ its owner and config/Home generation. Missing, retired, or mismatched Bindings
fail fast; Dify API does not search by Agent, Workspace, candidate count, or
recency, and it does not create a replacement implicitly.
`POST /execution-bindings` materializes the selected Home Snapshot and returns
opaque Binding and Workspace refs. Every create request represents a new
participant, even when the Agent, Snapshot, config generation, and Workspace
match another Binding. The request composition contains:
`POST /execution-bindings` accepts either an exact `home_snapshot_ref` or
`null`. An exact ref must be materialized without fallback; `null` selects the
backend's deployment-default Home. It returns opaque Binding and Workspace
refs. Every create request represents a new participant, even when the Agent,
Snapshot, config generation, and Workspace match another Binding. The request
composition contains:
```json
{
@@ -174,9 +181,9 @@ own Home plus the shared Workspace.
| Backend | Home Snapshot operations | Binding operations | Physical relationship |
| --- | --- | --- | --- |
| Local | Supported | Supported, including attaching multiple Bindings to one Workspace | Snapshot directory, per-Binding materialized Home, and Workspace directory are separate. |
| E2B | Supported | Supported without shared-Workspace attachment | Binding and Workspace refs map to the same E2B resource; Home initialization/checkpoint uses E2B snapshots. |
| Enterprise | Not implemented | Not implemented | Configuration is accepted, but every resource operation fails fast with `NotImplementedError`. |
| Local | Supported | Supported, including default empty Homes and attaching multiple Bindings to one Workspace | Snapshot directory, per-Binding materialized Home, and Workspace directory are separate. |
| E2B | Supported | Supported with template-backed default Homes, without shared-Workspace attachment | Binding and Workspace refs map to the same E2B resource; checkpoints use E2B snapshots. |
| Enterprise | Not implemented | Default-Home Binding creation, acquire, and coupled destroy are supported | Binding and Workspace refs map to one Gateway sandbox. Explicit Home Snapshot materialization fails fast. |
Local creates a new Home for every Binding id. Destroying one Binding without
the Workspace leaves sibling Homes and the shared Workspace intact. Current E2B
@@ -185,9 +192,8 @@ its Binding and Workspace are one Sandbox. It also rejects binding-only destroy.
Neither path creates a fallback Workspace or switches backends.
`DIFY_AGENT_E2B_ACTIVE_TIMEOUT_SECONDS` limits continuous active time for an E2B
resource. Runtime resources pause on timeout; temporary Home initialization
resources are killed. It is not a retention TTL and does not delete paused
resources or immutable snapshots.
resource. Runtime resources pause on timeout. It is not a retention TTL and
does not delete paused resources or immutable snapshots.
See the [Shell layer](../../user-manual/shell-layer/index.md) for request
composition and the [Operations Guide](../../guide/index.md) for Local and E2B
dify-agent/docs/dify-agent/get-started/index.md
+3 -4
View File
@@ -85,10 +85,9 @@ DIFY_AGENT_SANDBOX_FILE_UPLOAD_MAX_BYTES=52428800
E2B requires `DIFY_AGENT_E2B_API_KEY` and defaults to the prepared
`difys-default-team/dify-agent-local-sandbox` template. The E2B active timeout
pauses the physical resource behind a Binding or kills temporary Home
initialization resources; it is not a retention TTL. Enterprise settings are
accepted, but current Home Snapshot and Binding operations fail fast with
`NotImplementedError`.
pauses the physical resource behind a Binding; it is not a retention TTL.
Enterprise supports Bindings created from its deployment-default Home.
Immutable Home Snapshot creation and materialization remain unsupported there.
A shell-enabled request includes Execution Context, `dify.runtime`, and
`dify.shell`. Dify API creates or resolves the specific persistent Binding for
dify-agent/docs/dify-agent/guide/index.md
+6 -9
View File
@@ -42,13 +42,13 @@ also reads `.env` and `dify-agent/.env` when present.
| `DIFY_AGENT_RUNTIME_BACKEND` | `local` | Selects one coherent `local`, `enterprise`, or `e2b` Home Snapshot + Execution Binding backend profile. |
| `DIFY_AGENT_LOCAL_SANDBOX_ENDPOINT` | empty | Local shellctl data-plane URL. With the default Local selection, leaving it empty disables `dify.runtime` and resource endpoints. |
| `DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN` | empty | Optional bearer token sent to Local shellctl. |
| `DIFY_AGENT_ENTERPRISE_SANDBOX_GATEWAY_ENDPOINT` | empty | Enterprise Gateway endpoint required by configuration. Current Home Snapshot and Binding operations fail fast with `NotImplementedError`. |
| `DIFY_AGENT_ENTERPRISE_SANDBOX_GATEWAY_ENDPOINT` | empty | Enterprise Gateway endpoint required by configuration. Default-Home Bindings are supported; immutable Home Snapshot operations remain unsupported. |
| `DIFY_AGENT_ENTERPRISE_SANDBOX_GATEWAY_AUTH_TOKEN` | empty | Optional `X-Inner-Api-Key` sent to the Enterprise Gateway. |
| `DIFY_AGENT_ENTERPRISE_SANDBOX_GATEWAY_TIMEOUT` | `30` | Enterprise control-plane timeout in seconds. |
| `DIFY_AGENT_ENTERPRISE_SANDBOX_PROXY_TIMEOUT` | `60` | Enterprise shellctl-proxy timeout in seconds. |
| `DIFY_AGENT_E2B_API_KEY` | empty | E2B API key; required for E2B. |
| `DIFY_AGENT_E2B_TEMPLATE` | `difys-default-team/dify-agent-local-sandbox` | Prepared E2B template containing shellctl and the initial Home environment. |
| `DIFY_AGENT_E2B_ACTIVE_TIMEOUT_SECONDS` | `3600` | Maximum continuous active time, up to 3600 seconds. Binding resources pause on timeout; temporary Home initialization resources are killed. This is not a retention TTL. |
| `DIFY_AGENT_E2B_TEMPLATE` | `difys-default-team/dify-agent-local-sandbox` | Prepared E2B template containing shellctl and the deployment-default Home environment. |
| `DIFY_AGENT_E2B_ACTIVE_TIMEOUT_SECONDS` | `3600` | Maximum continuous active time, up to 3600 seconds. Binding resources pause on timeout. This is not a retention TTL. |
| `DIFY_AGENT_E2B_SHELLCTL_AUTH_TOKEN` | empty | Optional bearer token expected by shellctl inside the E2B template. |
| `DIFY_AGENT_E2B_SHELLCTL_PORT` | `5004` | shellctl port exposed by the E2B template. |
| `DIFY_AGENT_SANDBOX_FILE_UPLOAD_MAX_BYTES` | `52428800` | Standalone Dify Agent maximum for whole-file Workspace upload capture; 50 MiB by default. Docker Compose derives it from `PLUGIN_MAX_FILE_SIZE`. |
@@ -177,12 +177,9 @@ docker compose \
```
`DIFY_AGENT_E2B_ACTIVE_TIMEOUT_SECONDS` controls continuous active E2B time.
The physical resource behind a Binding pauses when that timeout fires; a
temporary Home initialization resource is killed. Pausing preserves the current
Workspace. The setting does not delete an aged paused resource or immutable
snapshot, and Dify Agent currently has no resource-age TTL, reconciler, or
eventual cleanup guarantee. Dify API retirement followed by Binding collection
kills the coupled E2B resource.
The physical resource behind a Binding pauses when that timeout fires, preserving
the current Workspace. The setting is not a resource-age TTL and does not delete
paused resources or immutable snapshots.
## Run runtime-backend integration contracts
dify-agent/docs/dify-agent/user-manual/shell-layer/index.md
+4 -4
View File
@@ -41,10 +41,10 @@ the opaque Binding ref belongs to `DifyRuntimeLayerConfig`.
## Runtime requirements
The server constructs one coherent runtime backend profile. Local and E2B
implement Home Snapshot and Execution Binding operations. Enterprise settings
can be selected, but resource operations currently fail fast with
`NotImplementedError`; there is no compatibility fallback to the retired
Sandbox protocol.
implement Home Snapshot and Execution Binding operations. Enterprise implements
default-Home Binding creation, acquisition, and coupled destruction, while
immutable Home Snapshot operations fail fast; there is no compatibility
fallback to the retired Sandbox protocol.
```python
from dify_agent.runtime.compositor_factory import create_default_layer_providers
dify-agent/src/dify_agent/client/_client.py
-19
View File
@@ -36,7 +36,6 @@ from dify_agent.protocol import (
DeleteHomeSnapshotRequest,
DestroyExecutionBindingRequest,
HomeSnapshotResponse,
InitializeHomeSnapshotRequest,
RUN_EVENT_ADAPTER,
RunEvent,
RunEventsResponse,
@@ -529,24 +528,6 @@ class Client:
response = self._post_sync_json("destroy_execution_binding_sync", "/execution-bindings/destroy", request)
_raise_for_status(response)
async def initialize_home_snapshot(self, request: InitializeHomeSnapshotRequest) -> HomeSnapshotResponse:
"""Create a backend-native initial Home Snapshot."""
response = await self._post_async_json(
"initialize_home_snapshot",
"/home-snapshots/initialize",
request,
)
return _parse_model_response(response, HomeSnapshotResponse)
def initialize_home_snapshot_sync(self, request: InitializeHomeSnapshotRequest) -> HomeSnapshotResponse:
"""Synchronous variant of ``initialize_home_snapshot``."""
response = self._post_sync_json(
"initialize_home_snapshot_sync",
"/home-snapshots/initialize",
request,
)
return _parse_model_response(response, HomeSnapshotResponse)
async def create_home_snapshot_from_binding(
self,
request: CreateHomeSnapshotFromBindingRequest,
dify-agent/src/dify_agent/protocol/__init__.py
-2
View File
@@ -46,7 +46,6 @@ from .home_snapshot import (
CreateHomeSnapshotFromBindingRequest,
DeleteHomeSnapshotRequest,
HomeSnapshotResponse,
InitializeHomeSnapshotRequest,
)
from .workspace import (
WorkspaceFileEntry,
@@ -79,7 +78,6 @@ __all__ = [
"EmptyRunEventData",
"LayerExitSignals",
"HomeSnapshotResponse",
"InitializeHomeSnapshotRequest",
"PydanticAIStreamRunEvent",
"RUN_EVENT_ADAPTER",
"RunCancelledEvent",
dify-agent/src/dify_agent/protocol/execution_binding.py
+1 -1
View File
@@ -11,7 +11,7 @@ class CreateExecutionBindingRequest(BaseModel):
binding_id: str = Field(min_length=1)
workspace_id: str = Field(min_length=1)
existing_workspace_ref: str | None = None
home_snapshot_ref: str = Field(min_length=1)
home_snapshot_ref: str | None = Field(default=None, min_length=1)
model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid")
dify-agent/src/dify_agent/protocol/home_snapshot.py
-9
View File
@@ -5,14 +5,6 @@ from typing import ClassVar
from pydantic import BaseModel, ConfigDict, Field
class InitializeHomeSnapshotRequest(BaseModel):
tenant_id: str = Field(min_length=1)
agent_id: str = Field(min_length=1)
home_snapshot_id: str = Field(min_length=1)
model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid")
class CreateHomeSnapshotFromBindingRequest(BaseModel):
tenant_id: str = Field(min_length=1)
agent_id: str = Field(min_length=1)
@@ -38,5 +30,4 @@ __all__ = [
"CreateHomeSnapshotFromBindingRequest",
"DeleteHomeSnapshotRequest",
"HomeSnapshotResponse",
"InitializeHomeSnapshotRequest",
]
dify-agent/src/dify_agent/runtime_backend/__init__.py
-2
View File
@@ -22,7 +22,6 @@ from .protocols import (
FileSystem,
HomeSnapshotBackend,
HomeSnapshotCreateSpec,
InitializeHomeSnapshotSpec,
RuntimeBackendProfile,
RuntimeLayout,
RuntimeLease,
@@ -46,7 +45,6 @@ __all__ = [
"HomeSnapshotCreateError",
"HomeSnapshotCreateSpec",
"HomeSnapshotNotFoundError",
"InitializeHomeSnapshotSpec",
"RuntimeBackendError",
"RuntimeBackendProfile",
"RuntimeLayout",
dify-agent/src/dify_agent/runtime_backend/e2b.py
+7 -39
View File
@@ -30,7 +30,6 @@ from dify_agent.runtime_backend.protocols import (
ExecutionBindingDestroySpec,
FileSystem,
HomeSnapshotCreateSpec,
InitializeHomeSnapshotSpec,
RuntimeLayout,
RuntimeLease,
)
@@ -163,44 +162,12 @@ class E2BSDKControlPlane:
class E2BHomeSnapshotBackend:
"""Implement immutable Home Snapshot operations with E2B snapshots.
Initialization snapshots the prepared deployment template and releases its
temporary E2B resource. Build Apply snapshots the E2B resource behind the
supplied ``RuntimeLease``. Dify API stores the returned value as an opaque
backend ref; this adapter keeps no cross-request state.
Build Apply snapshots the E2B resource behind the supplied ``RuntimeLease``.
Dify API stores the returned value as an opaque backend ref; this adapter
keeps no cross-request state.
"""
control_plane: E2BControlPlane
template: str
active_timeout_seconds: int
home_dir: str = "/home/dify"
async def initialize(self, spec: InitializeHomeSnapshotSpec) -> str:
sandbox: _E2BSandbox | None = None
try:
sandbox = await self.control_plane.create(
self.template,
timeout=self.active_timeout_seconds,
metadata={
"dify.resource": "home-snapshot-initialize",
"dify.tenant_id": spec.tenant_id,
"dify.agent_id": spec.agent_id,
"dify.home_snapshot_id": spec.home_snapshot_id,
},
on_timeout="kill",
)
_ = await sandbox.files.make_dir(self.home_dir)
snapshot = await sandbox.create_snapshot()
return snapshot.snapshot_id
except BaseException as exc:
if isinstance(exc, Exception):
raise HomeSnapshotCreateError(str(exc)) from exc
raise
finally:
if sandbox is not None:
try:
_ = await sandbox.kill()
except BaseException:
pass
async def create_from_runtime(self, *, spec: HomeSnapshotCreateSpec, source: RuntimeLease) -> str:
"""Create an immutable E2B snapshot from the source Binding's active lease."""
@@ -236,6 +203,7 @@ class E2BExecutionBindingBackend:
"""
control_plane: E2BControlPlane
template: str
active_timeout_seconds: int
shellctl_auth_token: str = ""
shellctl_port: int = 5004
@@ -244,13 +212,13 @@ class E2BExecutionBindingBackend:
)
async def create_binding(self, spec: ExecutionBindingCreateSpec) -> ExecutionBindingAllocation:
"""Create one paused E2B resource from an immutable Home Snapshot ref."""
"""Create one paused E2B resource from a snapshot or deployment template."""
if spec.existing_workspace_ref is not None:
raise SharedWorkspaceUnsupportedError("current E2B backend cannot attach to an existing Workspace")
sandbox: _E2BSandbox | None = None
try:
sandbox = await self.control_plane.create(
spec.home_snapshot_ref,
self.template if spec.home_snapshot_ref is None else spec.home_snapshot_ref,
timeout=self.active_timeout_seconds,
metadata={
"dify.resource": "runtime-sandbox",
@@ -274,7 +242,7 @@ class E2BExecutionBindingBackend:
except BaseException:
pass
if isinstance(exc, Exception):
if isinstance(exc, (BindingCreateError, SharedWorkspaceUnsupportedError)):
if isinstance(exc, BindingCreateError):
raise
raise BindingCreateError(str(exc)) from exc
raise
dify-agent/src/dify_agent/runtime_backend/enterprise.py
+83 -29
View File
@@ -1,10 +1,10 @@
"""Enterprise Gateway adapter for the working-environment protocol.
The existing Gateway can reconnect to and delete an already allocated sandbox,
but it cannot materialize a Home Snapshot or create a protocol-compliant
Binding. One physical sandbox owns both the materialized Home and Workspace, so
their cleanup is coupled. Runtime access remains operation-local and is routed
through the Gateway's shellctl proxy.
The existing Gateway can allocate, reconnect to, and delete a sandbox, but it
does not expose immutable Home Snapshot operations. One physical sandbox owns
both the materialized Home and Workspace, so their cleanup is coupled. Runtime
access remains operation-local and is routed through the Gateway's shellctl
proxy.
"""
from __future__ import annotations
@@ -21,8 +21,10 @@ from dify_agent.adapters.shell.protocols import ShellCommandProtocol, ShellProvi
from dify_agent.adapters.shell.shellctl import ShellctlClientProtocol, ShellctlCommands
from dify_agent.runtime_backend.errors import (
BindingAcquireError,
BindingCreateError,
BindingDestroyError,
BindingLostError,
SharedWorkspaceUnsupportedError,
WorkspacePreservationUnsupportedError,
)
from dify_agent.runtime_backend.protocols import (
@@ -31,7 +33,6 @@ from dify_agent.runtime_backend.protocols import (
ExecutionBindingDestroySpec,
FileSystem,
HomeSnapshotCreateSpec,
InitializeHomeSnapshotSpec,
RuntimeLayout,
RuntimeLease,
)
@@ -45,21 +46,13 @@ logger = logging.getLogger(__name__)
def _not_implemented() -> NotImplementedError:
return NotImplementedError("Enterprise Gateway does not implement the Execution Binding protocol")
return NotImplementedError("Enterprise Gateway does not implement immutable Home Snapshot operations")
@dataclass(slots=True)
class EnterpriseHomeSnapshotBackend:
"""Reject Home Snapshot operations until the Gateway exposes immutable snapshots."""
gateway_endpoint: str
auth_token: str
gateway_timeout: float = 30.0
async def initialize(self, spec: InitializeHomeSnapshotSpec) -> str:
del spec
raise _not_implemented()
async def create_from_runtime(self, *, spec: HomeSnapshotCreateSpec, source: RuntimeLease) -> str:
del spec, source
raise _not_implemented()
@@ -71,7 +64,7 @@ class EnterpriseHomeSnapshotBackend:
@dataclass(slots=True)
class EnterpriseExecutionBindingBackend:
"""Access and destroy legacy Gateway sandboxes as coupled physical Bindings."""
"""Manage Gateway sandboxes as coupled physical Bindings and Workspaces."""
gateway_endpoint: str
auth_token: str
@@ -82,8 +75,56 @@ class EnterpriseExecutionBindingBackend:
)
async def create_binding(self, spec: ExecutionBindingCreateSpec) -> ExecutionBindingAllocation:
del spec
raise _not_implemented()
"""Create a default Gateway sandbox and initialize its canonical layout."""
if spec.existing_workspace_ref is not None:
raise SharedWorkspaceUnsupportedError("current Enterprise backend cannot attach to an existing Workspace")
if spec.home_snapshot_ref is not None:
raise BindingCreateError("current Enterprise backend cannot materialize an immutable Home Snapshot")
sandbox_id: str | None = None
data_plane: ShellctlRuntimeLease | None = None
headers = {"X-Inner-Api-Key": self.auth_token} if self.auth_token else {}
try:
async with httpx.AsyncClient(
base_url=self.gateway_endpoint.rstrip("/"),
headers=headers,
timeout=httpx.Timeout(self.gateway_timeout),
) as client:
response = await client.post("/v1/sandboxes", json={"tenantId": spec.tenant_id})
_ = response.raise_for_status()
payload = response.json()
sandbox_id_value = payload.get("sandboxId") if isinstance(payload, dict) else None
if not isinstance(sandbox_id_value, str) or not sandbox_id_value:
raise BindingCreateError("Enterprise Gateway returned an invalid sandbox id")
sandbox_id = sandbox_id_value
data_plane = await self._create_data_plane(sandbox_id)
result = await run_shellctl_control_command(
ShellctlCommands(client=data_plane.client),
"\n".join(
[
"set -eu",
f"mkdir -p {shlex.quote(self.layout.home_dir)}",
f"rm -rf -- {shlex.quote(self.layout.workspace_dir)}",
f"mkdir -p {shlex.quote(self.layout.workspace_dir)}",
f"chmod 700 {shlex.quote(self.layout.home_dir)} {shlex.quote(self.layout.workspace_dir)}",
]
),
)
if result.exit_code != 0:
raise BindingCreateError(result.output)
await data_plane.close()
data_plane = None
return ExecutionBindingAllocation(binding_ref=sandbox_id, workspace_ref=sandbox_id)
except BaseException as exc:
await _close_best_effort(data_plane, binding_ref=sandbox_id or spec.binding_id)
if sandbox_id is not None:
await self._delete_sandbox_best_effort(sandbox_id)
if isinstance(exc, BindingCreateError):
raise
if isinstance(exc, Exception):
raise BindingCreateError(str(exc)) from exc
raise
async def acquire(self, binding_ref: str) -> RuntimeLease:
"""Reconnect to one existing Gateway sandbox without creating a replacement."""
@@ -137,21 +178,34 @@ class EnterpriseExecutionBindingBackend:
if spec.workspace_ref != spec.binding_ref:
raise BindingDestroyError("Enterprise Workspace ref must equal its Binding ref")
headers = {"X-Inner-Api-Key": self.auth_token} if self.auth_token else {}
encoded_binding_ref = quote(spec.binding_ref, safe="")
try:
async with httpx.AsyncClient(
base_url=self.gateway_endpoint.rstrip("/"),
headers=headers,
timeout=httpx.Timeout(self.gateway_timeout),
) as client:
response = await client.delete(f"/v1/sandboxes/{encoded_binding_ref}")
if response.status_code == 404:
return
_ = response.raise_for_status()
await self._delete_sandbox(spec.binding_ref)
except (httpx.TimeoutException, httpx.RequestError, httpx.HTTPStatusError) as exc:
raise BindingDestroyError(str(exc)) from exc
async def _delete_sandbox(self, sandbox_id: str) -> None:
headers = {"X-Inner-Api-Key": self.auth_token} if self.auth_token else {}
encoded_sandbox_id = quote(sandbox_id, safe="")
async with httpx.AsyncClient(
base_url=self.gateway_endpoint.rstrip("/"),
headers=headers,
timeout=httpx.Timeout(self.gateway_timeout),
) as client:
response = await client.delete(f"/v1/sandboxes/{encoded_sandbox_id}")
if response.status_code == 404:
return
_ = response.raise_for_status()
async def _delete_sandbox_best_effort(self, sandbox_id: str) -> None:
try:
await self._delete_sandbox(sandbox_id)
except BaseException:
logger.warning(
"failed to delete Enterprise sandbox after Binding creation failed",
exc_info=True,
extra={"binding_ref": sandbox_id},
)
async def _create_data_plane(self, binding_ref: str) -> ShellctlRuntimeLease:
proxy_base_url = f"{self.gateway_endpoint.rstrip('/')}/proxy/"
headers = {"X-Sandbox-Id": binding_ref}
dify-agent/src/dify_agent/runtime_backend/local.py
+13 -36
View File
@@ -27,7 +27,6 @@ from dify_agent.runtime_backend.protocols import (
ExecutionBindingCreateSpec,
ExecutionBindingDestroySpec,
HomeSnapshotCreateSpec,
InitializeHomeSnapshotSpec,
RuntimeLayout,
RuntimeLease,
)
@@ -49,28 +48,6 @@ class LocalHomeSnapshotBackend:
snapshot_root: str = "/home/dify/.dify-agent-home-snapshots"
client_factory: ShellctlClientFactory | None = None
async def initialize(self, spec: InitializeHomeSnapshotSpec) -> str:
snapshot_ref = _local_snapshot_ref(spec.home_snapshot_id)
lease = self._control_lease(snapshot_ref)
target = self._snapshot_dir(snapshot_ref)
try:
result = await run_shellctl_control_command(
lease.commands,
f"set -eu\nmkdir -p {shlex.quote(target)}\nchmod 700 {shlex.quote(target)}",
)
if result.exit_code != 0:
raise HomeSnapshotCreateError(result.output)
return snapshot_ref
except BaseException as exc:
await _remove_partial(lease.commands, target=target, resource_ref=snapshot_ref)
if isinstance(exc, HomeSnapshotCreateError):
raise
if isinstance(exc, Exception):
raise HomeSnapshotCreateError(str(exc)) from exc
raise
finally:
await _close_best_effort(lease, resource_ref=snapshot_ref)
async def create_from_runtime(self, *, spec: HomeSnapshotCreateSpec, source: RuntimeLease) -> str:
snapshot_ref = _local_snapshot_ref(spec.home_snapshot_id)
target = self._snapshot_dir(snapshot_ref)
@@ -142,31 +119,31 @@ class LocalExecutionBindingBackend:
async def create_binding(self, spec: ExecutionBindingCreateSpec) -> ExecutionBindingAllocation:
binding_id = _validated_ref_part(spec.binding_id)
workspace_id = _validated_ref_part(spec.workspace_id)
snapshot_ref = _validated_ref_part(spec.home_snapshot_ref)
workspace_ref = workspace_id
if spec.existing_workspace_ref is not None:
existing_workspace_ref = _validated_ref_part(spec.existing_workspace_ref)
if existing_workspace_ref != workspace_ref:
raise BindingCreateError("existing Workspace ref does not match workspace_id")
snapshot_dir: str | None = None
if spec.home_snapshot_ref is not None:
snapshot_ref = _validated_ref_part(spec.home_snapshot_ref)
snapshot_dir = f"{self.snapshot_root.rstrip('/')}/{snapshot_ref}"
binding_ref = _local_binding_ref(binding_id=binding_id, workspace_id=workspace_id)
lease = self._control_lease(binding_ref)
home_dir = self._home_dir(binding_id)
workspace_dir = self._workspace_dir(workspace_id)
snapshot_dir = f"{self.snapshot_root.rstrip('/')}/{snapshot_ref}"
creates_workspace = spec.existing_workspace_ref is None
workspace_setup = (
f"mkdir -p {shlex.quote(workspace_dir)}" if creates_workspace else f"test -d {shlex.quote(workspace_dir)}"
)
script = "\n".join(
[
"set -eu",
f"test -d {shlex.quote(snapshot_dir)}",
workspace_setup,
f"mkdir -p {shlex.quote(home_dir)}",
f"cp -a {shlex.quote(snapshot_dir)}/. {shlex.quote(home_dir)}/",
f"chmod 700 {shlex.quote(home_dir)} {shlex.quote(workspace_dir)}",
]
)
setup = ["set -eu"]
if snapshot_dir is not None:
setup.append(f"test -d {shlex.quote(snapshot_dir)}")
setup.extend([workspace_setup, f"mkdir -p {shlex.quote(home_dir)}"])
if snapshot_dir is not None:
setup.append(f"cp -a {shlex.quote(snapshot_dir)}/. {shlex.quote(home_dir)}/")
setup.append(f"chmod 700 {shlex.quote(home_dir)} {shlex.quote(workspace_dir)}")
script = "\n".join(setup)
lease = self._control_lease(binding_ref)
try:
result = await run_shellctl_control_command(lease.commands, script)
if result.exit_code != 0:
dify-agent/src/dify_agent/runtime_backend/profile.py
+2 -7
View File
@@ -99,11 +99,7 @@ def create_runtime_backend_profile(settings: RuntimeBackendSettings) -> RuntimeB
endpoint = settings.enterprise_sandbox_gateway_endpoint or ""
token = settings.enterprise_sandbox_gateway_auth_token or ""
return RuntimeBackendProfile(
home_snapshots=EnterpriseHomeSnapshotBackend(
gateway_endpoint=endpoint,
auth_token=token,
gateway_timeout=settings.enterprise_sandbox_gateway_timeout,
),
home_snapshots=EnterpriseHomeSnapshotBackend(),
execution_bindings=EnterpriseExecutionBindingBackend(
gateway_endpoint=endpoint,
auth_token=token,
@@ -116,11 +112,10 @@ def create_runtime_backend_profile(settings: RuntimeBackendSettings) -> RuntimeB
return RuntimeBackendProfile(
home_snapshots=E2BHomeSnapshotBackend(
control_plane=control_plane,
template=settings.e2b_template,
active_timeout_seconds=settings.e2b_active_timeout_seconds,
),
execution_bindings=E2BExecutionBindingBackend(
control_plane=control_plane,
template=settings.e2b_template,
active_timeout_seconds=settings.e2b_active_timeout_seconds,
shellctl_auth_token=settings.e2b_shellctl_auth_token,
shellctl_port=settings.e2b_shellctl_port,
dify-agent/src/dify_agent/runtime_backend/protocols.py
+13 -25
View File
@@ -13,13 +13,6 @@ from typing import Protocol
from dify_agent.adapters.shell.protocols import ShellCommandProtocol
@dataclass(frozen=True, slots=True)
class InitializeHomeSnapshotSpec:
tenant_id: str
agent_id: str
home_snapshot_id: str
@dataclass(frozen=True, slots=True)
class HomeSnapshotCreateSpec:
tenant_id: str
@@ -100,7 +93,7 @@ class ExecutionBindingCreateSpec:
binding_id: str
workspace_id: str
existing_workspace_ref: str | None
home_snapshot_ref: str
home_snapshot_ref: str | None = None
@dataclass(frozen=True, slots=True)
@@ -126,12 +119,18 @@ class ExecutionBindingBackend(Protocol):
async def create_binding(self, spec: ExecutionBindingCreateSpec) -> ExecutionBindingAllocation:
"""Materialize a mutable Home and make the requested Workspace ready.
Implementations must initialize the Home from ``home_snapshot_ref`` and
return stable opaque refs only after both resources are usable. With an
``existing_workspace_ref``, they must attach that Workspace without
clearing or replacing its contents; unsupported sharing must fail before
mutating it. On failure, implementations should clean up newly allocated
partial resources and must not damage a pre-existing Workspace.
With a non-null ``home_snapshot_ref``, implementations must initialize
the Home from that exact immutable snapshot and must fail rather than
fall back when it is unavailable. With ``None``, implementations must
create an independent mutable Home from their deployment default without
implicitly creating an immutable snapshot.
Implementations must return stable opaque refs only after Home and
Workspace are usable. With an ``existing_workspace_ref``, they must
attach that Workspace without clearing or replacing its contents;
unsupported sharing must fail before mutating it. On failure,
implementations should clean up newly allocated partial resources and
must not damage a pre-existing Workspace.
"""
...
@@ -171,16 +170,6 @@ class ExecutionBindingBackend(Protocol):
class HomeSnapshotBackend(Protocol):
"""Manage immutable backend-native Home resources."""
async def initialize(self, spec: InitializeHomeSnapshotSpec) -> str:
"""Create the deployment-defined baseline Home Snapshot.
Implementations may use any backend-native bootstrap mechanism, but must
return a stable opaque ref only after an immutable snapshot is ready for
future Binding creation. Temporary bootstrap resources must not become
part of the logical snapshot lifecycle.
"""
...
async def create_from_runtime(self, *, spec: HomeSnapshotCreateSpec, source: RuntimeLease) -> str:
"""Capture the source lease's current Home as a new immutable snapshot.
@@ -217,7 +206,6 @@ __all__ = [
"FileSystem",
"HomeSnapshotBackend",
"HomeSnapshotCreateSpec",
"InitializeHomeSnapshotSpec",
"RuntimeBackendProfile",
"RuntimeLayout",
"RuntimeLease",
dify-agent/src/dify_agent/server/home_snapshots.py
-15
View File
@@ -6,7 +6,6 @@ from dify_agent.protocol.home_snapshot import (
CreateHomeSnapshotFromBindingRequest,
DeleteHomeSnapshotRequest,
HomeSnapshotResponse,
InitializeHomeSnapshotRequest,
)
from dify_agent.runtime_backend import (
BindingAcquireError,
@@ -16,7 +15,6 @@ from dify_agent.runtime_backend import (
HomeSnapshotCreateError,
HomeSnapshotCreateSpec,
HomeSnapshotNotFoundError,
InitializeHomeSnapshotSpec,
)
from dify_agent.runtime_backend.leases import open_runtime_lease
@@ -38,19 +36,6 @@ class HomeSnapshotService:
home_snapshots: HomeSnapshotBackend
execution_bindings: ExecutionBindingBackend
async def initialize(self, request: InitializeHomeSnapshotRequest) -> HomeSnapshotResponse:
try:
snapshot_ref = await self.home_snapshots.initialize(
InitializeHomeSnapshotSpec(
tenant_id=request.tenant_id,
agent_id=request.agent_id,
home_snapshot_id=request.home_snapshot_id,
)
)
except HomeSnapshotCreateError as exc:
raise HomeSnapshotServiceError("home_snapshot_create_failed", str(exc), status_code=502) from exc
return HomeSnapshotResponse(snapshot_ref=snapshot_ref)
async def create_from_binding(
self,
request: CreateHomeSnapshotFromBindingRequest,
dify-agent/src/dify_agent/server/routes/home_snapshots.py
-14
View File
@@ -9,7 +9,6 @@ from dify_agent.protocol.home_snapshot import (
CreateHomeSnapshotFromBindingRequest,
DeleteHomeSnapshotRequest,
HomeSnapshotResponse,
InitializeHomeSnapshotRequest,
)
from dify_agent.server.home_snapshots import HomeSnapshotService, HomeSnapshotServiceError
@@ -26,19 +25,6 @@ def create_home_snapshots_router(get_service: Callable[[], HomeSnapshotService |
)
return service
@router.post("/initialize", response_model=HomeSnapshotResponse, status_code=status.HTTP_201_CREATED)
async def initialize_snapshot(
request: InitializeHomeSnapshotRequest,
service: Annotated[HomeSnapshotService, Depends(service_dep)],
) -> HomeSnapshotResponse:
try:
return await service.initialize(request)
except HomeSnapshotServiceError as exc:
raise HTTPException(
status_code=exc.status_code,
detail={"code": exc.code, "message": exc.message},
) from exc
@router.post("/from-binding", response_model=HomeSnapshotResponse, status_code=status.HTTP_201_CREATED)
async def create_snapshot_from_binding(
request: CreateHomeSnapshotFromBindingRequest,
dify-agent/tests/integration/dify_agent/runtime_backend/test_working_environment.py
+11 -35
View File
@@ -12,10 +12,9 @@ from dify_agent.runtime_backend import (
ExecutionBindingCreateSpec,
ExecutionBindingDestroySpec,
HomeSnapshotCreateSpec,
InitializeHomeSnapshotSpec,
)
from dify_agent.runtime_backend.e2b import E2BExecutionBindingBackend, E2BHomeSnapshotBackend, E2BSDKControlPlane
from dify_agent.runtime_backend.local import LocalExecutionBindingBackend, LocalHomeSnapshotBackend
from dify_agent.runtime_backend.local import LocalExecutionBindingBackend
pytestmark = pytest.mark.integration
@@ -32,19 +31,10 @@ async def test_local_two_agents_share_workspace_but_not_home() -> None:
endpoint = _required_env("DIFY_AGENT_TEST_LOCAL_SHELLCTL_ENDPOINT", "real Local shellctl")
token = os.environ.get("DIFY_AGENT_TEST_LOCAL_SHELLCTL_AUTH_TOKEN", "")
marker = uuid.uuid4().hex
snapshots = LocalHomeSnapshotBackend(endpoint=endpoint, auth_token=token)
bindings = LocalExecutionBindingBackend(endpoint=endpoint, auth_token=token)
snapshot_ref: str | None = None
allocations = []
active_leases = []
try:
snapshot_ref = await snapshots.initialize(
InitializeHomeSnapshotSpec(
tenant_id="integration-tenant",
agent_id="integration-agent",
home_snapshot_id=marker,
)
)
first = await bindings.create_binding(
ExecutionBindingCreateSpec(
tenant_id="integration-tenant",
@@ -52,7 +42,7 @@ async def test_local_two_agents_share_workspace_but_not_home() -> None:
binding_id=f"binding-a-{marker}",
workspace_id=f"workspace-{marker}",
existing_workspace_ref=None,
home_snapshot_ref=snapshot_ref,
home_snapshot_ref=None,
)
)
allocations.append(first)
@@ -71,7 +61,7 @@ async def test_local_two_agents_share_workspace_but_not_home() -> None:
binding_id=f"binding-b-{marker}",
workspace_id=f"workspace-{marker}",
existing_workspace_ref=first.workspace_ref,
home_snapshot_ref=snapshot_ref,
home_snapshot_ref=None,
)
)
allocations.append(second)
@@ -102,11 +92,6 @@ async def test_local_two_agents_share_workspace_but_not_home() -> None:
)
except BaseException as exc:
cleanup_errors.append(exc)
if snapshot_ref is not None:
try:
await snapshots.delete(snapshot_ref)
except BaseException as exc:
cleanup_errors.append(exc)
if cleanup_errors and not primary_error:
raise cleanup_errors[0]
@@ -120,22 +105,14 @@ async def test_e2b_binding_checkpoint_and_collection() -> None:
)
marker = uuid.uuid4().hex
control = E2BSDKControlPlane(api_key=api_key)
snapshots = E2BHomeSnapshotBackend(control_plane=control, template=template, active_timeout_seconds=3600)
bindings = E2BExecutionBindingBackend(control_plane=control, active_timeout_seconds=3600)
snapshot_ref: str | None = None
snapshots = E2BHomeSnapshotBackend(control_plane=control)
bindings = E2BExecutionBindingBackend(control_plane=control, template=template, active_timeout_seconds=3600)
checkpoint_ref: str | None = None
allocation = None
checkpoint_allocation = None
lease = None
checkpoint_lease = None
try:
snapshot_ref = await snapshots.initialize(
InitializeHomeSnapshotSpec(
tenant_id="integration-tenant",
agent_id="integration-agent",
home_snapshot_id=marker,
)
)
allocation = await bindings.create_binding(
ExecutionBindingCreateSpec(
tenant_id="integration-tenant",
@@ -143,7 +120,7 @@ async def test_e2b_binding_checkpoint_and_collection() -> None:
binding_id=marker,
workspace_id=marker,
existing_workspace_ref=None,
home_snapshot_ref=snapshot_ref,
home_snapshot_ref=None,
)
)
lease = await bindings.acquire(allocation.binding_ref)
@@ -211,11 +188,10 @@ async def test_e2b_binding_checkpoint_and_collection() -> None:
)
except BaseException as exc:
cleanup_errors.append(exc)
for ref in (checkpoint_ref, snapshot_ref):
if ref is not None:
try:
await snapshots.delete(ref)
except BaseException as exc:
cleanup_errors.append(exc)
if checkpoint_ref is not None:
try:
await snapshots.delete(checkpoint_ref)
except BaseException as exc:
cleanup_errors.append(exc)
if cleanup_errors and not primary_error:
raise cleanup_errors[0]
dify-agent/tests/local/dify_agent/client/test_client.py
+3 -21
View File
@@ -28,7 +28,6 @@ from dify_agent.protocol import (
CreateHomeSnapshotFromBindingRequest,
CreateRunRequest,
DestroyExecutionBindingRequest,
InitializeHomeSnapshotRequest,
RUN_EVENT_ADAPTER,
RunCancelledEvent,
RunEvent,
@@ -311,14 +310,6 @@ def test_async_workspace_methods_post_dtos_and_parse_responses() -> None:
asyncio.run(scenario())
def _initialize_home_snapshot_request() -> InitializeHomeSnapshotRequest:
return InitializeHomeSnapshotRequest(
tenant_id="tenant-1",
agent_id="agent-1",
home_snapshot_id="home-1",
)
def test_sync_execution_binding_client_uses_private_binding_routes() -> None:
def handler(request: httpx.Request) -> httpx.Response:
payload = cast(dict[str, object], json.loads(request.content))
@@ -368,12 +359,9 @@ def _create_home_snapshot_from_binding_request() -> CreateHomeSnapshotFromBindin
)
def test_sync_home_snapshot_client_parses_initialize_checkpoint_and_delete() -> None:
def test_sync_home_snapshot_client_parses_checkpoint_and_delete() -> None:
def handler(request: httpx.Request) -> httpx.Response:
if request.method == "POST":
if request.url.path == "/home-snapshots/initialize":
assert json.loads(request.content) == _initialize_home_snapshot_request().model_dump(mode="json")
return httpx.Response(201, json={"snapshot_ref": "initial-home"})
if request.url.path == "/home-snapshots/from-binding":
assert json.loads(request.content) == _create_home_snapshot_from_binding_request().model_dump(
mode="json"
@@ -387,20 +375,16 @@ def test_sync_home_snapshot_client_parses_initialize_checkpoint_and_delete() ->
http_client = httpx.Client(transport=httpx.MockTransport(handler))
client = Client(base_url="http://testserver", sync_http_client=http_client)
initialized = client.initialize_home_snapshot_sync(_initialize_home_snapshot_request())
created = client.create_home_snapshot_from_binding_sync(_create_home_snapshot_from_binding_request())
client.delete_home_snapshot_sync(created.snapshot_ref)
assert initialized.snapshot_ref == "initial-home"
assert created.snapshot_ref == "team/home 1"
http_client.close()
def test_async_home_snapshot_client_parses_initialize_checkpoint_and_delete() -> None:
def test_async_home_snapshot_client_parses_checkpoint_and_delete() -> None:
def handler(request: httpx.Request) -> httpx.Response:
if request.method == "POST":
if request.url.path == "/home-snapshots/initialize":
return httpx.Response(201, json={"snapshot_ref": "initial-home"})
if request.url.path == "/home-snapshots/from-binding":
return httpx.Response(201, json={"snapshot_ref": "team/home 1"})
assert request.url.path == "/home-snapshots/delete"
@@ -411,11 +395,9 @@ def test_async_home_snapshot_client_parses_initialize_checkpoint_and_delete() ->
http_client = httpx.AsyncClient(transport=httpx.MockTransport(handler))
client = Client(base_url="http://testserver", async_http_client=http_client)
initialized = await client.initialize_home_snapshot(_initialize_home_snapshot_request())
created = await client.create_home_snapshot_from_binding(_create_home_snapshot_from_binding_request())
await client.delete_home_snapshot(created.snapshot_ref)
assert initialized.snapshot_ref == "initial-home"
assert created.snapshot_ref == "team/home 1"
await http_client.aclose()
@@ -430,7 +412,7 @@ def test_home_snapshot_client_maps_sync_validation_and_async_http_errors() -> No
)
with pytest.raises(DifyAgentValidationError):
_ = sync_client.initialize_home_snapshot_sync(_initialize_home_snapshot_request())
_ = sync_client.create_home_snapshot_from_binding_sync(_create_home_snapshot_from_binding_request())
sync_http_client.close()
async def scenario() -> None:
dify-agent/tests/local/dify_agent/protocol/test_working_environment.py
+23
View File
@@ -29,6 +29,29 @@ def test_execution_binding_request_uses_opaque_backend_refs() -> None:
}
def test_execution_binding_request_accepts_missing_or_null_home_snapshot_ref() -> None:
fields = {
"tenant_id": "tenant-1",
"agent_id": "agent-1",
"binding_id": "binding-1",
"workspace_id": "workspace-1",
}
assert CreateExecutionBindingRequest(**fields).home_snapshot_ref is None
assert CreateExecutionBindingRequest(**fields, home_snapshot_ref=None).home_snapshot_ref is None
def test_execution_binding_request_rejects_empty_home_snapshot_ref() -> None:
with pytest.raises(ValidationError, match="home_snapshot_ref"):
CreateExecutionBindingRequest(
tenant_id="tenant-1",
agent_id="agent-1",
binding_id="binding-1",
workspace_id="workspace-1",
home_snapshot_ref="",
)
def test_destroy_workspace_requires_workspace_ref() -> None:
with pytest.raises(ValidationError, match="workspace_ref"):
DestroyExecutionBindingRequest(binding_ref="binding-1", destroy_workspace=True)
dify-agent/tests/local/dify_agent/runtime_backend/test_e2b.py
+63 -26
View File
@@ -10,7 +10,6 @@ from dify_agent.runtime_backend import (
ExecutionBindingCreateSpec,
ExecutionBindingDestroySpec,
HomeSnapshotCreateSpec,
InitializeHomeSnapshotSpec,
SharedWorkspaceUnsupportedError,
WorkspacePreservationUnsupportedError,
)
@@ -86,7 +85,7 @@ class _ControlPlane:
sandbox = _Sandbox(sandbox_id=sandbox_id, pause_error=self.pause_error)
self.sandboxes[sandbox_id] = sandbox
self.created.append((template, on_timeout))
assert metadata["dify.resource"] in {"home-snapshot-initialize", "runtime-sandbox"}
assert metadata["dify.resource"] == "runtime-sandbox"
return sandbox
async def connect(self, handle: str, *, timeout: int) -> _Sandbox:
@@ -103,49 +102,57 @@ class _ControlPlane:
@pytest.mark.anyio
async def test_e2b_profile_uses_snapshot_as_runtime_template_and_couples_refs() -> None:
async def test_e2b_binding_uses_default_template_or_exact_snapshot_and_couples_refs() -> None:
control = _ControlPlane()
snapshots = E2BHomeSnapshotBackend(
control_plane=control, # pyright: ignore[reportArgumentType]
)
bindings = E2BExecutionBindingBackend(
control_plane=control, # pyright: ignore[reportArgumentType]
template="prepared-template",
active_timeout_seconds=3600,
)
bindings = E2BExecutionBindingBackend(
control_plane=control, # pyright: ignore[reportArgumentType]
active_timeout_seconds=3600,
)
snapshot_ref = await snapshots.initialize(
InitializeHomeSnapshotSpec(tenant_id="tenant-1", agent_id="agent-1", home_snapshot_id="home-1")
)
allocation = await bindings.create_binding(
default_allocation = await bindings.create_binding(
ExecutionBindingCreateSpec(
tenant_id="tenant-1",
agent_id="agent-1",
binding_id="binding-1",
workspace_id="workspace-1",
existing_workspace_ref=None,
home_snapshot_ref=snapshot_ref,
home_snapshot_ref=None,
)
)
snapshot_allocation = await bindings.create_binding(
ExecutionBindingCreateSpec(
tenant_id="tenant-1",
agent_id="agent-1",
binding_id="binding-2",
workspace_id="workspace-2",
existing_workspace_ref=None,
home_snapshot_ref="snapshot-1",
)
)
assert control.created == [("prepared-template", "kill"), (snapshot_ref, "pause")]
assert allocation.binding_ref == allocation.workspace_ref
runtime = control.sandboxes[allocation.binding_ref]
assert control.created == [("prepared-template", "pause"), ("snapshot-1", "pause")]
assert default_allocation.binding_ref == default_allocation.workspace_ref
assert snapshot_allocation.binding_ref == snapshot_allocation.workspace_ref
runtime = control.sandboxes[default_allocation.binding_ref]
assert runtime.files.paths == {"/home/dify/workspace"}
assert runtime.pauses == [True]
await bindings.destroy_binding(
ExecutionBindingDestroySpec(
binding_ref=allocation.binding_ref,
workspace_ref=allocation.workspace_ref,
destroy_workspace=True,
for allocation in (default_allocation, snapshot_allocation):
await bindings.destroy_binding(
ExecutionBindingDestroySpec(
binding_ref=allocation.binding_ref,
workspace_ref=allocation.workspace_ref,
destroy_workspace=True,
)
)
)
await snapshots.delete(snapshot_ref)
await snapshots.delete("snapshot-1")
assert control.killed == [allocation.binding_ref]
assert control.deleted_snapshots == [snapshot_ref]
assert control.killed == [default_allocation.binding_ref, snapshot_allocation.binding_ref]
assert control.deleted_snapshots == ["snapshot-1"]
@pytest.mark.anyio
@@ -153,6 +160,7 @@ async def test_e2b_rejects_shared_workspace_and_binding_only_destroy() -> None:
control = _ControlPlane()
backend = E2BExecutionBindingBackend(
control_plane=control, # pyright: ignore[reportArgumentType]
template="prepared-template",
active_timeout_seconds=3600,
)
spec = ExecutionBindingCreateSpec(
@@ -177,6 +185,7 @@ async def test_e2b_binding_create_kills_sandbox_when_initialization_fails() -> N
control = _ControlPlane(pause_error=RuntimeError("pause failed"))
backend = E2BExecutionBindingBackend(
control_plane=control, # pyright: ignore[reportArgumentType]
template="prepared-template",
active_timeout_seconds=3600,
)
@@ -196,6 +205,36 @@ async def test_e2b_binding_create_kills_sandbox_when_initialization_fails() -> N
assert sandbox.killed == 1
@pytest.mark.anyio
async def test_e2b_missing_explicit_snapshot_does_not_fall_back_to_template() -> None:
class _FailingControlPlane(_ControlPlane):
async def create(self, template: str, *, timeout: int, metadata: dict[str, str], on_timeout: str) -> _Sandbox:
del timeout, metadata
self.created.append((template, on_timeout))
raise RuntimeError("snapshot unavailable")
control = _FailingControlPlane()
backend = E2BExecutionBindingBackend(
control_plane=control, # pyright: ignore[reportArgumentType]
template="prepared-template",
active_timeout_seconds=3600,
)
with pytest.raises(BindingCreateError, match="snapshot unavailable"):
await backend.create_binding(
ExecutionBindingCreateSpec(
tenant_id="tenant-1",
agent_id="agent-1",
binding_id="binding-1",
workspace_id="workspace-1",
existing_workspace_ref=None,
home_snapshot_ref="missing-snapshot",
)
)
assert control.created == [("missing-snapshot", "pause")]
@pytest.mark.anyio
async def test_e2b_checkpoint_uses_exact_source_runtime() -> None:
control = _ControlPlane()
@@ -206,8 +245,6 @@ async def test_e2b_checkpoint_uses_exact_source_runtime() -> None:
)
backend = E2BHomeSnapshotBackend(
control_plane=control, # pyright: ignore[reportArgumentType]
template="prepared-template",
active_timeout_seconds=3600,
)
snapshot_ref = await backend.create_from_runtime(
dify-agent/tests/local/dify_agent/runtime_backend/test_enterprise_backend.py
+109 -16
View File
@@ -9,13 +9,14 @@ import pytest
from dify_agent.runtime_backend import (
BindingAcquireError,
BindingCreateError,
BindingDestroyError,
BindingLostError,
ExecutionBindingCreateSpec,
ExecutionBindingDestroySpec,
HomeSnapshotCreateSpec,
InitializeHomeSnapshotSpec,
RuntimeLease,
SharedWorkspaceUnsupportedError,
WorkspacePreservationUnsupportedError,
)
from dify_agent.runtime_backend.enterprise import (
@@ -282,23 +283,55 @@ async def test_enterprise_destroy_propagates_gateway_failure(
@pytest.mark.anyio
async def test_enterprise_allocation_and_home_snapshots_remain_explicitly_not_implemented() -> None:
snapshots = EnterpriseHomeSnapshotBackend(gateway_endpoint="https://gateway", auth_token="secret")
bindings = EnterpriseExecutionBindingBackend(gateway_endpoint="https://gateway", auth_token="secret")
async def test_enterprise_default_binding_creates_gateway_sandbox_and_layout(
monkeypatch: pytest.MonkeyPatch,
) -> None:
requests: list[httpx.Request] = []
with pytest.raises(NotImplementedError, match="Execution Binding protocol"):
_ = await snapshots.initialize(
InitializeHomeSnapshotSpec(tenant_id="tenant-1", agent_id="agent-1", home_snapshot_id="home-1")
def handler(request: httpx.Request) -> httpx.Response:
requests.append(request)
if request.url.path == "/v1/sandboxes":
assert json.loads(request.content) == {"tenantId": "tenant-1"}
return httpx.Response(201, json={"sandboxId": "sandbox-1", "status": "running"})
if request.url.path == "/proxy/v1/jobs/run":
assert request.headers["X-Sandbox-Id"] == "sandbox-1"
payload = cast(dict[str, object], json.loads(request.content))
script = payload["script"]
assert isinstance(script, str)
assert "mkdir -p /home/dify" in script
assert "rm -rf -- /home/dify/workspace" in script
return _job_response()
return httpx.Response(200, json={"job_id": "job-1"})
clients = _mock_http(monkeypatch, handler)
backend = EnterpriseExecutionBindingBackend(gateway_endpoint="http://gateway.example", auth_token="secret")
allocation = await backend.create_binding(
ExecutionBindingCreateSpec(
tenant_id="tenant-1",
agent_id="agent-1",
binding_id="binding-1",
workspace_id="workspace-1",
existing_workspace_ref=None,
home_snapshot_ref=None,
)
with pytest.raises(NotImplementedError, match="Execution Binding protocol"):
_ = await snapshots.create_from_runtime(
spec=HomeSnapshotCreateSpec(tenant_id="tenant-1", agent_id="agent-1", home_snapshot_id="home-2"),
source=cast(RuntimeLease, object()),
)
with pytest.raises(NotImplementedError, match="Execution Binding protocol"):
await snapshots.delete("snapshot-1")
with pytest.raises(NotImplementedError, match="Execution Binding protocol"):
_ = await bindings.create_binding(
)
assert allocation.binding_ref == allocation.workspace_ref == "sandbox-1"
assert requests[0].headers["X-Inner-Api-Key"] == "secret"
assert all(client.is_closed for client in clients)
@pytest.mark.anyio
async def test_enterprise_binding_rejects_snapshot_and_shared_workspace_before_gateway_call(
monkeypatch: pytest.MonkeyPatch,
) -> None:
requests: list[httpx.Request] = []
_ = _mock_http(monkeypatch, lambda request: requests.append(request) or httpx.Response(500))
backend = EnterpriseExecutionBindingBackend(gateway_endpoint="http://gateway.example", auth_token="secret")
with pytest.raises(BindingCreateError, match="immutable Home Snapshot"):
await backend.create_binding(
ExecutionBindingCreateSpec(
tenant_id="tenant-1",
agent_id="agent-1",
@@ -308,3 +341,63 @@ async def test_enterprise_allocation_and_home_snapshots_remain_explicitly_not_im
home_snapshot_ref="snapshot-1",
)
)
with pytest.raises(SharedWorkspaceUnsupportedError):
await backend.create_binding(
ExecutionBindingCreateSpec(
tenant_id="tenant-1",
agent_id="agent-1",
binding_id="binding-1",
workspace_id="workspace-1",
existing_workspace_ref="workspace-1",
home_snapshot_ref=None,
)
)
assert requests == []
@pytest.mark.anyio
async def test_enterprise_binding_create_deletes_new_sandbox_when_layout_setup_fails(
monkeypatch: pytest.MonkeyPatch,
) -> None:
requests: list[httpx.Request] = []
def handler(request: httpx.Request) -> httpx.Response:
requests.append(request)
if request.url.path == "/v1/sandboxes":
return httpx.Response(201, json={"sandboxId": "sandbox-1"})
if request.url.path == "/proxy/v1/jobs/run":
return _job_response(exit_code=1)
if request.method == "DELETE":
return httpx.Response(204)
return httpx.Response(200, json={"job_id": "job-1"})
_ = _mock_http(monkeypatch, handler)
backend = EnterpriseExecutionBindingBackend(gateway_endpoint="http://gateway.example", auth_token="secret")
with pytest.raises(BindingCreateError):
await backend.create_binding(
ExecutionBindingCreateSpec(
tenant_id="tenant-1",
agent_id="agent-1",
binding_id="binding-1",
workspace_id="workspace-1",
existing_workspace_ref=None,
home_snapshot_ref=None,
)
)
assert any(request.method == "DELETE" and request.url.path == "/v1/sandboxes/sandbox-1" for request in requests)
@pytest.mark.anyio
async def test_enterprise_home_snapshots_remain_explicitly_not_implemented() -> None:
snapshots = EnterpriseHomeSnapshotBackend()
with pytest.raises(NotImplementedError, match="immutable Home Snapshot"):
_ = await snapshots.create_from_runtime(
spec=HomeSnapshotCreateSpec(tenant_id="tenant-1", agent_id="agent-1", home_snapshot_id="home-2"),
source=cast(RuntimeLease, object()),
)
with pytest.raises(NotImplementedError, match="immutable Home Snapshot"):
await snapshots.delete("snapshot-1")
dify-agent/tests/local/dify_agent/runtime_backend/test_local.py
+29 -39
View File
@@ -13,8 +13,6 @@ from dify_agent.runtime_backend import (
ExecutionBindingCreateSpec,
ExecutionBindingDestroySpec,
HomeSnapshotCreateSpec,
HomeSnapshotCreateError,
InitializeHomeSnapshotSpec,
)
from dify_agent.runtime_backend.local import LocalExecutionBindingBackend, LocalHomeSnapshotBackend
@@ -137,42 +135,6 @@ class _FailThenSucceedFactory:
return tuple(command for run in self.runs for command in run.commands)
@pytest.mark.anyio
async def test_local_snapshot_initialize_creates_private_snapshot_directory() -> None:
factory = _Factory()
snapshots = LocalHomeSnapshotBackend(
endpoint="http://shellctl",
auth_token="",
snapshot_root="/snapshots",
client_factory=factory, # pyright: ignore[reportArgumentType]
)
snapshot_ref = await snapshots.initialize(
InitializeHomeSnapshotSpec(tenant_id="tenant-1", agent_id="agent-1", home_snapshot_id="home-1")
)
assert snapshot_ref == "home-home-1"
assert ("mkdir", "-p", "/snapshots/home-home-1") in factory.commands
assert ("chmod", "700", "/snapshots/home-home-1") in factory.commands
@pytest.mark.anyio
async def test_local_snapshot_create_failure_removes_partial_snapshot() -> None:
factory = _FailThenSucceedFactory()
snapshots = LocalHomeSnapshotBackend(
endpoint="http://shellctl",
auth_token="",
snapshot_root="/snapshots",
client_factory=factory, # pyright: ignore[reportArgumentType]
)
with pytest.raises(HomeSnapshotCreateError, match="primary shellctl failure"):
await snapshots.initialize(
InitializeHomeSnapshotSpec(tenant_id="tenant-1", agent_id="agent-1", home_snapshot_id="home-1")
)
assert ("rm", "-rf", "--", "/snapshots/home-home-1") in factory.commands
@pytest.mark.anyio
async def test_local_binding_create_materializes_home_and_new_workspace() -> None:
factory = _Factory()
@@ -198,13 +160,41 @@ async def test_local_binding_create_materializes_home_and_new_workspace() -> Non
assert allocation.binding_ref == "binding-1:workspace-1"
assert allocation.workspace_ref == "workspace-1"
assert ("test", "-d", "/snapshots/home-home-1") in factory.commands
assert factory.commands[0] == ("test", "-d", "/snapshots/home-home-1")
assert ("mkdir", "-p", "/workspaces/workspace-1") in factory.commands
assert ("mkdir", "-p", "/homes/binding-1") in factory.commands
assert ("cp", "-a", "/snapshots/home-home-1/.", "/homes/binding-1/") in factory.commands
assert ("chmod", "700", "/homes/binding-1", "/workspaces/workspace-1") in factory.commands
@pytest.mark.anyio
async def test_local_binding_create_uses_empty_default_home_without_snapshot_access() -> None:
factory = _Factory()
backend = LocalExecutionBindingBackend(
endpoint="http://shellctl",
auth_token="",
materialized_home_root="/homes",
workspace_root="/workspaces",
snapshot_root="/snapshots",
client_factory=factory, # pyright: ignore[reportArgumentType]
)
allocation = await backend.create_binding(
ExecutionBindingCreateSpec(
tenant_id="tenant-1",
agent_id="agent-1",
binding_id="binding-1",
workspace_id="workspace-1",
existing_workspace_ref=None,
home_snapshot_ref=None,
)
)
assert allocation.binding_ref == "binding-1:workspace-1"
assert ("mkdir", "-p", "/homes/binding-1") in factory.commands
assert all("/snapshots" not in part for command in factory.commands for part in command)
@pytest.mark.anyio
async def test_local_binding_create_failure_removes_partial_home_and_workspace() -> None:
factory = _FailThenSucceedFactory()
dify-agent/tests/local/dify_agent/server/test_home_snapshots.py
+2 -12
View File
@@ -8,22 +8,16 @@ import pytest
from dify_agent.protocol import (
CreateHomeSnapshotFromBindingRequest,
DeleteHomeSnapshotRequest,
InitializeHomeSnapshotRequest,
)
from dify_agent.runtime_backend import HomeSnapshotCreateSpec, InitializeHomeSnapshotSpec, RuntimeLease
from dify_agent.runtime_backend import HomeSnapshotCreateSpec, RuntimeLease
from dify_agent.server.home_snapshots import HomeSnapshotService
@dataclass(slots=True)
class _HomeBackend:
initialized: list[InitializeHomeSnapshotSpec] = field(default_factory=list)
checkpointed: list[tuple[HomeSnapshotCreateSpec, RuntimeLease]] = field(default_factory=list)
deleted: list[str] = field(default_factory=list)
async def initialize(self, spec: InitializeHomeSnapshotSpec) -> str:
self.initialized.append(spec)
return "snapshot-initial"
async def create_from_runtime(self, *, spec: HomeSnapshotCreateSpec, source: RuntimeLease) -> str:
self.checkpointed.append((spec, source))
return "snapshot-build"
@@ -47,7 +41,7 @@ class _BindingBackend:
@pytest.mark.anyio
async def test_home_snapshot_service_initializes_and_checkpoints_exact_binding() -> None:
async def test_home_snapshot_service_checkpoints_exact_binding() -> None:
lease = cast(RuntimeLease, object())
homes = _HomeBackend()
bindings = _BindingBackend(lease=lease)
@@ -56,9 +50,6 @@ async def test_home_snapshot_service_initializes_and_checkpoints_exact_binding()
execution_bindings=bindings, # pyright: ignore[reportArgumentType]
)
initial = await service.initialize(
InitializeHomeSnapshotRequest(tenant_id="tenant-1", agent_id="agent-1", home_snapshot_id="home-1")
)
checkpoint = await service.create_from_binding(
CreateHomeSnapshotFromBindingRequest(
tenant_id="tenant-1",
@@ -68,7 +59,6 @@ async def test_home_snapshot_service_initializes_and_checkpoints_exact_binding()
)
)
assert initial.snapshot_ref == "snapshot-initial"
assert checkpoint.snapshot_ref == "snapshot-build"
assert bindings.acquired == ["binding-ref"]
assert bindings.released == [lease]
dify-agent/tests/local/dify_agent/server/test_settings.py
+1
View File
@@ -301,6 +301,7 @@ def test_build_runtime_backend_profile_passes_e2b_active_timeout() -> None:
assert profile is not None
assert isinstance(profile.execution_bindings, E2BExecutionBindingBackend)
assert profile.execution_bindings.active_timeout_seconds == 900
assert profile.execution_bindings.template == "difys-default-team/dify-agent-local-sandbox"
def test_sandbox_file_upload_limit_defaults_to_tool_file_limit() -> None: