opensource/dify
3
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-10-16 07:20:34 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into deploy/konwledge

Browse Source 
Port KnowledgeFS onto main's native Pydantic AI agent modules (Agenton
removed), reattach KnowledgeFS RBAC invalidation to the split RBAC
services, and adapt the new-rag settings form to the composable
IconPicker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Stephen Zhou
2026-10-10 22:54:29 +08:00
co-authored by Claude Opus 5.5
parent 2928fb2a90 2ac4c221d5
commit 709e879e9a
1390 changed files with 30453 additions and 41957 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/frontend-code-review/references/data-query-contracts.md
+1
View File
@@ -10,6 +10,7 @@ Review explicit team conventions as contracts, including direct generated option
- Distinguish a pass-through wrapper from a feature hook with actual orchestration. Check whether an independent execution condition or Promise composition justifies the documented query/mutation exception.
- Trace generated input and output types through their boundaries. Identify the exact DTO mirror, field widening, placeholder input, or lost intentional empty value when reporting a violation.
- Check whether a local mutation callback owns feature feedback or replaces shared invalidation, retry, or cache defaults. Match optimistic changes to the current list/detail owner.
- Check whether a new or changed toast reports something the user cannot already see. Flag a success toast for a result visible in place, a second error toast for a failure another layer already reports, and validation reported through a toast.
## Imperative Access And SSR
.agents/skills/how-to-write-component/references/data.md
+1
View File
@@ -35,6 +35,7 @@ Read this document when a component consumes generated contracts, nullable API v
- Do not introduce deprecated `useInvalid` or `useReset` APIs.
- Put shared invalidation, retries, and cache behavior in `createTanstackQueryUtils(...experimental_defaults...)`. In oRPC v1, caller options override defaults, including callbacks. Local callbacks may own toast, close, and navigation effects but must preserve shared cache behavior.
- Workflow-specific invalidation belongs beside the mutation/command that understands which caches become stale. Construct generated query options or keys from graph identity there instead of receiving keys from a rendering parent.
- A mutation callback is not a reason to show a toast. Show one only for a result the user cannot see where they acted; a control that displays its new value, or a dialog that closes onto the updated list, is already the feedback. Do not report a failure that another layer already reports.
- Prefer `mutate(...)`. Use `mutateAsync(...)` only when Promise composition is required, and catch awaited failures.
- Preserve intentional empty values and current list/detail ownership when updating data. Do not add optimistic updates without a verified owner contract.
.devcontainer/Dockerfile
+1 -1
View File
@@ -1,4 +1,4 @@
FROM mcr.microsoft.com/devcontainers/python:3.12-bookworm
FROM mcr.microsoft.com/devcontainers/python:3.13-bookworm
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install libgmp-dev libmpfr-dev libmpc-dev
.devcontainer/devcontainer.json
+1 -1
View File
@@ -1,7 +1,7 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/anaconda
{
"name": "Python 3.12",
"name": "Python 3.13",
"build": {
"context": "..",
"dockerfile": "Dockerfile"
.github/workflows/accessibility-e2e.yml
+3 -3
View File
@@ -57,7 +57,7 @@ jobs:
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
python-version: '3.12'
python-version: '3.13'
cache-dependency-glob: |
api/uv.lock
@@ -99,7 +99,7 @@ jobs:
- name: Upload accessibility Cucumber report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: cucumber-report-accessibility-${{ inputs.level }}-${{ inputs.page }}
path: e2e/cucumber-report
@@ -107,7 +107,7 @@ jobs:
- name: Upload accessibility E2E logs
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-logs-accessibility-${{ inputs.level }}-${{ inputs.page }}
path: e2e/.logs/*.log
.github/workflows/api-tests.yml
+47 -108
View File
@@ -25,7 +25,7 @@ jobs:
- name: Setup UV and Python
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
python-version: '3.12'
python-version: '3.13'
enable-cache: false
- name: Download shared timing baseline
# Public statistics are independent of PR refs and require no credentials.
@@ -49,7 +49,7 @@ jobs:
api/providers/vdb/*/tests/unit_tests \
api/providers/trace/*/tests/unit_tests
- name: Upload shared plan
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: api-unit-plan
path: |
@@ -72,7 +72,7 @@ jobs:
fail-fast: false
matrix:
python-version:
- '3.12'
- '3.13'
shardIndex:
- 1
- 2
@@ -99,7 +99,7 @@ jobs:
run: uv sync --project api --dev
- name: Download shared plan
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: api-unit-plan
@@ -128,7 +128,7 @@ jobs:
- name: Upload unit test reports
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: api-test-results-unit-${{ matrix.python-version }}-${{ matrix.shardIndex }}
path: test-results/*.xml
@@ -136,27 +136,34 @@ jobs:
retention-days: 7
- name: Upload successful timing observations
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: api-unit-durations-${{ matrix.shardIndex }}
path: test-results/durations-*.json
retention-days: 7
if-no-files-found: error
- name: Upload unit coverage data
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
- name: Generate coverage report
run: uv run --project api coverage xml -o coverage.xml
# Codecov combines all backend uploads under the same commit and api flag.
- name: Report coverage
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
with:
name: api-coverage-unit-${{ matrix.python-version }}-${{ matrix.shardIndex }}
path: ${{ env.COVERAGE_FILE }}
if-no-files-found: error
retention-days: 1
files: ./coverage.xml
disable_search: true
fail_ci_if_error: true
flags: api
name: api-unit-${{ matrix.python-version }}-${{ matrix.shardIndex }}-${{ matrix.shardTotal }}
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
api-controllers:
name: API Controller Tests (3.12)
name: API Controller Tests (3.13)
runs-on: depot-ubuntu-24.04
env:
PYTHONUNBUFFERED: '1'
COVERAGE_FILE: coverage-controllers-3.12
COVERAGE_FILE: coverage-controllers-3.13
steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -167,7 +174,7 @@ jobs:
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
python-version: '3.12'
python-version: '3.13'
cache-dependency-glob: api/uv.lock
- name: Check UV lockfile
@@ -177,10 +184,12 @@ jobs:
run: uv sync --project api --dev
- name: Run Controller Tests
# Controller tests register Flask routes at import time, so keep them out of xdist.
# Workers have separate Flask registries; keep collection separate from the other unit suites.
run: |
uv run --project api pytest \
-p no:benchmark \
--timeout "${PYTEST_TIMEOUT:-20}" \
-n auto \
-o 'addopts=-p no:opik -p no:langsmith_plugin --cov=./api --cov-branch --import-mode=importlib' \
--cov-report= \
--junitxml=test-results/controllers.xml \
@@ -188,20 +197,27 @@ jobs:
- name: Upload controller test reports
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: api-test-results-controllers-3.12
name: api-test-results-controllers-3.13
path: test-results/*.xml
if-no-files-found: warn
retention-days: 7
- name: Upload controller coverage data
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
- name: Generate coverage report
run: uv run --project api coverage xml -o coverage.xml
# Codecov combines all backend uploads under the same commit and api flag.
- name: Report coverage
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
with:
name: api-coverage-controllers-3.12
path: ${{ env.COVERAGE_FILE }}
if-no-files-found: error
retention-days: 1
files: ./coverage.xml
disable_search: true
fail_ci_if_error: true
flags: api
name: api-controllers-3.13
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
api-integration:
name: API Integration Tests (${{ matrix.python-version }})
@@ -218,7 +234,7 @@ jobs:
fail-fast: false
matrix:
python-version:
- '3.12'
- '3.13'
steps:
- name: Checkout code
@@ -253,94 +269,17 @@ jobs:
api/tests/integration_tests/tools \
api/tests/test_containers_integration_tests
- name: Upload integration coverage data
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: api-coverage-integration-${{ matrix.python-version }}
path: ${{ env.COVERAGE_FILE }}
if-no-files-found: error
retention-days: 1
api-coverage:
name: API Coverage
runs-on: depot-ubuntu-24.04
needs:
- api-unit
- api-controllers
- api-integration
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
COVERAGE_FILE: .coverage
defaults:
run:
shell: bash
steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Setup UV and Python
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
python-version: '3.12'
- name: Download coverage data
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: coverage-data
pattern: api-coverage-*
merge-multiple: true
- name: Combine coverage
run: |
set -euo pipefail
coverage_version=$(python3 -c 'import tomllib; print(next(p["version"] for p in tomllib.load(open("api/uv.lock", "rb"))["package"] if p["name"] == "coverage"))')
mapfile -t unit_coverage < <(find coverage-data -type f -name 'coverage-unit-*' -print | sort)
mapfile -t controller_coverage < <(find coverage-data -type f -name 'coverage-controllers-*' -print | sort)
mapfile -t integration_coverage < <(find coverage-data -type f -name 'coverage-integration-*' -print | sort)
if [[ "${#unit_coverage[@]}" -ne 2 ]]; then
echo "expected 2 unit coverage artifacts, found ${#unit_coverage[@]}" >&2
exit 1
fi
if [[ "${#controller_coverage[@]}" -ne 1 ]]; then
echo "expected 1 controller coverage artifact, found ${#controller_coverage[@]}" >&2
exit 1
fi
if [[ "${#integration_coverage[@]}" -ne 1 ]]; then
echo "expected 1 integration coverage artifact, found ${#integration_coverage[@]}" >&2
exit 1
fi
uvx --from "coverage==$coverage_version" coverage combine "${unit_coverage[@]}" "${controller_coverage[@]}" "${integration_coverage[@]}"
uvx --from "coverage==$coverage_version" coverage xml -o coverage.xml
# Reuse the XML totals instead of analyzing every source file again for a text report.
python3 - <<'PY'
import os
import xml.etree.ElementTree as ET
coverage = ET.parse("coverage.xml").getroot()
with open(os.environ["GITHUB_STEP_SUMMARY"], "a") as summary:
summary.write("### API Coverage\n\n")
summary.write("Merged backend coverage report generated for Codecov project status.\n\n")
summary.write("| Metric | Covered | Total | Coverage |\n")
summary.write("| --- | ---: | ---: | ---: |\n")
for label, metric in (("Lines", "lines"), ("Branches", "branches")):
covered = int(coverage.attrib[f"{metric}-covered"])
total = int(coverage.attrib[f"{metric}-valid"])
percentage = f"{covered / total:.2%}" if total else "N/A"
summary.write(f"| {label} | {covered} | {total} | {percentage} |\n")
summary.write("\nSee Codecov for per-file coverage details.\n")
PY
- name: Generate coverage report
run: uv run --project api coverage xml -o coverage.xml
# Codecov combines all backend uploads under the same commit and api flag.
- name: Report coverage
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
with:
files: ./coverage.xml
disable_search: true
fail_ci_if_error: true
flags: api
name: api-integration-${{ matrix.python-version }}
env:
CODECOV_TOKEN: ${{ env.CODECOV_TOKEN }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
.github/workflows/autofix.yml
+1 -1
View File
@@ -61,7 +61,7 @@ jobs:
- if: github.event_name != 'merge_group' && (steps.docker-compose-changes.outputs.any_changed == 'true' || steps.api-changes.outputs.any_changed == 'true' || steps.frontend-contract-changes.outputs.any_changed == 'true' || steps.dify-agent-changes.outputs.any_changed == 'true')
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.11'
python-version: '3.13'
- if: github.event_name != 'merge_group' && (steps.api-changes.outputs.any_changed == 'true' || steps.frontend-contract-changes.outputs.any_changed == 'true' || steps.dify-agent-changes.outputs.any_changed == 'true')
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
.github/workflows/build-push.yml
+3 -3
View File
@@ -133,7 +133,7 @@ jobs:
touch "/tmp/digests/${sanitized_digest}"
- name: Upload digest
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: digests-${{ matrix.artifact_context }}-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
@@ -192,7 +192,7 @@ jobs:
context: 'local-sandbox'
steps:
- name: Download digests
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
path: /tmp/digests
pattern: digests-${{ matrix.context }}-*
@@ -246,7 +246,7 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Use Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version-file: package.json
cache: ''
.github/workflows/cli-e2e.yml
+2 -2
View File
@@ -266,7 +266,7 @@ jobs:
- name: Upload results on failure
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-run-${{ matrix.name }}-${{ github.run_id }}
path: cli/test-results/
@@ -396,7 +396,7 @@ jobs:
- name: Upload results on failure
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-last-${{ github.run_id }}
path: cli/test-results/
.github/workflows/cli-tests.yml
+6 -5
View File
@@ -21,8 +21,6 @@ jobs:
fail-fast: false
matrix:
os: [depot-ubuntu-24.04, windows-latest, macos-latest]
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
defaults:
run:
shell: bash
@@ -50,10 +48,13 @@ jobs:
run: pnpm run ci
- name: Report coverage
if: ${{ env.CODECOV_TOKEN != '' && matrix.os == 'depot-ubuntu-24.04' }}
if: matrix.os == 'depot-ubuntu-24.04'
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
with:
directory: cli/coverage
files: cli/coverage/coverage-final.json
disable_search: true
fail_ci_if_error: true
flags: cli
name: cli-linux
env:
CODECOV_TOKEN: ${{ env.CODECOV_TOKEN }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
.github/workflows/db-migration-test.yml
+2 -2
View File
@@ -22,7 +22,7 @@ jobs:
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
python-version: '3.12'
python-version: '3.13'
cache-dependency-glob: api/uv.lock
- name: Install dependencies
@@ -72,7 +72,7 @@ jobs:
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
python-version: '3.12'
python-version: '3.13'
cache-dependency-glob: api/uv.lock
- name: Install dependencies
.github/workflows/dify-agent-tests.yml
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
strategy:
matrix:
python-version:
- '3.12'
- '3.13'
steps:
- name: Checkout code
.github/workflows/pyrefly-diff.yml
+1 -1
View File
@@ -66,7 +66,7 @@ jobs:
echo ${{ github.event.pull_request.number }} > pr_number.txt
- name: Upload pyrefly diff
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: pyrefly_diff
path: |
.github/workflows/pyrefly-type-coverage.yml
+1 -1
View File
@@ -74,7 +74,7 @@ jobs:
echo ${{ github.event.pull_request.number }} > pr_number.txt
- name: Upload type coverage artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: pyrefly_type_coverage
path: |
.github/workflows/style.yml
+2 -2
View File
@@ -64,7 +64,7 @@ jobs:
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: false
python-version: '3.12'
python-version: '3.13'
cache-dependency-glob: |
api/uv.lock
dify-agent/uv.lock
@@ -177,7 +177,7 @@ jobs:
name: Web Style
if: ${{ always() && !inputs.skip-checks && (!inputs.classification-succeeded || inputs.run-web-style) }}
needs: [web-build, web-style-checks]
runs-on: ubuntu-24.04
runs-on: depot-ubuntu-24.04
permissions: {}
steps:
- name: Verify web checks
.github/workflows/translate-i18n-claude.yml
+4 -8
View File
@@ -8,19 +8,15 @@ on:
workflow_dispatch:
inputs:
files:
description: 'Specific files to translate (space-separated, e.g., "app common").
Required for full mode; leave empty in incremental mode to use en-US
files changed since HEAD~1.'
description: 'Specific files to translate (space-separated, e.g., "app common"). Required for full mode; leave empty in incremental mode to use en-US files changed since HEAD~1.'
required: false
type: string
languages:
description: 'Specific languages to translate (space-separated, e.g., "zh-Hans
ja-JP"). Leave empty for all supported target languages except en-US.'
description: 'Specific languages to translate (space-separated, e.g., "zh-Hans ja-JP"). Leave empty for all supported target languages except en-US.'
required: false
type: string
mode:
description: 'Sync mode: incremental (compare with previous en-US revision) or
full (sync all keys in scope)'
description: 'Sync mode: incremental (compare with previous en-US revision) or full (sync all keys in scope)'
required: false
default: incremental
type: choice
@@ -163,7 +159,7 @@ jobs:
- name: Run Claude Code for Translation Sync
if: steps.context.outputs.CHANGED_FILES != ''
uses: anthropics/claude-code-action@ed670b4cf9de2a5a570d130d2f6197b9e543cd64 # v1.0.240
uses: anthropics/claude-code-action@2dca132ff0e0c4094ce6048b422c6915a071210b # v1.0.247
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }}
.github/workflows/vdb-tests-full.yml
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
strategy:
matrix:
python-version:
- '3.12'
- '3.13'
steps:
- name: Checkout code
.github/workflows/vdb-tests.yml
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
strategy:
matrix:
python-version:
- '3.12'
- '3.13'
steps:
- name: Checkout code
.github/workflows/web-bundle-size.yml
+5 -5
View File
@@ -15,7 +15,7 @@ permissions:
jobs:
measure:
name: Measure ${{ matrix.revision }}
runs-on: ubuntu-24.04
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 30
permissions:
contents: read
@@ -67,7 +67,7 @@ jobs:
run: node "$RUNNER_TEMP/bundle-analysis.ts" collect web "$COMMIT_SHA" "$RUNNER_TEMP/bundle-size/summary.json"
- name: Upload size snapshot
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: web-bundle-summary-${{ matrix.revision }}
path: ${{ runner.temp }}/bundle-size/summary.json
@@ -75,7 +75,7 @@ jobs:
retention-days: 14
- name: Upload official Rolldown analysis
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: web-bundle-analyzer-${{ matrix.revision }}
path: |
@@ -89,7 +89,7 @@ jobs:
report:
name: Compare bundle sizes
needs: measure
runs-on: ubuntu-24.04
runs-on: depot-ubuntu-24.04
timeout-minutes: 5
permissions:
contents: read
@@ -106,7 +106,7 @@ jobs:
install: false
- name: Download size snapshots
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
pattern: web-bundle-summary-*
path: bundle-results
.github/workflows/web-e2e.yml
+14 -18
View File
@@ -71,14 +71,10 @@ jobs:
- name: Package Web build
run: >-
tar -cf e2e-web-build.tar -C web
.next/BUILD_ID
.next/e2e-web-build.sha256
.next/standalone
.next/static
tar -cf e2e-web-build.tar -C web .next/BUILD_ID .next/e2e-web-build.sha256 .next/standalone .next/static
- name: Upload Web build
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: core-e2e-web-build-${{ github.run_attempt }}
path: e2e-web-build.tar
@@ -95,7 +91,7 @@ jobs:
chromium-full:
name: Chromium Full (${{ matrix.shard }}/3)
if: ${{ !inputs.run-external-runtime }}
runs-on: ubuntu-24.04
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 45
strategy:
fail-fast: false
@@ -122,7 +118,7 @@ jobs:
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
python-version: '3.12'
python-version: '3.13'
cache-dependency-glob: api/uv.lock
# API and web installs are independent, so overlap them; the browser
@@ -154,7 +150,7 @@ jobs:
- name: Upload Cucumber report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: cucumber-report-chromium-${{ matrix.shard }}
path: e2e/cucumber-report
@@ -164,7 +160,7 @@ jobs:
- name: Upload E2E logs
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-logs-chromium-${{ matrix.shard }}
path: e2e/.logs/*.log
@@ -176,7 +172,7 @@ jobs:
webkit-smoke:
name: WebKit Browser Smoke
if: ${{ !inputs.run-external-runtime }}
runs-on: ubuntu-24.04
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 45
env:
E2E_ADMIN_EMAIL: e2e-admin@example.com
@@ -200,7 +196,7 @@ jobs:
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
python-version: '3.12'
python-version: '3.13'
cache-dependency-glob: api/uv.lock
# API and web installs are independent, so overlap them; the browser
@@ -232,7 +228,7 @@ jobs:
- name: Upload Cucumber report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: cucumber-report-webkit
path: e2e/cucumber-report
@@ -242,7 +238,7 @@ jobs:
- name: Upload E2E logs
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-logs-webkit
path: e2e/.logs/*.log
@@ -272,7 +268,7 @@ jobs:
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
python-version: '3.12'
python-version: '3.13'
cache-dependency-glob: |
api/uv.lock
dify-agent/uv.lock
@@ -327,7 +323,7 @@ jobs:
- name: Upload Cucumber report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: cucumber-report
path: e2e/cucumber-report
@@ -336,7 +332,7 @@ jobs:
- name: Upload E2E logs
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-logs
path: e2e/.logs/*.log
@@ -346,7 +342,7 @@ jobs:
- name: Upload E2E seed report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-seed-report
path: e2e/seed-report
.github/workflows/web-tests.yml
+28 -59
View File
@@ -33,20 +33,23 @@ jobs:
uses: ./.github/actions/setup-web
- name: Run tests
run: vp test run --project unit --reporter=blob --reporter=minimal --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --coverage
run: vp test run --project unit --reporter=minimal --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --coverage
- name: Upload blob report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
# Codecov combines the shard uploads under the same commit and web flag.
- name: Report coverage
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
with:
name: blob-report-${{ matrix.shardIndex }}
path: web/.vitest/blob/*
include-hidden-files: true
retention-days: 1
files: web/coverage/coverage-final.json
disable_search: true
fail_ci_if_error: true
flags: web
name: web-${{ matrix.shardIndex }}-${{ matrix.shardTotal }}
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
browser-test:
name: Web Browser Tests
runs-on: ubuntu-24.04
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 20
defaults:
run:
@@ -71,7 +74,7 @@ jobs:
- name: Upload browser failure artifacts
if: ${{ failure() && !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: browser-test-failures
path: web/.vitest-browser/
@@ -79,51 +82,10 @@ jobs:
include-hidden-files: true
retention-days: 7
merge-reports:
name: Merge Test Reports
if: ${{ !cancelled() }}
needs: [test]
runs-on: depot-ubuntu-24.04-4
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
defaults:
run:
shell: bash
working-directory: ./web
steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Setup web environment
uses: ./.github/actions/setup-web
- name: Download blob reports
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: web/.vitest/blob
pattern: blob-report-*
merge-multiple: true
- name: Merge reports
run: vp test --merge-reports --coverage --silent=passed-only
- name: Report coverage
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
with:
directory: web/coverage
flags: web
env:
CODECOV_TOKEN: ${{ env.CODECOV_TOKEN }}
dify-ui-test:
name: dify-ui Tests
runs-on: ubuntu-24.04
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 20
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
defaults:
run:
shell: bash
@@ -147,10 +109,12 @@ jobs:
- name: Upload dify-ui test failure artifacts
if: ${{ failure() && !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: dify-ui-test-failures
path: packages/dify-ui/.vitest-browser/
path: |
packages/dify-ui/.vitest/attachments/
packages/dify-ui/.vitest-browser/
if-no-files-found: ignore
include-hidden-files: true
retention-days: 7
@@ -158,14 +122,17 @@ jobs:
- name: Report coverage
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
with:
directory: packages/dify-ui/coverage
files: packages/dify-ui/coverage/coverage-final.json
disable_search: true
fail_ci_if_error: true
flags: dify-ui
name: dify-ui
env:
CODECOV_TOKEN: ${{ env.CODECOV_TOKEN }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
dify-ui-storybook-test:
name: dify-ui Storybook Tests
runs-on: ubuntu-24.04
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 20
defaults:
run:
@@ -190,10 +157,12 @@ jobs:
- name: Upload dify-ui Storybook test failure artifacts
if: ${{ failure() && !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: dify-ui-storybook-test-failures
path: packages/dify-ui/.vitest-browser/
path: |
packages/dify-ui/.vitest/attachments/
packages/dify-ui/.vitest-browser/
if-no-files-found: ignore
include-hidden-files: true
retention-days: 7
Makefile
+2 -2
View File
@@ -113,7 +113,7 @@ test:
api/providers/vdb/*/tests/unit_tests \
api/providers/trace/*/tests/unit_tests \
--ignore=api/tests/unit_tests/controllers; \
uv run --project api --dev pytest --timeout "$${PYTEST_TIMEOUT:-20}" --cov-append \
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto --cov-append \
api/tests/unit_tests/controllers; \
fi
@echo "✅ Unit tests complete"
@@ -131,7 +131,7 @@ test-all:
api/providers/vdb/*/tests/unit_tests \
api/providers/trace/*/tests/unit_tests \
--ignore=api/tests/unit_tests/controllers; \
uv run --project api --dev pytest --timeout "$${PYTEST_TIMEOUT:-20}" --cov-append \
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto --cov-append \
api/tests/unit_tests/controllers; \
echo "Running backend integration tests"; \
uv run --project api --dev pytest -p no:benchmark --start-middleware -n auto \
README.md
+1 -7
View File
@@ -161,13 +161,7 @@ Choose the channel that best fits your question:
## Star History
<a href="https://star-history.dera.page/#langgenius/dify&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://star-history.dera.page/svg?repos=langgenius/dify&type=date&theme=dark&legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://star-history.dera.page/svg?repos=langgenius/dify&type=date&legend=top-left" />
<img alt="Star History Chart" src="https://star-history.dera.page/svg?repos=langgenius/dify&type=date&legend=top-left" />
</picture>
</a>
[![Star History Chart](https://star-history.dera.page/svg?repos=langgenius/dify&type=date&legend=top-left)](https://star-history.dera.page/langgenius/dify?type=date&legend=top-left#gh-light-mode-only) [![Star History Chart](https://star-history.dera.page/svg?repos=langgenius/dify&type=date&theme=dark&legend=top-left)](https://star-history.dera.page/langgenius/dify?type=date&legend=top-left#gh-dark-mode-only)
## Security disclosure
api/Dockerfile
+1 -1
View File
@@ -2,7 +2,7 @@
# engine bundles and native dependencies at /app/knowledge-fs in the dedicated Celery workers.
# Ordinary Dify API images do not rebuild or download a second copy of the private engine.
# base image
FROM python:3.12-slim-bookworm AS base
FROM python:3.13-slim-bookworm AS base
WORKDIR /app/api
api/clients/agent_backend/__init__.py
-2
View File
@@ -17,7 +17,6 @@ from clients.agent_backend.errors import (
)
from clients.agent_backend.event_adapter import (
AgentBackendAgentMessageDeltaInternalEvent,
AgentBackendDeferredToolCallInternalEvent,
AgentBackendInternalEvent,
AgentBackendInternalEventType,
AgentBackendRunCancelledInternalEvent,
@@ -57,7 +56,6 @@ __all__ = [
"WORKFLOW_USER_PROMPT_LAYER_ID",
"AgentBackendAgentAppRunInput",
"AgentBackendAgentMessageDeltaInternalEvent",
"AgentBackendDeferredToolCallInternalEvent",
"AgentBackendError",
"AgentBackendHTTPError",
"AgentBackendInternalEvent",
api/clients/agent_backend/event_adapter.py
+5 -48
View File
@@ -2,9 +2,7 @@
The adapter does not define a new cross-service event contract. It consumes
``dify_agent.protocol.RunEvent`` and produces small API-internal models that the
workflow Agent Node maps to Graphon/AppQueue events. Deferred external tool calls
remain Dify Agent ``run_succeeded`` payloads on the wire; API code turns them
into an internal event so workflow pause/session handling stays local to API.
workflow Agent Node maps to Graphon/AppQueue events.
Agent-message deltas are exposed as annotations on ``PydanticAIStreamRunEvent``
so API code does not have to parse Pydantic AI stream-event internals to
preserve streaming. The terminal answer remains the ``run_succeeded`` output.
@@ -15,9 +13,7 @@ from __future__ import annotations
from enum import StrEnum
from typing import Annotated, Literal, cast
from agenton.compositor import CompositorSessionSnapshot
from dify_agent.protocol import (
DeferredToolCallPayload,
PydanticAIStreamRunEvent,
RunCancelledEvent,
RunEvent,
@@ -26,6 +22,7 @@ from dify_agent.protocol import (
RunStartedEvent,
RunSucceededEvent,
)
from dify_agent.protocol.snapshot import SessionSnapshot
from pydantic import BaseModel, ConfigDict, Field, JsonValue, TypeAdapter
_EVENT_DATA_ADAPTER = TypeAdapter(object)
@@ -37,7 +34,6 @@ class AgentBackendInternalEventType(StrEnum):
RUN_STARTED = "run_started"
STREAM_EVENT = "stream_event"
AGENT_MESSAGE_DELTA = "agent_message_delta"
DEFERRED_TOOL_CALL = "deferred_tool_call"
RUN_SUCCEEDED = "run_succeeded"
RUN_FAILED = "run_failed"
RUN_CANCELLED = "run_cancelled"
@@ -78,17 +74,7 @@ class AgentBackendRunSucceededInternalEvent(AgentBackendInternalEventBase):
type: Literal[AgentBackendInternalEventType.RUN_SUCCEEDED] = AgentBackendInternalEventType.RUN_SUCCEEDED
output: JsonValue
session_snapshot: CompositorSessionSnapshot
usage: dict[str, JsonValue] | None = None
class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
"""API-internal representation of a Dify Agent deferred external tool call."""
type: Literal[AgentBackendInternalEventType.DEFERRED_TOOL_CALL] = AgentBackendInternalEventType.DEFERRED_TOOL_CALL
deferred_tool_call: DeferredToolCallPayload
message: str | None = None
session_snapshot: CompositorSessionSnapshot
session_snapshot: SessionSnapshot
usage: dict[str, JsonValue] | None = None
@@ -99,7 +85,7 @@ class AgentBackendRunFailedInternalEvent(AgentBackendInternalEventBase):
error: str
error_type: RunFailureType | None = None
reason: str | None = None
session_snapshot: CompositorSessionSnapshot | None = None
session_snapshot: SessionSnapshot | None = None
usage: dict[str, JsonValue] | None = None
@@ -109,7 +95,7 @@ class AgentBackendRunCancelledInternalEvent(AgentBackendInternalEventBase):
type: Literal[AgentBackendInternalEventType.RUN_CANCELLED] = AgentBackendInternalEventType.RUN_CANCELLED
reason: str | None = None
message: str | None = None
session_snapshot: CompositorSessionSnapshot | None = None
session_snapshot: SessionSnapshot | None = None
usage: dict[str, JsonValue] | None = None
@@ -117,7 +103,6 @@ type AgentBackendInternalEvent = Annotated[
AgentBackendRunStartedInternalEvent
| AgentBackendStreamInternalEvent
| AgentBackendAgentMessageDeltaInternalEvent
| AgentBackendDeferredToolCallInternalEvent
| AgentBackendRunSucceededInternalEvent
| AgentBackendRunFailedInternalEvent
| AgentBackendRunCancelledInternalEvent,
@@ -158,19 +143,6 @@ class AgentBackendRunEventAdapter:
)
]
case RunSucceededEvent():
if "deferred_tool_call" in event.data.model_fields_set:
if event.data.deferred_tool_call is None:
raise TypeError("run_succeeded deferred_tool_call branch is missing payload")
return [
AgentBackendDeferredToolCallInternalEvent(
run_id=event.run_id,
source_event_id=event.id,
deferred_tool_call=event.data.deferred_tool_call,
message=_deferred_tool_call_message(event.data.deferred_tool_call),
session_snapshot=event.data.session_snapshot,
usage=_agent_run_usage(event.data.usage),
)
]
return [
AgentBackendRunSucceededInternalEvent(
run_id=event.run_id,
@@ -206,21 +178,6 @@ class AgentBackendRunEventAdapter:
raise TypeError(f"unsupported agent backend run event: {type(event).__name__}")
def _deferred_tool_call_message(payload: DeferredToolCallPayload) -> str:
"""Return a concise workflow pause message from deferred-tool arguments."""
args = payload.args
if isinstance(args, dict):
question = args.get("question")
if isinstance(question, str) and question.strip():
return question
title = args.get("title")
if isinstance(title, str) and title.strip():
return title
return f"Agent backend requested external input via deferred tool '{payload.tool_name}'."
def _agent_run_usage(usage: object | None) -> dict[str, JsonValue] | None:
"""Return JSON-safe usage metadata from optional Agent backend usage."""
if usage is None:
api/clients/agent_backend/fake_client.py
+4 -31
View File
@@ -11,13 +11,11 @@ from collections.abc import Callable, Iterator
from datetime import UTC, datetime
from enum import StrEnum
from agenton.compositor import CompositorSessionSnapshot
from dify_agent.protocol import (
CancelRunRequest,
CancelRunResponse,
CreateRunRequest,
CreateRunResponse,
DeferredToolCallPayload,
RunCancelledEvent,
RunCancelledEventData,
RunEvent,
@@ -28,6 +26,7 @@ from dify_agent.protocol import (
RunSucceededEvent,
RunSucceededEventData,
)
from dify_agent.protocol.snapshot import SessionSnapshot
_FIXED_TIME = datetime(2026, 1, 1, tzinfo=UTC)
@@ -41,7 +40,6 @@ class FakeAgentBackendScenario(StrEnum):
SUCCESS = "success"
FAILED = "failed"
PAUSED = "paused"
class FakeAgentBackendRunClient:
@@ -89,7 +87,7 @@ class FakeAgentBackendRunClient:
data=RunCancelledEventData(
reason=request.reason,
message=request.message,
session_snapshot=CompositorSessionSnapshot(layers=[]),
session_snapshot=SessionSnapshot(),
),
)
@@ -127,13 +125,6 @@ class FakeAgentBackendRunClient:
updated_at=_FIXED_TIME,
error="fake failure",
)
case FakeAgentBackendScenario.PAUSED:
return RunStatusResponse(
run_id=run_id,
status="succeeded",
created_at=_FIXED_TIME,
updated_at=_FIXED_TIME,
)
def _events(self, run_id: str) -> tuple[RunEvent, ...]:
match self.scenario:
@@ -146,7 +137,7 @@ class FakeAgentBackendRunClient:
created_at=_FIXED_TIME,
data=RunSucceededEventData(
output={"text": "hello agent"},
session_snapshot=CompositorSessionSnapshot(layers=[]),
session_snapshot=SessionSnapshot(),
),
),
)
@@ -160,25 +151,7 @@ class FakeAgentBackendRunClient:
data=RunFailedEventData(
error="fake failure",
reason="unit_test",
session_snapshot=CompositorSessionSnapshot(layers=[]),
),
),
)
case FakeAgentBackendScenario.PAUSED:
return (
RunStartedEvent(id="1-0", run_id=run_id, created_at=_FIXED_TIME),
RunSucceededEvent(
id="2-0",
run_id=run_id,
created_at=_FIXED_TIME,
data=RunSucceededEventData(
deferred_tool_call=DeferredToolCallPayload(
tool_call_id="fake-ask-human-1",
tool_name="ask_human",
args={"question": "Agent requested human input."},
metadata={"layer_type": "dify.ask_human", "schema_version": 1},
),
session_snapshot=CompositorSessionSnapshot(layers=[]),
session_snapshot=SessionSnapshot(),
),
),
)
api/clients/agent_backend/request_builder.py
+81 -201
View File
@@ -3,10 +3,9 @@
This module is intentionally an adapter, not a wire DTO package. The emitted
object is always ``dify_agent.protocol.CreateRunRequest`` so the Agent backend
protocol has a single owner. API-only context such as Agent Soul vs workflow job
prompt is preserved in layer names and metadata until the dedicated product
schemas land in later phases. Dify-owned execution identifiers are emitted as an
explicit ``dify.execution_context`` layer so the run request stays fully
composition-driven.
prompt is preserved in registered module names. Execution identifiers live in
execution_context Config; model, tool and resource references are explicit Config
fields. The wire request contains current Config and optional state-only snapshot.
"""
from __future__ import annotations
@@ -15,30 +14,22 @@ import re
from collections.abc import Mapping
from typing import ClassVar, Literal
from agenton.compositor import CompositorSessionSnapshot
from agenton.layers import ExitIntent
from agenton_collections.layers.plain import PLAIN_PROMPT_LAYER_TYPE_ID, PromptLayerConfig
from agenton_collections.layers.pydantic_ai import PYDANTIC_AI_HISTORY_LAYER_TYPE_ID
from dify_agent.layers.ask_human import DIFY_ASK_HUMAN_LAYER_TYPE_ID, DifyAskHumanLayerConfig
from dify_agent.layers.config import DIFY_CONFIG_LAYER_TYPE_ID, DifyConfigLayerConfig
from dify_agent.layers.dify_core_tools import DIFY_CORE_TOOLS_LAYER_TYPE_ID, DifyCoreToolsLayerConfig
from dify_agent.layers.config import DifyConfigLayerConfig
from dify_agent.layers.dify_core_tools import DifyCoreToolsLayerConfig
from dify_agent.layers.dify_plugin import (
DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
DifyPluginLLMLayerConfig,
DifyPluginToolsLayerConfig,
)
from dify_agent.layers.execution_context import (
DIFY_EXECUTION_CONTEXT_LAYER_TYPE_ID,
DifyExecutionContextLayerConfig,
)
from dify_agent.layers.knowledge import DIFY_KNOWLEDGE_BASE_LAYER_TYPE_ID, DifyKnowledgeBaseLayerConfig
from dify_agent.layers.knowledge_fs import DIFY_KNOWLEDGE_FS_LAYER_TYPE_ID, DifyKnowledgeFsLayerConfig
from dify_agent.layers.output import DIFY_OUTPUT_LAYER_TYPE_ID, DifyOutputLayerConfig
from dify_agent.layers.runtime import DIFY_RUNTIME_LAYER_TYPE_ID, DifyRuntimeLayerConfig
from dify_agent.layers.shell import DIFY_SHELL_LAYER_TYPE_ID, DifyShellLayerConfig
from dify_agent.layers.knowledge import DifyKnowledgeBaseLayerConfig
from dify_agent.layers.knowledge_fs import DifyKnowledgeFsLayerConfig
from dify_agent.layers.output import DifyOutputLayerConfig
from dify_agent.layers.prompt import Config as PromptConfig
from dify_agent.layers.runtime import DifyRuntimeLayerConfig
from dify_agent.layers.shell import DifyShellLayerConfig
from dify_agent.layers.user_prompt import (
DIFY_USER_PROMPT_LAYER_TYPE_ID,
DifyUserPromptFileConfig,
DifyUserPromptLayerConfig,
)
@@ -47,11 +38,10 @@ from dify_agent.protocol import (
DIFY_AGENT_MODEL_LAYER_ID,
DIFY_AGENT_OUTPUT_LAYER_ID,
CreateRunRequest,
DeferredToolResultsPayload,
LayerExitSignals,
RunComposition,
RunLayerSpec,
)
from dify_agent.protocol.snapshot import SessionSnapshot
from pydantic import BaseModel, ConfigDict, Field, JsonValue, field_validator
AGENT_SOUL_PROMPT_LAYER_ID = "agent_soul_prompt"
@@ -64,30 +54,17 @@ DIFY_CONFIG_LAYER_ID = "config"
DIFY_PLUGIN_TOOLS_LAYER_ID = "tools"
DIFY_CORE_TOOLS_LAYER_ID = "core_tools"
DIFY_KNOWLEDGE_BASE_LAYER_ID = "knowledge"
DIFY_ASK_HUMAN_LAYER_ID = "ask_human"
DIFY_KNOWLEDGE_FS_LAYER_ID = "knowledge_fs"
DIFY_SHELL_LAYER_ID = "shell"
type AgentConfigVersionKind = Literal["snapshot", "draft", "build_draft"]
def _shell_layer_deps() -> dict[str, str]:
return {
"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID,
"runtime": DIFY_RUNTIME_LAYER_ID,
}
def _config_layer_deps() -> dict[str, str]:
return {"shell": DIFY_SHELL_LAYER_ID}
def _knowledge_fs_spec(run_input: AgentBackendWorkflowNodeRunInput | AgentBackendAgentAppRunInput) -> RunLayerSpec:
if not run_input.include_shell or run_input.knowledge is not None:
if not run_input.include_shell or run_input.knowledge is not None or run_input.knowledge_fs is None:
raise ValueError("KnowledgeFS requires shell and cannot be combined with legacy knowledge retrieval")
return RunLayerSpec(
name="knowledge_fs",
type=DIFY_KNOWLEDGE_FS_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID, "shell": DIFY_SHELL_LAYER_ID},
config=run_input.knowledge_fs,
name=DIFY_KNOWLEDGE_FS_LAYER_ID,
config=run_input.knowledge_fs.model_dump(mode="json"),
)
@@ -226,18 +203,9 @@ class AgentBackendWorkflowNodeRunInput(BaseModel):
knowledge: DifyKnowledgeBaseLayerConfig | None = None
knowledge_fs: DifyKnowledgeFsLayerConfig | None = None
config_layer_config: DifyConfigLayerConfig | None = None
# Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
# the Agent Soul configures human involvement; a deferred call ends the run and
# the workflow pauses via the existing HITL form mechanism (ENG-635).
ask_human_config: DifyAskHumanLayerConfig | None = None
# Inject the sandboxed shell graph. Requires a deployment-selected runtime
# backend plus the product-resolved persistent Binding.
include_shell: bool = False
shell_config: DifyShellLayerConfig | None = None
session_snapshot: CompositorSessionSnapshot | None = None
# Human tool results fed back into a continuation run after a HITL submission
# (ENG-638). Keyed by the original deferred tool_call_id.
deferred_tool_results: DeferredToolResultsPayload | None = None
session_snapshot: SessionSnapshot | None = None
include_history: bool = True
metadata: dict[str, JsonValue] = Field(default_factory=dict)
@@ -274,17 +242,9 @@ class AgentBackendAgentAppRunInput(BaseModel):
knowledge: DifyKnowledgeBaseLayerConfig | None = None
knowledge_fs: DifyKnowledgeFsLayerConfig | None = None
config_layer_config: DifyConfigLayerConfig | None = None
# Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
# the Agent Soul configures human involvement (ENG-635).
ask_human_config: DifyAskHumanLayerConfig | None = None
# Inject the sandboxed shell graph. Requires a deployment-selected runtime
# backend plus the product-resolved persistent Binding.
include_shell: bool = False
shell_config: DifyShellLayerConfig | None = None
session_snapshot: CompositorSessionSnapshot | None = None
# Human tool results fed back into a continuation run after a HITL submission
# (ENG-638). Keyed by the original deferred tool_call_id.
deferred_tool_results: DeferredToolResultsPayload | None = None
session_snapshot: SessionSnapshot | None = None
include_history: bool = True
metadata: dict[str, JsonValue] = Field(default_factory=dict)
@@ -307,7 +267,7 @@ class AgentBackendRunRequestBuilder:
Layer graph: optional Agent Soul system prompt → user prompt →
execution context → optional shell / config / history
(multi-turn) → LLM → optional plugin-direct tools / core-routed tools /
knowledge search / ask_human / structured output. Mirrors the
knowledge search / structured output. Mirrors the
workflow-node layer ordering minus the workflow-job / previous-node
prompt.
"""
@@ -320,9 +280,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=AGENT_SOUL_PROMPT_LAYER_ID,
type=PLAIN_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "agent_soul"},
config=PromptLayerConfig(prefix=agent_soul_prompt),
config=(PromptConfig(prefix=agent_soul_prompt)).model_dump(mode="json"),
)
)
@@ -330,15 +288,13 @@ class AgentBackendRunRequestBuilder:
[
RunLayerSpec(
name=AGENT_APP_USER_PROMPT_LAYER_ID,
type=DIFY_USER_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "agent_app_user_prompt"},
config=DifyUserPromptLayerConfig(text=run_input.user_prompt, files=run_input.user_files),
config=(
DifyUserPromptLayerConfig(text=run_input.user_prompt, files=run_input.user_files)
).model_dump(mode="json"),
),
RunLayerSpec(
name=DIFY_EXECUTION_CONTEXT_LAYER_ID,
type=DIFY_EXECUTION_CONTEXT_LAYER_TYPE_ID,
metadata=run_input.metadata,
config=run_input.execution_context,
config=(run_input.execution_context).model_dump(mode="json"),
),
]
)
@@ -348,9 +304,9 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_RUNTIME_LAYER_ID,
type=DIFY_RUNTIME_LAYER_TYPE_ID,
metadata=run_input.metadata,
config=DifyRuntimeLayerConfig(backend_binding_ref=run_input.backend_binding_ref),
config=(DifyRuntimeLayerConfig(backend_binding_ref=run_input.backend_binding_ref)).model_dump(
mode="json"
),
)
)
# Sandboxed bash workspace (dify.shell). It enters before config so
@@ -358,10 +314,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(),
metadata=run_input.metadata,
config=run_input.shell_config or DifyShellLayerConfig(),
config=(run_input.shell_config or DifyShellLayerConfig()).model_dump(mode="json"),
)
)
@@ -369,10 +322,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_CONFIG_LAYER_ID,
type=DIFY_CONFIG_LAYER_TYPE_ID,
deps=_config_layer_deps(),
metadata=run_input.metadata,
config=run_input.config_layer_config,
config=(run_input.config_layer_config).model_dump(mode="json"),
)
)
@@ -380,38 +330,31 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_AGENT_HISTORY_LAYER_ID,
type=PYDANTIC_AI_HISTORY_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "agent_session_history"},
)
)
layers.append(
RunLayerSpec(
name=DIFY_AGENT_MODEL_LAYER_ID,
type=DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=DifyPluginLLMLayerConfig(
plugin_id=run_input.model.plugin_id,
model_provider=run_input.model.model_provider,
model=run_input.model.model,
model_settings=_agent_model_settings(run_input.model.model_settings),
context_window_tokens=run_input.model.context_window_tokens,
),
config=(
DifyPluginLLMLayerConfig(
plugin_id=run_input.model.plugin_id,
model_provider=run_input.model.model_provider,
model=run_input.model.model,
model_settings=_agent_model_settings(run_input.model.model_settings),
context_window_tokens=run_input.model.context_window_tokens,
)
).model_dump(mode="json"),
)
)
if run_input.tools is not None and run_input.tools.tools:
plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
if include_shell:
plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
layers.append(
RunLayerSpec(
name=DIFY_PLUGIN_TOOLS_LAYER_ID,
type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
deps=plugin_tool_deps,
metadata=run_input.metadata,
config=run_input.tools,
config=(
run_input.tools.model_copy(update={"shell": DIFY_SHELL_LAYER_ID if include_shell else None})
).model_dump(mode="json"),
)
)
@@ -419,10 +362,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_CORE_TOOLS_LAYER_ID,
type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.core_tools,
config=(run_input.core_tools).model_dump(mode="json"),
)
)
@@ -432,23 +372,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
type=DIFY_KNOWLEDGE_BASE_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.knowledge,
)
)
if run_input.ask_human_config is not None:
# Human-in-the-loop ask_human deferred tool (dify.ask_human). A call ends
# the run with a deferred_tool_call; the caller pauses (workflow HITL) and
# later resumes with deferred_tool_results. Needs the history layer above.
layers.append(
RunLayerSpec(
name=DIFY_ASK_HUMAN_LAYER_ID,
type=DIFY_ASK_HUMAN_LAYER_TYPE_ID,
metadata=run_input.metadata,
config=run_input.ask_human_config,
config=(run_input.knowledge).model_dump(mode="json"),
)
)
@@ -456,13 +380,13 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_AGENT_OUTPUT_LAYER_ID,
type=DIFY_OUTPUT_LAYER_TYPE_ID,
metadata=run_input.metadata,
config=DifyOutputLayerConfig(
json_schema=run_input.output.json_schema,
description=run_input.output.description,
strict=run_input.output.strict,
),
config=(
DifyOutputLayerConfig(
json_schema=run_input.output.json_schema,
description=run_input.output.description,
strict=run_input.output.strict,
)
).model_dump(mode="json"),
)
)
@@ -471,8 +395,6 @@ class AgentBackendRunRequestBuilder:
idempotency_key=run_input.idempotency_key,
metadata=run_input.metadata,
session_snapshot=run_input.session_snapshot,
deferred_tool_results=run_input.deferred_tool_results,
on_exit=LayerExitSignals(default=ExitIntent.SUSPEND),
)
def build_for_workflow_node(self, run_input: AgentBackendWorkflowNodeRunInput) -> CreateRunRequest:
@@ -481,7 +403,7 @@ class AgentBackendRunRequestBuilder:
Layer graph mirrors the workflow surface: prompts → execution context →
optional shell / config / history → LLM → optional
plugin-direct tools / core-routed tools / knowledge search /
ask_human / structured output.
structured output.
"""
layers: list[RunLayerSpec] = []
agent_soul_prompt = _agent_soul_prompt_for_layer(
@@ -492,9 +414,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=AGENT_SOUL_PROMPT_LAYER_ID,
type=PLAIN_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "agent_soul"},
config=PromptLayerConfig(prefix=agent_soul_prompt),
config=(PromptConfig(prefix=agent_soul_prompt)).model_dump(mode="json"),
)
)
@@ -502,21 +422,15 @@ class AgentBackendRunRequestBuilder:
[
RunLayerSpec(
name=WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
type=PLAIN_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "workflow_node_job"},
config=PromptLayerConfig(user=run_input.workflow_node_job_prompt),
config=(PromptConfig(user=run_input.workflow_node_job_prompt)).model_dump(mode="json"),
),
RunLayerSpec(
name=WORKFLOW_USER_PROMPT_LAYER_ID,
type=PLAIN_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "workflow_user_prompt"},
config=PromptLayerConfig(user=run_input.user_prompt),
config=(PromptConfig(user=run_input.user_prompt)).model_dump(mode="json"),
),
RunLayerSpec(
name=DIFY_EXECUTION_CONTEXT_LAYER_ID,
type=DIFY_EXECUTION_CONTEXT_LAYER_TYPE_ID,
metadata=run_input.metadata,
config=run_input.execution_context,
config=(run_input.execution_context).model_dump(mode="json"),
),
]
)
@@ -526,9 +440,9 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_RUNTIME_LAYER_ID,
type=DIFY_RUNTIME_LAYER_TYPE_ID,
metadata=run_input.metadata,
config=DifyRuntimeLayerConfig(backend_binding_ref=run_input.backend_binding_ref),
config=(DifyRuntimeLayerConfig(backend_binding_ref=run_input.backend_binding_ref)).model_dump(
mode="json"
),
)
)
# Sandboxed bash workspace (dify.shell). It enters before config so
@@ -536,10 +450,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(),
metadata=run_input.metadata,
config=run_input.shell_config or DifyShellLayerConfig(),
config=(run_input.shell_config or DifyShellLayerConfig()).model_dump(mode="json"),
)
)
@@ -547,10 +458,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_CONFIG_LAYER_ID,
type=DIFY_CONFIG_LAYER_TYPE_ID,
deps=_config_layer_deps(),
metadata=run_input.metadata,
config=run_input.config_layer_config,
config=(run_input.config_layer_config).model_dump(mode="json"),
)
)
@@ -558,8 +466,6 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_AGENT_HISTORY_LAYER_ID,
type=PYDANTIC_AI_HISTORY_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "agent_session_history"},
)
)
@@ -567,31 +473,26 @@ class AgentBackendRunRequestBuilder:
[
RunLayerSpec(
name=DIFY_AGENT_MODEL_LAYER_ID,
type=DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=DifyPluginLLMLayerConfig(
plugin_id=run_input.model.plugin_id,
model_provider=run_input.model.model_provider,
model=run_input.model.model,
model_settings=_agent_model_settings(run_input.model.model_settings),
context_window_tokens=run_input.model.context_window_tokens,
),
config=(
DifyPluginLLMLayerConfig(
plugin_id=run_input.model.plugin_id,
model_provider=run_input.model.model_provider,
model=run_input.model.model,
model_settings=_agent_model_settings(run_input.model.model_settings),
context_window_tokens=run_input.model.context_window_tokens,
)
).model_dump(mode="json"),
),
]
)
if run_input.tools is not None and run_input.tools.tools:
plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
if include_shell:
plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
layers.append(
RunLayerSpec(
name=DIFY_PLUGIN_TOOLS_LAYER_ID,
type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
deps=plugin_tool_deps,
metadata=run_input.metadata,
config=run_input.tools,
config=(
run_input.tools.model_copy(update={"shell": DIFY_SHELL_LAYER_ID if include_shell else None})
).model_dump(mode="json"),
)
)
@@ -599,10 +500,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_CORE_TOOLS_LAYER_ID,
type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.core_tools,
config=(run_input.core_tools).model_dump(mode="json"),
)
)
@@ -612,23 +510,7 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
type=DIFY_KNOWLEDGE_BASE_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.knowledge,
)
)
if run_input.ask_human_config is not None:
# Human-in-the-loop ask_human deferred tool (dify.ask_human). A call ends
# the run with a deferred_tool_call; the caller pauses (workflow HITL) and
# later resumes with deferred_tool_results. Needs the history layer above.
layers.append(
RunLayerSpec(
name=DIFY_ASK_HUMAN_LAYER_ID,
type=DIFY_ASK_HUMAN_LAYER_TYPE_ID,
metadata=run_input.metadata,
config=run_input.ask_human_config,
config=(run_input.knowledge).model_dump(mode="json"),
)
)
@@ -636,13 +518,13 @@ class AgentBackendRunRequestBuilder:
layers.append(
RunLayerSpec(
name=DIFY_AGENT_OUTPUT_LAYER_ID,
type=DIFY_OUTPUT_LAYER_TYPE_ID,
metadata=run_input.metadata,
config=DifyOutputLayerConfig(
json_schema=run_input.output.json_schema,
description=run_input.output.description,
strict=run_input.output.strict,
),
config=(
DifyOutputLayerConfig(
json_schema=run_input.output.json_schema,
description=run_input.output.description,
strict=run_input.output.strict,
)
).model_dump(mode="json"),
)
)
@@ -651,8 +533,6 @@ class AgentBackendRunRequestBuilder:
idempotency_key=run_input.idempotency_key,
metadata=run_input.metadata,
session_snapshot=run_input.session_snapshot,
deferred_tool_results=run_input.deferred_tool_results,
on_exit=LayerExitSignals(default=ExitIntent.SUSPEND),
)
api/commands/__init__.py
+2 -1
View File
@@ -3,11 +3,12 @@ CLI command modules extracted from `commands.py`.
"""
from .account import create_tenant, reset_email, reset_password
from .data_migrate import data_migrate, legacy_model_types
from .data_migration import (
data_migrate,
export_migration_data,
export_migration_data_template,
import_migration_data,
legacy_model_types,
migration_data_wizard,
)
from .knowledge_fs import knowledge_fs_control_space
api/commands/data_migrate.py
-186
View File
@@ -1,186 +0,0 @@
import io
import os
import sys
from contextlib import AbstractContextManager, nullcontext
from pathlib import Path
from typing import cast
import click
from commands.rbac import (
migrate_dataset_permissions_to_rbac,
migrate_only_me_resource_whitelist_scopes_to_automatic_include,
migrate_resource_whitelist_scopes_to_automatic_include, # noqa: F401
)
from extensions.ext_database import db
from graphon.model_runtime.entities.model_entities import ModelType
from services.legacy_model_type_migration import (
VALID_TABLE_NAMES,
LegacyModelTypeMigrationService,
load_tenant_ids_from_file,
)
_SUPPORTED_MODEL_TYPE_CHOICES = (
ModelType.LLM.value,
ModelType.TEXT_EMBEDDING.value,
ModelType.RERANK.value,
)
_DEFAULT_CONCURRENCY = os.cpu_count() or 1
def _normalize_multi_value_option(
values: tuple[str, ...],
*,
valid_values: tuple[str, ...],
option_name: str,
) -> tuple[str, ...]:
normalized_values: list[str] = []
seen_values: set[str] = set()
for value in values:
for item in value.split(","):
normalized_item = item.strip()
if not normalized_item:
continue
if normalized_item not in valid_values:
raise click.BadParameter(
f"invalid value '{normalized_item}'. valid values: {', '.join(valid_values)}",
param_hint=option_name,
)
if normalized_item in seen_values:
continue
seen_values.add(normalized_item)
normalized_values.append(normalized_item)
return tuple(normalized_values)
@click.group(
"data-migrate",
help="Online data migration commands.",
)
def data_migrate() -> None:
"""Namespace for production data migration commands."""
@click.command(
"legacy-model-types",
help=(
"Migrate legacy provider model_type values to canonical values. "
"Default is dry-run and emits JSON lines only. "
"If --tables includes provider_model_credentials, the command may also update "
"provider_models and load_balancing_model_configs references so merged credentials stay reachable."
),
)
@click.option(
"--apply",
is_flag=True,
default=False,
help="Apply the migration. Default is dry-run.",
)
@click.option(
"--tables",
"tables",
multiple=True,
type=str,
help=(
"Limit migration to specific tables. Accepts comma-separated values or repeated flags.\n"
"\n"
"Options: load_balancing_model_configs, provider_model_credentials, "
"provider_model_settings, provider_models, tenant_default_models.\n\n"
"When provider_model_credentials is selected, provider_models and "
"load_balancing_model_configs may also be updated for credential reference rewrites.\n"
"\n"
"If unspecified, all relevant tables are migrated."
),
)
@click.option(
"--model-types",
"model_types",
multiple=True,
type=str,
help=(
"Canonical model types to migrate. Accepts comma-separated values or repeated flags.\n"
"\n"
"Options: llm,text-embedding,rerank\n"
"\n"
"If unspecified, all relevant legacy model types are migrated."
),
)
@click.option(
"--tenant-id-file",
type=click.Path(exists=True, dir_okay=False, readable=True, resolve_path=True),
help="Optional file containing tenant ids, one per line.",
)
@click.option(
"--output",
type=click.Path(dir_okay=False, resolve_path=True, path_type=Path),
help=(
"Optional file path for JSON lines event logs. Defaults to stdout.\n"
"It's highly recommended to save the event logs to a file and preserve it for a period of time."
),
)
@click.option(
"--concurrency",
type=click.IntRange(min=1),
default=_DEFAULT_CONCURRENCY,
show_default=True,
help="Number of tenant-level worker threads to run in parallel.",
)
def legacy_model_types(
apply: bool,
tables: tuple[str, ...],
model_types: tuple[str, ...],
tenant_id_file: str | None,
output: Path | None,
concurrency: int = _DEFAULT_CONCURRENCY,
) -> None:
"""
Migrate legacy provider-related model_type values and emit JSON lines events.
"""
normalized_tables = _normalize_multi_value_option(
tables,
valid_values=VALID_TABLE_NAMES,
option_name="--tables",
)
normalized_model_types = _normalize_multi_value_option(
model_types,
valid_values=_SUPPORTED_MODEL_TYPE_CHOICES,
option_name="--model-types",
)
selected_model_types = (
tuple(ModelType(model_type) for model_type in normalized_model_types)
if normalized_model_types
else (
ModelType.LLM,
ModelType.TEXT_EMBEDDING,
ModelType.RERANK,
)
)
tenant_ids = load_tenant_ids_from_file(tenant_id_file) if tenant_id_file else None
output_context: AbstractContextManager[io.TextIOBase]
if output is None:
output_context = nullcontext(cast(io.TextIOBase, sys.stdout))
else:
try:
output_context = output.open("w", encoding="utf-8")
except OSError as exc:
raise click.ClickException(f"failed to open output file '{output}': {exc.strerror or exc}") from exc
with output_context as output_stream:
LegacyModelTypeMigrationService(
engine=db.engine,
apply=apply,
concurrency=concurrency,
output=cast(io.TextIOBase, output_stream),
tables=normalized_tables or None,
model_types=selected_model_types,
tenant_ids=tenant_ids,
).migrate()
data_migrate.add_command(legacy_model_types)
data_migrate.add_command(migrate_dataset_permissions_to_rbac)
data_migrate.add_command(migrate_only_me_resource_whitelist_scopes_to_automatic_include)
api/commands/data_migration.py
+181
View File
@@ -1,6 +1,12 @@
"""Online data migration and application import/export CLI commands."""
from __future__ import annotations
import io
import json
import os
import sys
from contextlib import AbstractContextManager, nullcontext
from datetime import datetime
from pathlib import Path
from typing import Any, cast
@@ -11,8 +17,14 @@ import sqlalchemy as sa
import yaml
from sqlalchemy.orm import Session
from commands.rbac import (
migrate_dataset_permissions_to_rbac,
migrate_only_me_resource_whitelist_scopes_to_automatic_include,
migrate_resource_whitelist_scopes_to_automatic_include, # noqa: F401
)
from core.db.session_factory import session_factory
from extensions.ext_database import db
from graphon.model_runtime.entities.model_entities import ModelType
from models import Tenant
from models.model import App
from models.tools import ApiToolProvider, MCPToolProvider, WorkflowToolProvider
@@ -29,7 +41,18 @@ from services.data_migration.export_service import ExportConfigParser, Migration
from services.data_migration.import_service import ImportRequest, MigrationImportService
from services.data_migration.package_service import MigrationPackageService
from services.data_migration.report_service import MigrationReportService
from services.legacy_model_type_migration import (
VALID_TABLE_NAMES,
LegacyModelTypeMigrationService,
load_tenant_ids_from_file,
)
_SUPPORTED_MODEL_TYPE_CHOICES = (
ModelType.LLM.value,
ModelType.TEXT_EMBEDDING.value,
ModelType.RERANK.value,
)
_DEFAULT_CONCURRENCY = os.cpu_count() or 1
ID_STRATEGY_CHOICES = ["preserve-id", "generate-new-id"]
CONFLICT_STRATEGY_CHOICES = ["fail", "skip", "update"]
SUPPORTED_WIZARD_APP_MODES = ["workflow", "advanced-chat"]
@@ -37,6 +60,164 @@ WizardToolMap = dict[str, dict[str, str | None]]
WizardToolSelection = dict[str, list[str]]
def _normalize_multi_value_option(
values: tuple[str, ...],
*,
valid_values: tuple[str, ...],
option_name: str,
) -> tuple[str, ...]:
normalized_values: list[str] = []
seen_values: set[str] = set()
for value in values:
for item in value.split(","):
normalized_item = item.strip()
if not normalized_item:
continue
if normalized_item not in valid_values:
raise click.BadParameter(
f"invalid value '{normalized_item}'. valid values: {', '.join(valid_values)}",
param_hint=option_name,
)
if normalized_item in seen_values:
continue
seen_values.add(normalized_item)
normalized_values.append(normalized_item)
return tuple(normalized_values)
@click.group(
"data-migrate",
help="Online data migration commands.",
)
def data_migrate() -> None:
"""Namespace for production data migration commands."""
@click.command(
"legacy-model-types",
help=(
"Migrate legacy provider model_type values to canonical values. "
"Default is dry-run and emits JSON lines only. "
"If --tables includes provider_model_credentials, the command may also update "
"provider_models and load_balancing_model_configs references so merged credentials stay reachable."
),
)
@click.option(
"--apply",
is_flag=True,
default=False,
help="Apply the migration. Default is dry-run.",
)
@click.option(
"--tables",
"tables",
multiple=True,
type=str,
help=(
"Limit migration to specific tables. Accepts comma-separated values or repeated flags.\n"
"\n"
"Options: load_balancing_model_configs, provider_model_credentials, "
"provider_model_settings, provider_models, tenant_default_models.\n\n"
"When provider_model_credentials is selected, provider_models and "
"load_balancing_model_configs may also be updated for credential reference rewrites.\n"
"\n"
"If unspecified, all relevant tables are migrated."
),
)
@click.option(
"--model-types",
"model_types",
multiple=True,
type=str,
help=(
"Canonical model types to migrate. Accepts comma-separated values or repeated flags.\n"
"\n"
"Options: llm,text-embedding,rerank\n"
"\n"
"If unspecified, all relevant legacy model types are migrated."
),
)
@click.option(
"--tenant-id-file",
type=click.Path(exists=True, dir_okay=False, readable=True, resolve_path=True),
help="Optional file containing tenant ids, one per line.",
)
@click.option(
"--output",
type=click.Path(dir_okay=False, resolve_path=True, path_type=Path),
help=(
"Optional file path for JSON lines event logs. Defaults to stdout.\n"
"It's highly recommended to save the event logs to a file and preserve it for a period of time."
),
)
@click.option(
"--concurrency",
type=click.IntRange(min=1),
default=_DEFAULT_CONCURRENCY,
show_default=True,
help="Number of tenant-level worker threads to run in parallel.",
)
def legacy_model_types(
apply: bool,
tables: tuple[str, ...],
model_types: tuple[str, ...],
tenant_id_file: str | None,
output: Path | None,
concurrency: int = _DEFAULT_CONCURRENCY,
) -> None:
"""
Migrate legacy provider-related model_type values and emit JSON lines events.
"""
normalized_tables = _normalize_multi_value_option(
tables,
valid_values=VALID_TABLE_NAMES,
option_name="--tables",
)
normalized_model_types = _normalize_multi_value_option(
model_types,
valid_values=_SUPPORTED_MODEL_TYPE_CHOICES,
option_name="--model-types",
)
selected_model_types = (
tuple(ModelType(model_type) for model_type in normalized_model_types)
if normalized_model_types
else (
ModelType.LLM,
ModelType.TEXT_EMBEDDING,
ModelType.RERANK,
)
)
tenant_ids = load_tenant_ids_from_file(tenant_id_file) if tenant_id_file else None
output_context: AbstractContextManager[io.TextIOBase]
if output is None:
output_context = nullcontext(cast(io.TextIOBase, sys.stdout))
else:
try:
output_context = output.open("w", encoding="utf-8")
except OSError as exc:
raise click.ClickException(f"failed to open output file '{output}': {exc.strerror or exc}") from exc
with output_context as output_stream:
LegacyModelTypeMigrationService(
engine=db.engine,
apply=apply,
concurrency=concurrency,
output=cast(io.TextIOBase, output_stream),
tables=normalized_tables or None,
model_types=selected_model_types,
tenant_ids=tenant_ids,
).migrate()
data_migrate.add_command(legacy_model_types)
data_migrate.add_command(migrate_dataset_permissions_to_rbac)
data_migrate.add_command(migrate_only_me_resource_whitelist_scopes_to_automatic_include)
def _scripted_export_template() -> dict[str, Any]:
return {
"source_tenant": {
api/commands/rbac.py
+8 -34
View File
@@ -24,11 +24,11 @@ from models import (
TenantAccountJoin,
TenantAccountRole,
)
from services.enterprise.rbac_service import (
LegacyAgentRoleMigration,
from services.enterprise.rbac_service import RBACService
from services.rbac.contracts import (
AgentRoleMigration,
ListOption,
RBACResourceType,
RBACService,
ReplaceMemberBindings,
ReplaceUserAccessPolicies,
)
@@ -148,7 +148,7 @@ class _AgentAccessBootstrapReason(StrEnum):
def _agent_manage_role_event(
tenant_id: str | None,
entry: LegacyAgentRoleMigration,
entry: AgentRoleMigration,
*,
apply: bool,
kind: _AgentMigrationEventKind,
@@ -239,35 +239,16 @@ def _replace_member_role(
operator_account_id: str,
member_account_id: str,
role_id: str,
*,
session: Session,
) -> str:
RBACService.MemberRoles.replace(
tenant_id=tenant_id,
account_id=operator_account_id,
member_account_id=member_account_id,
role_ids=[role_id],
session=session,
)
return member_account_id
def _replace_member_role_with_new_session(
tenant_id: str,
operator_account_id: str,
member_account_id: str,
role_id: str,
) -> str:
with session_factory.create_session() as session:
return _replace_member_role(
tenant_id=tenant_id,
operator_account_id=operator_account_id,
member_account_id=member_account_id,
role_id=role_id,
session=session,
)
@click.command(
"rbac-migrate-member-roles", help="Migrate legacy workspace member roles into RBAC member-role bindings."
)
@@ -342,21 +323,14 @@ def migrate_member_roles_to_rbac(
if replace_jobs:
if workers == 1:
with session_factory.create_session() as session:
for member_account_id, resolved_role_id in replace_jobs:
_replace_member_role(
workspace_id,
owner_account_id,
member_account_id,
resolved_role_id,
session=session,
)
migrated_count += 1
for member_account_id, resolved_role_id in replace_jobs:
_replace_member_role(workspace_id, owner_account_id, member_account_id, resolved_role_id)
migrated_count += 1
else:
with ThreadPoolExecutor(max_workers=workers) as executor:
futures = [
executor.submit(
_replace_member_role_with_new_session,
_replace_member_role,
workspace_id,
owner_account_id,
member_account_id,
api/constants/oauth_bearer.py
+2
View File
@@ -28,6 +28,7 @@ class SubjectType(StrEnum):
# not is keyed by its external issuer instead.
bound_to_account: bool
RESOURCE_ACCESS = ("resource_access", frozenset({Scope.WORKSPACE_READ, Scope.APPS_READ, Scope.APPS_RUN}), False)
ACCOUNT = ("account", frozenset({Scope.FULL}), True)
EXTERNAL_SSO = ("external_sso", frozenset({Scope.APPS_RUN, Scope.APPS_READ_PERMITTED_EXTERNAL}), False)
@@ -47,6 +48,7 @@ class TokenType(StrEnum):
prefix: str
subject: SubjectType
RESOURCE_ACCESS = ("resource_access", "sk-", SubjectType.RESOURCE_ACCESS)
OAUTH_ACCOUNT = ("oauth_account", "dfoa_", SubjectType.ACCOUNT)
OAUTH_EXTERNAL_SSO = ("oauth_external_sso", "dfoe_", SubjectType.EXTERNAL_SSO)
api/constants/resource_access_token.py
+10
View File
@@ -0,0 +1,10 @@
"""Resource access token vocabulary shared by persistence and application services."""
from enum import StrEnum
TOKEN_PREFIX = "sk-"
class ResourceAccessTokenResourceType(StrEnum):
APP = "app"
KNOWLEDGE = "knowledge"
api/context/execution_context.py
+2 -2
View File
@@ -94,7 +94,7 @@ class ExecutionContext:
return self._user
@contextmanager
def enter(self) -> Generator[None, None, None]:
def enter(self) -> Generator[None]:
"""Enter this execution context."""
if self._context_vars:
for var, val in self._context_vars.items():
@@ -145,7 +145,7 @@ class NullAppContext(AppContext):
@contextmanager
@override
def enter(self) -> Generator[None, None, None]:
def enter(self) -> Generator[None]:
"""Enter null context (no-op)."""
yield
api/context/flask_app_context.py
+2 -2
View File
@@ -42,7 +42,7 @@ class FlaskAppContext(AppContext):
@contextmanager
@override
def enter(self) -> Generator[None, None, None]:
def enter(self) -> Generator[None]:
"""Enter Flask app context."""
with self._flask_app.app_context():
yield
@@ -159,7 +159,7 @@ class FlaskExecutionContext:
cm.__exit__(*args)
@contextmanager
def enter(self) -> Generator[None, None, None]:
def enter(self) -> Generator[None]:
"""Enter Flask execution context as context manager."""
# Restore non-Flask context variables to avoid leaking Flask tokens across threads
for var, val in self._context_vars.items():
api/controllers/common/agent_access.py
+2 -8
View File
@@ -2,17 +2,13 @@ from __future__ import annotations
from collections.abc import Sequence
from dataclasses import dataclass
from typing import TYPE_CHECKING
from sqlalchemy.orm import Session
from services.agent.roster_service import AgentRosterService
from services.enterprise import rbac_service as enterprise_rbac_service
if TYPE_CHECKING:
from services.app_service import AppListBaseParams
from services.enterprise.rbac_service import MyPermissionsResponse
from services.entities.app_entities import AppListBaseParams
from services.rbac.contracts import MyPermissionsResponse
AGENT_LIST_PERMISSION_KEYS: frozenset[str] = frozenset({"agent.acl.preview"})
@@ -54,7 +50,6 @@ def resolve_agent_access_filter(
tenant_id: str,
account_id: str,
*,
session: Session,
permissions: MyPermissionsResponse | None = None,
) -> AgentAccessFilter:
"""Compute Agent resources visible to an account in a workspace."""
@@ -62,7 +57,6 @@ def resolve_agent_access_filter(
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
tenant_id,
account_id,
session=session,
)
whitelist_scope = enterprise_rbac_service.RBACService.AgentAccess.whitelist_resources(tenant_id, account_id)
has_default_preview = has_agent_list_permission(
api/controllers/common/rbac/__init__.py
+9
View File
@@ -1,3 +1,5 @@
from flask import request
from core.rbac import RBACPermission, RBACResourceScope
from .checks import RBAC_CHECKS_ATTR, RBACCheck, enforce_rbac_checks
@@ -28,4 +30,11 @@ __all__ = [
"ResourceLocator",
"Workspace",
"enforce_rbac_checks",
"rbac_language",
]
def rbac_language() -> str | None:
"""Read and normalize the optional RBAC label language from the query string."""
value = (request.args.get("language") or "").strip().lower()
return value if value in {"en", "ja", "zh"} else None
api/controllers/common/rbac/checks.py
+6 -2
View File
@@ -10,11 +10,15 @@ from services.enterprise.rbac_service import RBACService
from .locators import ResourceLocator, Workspace
__all__ = ["RBAC_CHECKS_ATTR", "RBACCheck", "enforce_rbac_checks"]
__all__ = ["RBAC_CHECKS_ATTR", "InvalidRBACCheckError", "RBACCheck", "enforce_rbac_checks"]
RBAC_CHECKS_ATTR = "rbac_checks"
class InvalidRBACCheckError(RuntimeError):
"""A route declares a permission scene with an incompatible resource locator."""
@dataclass(frozen=True)
class RBACCheck:
scene: RBACPermission
@@ -24,7 +28,7 @@ class RBACCheck:
if isinstance(self.locator, Workspace) or self.scene.scope is RBACResourceScope.WORKSPACE:
return
if self.locator.scope is not self.scene.scope:
raise ValueError(
raise InvalidRBACCheckError(
f"{self.scene} is a {self.scene.scope} scene but {self.locator!r} locates {self.locator.scope}"
)
api/controllers/common/rbac/locators.py
+13 -17
View File
@@ -1,16 +1,14 @@
from collections.abc import Mapping
from dataclasses import dataclass
from typing import TYPE_CHECKING, Protocol, override, runtime_checkable
from typing import Protocol, override, runtime_checkable
from flask import g
from werkzeug.exceptions import NotFound
from controllers.common.errors import InvalidArgumentError
from core.rbac import RBACResourceScope
from extensions.ext_database import db
from services.rbac_resource_service import RBACResourceService
if TYPE_CHECKING:
from models.agent import Agent
from extensions.ext_application_services import application_services
from services.entities.app_entities import AppAgentBinding
__all__ = [
"AgentBehindApp",
@@ -46,15 +44,15 @@ class ResourceLocator(Protocol):
def _required(path_args: Mapping[str, object], param: str) -> str:
value = path_args.get(param)
if not value:
raise ValueError(f"Missing {param} in request path")
raise InvalidArgumentError(description=f"Missing {param} in request path")
return str(value)
def agent_binding(tenant_id: str, app_id: str) -> "Agent | None":
cache: dict[tuple[str, str], Agent | None] = g.setdefault(_AGENT_BINDING_CACHE_KEY, {})
def agent_binding(tenant_id: str, app_id: str) -> AppAgentBinding | None:
cache: dict[tuple[str, str], AppAgentBinding | None] = g.setdefault(_AGENT_BINDING_CACHE_KEY, {})
key = (tenant_id, app_id)
if key not in cache:
cache[key] = RBACResourceService.get_app_agent_binding(db.session, tenant_id, app_id)
cache[key] = application_services().rbac.queries.get_app_agent_binding(tenant_id, app_id)
return cache[key]
@@ -104,7 +102,7 @@ class PlainApp(_ParamLocator):
return ResourceIdentity(self.scope, app_id)
def owner_id(self, tenant_id: str, identity: ResourceIdentity) -> str | None:
return RBACResourceService.get_app_maintainer(db.session, tenant_id, identity.id)
return application_services().rbac.queries.get_app_maintainer(tenant_id, identity.id)
class AgentBehindApp(_ParamLocator):
@@ -112,10 +110,8 @@ class AgentBehindApp(_ParamLocator):
default_param = "app_id"
def locate(self, tenant_id: str, path_args: Mapping[str, object]) -> ResourceIdentity | None:
from models.agent import AgentScope
binding = agent_binding(tenant_id, _required(path_args, self.param))
if binding is None or binding.scope == AgentScope.WORKFLOW_ONLY:
if binding is None or binding.workflow_only:
return None
return ResourceIdentity(self.scope, binding.id)
@@ -131,7 +127,7 @@ class DatasetId(_ParamLocator):
return ResourceIdentity(self.scope, _required(path_args, self.param))
def owner_id(self, tenant_id: str, identity: ResourceIdentity) -> str | None:
return RBACResourceService.get_dataset_maintainer(db.session, tenant_id, identity.id)
return application_services().rbac.queries.get_dataset_maintainer(tenant_id, identity.id)
class DatasetByDocument(DatasetId):
@@ -140,7 +136,7 @@ class DatasetByDocument(DatasetId):
@override
def locate(self, tenant_id: str, path_args: Mapping[str, object]) -> ResourceIdentity | None:
document_id = _required(path_args, self.param)
dataset_id = RBACResourceService.get_dataset_id_by_document(db.session, tenant_id, document_id)
dataset_id = application_services().rbac.queries.get_dataset_id_by_document(tenant_id, document_id)
if dataset_id is None:
raise NotFound("Document not found")
return ResourceIdentity(self.scope, dataset_id)
@@ -152,7 +148,7 @@ class DatasetByPipeline(DatasetId):
@override
def locate(self, tenant_id: str, path_args: Mapping[str, object]) -> ResourceIdentity | None:
pipeline_id = _required(path_args, self.param)
dataset_id = RBACResourceService.get_dataset_id_by_pipeline(db.session, tenant_id, pipeline_id)
dataset_id = application_services().rbac.queries.get_dataset_id_by_pipeline(tenant_id, pipeline_id)
if dataset_id is None:
raise NotFound("Dataset not found for pipeline")
return ResourceIdentity(self.scope, dataset_id)
api/controllers/common/resource_access_token_errors.py
+27
View File
@@ -0,0 +1,27 @@
"""Translate machine credential application failures at HTTP transport boundaries."""
from collections.abc import Generator
from contextlib import contextmanager
from werkzeug.exceptions import BadRequest, Forbidden, NotFound, Unauthorized
from services.auth.resource_access_token_contracts import (
ResourceAccessTokenForbiddenError,
ResourceAccessTokenInputError,
ResourceAccessTokenInvalidError,
ResourceAccessTokenNotFoundError,
)
@contextmanager
def resource_access_token_errors() -> Generator[None]:
try:
yield
except ResourceAccessTokenInputError as error:
raise BadRequest(str(error)) from error
except ResourceAccessTokenForbiddenError as error:
raise Forbidden(str(error)) from error
except ResourceAccessTokenNotFoundError as error:
raise NotFound(str(error)) from error
except ResourceAccessTokenInvalidError as error:
raise Unauthorized(str(error)) from error
api/controllers/console/__init__.py
+2
View File
@@ -64,6 +64,7 @@ from . import (
init_validate,
notification,
onboarding,
resource_access_token,
setup,
spec,
system,
@@ -243,6 +244,7 @@ __all__ = [
"rag_pipeline_workflow",
"rbac",
"recommended_app",
"resource_access_token",
"saved_message",
"setup",
"site",
api/controllers/console/agent/roster.py
+5 -10
View File
@@ -85,10 +85,10 @@ from services.agent.observability_service import (
)
from services.agent.roster_service import AgentRosterService
from services.app_service import AgentAppPublicationCounts, AppListParams, AppResponseView, AppService, CreateAppParams
from services.enterprise import rbac_service as enterprise_rbac_service
from services.enterprise.enterprise_service import EnterpriseService
from services.entities.agent_entities import ComposerSavePayload, RosterListQuery
from services.feature_service import FeatureService
from services.rbac import contracts as rbac_contracts
from services.system_feature_service import SystemFeatureService
AgentPublicationStatus = Literal["published", "drafts"]
@@ -447,11 +447,10 @@ def _serialize_agent_app_detail(
payload["debug_conversation_message_count"] = message_count
payload["role"] = agent.role or ""
payload["access_ready"] = agent_has_workflow_callable_active_snapshot(session=session, agent=agent)
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
permissions = application_services().rbac.members.permissions(
app_model.tenant_id,
current_user.id,
agent_id=agent.id,
session=session,
)
payload["permission_keys"] = permissions.agent.permission_keys_by_resource_ids([agent.id]).get(agent.id, [])
return AgentAppDetailWithSite.model_validate(payload).model_dump(mode="json", exclude={"bound_agent_id"})
@@ -464,7 +463,7 @@ def _serialize_agent_app_pagination(
tenant_id: str,
current_user: Account,
publication_counts: AgentAppPublicationCounts,
agent_permissions: enterprise_rbac_service.ResourcePermissionSnapshot,
agent_permissions: rbac_contracts.ResourcePermissionSnapshot,
) -> dict:
"""Serialize Agent App lists with roster-shaped items.
@@ -656,16 +655,14 @@ class AgentAppListApi(Resource):
agent_is_published=agent_is_published,
)
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
permissions = application_services().rbac.members.permissions(
current_tenant_id,
current_user.id,
session=session,
)
if dify_config.RBAC_ENABLED:
access_filter = resolve_agent_access_filter(
current_tenant_id,
current_user.id,
session=session,
permissions=permissions,
)
access_filter.apply_to_app_params(params, tenant_id=current_tenant_id, session=session)
@@ -1103,15 +1100,13 @@ class AgentInviteOptionsApi(Resource):
):
accessible_agent_ids = None
if dify_config.RBAC_ENABLED:
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
permissions = application_services().rbac.members.permissions(
tenant_id,
current_user.id,
session=session,
)
access_filter = resolve_agent_access_filter(
tenant_id,
current_user.id,
session=session,
permissions=permissions,
)
if access_filter.accessible_agent_ids is not None:
api/controllers/console/app/app.py
+1
View File
@@ -109,6 +109,7 @@ class AppListBaseQuery(BaseModel):
description="Sort apps by last modified, recently created, or earliest created",
)
name: str | None = Field(default=None, description="Filter by app name")
openapi_visible: bool = Field(default=False, description="Only return apps with API access enabled")
tag_ids: list[str] | None = Field(default=None, description="Filter by tag IDs")
creator_ids: list[str] | None = Field(default=None, description="Filter by creator account IDs")
is_created_by_me: bool | None = Field(default=None, description="Filter by creator")
api/controllers/console/app/completion.py
+2 -7
View File
@@ -17,7 +17,6 @@ from controllers.common.schema import register_response_schema_models, register_
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.error import (
AgentSessionConfigurationChangedError,
AppUnavailableError,
CompletionRequestError,
ConversationCompletedError,
@@ -473,7 +472,7 @@ def _create_build_chat_finalization_message(
return {"result": "success"}, 200
def _drain_streaming_generate_response(response: RateLimitGenerator | Generator[str, None, None]) -> None:
def _drain_streaming_generate_response(response: RateLimitGenerator | Generator[str]) -> None:
"""Consume a streamed app-generate response until a terminal message event arrives.
Finalize keeps the normal Agent App streaming path so the existing queue,
@@ -622,16 +621,12 @@ def _raise_agent_stream_error_before_response(response):
if isinstance(response, _ClosableStream):
response.close()
message = error_payload.get("message")
if error_payload.get("code") == AgentSessionConfigurationChangedError.error_code:
raise AgentSessionConfigurationChangedError(
str(message or AgentSessionConfigurationChangedError.description)
)
raise CompletionRequestError(str(message or "Agent App chat failed."))
return _prepend_stream_chunks(buffered, chunk, iterator)
def _prepend_stream_chunks(buffered: list[Any], first: Any, iterator: Iterator[Any]) -> Generator[Any, None, None]:
def _prepend_stream_chunks(buffered: list[Any], first: Any, iterator: Iterator[Any]) -> Generator[Any]:
yield from buffered
yield first
yield from iterator
api/controllers/console/app/error.py
-10
View File
@@ -1,7 +1,3 @@
from core.app.apps.agent_app.errors import (
AGENT_SESSION_CONFIGURATION_CHANGED_ERROR_CODE,
AGENT_SESSION_CONFIGURATION_CHANGED_MESSAGE,
)
from libs.exception import BaseHTTPException
@@ -53,12 +49,6 @@ class CompletionRequestError(BaseHTTPException):
code = 400
class AgentSessionConfigurationChangedError(BaseHTTPException):
error_code = AGENT_SESSION_CONFIGURATION_CHANGED_ERROR_CODE
description = AGENT_SESSION_CONFIGURATION_CHANGED_MESSAGE
code = 409
class AppMoreLikeThisDisabledError(BaseHTTPException):
error_code = "app_more_like_this_disabled"
description = "The 'More like this' feature is disabled. Please refresh your page."
api/controllers/console/app/generator.py
+1 -1
View File
@@ -602,7 +602,7 @@ class WorkflowGenerateStreamApi(Resource):
if guard is not None:
return guard
def generate() -> Generator[str, None, None]:
def generate() -> Generator[str]:
try:
for event_name, payload in WorkflowGeneratorService.generate_workflow_graph_stream(
tenant_id=current_tenant_id,
api/controllers/console/app/permission_keys.py
+4 -4
View File
@@ -1,9 +1,9 @@
from extensions.ext_database import db
from services.enterprise import rbac_service as enterprise_rbac_service
from extensions.ext_application_services import application_services
from services.rbac.contracts import RBACResourceType
def get_app_permission_keys(tenant_id: str, account_id: str | None, app_id: str) -> list[str]:
permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
tenant_id, account_id, [app_id], session=db.session()
permission_keys_map = application_services().rbac.members.resource_permissions(
tenant_id, account_id, RBACResourceType.APP, [app_id]
)
return permission_keys_map.get(app_id, [])
api/controllers/console/app/workflow.py
+9 -2
View File
@@ -152,6 +152,7 @@ class SyncDraftWorkflowPayload(BaseModel):
graph: dict[str, Any]
features: dict[str, Any]
hash: str | None = None
force: bool = Field(default=False, strict=True, description="Explicitly confirm saving an empty workflow graph.")
is_collaborative: bool = Field(default=False, alias="_is_collaborative")
environment_variable_patch: SyncEnvironmentVariablePatchPayload | None = None
conversation_variables: list[dict[str, Any]] = Field(
@@ -654,6 +655,12 @@ class DraftWorkflowApi(Resource):
return {"message": "Invalid JSON data"}, 400
else:
abort(415)
# Empty autosaves can otherwise replace a populated draft, even with a matching hash.
# Require explicit confirmation for both regular saves and text/plain unload beacons.
if args_model.graph.get("nodes") == [] and args_model.graph.get("edges") == [] and not args_model.force:
raise InvalidArgumentError(description="Saving an empty workflow requires force=true.")
workflow_service = WorkflowService()
try:
@@ -1964,10 +1971,10 @@ class WorkflowOnlineUsersApi(Resource):
return {"data": []}
access_filter = None
if dify_config.RBAC_ENABLED:
access_filter = resolve_app_access_filter(current_tenant_id, current_user.id)
workflow_service = WorkflowService()
with session_factory.create_session() as session:
if dify_config.RBAC_ENABLED:
access_filter = resolve_app_access_filter(current_tenant_id, current_user.id, session=session)
app_maintainers = workflow_service.get_tenant_app_maintainers(app_ids, current_tenant_id, session=session)
accessible_app_ids = set(app_maintainers)
api/controllers/console/datasets/datasets.py
+1 -1
View File
@@ -106,7 +106,7 @@ register_response_schema_models(console_ns, ApiBaseUrlResponse, SimpleResultResp
@contextmanager
def _upgrade_errors() -> Generator[None, None, None]:
def _upgrade_errors() -> Generator[None]:
try:
yield
except (DatasetNotFoundError, DatasetAccessDeniedError) as error:
api/controllers/console/datasets/rag_pipeline/rag_pipeline_datasets.py
+3 -2
View File
@@ -29,6 +29,7 @@ from services.entities.knowledge_entities.rag_pipeline_entities import IconInfo,
from services.knowledge.dataset_read_service import load_dataset_detail
from services.knowledge.dataset_service import DatasetPermissionService, DatasetService
from services.rag_pipeline.rag_pipeline_dsl_service import RagPipelineDslService
from services.rbac import contracts as rbac_contracts
class RagPipelineDatasetImportPayload(BaseModel):
@@ -98,12 +99,12 @@ class CreateRagPipelineDatasetApi(Resource):
current_tenant_id,
current_user.id,
dataset_id,
enterprise_rbac_service.ReplaceMemberBindings(automatic_include_workspace_members=False),
rbac_contracts.ReplaceMemberBindings(automatic_include_workspace_members=False),
)
enterprise_rbac_service.try_sync_creator_access_policy_member_bindings(
current_tenant_id,
current_user.id,
enterprise_rbac_service.RBACResourceType.DATASET,
rbac_contracts.RBACResourceType.DATASET,
dataset_id,
)
api/controllers/console/files.py
+1 -1
View File
@@ -85,7 +85,7 @@ def _parse_file_upload() -> tuple[FileStorage, str, Literal["datasets"] | None]:
@contextmanager
def _file_upload_errors() -> Generator[None, None, None]:
def _file_upload_errors() -> Generator[None]:
try:
yield
except services.errors.file.FileTooLargeError as file_too_large_error:
api/controllers/console/human_input_form.py
+1 -1
View File
@@ -198,7 +198,7 @@ class ConsoleWorkflowEventsApi(Resource):
payload = response.model_dump(mode="json")
payload["event"] = response.event.value
def _generate_finished_events() -> Generator[str, None, None]:
def _generate_finished_events() -> Generator[str]:
yield f"data: {json.dumps(payload)}\n\n"
event_generator = _generate_finished_events
api/controllers/console/resource_access_token.py
+221
View File
@@ -0,0 +1,221 @@
from __future__ import annotations
from datetime import datetime
from uuid import UUID
from flask_restx import Resource
from pydantic import Field, field_validator
from constants.resource_access_token import ResourceAccessTokenResourceType
from controllers.common.resource_access_token_errors import resource_access_token_errors
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.flask_admission import console_account_admission
from controllers.console.wraps import model_validate
from extensions.ext_application_services import application_services
from fields.base import ResponseModel
from libs.helper import dump_response, to_timestamp
from machinery.context import RequestContext
from models.account import TenantAccountRole
from services.auth.resource_access_token_contracts import (
ResourceAccessTokenResource,
ResourceAccessTokenRow,
)
class ResourceAccessTokenResourcePayload(ResponseModel):
type: ResourceAccessTokenResourceType
id: str
class ResourceAccessTokenCreatePayload(ResponseModel):
name: str = Field(min_length=1)
resources: list[ResourceAccessTokenResourcePayload] = Field(min_length=1)
class ResourceAccessTokenUpdatePayload(ResponseModel):
name: str = Field(min_length=1)
resources: list[ResourceAccessTokenResourcePayload] = Field(min_length=1)
class ResourceAccessTokenListQuery(ResponseModel):
page: int = Field(default=1, ge=1)
limit: int = Field(default=20, ge=1, le=100)
keyword: str | None = Field(default=None, min_length=1, max_length=100)
class ResourceAccessTokenRowResponse(ResponseModel):
token_id: str
relation_id: str
name: str
track_id: str
token: str | None = None
masked_token: str
resource_type: ResourceAccessTokenResourceType
resource_id: str
resource_name: str
created_at: int
last_used_at: int | None = None
@field_validator("created_at", "last_used_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
return to_timestamp(value)
class ResourceAccessTokenListResponse(ResponseModel):
data: list[ResourceAccessTokenRowResponse]
page: int
limit: int
total: int
has_more: bool
class ResourceAccessTokenCreateResponse(ResponseModel):
token: str
data: list[ResourceAccessTokenRowResponse]
register_schema_models(
console_ns,
ResourceAccessTokenResourcePayload,
ResourceAccessTokenCreatePayload,
ResourceAccessTokenUpdatePayload,
ResourceAccessTokenListQuery,
)
register_response_schema_models(
console_ns,
ResourceAccessTokenRowResponse,
ResourceAccessTokenListResponse,
ResourceAccessTokenCreateResponse,
)
_OWNER_ROLES = frozenset({TenantAccountRole.OWNER})
def _dump_rows(rows: tuple[ResourceAccessTokenRow, ...]) -> list[dict[str, object]]:
return [
dump_response(
ResourceAccessTokenRowResponse,
ResourceAccessTokenRowResponse.model_validate(row, from_attributes=True),
)
for row in rows
]
@console_ns.route("/resource-access-tokens")
class ResourceAccessTokenListApi(Resource):
@console_ns.doc("list_resource_access_tokens")
@console_ns.doc(params=query_params_from_model(ResourceAccessTokenListQuery))
@console_ns.response(200, "Resource access tokens", console_ns.models[ResourceAccessTokenListResponse.__name__])
@console_account_admission(allowed_roles=_OWNER_ROLES)
@model_validate(ResourceAccessTokenListQuery)
def get(
self,
query: ResourceAccessTokenListQuery,
context: RequestContext,
) -> dict[str, object]:
with resource_access_token_errors():
rows = application_services().resource_access_tokens.list_rows(
context,
page=query.page,
limit=query.limit,
keyword=query.keyword,
)
total = application_services().resource_access_tokens.count_rows(context, keyword=query.keyword)
return dump_response(
ResourceAccessTokenListResponse,
{
"data": _dump_rows(rows),
"page": query.page,
"limit": query.limit,
"total": total,
"has_more": query.page * query.limit < total,
},
)
@console_ns.doc("create_resource_access_token")
@console_ns.expect(console_ns.models[ResourceAccessTokenCreatePayload.__name__])
@console_ns.response(
201,
"Resource access token created",
console_ns.models[ResourceAccessTokenCreateResponse.__name__],
)
@console_account_admission(allowed_roles=_OWNER_ROLES)
@model_validate(ResourceAccessTokenCreatePayload)
def post(
self,
payload: ResourceAccessTokenCreatePayload,
context: RequestContext,
) -> tuple[dict[str, object], int]:
with resource_access_token_errors():
result = application_services().resource_access_tokens.create(
context,
name=payload.name,
resources=tuple(
ResourceAccessTokenResource(type=resource.type, id=resource.id) for resource in payload.resources
),
)
return dump_response(
ResourceAccessTokenCreateResponse,
{"token": result.token, "data": _dump_rows(result.rows)},
), 201
@console_ns.route("/resource-access-tokens/<uuid:token_id>")
class ResourceAccessTokenApi(Resource):
@console_ns.doc("update_resource_access_token")
@console_ns.expect(console_ns.models[ResourceAccessTokenUpdatePayload.__name__])
@console_ns.response(
200,
"Resource access token updated",
console_ns.models[ResourceAccessTokenListResponse.__name__],
)
@console_account_admission(allowed_roles=_OWNER_ROLES)
@model_validate(ResourceAccessTokenUpdatePayload)
def patch(
self,
payload: ResourceAccessTokenUpdatePayload,
context: RequestContext,
token_id: UUID,
) -> dict[str, object]:
with resource_access_token_errors():
rows = application_services().resource_access_tokens.update(
context,
token_id=str(token_id),
name=payload.name,
resources=tuple(
ResourceAccessTokenResource(type=resource.type, id=resource.id) for resource in payload.resources
),
)
return dump_response(
ResourceAccessTokenListResponse,
{
"data": _dump_rows(rows),
"page": 1,
"limit": len(rows),
"total": len(rows),
"has_more": False,
},
)
@console_ns.route("/resource-access-tokens/<uuid:token_id>/relations/<uuid:relation_id>")
class ResourceAccessTokenRelationApi(Resource):
@console_ns.doc("delete_resource_access_token_relation")
@console_ns.response(204, "Resource access token relation deleted")
@console_account_admission(allowed_roles=_OWNER_ROLES)
def delete(
self,
context: RequestContext,
token_id: UUID,
relation_id: UUID,
) -> tuple[str, int]:
with resource_access_token_errors():
application_services().resource_access_tokens.delete_relation(
context,
token_id=str(token_id),
relation_id=str(relation_id),
)
return "", 204
api/controllers/console/workspace/members.py
+4 -1
View File
@@ -10,6 +10,7 @@ from werkzeug.exceptions import Forbidden, NotFound
from configs import dify_config
from controllers.common.fields import SimpleResultDataResponse, SimpleResultResponse, VerificationTokenResponse
from controllers.common.rbac import rbac_language
from controllers.common.schema import register_enum_models, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.auth.error import (
@@ -142,7 +143,9 @@ class MemberListApi(Resource):
@console_ns.response(HTTPStatus.OK, "Success", console_ns.models[AccountWithRoleListResponse.__name__])
@console_account_admission()
def get(self, request_context: RequestContext):
members = application_services().workspaces.member_queries.list_current(request_context)
members = application_services().workspaces.member_queries.list_current(
request_context, language=rbac_language()
)
serialized_members = [
{
"id": member.id,
api/controllers/console/workspace/rbac.py
-968
View File
@@ -1,968 +0,0 @@
from __future__ import annotations
from collections.abc import Callable, Mapping
from dataclasses import dataclass
from typing import Any, Literal
from flask import request
from flask_restx import Resource
from pydantic import AliasChoices, BaseModel, ConfigDict, Field, ValidationError, field_validator
from sqlalchemy import select
from werkzeug.exceptions import NotFound
from configs import dify_config
from controllers.common.rbac import RBACCheck, Workspace
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.wraps import RBACPermission, model_validate, rbac_permission_required
from core.db.session_factory import session_factory
from enums import DeploymentEdition
from extensions.ext_database import db
from libs.login import current_account_with_tenant, login_required
from models import Account
from services.enterprise import rbac_service as svc
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
class _RBACRoleList(svc.Paginated[svc.RBACRole]):
pass
class _RBACRoleAccountList(svc.Paginated[svc.RBACRoleAccount]):
pass
class _AccessPolicyList(svc.Paginated[svc.AccessPolicy]):
pass
class _MembersInRoleList(svc.Paginated[svc.MembersInRole]):
pass
register_response_schema_models(
console_ns,
svc.PermissionCatalogResponse,
svc.RBACRole,
_RBACRoleList,
_RBACRoleAccountList,
_MembersInRoleList,
svc.AccessPolicy,
_AccessPolicyList,
svc.AccessPolicyBindingState,
svc.MyPermissionsResponse,
svc.AppAccessMatrix,
svc.DatasetAccessMatrix,
svc.AgentAccessMatrix,
svc.WorkspaceAccessMatrix,
svc.ResourceWhitelist,
svc.ResourceWhitelistConfig,
svc.ResourceUserAccessPoliciesResponse,
svc.ReplaceUserAccessPoliciesResponse,
svc.RoleBindingsResponse,
svc.MemberBindingsResponse,
svc.MemberRolesResponse,
svc.AccessMatrixItem,
)
_LEGACY_ROLE_PERMISSION_KEYS: dict[str, list[str]] = {
# This is a compatibility projection from the pre-RBAC workspace roles into
# the 2.0 permission matrix documented in "权限整理2.0". It intentionally
# models the product-facing role surface for the new RBAC UI instead of the
# legacy backend's exact hard-authorization checks.
"owner": [
*svc._LEGACY_WORKSPACE_OWNER_KEYS,
*svc._LEGACY_APP_OWNER_KEYS,
*svc._LEGACY_DATASET_OWNER_KEYS,
],
"admin": [
*svc._LEGACY_WORKSPACE_ADMIN_KEYS,
*svc._LEGACY_APP_ADMIN_KEYS,
*svc._LEGACY_DATASET_ADMIN_KEYS,
],
"editor": [
*svc._LEGACY_WORKSPACE_EDITOR_KEYS,
*svc._LEGACY_APP_EDITOR_KEYS,
*svc._LEGACY_DATASET_EDITOR_KEYS,
],
"normal": [
*svc._LEGACY_WORKSPACE_NORMAL_KEYS,
*svc._LEGACY_APP_NORMAL_KEYS,
],
"dataset_operator": [
*svc._LEGACY_WORKSPACE_DATASET_OPERATOR_KEYS,
*svc._LEGACY_DATASET_DATASET_OPERATOR_KEYS,
],
}
def _current_ids() -> tuple[str, str]:
"""Return ``(tenant_id, account_id)`` for the authenticated user, or
raise a 404 when no tenant is associated with the session.
"""
user, tenant_id = current_account_with_tenant()
if not tenant_id:
raise NotFound("Current workspace not found")
return tenant_id, user.id
def _payload(model: type[BaseModel]) -> Any:
"""Validate the JSON body against ``model`` or raise ``ValidationError``.
``ValidationError`` bubbles up as HTTP 400 thanks to
``controllers/common/helpers.py`` error handling.
"""
try:
return model.model_validate(console_ns.payload or {})
except ValidationError as exc:
# Re-raise as-is so the upstream error handler renders a 400.
raise exc
def _dump(model: BaseModel) -> dict[str, Any]:
return model.model_dump(mode="json")
def _account_names_by_ids(account_ids: list[str]) -> dict[str, dict[str, str]]:
ids = sorted({account_id.strip() for account_id in account_ids if account_id and account_id.strip()})
if not ids:
return {}
with session_factory.create_session() as session:
rows = session.execute(
select(Account.id, Account.name, Account.avatar, Account.email).where(Account.id.in_(ids))
).all()
return {
account_id: {
"name": name or "",
"avatar": avatar or "",
"email": email or "",
}
for account_id, name, avatar, email in rows
}
def _hydrate_access_matrix_account_names(items: list[svc.AccessMatrixItem]) -> None:
account_ids: list[str] = []
for item in items:
for account in item.accounts:
account_id = account.account_id
if account_id and not account.account_name:
account_ids.append(account_id)
account_names = _account_names_by_ids(account_ids)
if not account_names:
return
for item in items:
for account in item.accounts:
account_id = str(account.account_id or "").strip()
if account_id and not account.account_name:
account.account_name = account_names.get(account_id, {}).get("name", "")
account.avatar = account_names.get(account_id, {}).get("avatar", "")
account.email = account_names.get(account_id, {}).get("email", "")
def _hydrate_resource_user_account_names(items: list[svc.ResourceUserAccessPolicies]) -> None:
account_names = _account_names_by_ids([item.account.account_id for item in items])
for item in items:
account_id = item.account.account_id
if account_id and not item.account.account_name:
item.account.account_name = account_names.get(account_id, {}).get("name", "")
item.account.avatar = account_names.get(account_id, {}).get("avatar", "")
item.account.email = account_names.get(account_id, {}).get("email", "")
def _move_resource_maintainer_first(items: list[svc.ResourceUserAccessPolicies], maintainer_id: str | None) -> None:
if not maintainer_id:
return
normalized_maintainer_id = str(maintainer_id).strip()
if not normalized_maintainer_id:
return
for index, item in enumerate(items):
account_id = str(item.account.account_id or "").strip()
if account_id == normalized_maintainer_id:
if index:
items.insert(0, items.pop(index))
return
class _PaginationQuery(BaseModel):
model_config = ConfigDict(extra="ignore")
page_number: int | None = Field(default=None, ge=1, validation_alias=AliasChoices("page", "page_number"))
results_per_page: int | None = Field(
default=None, ge=1, le=99999, validation_alias=AliasChoices("limit", "results_per_page")
)
reverse: bool | None = None
def to_inner_options(self) -> svc.ListOption:
return svc.ListOption.model_validate(self.model_dump())
class _RolesListQuery(_PaginationQuery):
include_owner: int = Field(default=0, ge=0, le=1)
class CopyRoleParam(BaseModel):
copy_member: bool = True
def _pagination_options() -> svc.ListOption:
return _PaginationQuery.model_validate(request.args.to_dict(flat=True)).to_inner_options()
def _legacy_workspace_roles(
options: svc.ListOption | None = None, *, include_owner: int = 0, billing_enabled: bool = True
) -> svc.Paginated[svc.RBACRole]:
"""Return the built-in legacy workspace roles in the RBAC list shape.
This keeps the new `/rbac/roles` endpoint compatible with the original
Dify role model when enterprise RBAC is disabled.
"""
legacy_roles = []
for role_name in ("owner", "admin", "editor", "normal", "dataset_operator"):
if not dify_config.DATASET_OPERATOR_ENABLED and role_name == "dataset_operator":
continue
permission_keys = _LEGACY_ROLE_PERMISSION_KEYS[role_name]
valid_permission_keys = []
for permission_key in permission_keys:
if not billing_enabled and "billing" in permission_key:
continue
valid_permission_keys.append(permission_key)
legacy_roles.append(
svc.RBACRole(
id=role_name,
tenant_id="",
type=svc.RBACRoleType.WORKSPACE.value,
category="global_system_default",
name=role_name,
description="",
is_builtin=True,
permission_keys=valid_permission_keys,
role_tag="owner" if role_name == "owner" else "",
)
)
if not include_owner:
legacy_roles = [r for r in legacy_roles if r.name != "owner"]
page_number = options.page_number if options and options.page_number is not None else 1
results_per_page = (
options.results_per_page if options and options.results_per_page is not None else len(legacy_roles)
)
reverse = options.reverse if options and options.reverse is not None else False
ordered_roles = list(reversed(legacy_roles)) if reverse else legacy_roles
start = max(page_number - 1, 0) * results_per_page
end = start + results_per_page
paged_roles = ordered_roles[start:end]
total_count = len(legacy_roles)
total_pages = (total_count + results_per_page - 1) // results_per_page if results_per_page > 0 else 0
return svc.Paginated[svc.RBACRole](
data=paged_roles,
pagination=svc.Pagination(
total_count=total_count,
per_page=results_per_page,
current_page=page_number,
total_pages=total_pages,
),
)
@console_ns.route("/workspaces/current/rbac/role-permissions/catalog")
class RBACWorkspaceCatalogApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.PermissionCatalogResponse.__name__])
def get(self):
tenant_id, account_id = _current_ids()
return _dump(svc.RBACService.Catalog.workspace(tenant_id, account_id))
# ---------------------------------------------------------------------------
# Roles.
# ---------------------------------------------------------------------------
class _RoleUpsertRequest(BaseModel):
"""Accepts the payload sent by the Create/Edit Role dialog."""
name: str
description: str = ""
permission_keys: list[str] = []
def to_mutation(self) -> svc.RoleMutation:
return svc.RoleMutation(
name=self.name,
description=self.description,
permission_keys=list(self.permission_keys),
)
@console_ns.route("/workspaces/current/rbac/roles")
class RBACRolesApi(Resource):
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(200, "Success", console_ns.models[_RBACRoleList.__name__])
@model_validate(_RolesListQuery)
def get(self, req_data: _RolesListQuery):
tenant_id, account_id = _current_ids()
options = req_data.to_inner_options()
if not dify_config.RBAC_ENABLED:
result = _legacy_workspace_roles(
options,
include_owner=req_data.include_owner,
billing_enabled=dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD,
)
else:
result = svc.RBACService.Roles.list(
tenant_id,
account_id,
include_owner=req_data.include_owner,
biiling_enabled=dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD,
options=options,
)
return _dump(result)
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(201, "Role created", console_ns.models[svc.RBACRole.__name__])
def post(self):
tenant_id, account_id = _current_ids()
request = _payload(_RoleUpsertRequest)
role = svc.RBACService.Roles.create(tenant_id, account_id, request.to_mutation())
return _dump(role), 201
@console_ns.route("/workspaces/current/rbac/roles/<uuid:role_id>")
class RBACRoleItemApi(Resource):
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(200, "Success", console_ns.models[svc.RBACRole.__name__])
def get(self, role_id):
tenant_id, account_id = _current_ids()
return _dump(
svc.RBACService.Roles.get(
tenant_id,
account_id,
role_id,
billing_enabled=dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD,
)
)
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(200, "Success", console_ns.models[svc.RBACRole.__name__])
def put(self, role_id):
tenant_id, account_id = _current_ids()
request = _payload(_RoleUpsertRequest)
role = svc.RBACService.KnowledgeFSRoleMutations.update_role(
tenant_id,
account_id,
str(role_id),
request.to_mutation(),
session=db.session(),
)
return _dump(role)
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(200, "Success", console_ns.models[svc.RBACRole.__name__])
def delete(self, role_id):
tenant_id, account_id = _current_ids()
svc.RBACService.KnowledgeFSRoleMutations.delete_role(
tenant_id,
account_id,
str(role_id),
session=db.session(),
)
return {"result": "success"}
@console_ns.route("/workspaces/current/rbac/roles/<uuid:role_id>/copy")
class RBACRoleCopyApi(Resource):
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(201, "Role copied", console_ns.models[svc.RBACRole.__name__])
def post(self, role_id):
tenant_id, account_id = _current_ids()
request = _payload(CopyRoleParam)
role = svc.RBACService.Roles.copy(tenant_id, account_id, str(role_id), copy_member=request.copy_member)
return _dump(role), 201
class _AccessPolicyCreateRequest(BaseModel):
name: str
resource_type: svc.RBACResourceType
description: str = ""
permission_keys: list[str] = []
class _AccessPolicyUpdateRequest(BaseModel):
name: str
description: str = ""
permission_keys: list[str] = []
register_schema_models(console_ns, _AccessPolicyCreateRequest, _AccessPolicyUpdateRequest)
@console_ns.route("/workspaces/current/rbac/access-policies")
class RBACAccessPoliciesApi(Resource):
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(200, "Success", console_ns.models[_AccessPolicyList.__name__])
def get(self):
tenant_id, account_id = _current_ids()
# `resource_type` is exposed as a query argument so the UI can show
# only app-scoped or only dataset-scoped permission sets.
resource_type = request.args.get("resource_type") or None
return _dump(
svc.RBACService.AccessPolicies.list(
tenant_id,
account_id,
resource_type=resource_type,
options=_pagination_options(),
)
)
@console_ns.expect(console_ns.models[_AccessPolicyCreateRequest.__name__])
@console_ns.response(201, "Policy created", console_ns.models[svc.AccessPolicy.__name__])
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
def post(self):
tenant_id, account_id = _current_ids()
request = _payload(_AccessPolicyCreateRequest)
policy = svc.RBACService.AccessPolicies.create(
tenant_id,
account_id,
svc.AccessPolicyCreate(
name=request.name,
resource_type=request.resource_type,
description=request.description,
permission_keys=list(request.permission_keys),
),
)
return _dump(policy), 201
@console_ns.route("/workspaces/current/rbac/access-policies/<uuid:policy_id>")
class RBACAccessPolicyItemApi(Resource):
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(200, "Success", console_ns.models[svc.AccessPolicy.__name__])
def get(self, policy_id):
tenant_id, account_id = _current_ids()
return _dump(svc.RBACService.AccessPolicies.get(tenant_id, account_id, str(policy_id)))
@console_ns.expect(console_ns.models[_AccessPolicyUpdateRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.AccessPolicy.__name__])
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
def put(self, policy_id):
tenant_id, account_id = _current_ids()
request = _payload(_AccessPolicyUpdateRequest)
policy = svc.RBACService.AccessPolicies.update(
tenant_id,
account_id,
str(policy_id),
svc.AccessPolicyUpdate(
name=request.name,
description=request.description,
permission_keys=list(request.permission_keys),
),
)
return _dump(policy)
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(200, "Success", console_ns.models[svc.AccessPolicy.__name__])
def delete(self, policy_id):
tenant_id, account_id = _current_ids()
svc.RBACService.AccessPolicies.delete(tenant_id, account_id, str(policy_id))
return {"result": "success"}
@console_ns.route("/workspaces/current/rbac/access-policies/<uuid:policy_id>/copy")
class RBACAccessPolicyCopyApi(Resource):
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(201, "Policy copied", console_ns.models[svc.AccessPolicy.__name__])
def post(self, policy_id):
tenant_id, account_id = _current_ids()
policy = svc.RBACService.AccessPolicies.copy(tenant_id, account_id, str(policy_id))
return _dump(policy), 201
@console_ns.route("/workspaces/current/rbac/access-policy-bindings/<uuid:binding_id>/lock")
class RBACAccessPolicyBindingLockApi(Resource):
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(200, "Success", console_ns.models[svc.AccessPolicyBindingState.__name__])
def put(self, binding_id):
tenant_id, account_id = _current_ids()
return _dump(svc.RBACService.AccessPolicyBindings.lock(tenant_id, account_id, str(binding_id)))
@console_ns.route("/workspaces/current/rbac/access-policy-bindings/<uuid:binding_id>/unlock")
class RBACAccessPolicyBindingUnlockApi(Resource):
@login_required
@rbac_permission_required(RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace()))
@console_ns.response(200, "Success", console_ns.models[svc.AccessPolicyBindingState.__name__])
def put(self, binding_id):
tenant_id, account_id = _current_ids()
return _dump(svc.RBACService.AccessPolicyBindings.unlock(tenant_id, account_id, str(binding_id)))
class _ResourceAccessScopeRequest(BaseModel):
automatic_include_workspace_members: bool
class _ReplaceBindingsRequest(BaseModel):
role_ids: list[str] = Field(default_factory=list)
account_ids: list[str] = Field(default_factory=list)
@field_validator("role_ids", "account_ids", mode="before")
@classmethod
def _coerce_bindings(cls, value: Any) -> list[str]:
if value is None:
return []
return value
class _DeleteMemberBindingsRequest(BaseModel):
account_ids: list[str] = Field(default_factory=list)
@field_validator("account_ids", mode="before")
@classmethod
def _coerce_account_ids(cls, value: Any) -> list[str]:
if value is None:
return []
return value
class _AccessControlLanguageQuery(BaseModel):
language: Literal["en", "ja", "zh"] | None = Field(default=None, description="Localized policy label language")
class _ResourceUserAccessPoliciesQuery(_PaginationQuery):
language: Literal["en", "ja", "zh"] | None = Field(default=None, description="Localized policy label language")
register_schema_models(
console_ns,
_ResourceAccessScopeRequest,
_ReplaceBindingsRequest,
_DeleteMemberBindingsRequest,
_AccessControlLanguageQuery,
_ResourceUserAccessPoliciesQuery,
svc.ReplaceUserAccessPolicies,
)
@console_ns.route("/workspaces/current/rbac/my-permissions")
class RBACMyPermissionsApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.MyPermissionsResponse.__name__])
def get(self):
tenant_id, account_id = _current_ids()
return _dump(
svc.RBACService.MyPermissions.get(
tenant_id,
account_id,
app_id=request.args.get("app_id") or None,
dataset_id=request.args.get("dataset_id") or None,
agent_id=request.args.get("agent_id") or None,
session=db.session(),
)
)
type _ResourceAccessClient = (
type[svc.RBACService.AppAccess] | type[svc.RBACService.DatasetAccess] | type[svc.RBACService.AgentAccess]
)
@dataclass(frozen=True)
class _ResourceAccessRoutes:
resource_type: svc.RBACResourceType
class_prefix: str
matrix_model: type[BaseModel]
access: _ResourceAccessClient
catalog: Callable[[str, str], svc.PermissionCatalogResponse]
workspace_matrix: Callable[[str, str, svc.ListOption], svc.WorkspaceAccessMatrix]
workspace_role_bindings: Callable[[str, str, str], svc.RoleBindingsResponse]
workspace_member_bindings: Callable[[str, str, str], svc.MemberBindingsResponse]
replace_workspace_bindings: Callable[[str, str, str, svc.ReplaceBindings], svc.AccessMatrixItem]
maintainer_id: Callable[[str, str], str | None] | None = None
@property
def url_segment(self) -> str:
return self.resource_type.route.segment
@property
def id_param(self) -> str:
return self.resource_type.route.id_param
@dataclass(frozen=True)
class _ResourceAccessApis:
catalog: type[Resource]
matrix: type[Resource]
whitelist: type[Resource]
whitelist_config: type[Resource]
user_access_policies: type[Resource]
user_access_policy_assignment: type[Resource]
role_bindings: type[Resource]
member_bindings: type[Resource]
workspace_matrix: type[Resource]
workspace_role_bindings: type[Resource]
workspace_bindings: type[Resource]
workspace_member_bindings: type[Resource]
_RESOURCE_ACCESS_ROUTES = (
_ResourceAccessRoutes(
resource_type=svc.RBACResourceType.APP,
class_prefix="App",
matrix_model=svc.AppAccessMatrix,
access=svc.RBACService.AppAccess,
catalog=lambda tenant_id, account_id: svc.RBACService.Catalog.app(tenant_id, account_id),
workspace_matrix=lambda tenant_id, account_id, options: svc.RBACService.WorkspaceAccess.app_matrix(
tenant_id, account_id, options=options
),
workspace_role_bindings=lambda tenant_id, account_id, policy_id: (
svc.RBACService.WorkspaceAccess.list_app_role_bindings(tenant_id, account_id, policy_id)
),
workspace_member_bindings=lambda tenant_id, account_id, policy_id: (
svc.RBACService.WorkspaceAccess.list_app_member_bindings(tenant_id, account_id, policy_id)
),
replace_workspace_bindings=lambda tenant_id, account_id, policy_id, payload: (
svc.RBACService.WorkspaceAccess.replace_app_bindings(tenant_id, account_id, policy_id, payload)
),
maintainer_id=lambda tenant_id, resource_id: svc.app_maintainer_id(tenant_id, resource_id),
),
_ResourceAccessRoutes(
resource_type=svc.RBACResourceType.DATASET,
class_prefix="Dataset",
matrix_model=svc.DatasetAccessMatrix,
access=svc.RBACService.DatasetAccess,
catalog=lambda tenant_id, account_id: svc.RBACService.Catalog.dataset(tenant_id, account_id),
workspace_matrix=lambda tenant_id, account_id, options: svc.RBACService.WorkspaceAccess.dataset_matrix(
tenant_id, account_id, options=options
),
workspace_role_bindings=lambda tenant_id, account_id, policy_id: (
svc.RBACService.WorkspaceAccess.list_dataset_role_bindings(tenant_id, account_id, policy_id)
),
workspace_member_bindings=lambda tenant_id, account_id, policy_id: (
svc.RBACService.WorkspaceAccess.list_dataset_member_bindings(tenant_id, account_id, policy_id)
),
replace_workspace_bindings=lambda tenant_id, account_id, policy_id, payload: (
svc.RBACService.WorkspaceAccess.replace_dataset_bindings(tenant_id, account_id, policy_id, payload)
),
maintainer_id=lambda tenant_id, resource_id: svc.dataset_maintainer_id(tenant_id, resource_id),
),
_ResourceAccessRoutes(
resource_type=svc.RBACResourceType.AGENT,
class_prefix="Agent",
matrix_model=svc.AgentAccessMatrix,
access=svc.RBACService.AgentAccess,
catalog=lambda tenant_id, account_id: svc.RBACService.Catalog.agent(tenant_id, account_id),
workspace_matrix=lambda tenant_id, account_id, options: svc.RBACService.WorkspaceAccess.agent_matrix(
tenant_id, account_id, options=options
),
workspace_role_bindings=lambda tenant_id, account_id, policy_id: (
svc.RBACService.WorkspaceAccess.list_agent_role_bindings(tenant_id, account_id, policy_id)
),
workspace_member_bindings=lambda tenant_id, account_id, policy_id: (
svc.RBACService.WorkspaceAccess.list_agent_member_bindings(tenant_id, account_id, policy_id)
),
replace_workspace_bindings=lambda tenant_id, account_id, policy_id, payload: (
svc.RBACService.WorkspaceAccess.replace_agent_bindings(tenant_id, account_id, policy_id, payload)
),
),
)
def _build_resource_access_apis(spec: _ResourceAccessRoutes) -> _ResourceAccessApis:
id_param = spec.id_param
resource_prefix = f"/workspaces/current/rbac/{spec.url_segment}/<uuid:{id_param}>"
workspace_prefix = f"/workspaces/current/rbac/workspace/{spec.url_segment}"
def resource_id(path_params: Mapping[str, object]) -> str:
return str(path_params[id_param])
def register(resource: type[Resource], name: str, url: str) -> type[Resource]:
resource.__name__ = name
resource.__qualname__ = name
console_ns.route(url)(resource)
return resource
class CatalogApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.PermissionCatalogResponse.__name__])
def get(self):
tenant_id, account_id = _current_ids()
return _dump(spec.catalog(tenant_id, account_id))
class MatrixApi(Resource):
@login_required
@console_ns.doc(params=query_params_from_model(_AccessControlLanguageQuery))
@console_ns.response(200, "Success", console_ns.models[spec.matrix_model.__name__])
def get(self, **path_params):
tenant_id, account_id = _current_ids()
result = spec.access.matrix(tenant_id, account_id, resource_id(path_params))
_hydrate_access_matrix_account_names(result.items)
return _dump(result)
class WhitelistApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.ResourceWhitelist.__name__])
def get(self, **path_params):
tenant_id, account_id = _current_ids()
return _dump(spec.access.whitelist(tenant_id, account_id, resource_id(path_params)))
@login_required
@console_ns.expect(console_ns.models[_ResourceAccessScopeRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.ResourceWhitelist.__name__])
def put(self, **path_params):
tenant_id, account_id = _current_ids()
target_id = resource_id(path_params)
scope = _payload(_ResourceAccessScopeRequest)
result = spec.access.replace_whitelist(
tenant_id,
account_id,
target_id,
svc.ReplaceMemberBindings(
automatic_include_workspace_members=scope.automatic_include_workspace_members
),
)
if scope.automatic_include_workspace_members:
initialize_created_app_rbac_access_task.delay(tenant_id, account_id, **{id_param: target_id})
return _dump(result)
class WhitelistConfigApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.ResourceWhitelistConfig.__name__])
def get(self, **path_params):
tenant_id, account_id = _current_ids()
return _dump(spec.access.whitelist_config(tenant_id, account_id, resource_id(path_params)))
class UserAccessPoliciesApi(Resource):
@login_required
@console_ns.doc(params=query_params_from_model(_ResourceUserAccessPoliciesQuery))
@console_ns.response(200, "Success", console_ns.models[svc.ResourceUserAccessPoliciesResponse.__name__])
def get(self, **path_params):
tenant_id, account_id = _current_ids()
target_id = resource_id(path_params)
options = _pagination_options()
result = spec.access.user_access_policies(tenant_id, account_id, target_id, options=options)
maintainer_id = spec.maintainer_id(tenant_id, target_id) if spec.maintainer_id else None
_move_resource_maintainer_first(result.data, maintainer_id)
_hydrate_resource_user_account_names(result.data)
return _dump(result)
class UserAccessPolicyAssignmentApi(Resource):
@login_required
@console_ns.expect(console_ns.models[svc.ReplaceUserAccessPolicies.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.ReplaceUserAccessPoliciesResponse.__name__])
def put(self, target_account_id, **path_params):
tenant_id, account_id = _current_ids()
payload = _payload(svc.ReplaceUserAccessPolicies)
return _dump(
spec.access.replace_user_access_policies(
tenant_id,
account_id,
resource_id(path_params),
str(target_account_id),
payload,
)
)
class RoleBindingsApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.RoleBindingsResponse.__name__])
def get(self, policy_id, **path_params):
tenant_id, account_id = _current_ids()
return _dump(
spec.access.list_role_bindings(tenant_id, account_id, resource_id(path_params), str(policy_id))
)
class MemberBindingsApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.MemberBindingsResponse.__name__])
def get(self, policy_id, **path_params):
tenant_id, account_id = _current_ids()
return _dump(
spec.access.list_member_bindings(tenant_id, account_id, resource_id(path_params), str(policy_id))
)
@login_required
@console_ns.expect(console_ns.models[_DeleteMemberBindingsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.MemberBindingsResponse.__name__])
def delete(self, policy_id, **path_params):
tenant_id, account_id = _current_ids()
body = _payload(_DeleteMemberBindingsRequest)
spec.access.delete_member_bindings(
tenant_id,
account_id,
resource_id(path_params),
str(policy_id),
svc.DeleteMemberBindings(account_ids=body.account_ids),
)
return {"result": "success"}
class WorkspaceMatrixApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.WorkspaceAccessMatrix.__name__])
def get(self):
tenant_id, account_id = _current_ids()
result = spec.workspace_matrix(tenant_id, account_id, _pagination_options())
_hydrate_access_matrix_account_names(result.items)
return _dump(result)
class WorkspaceRoleBindingsApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.RoleBindingsResponse.__name__])
def get(self, policy_id):
tenant_id, account_id = _current_ids()
return _dump(spec.workspace_role_bindings(tenant_id, account_id, str(policy_id)))
class WorkspaceBindingsApi(Resource):
@login_required
@console_ns.expect(console_ns.models[_ReplaceBindingsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.AccessMatrixItem.__name__])
def put(self, policy_id):
tenant_id, account_id = _current_ids()
body = _payload(_ReplaceBindingsRequest)
return _dump(
spec.replace_workspace_bindings(
tenant_id,
account_id,
str(policy_id),
svc.ReplaceBindings(role_ids=list(body.role_ids), account_ids=list(body.account_ids)),
)
)
class WorkspaceMemberBindingsApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.MemberBindingsResponse.__name__])
def get(self, policy_id):
tenant_id, account_id = _current_ids()
return _dump(spec.workspace_member_bindings(tenant_id, account_id, str(policy_id)))
prefix = spec.class_prefix
return _ResourceAccessApis(
catalog=register(
CatalogApi,
f"RBAC{prefix}CatalogApi",
f"/workspaces/current/rbac/role-permissions/catalog/{spec.resource_type.value}",
),
matrix=register(MatrixApi, f"RBAC{prefix}MatrixApi", f"{resource_prefix}/access-policy"),
whitelist=register(WhitelistApi, f"RBAC{prefix}WhitelistApi", f"{resource_prefix}/whitelist"),
whitelist_config=register(
WhitelistConfigApi, f"RBAC{prefix}WhitelistConfigApi", f"{resource_prefix}/whitelist_config"
),
user_access_policies=register(
UserAccessPoliciesApi, f"RBAC{prefix}UserAccessPoliciesApi", f"{resource_prefix}/user-access-policies"
),
user_access_policy_assignment=register(
UserAccessPolicyAssignmentApi,
f"RBAC{prefix}UserAccessPolicyAssignmentApi",
f"{resource_prefix}/users/<uuid:target_account_id>/access-policies",
),
role_bindings=register(
RoleBindingsApi,
f"RBAC{prefix}RoleBindingsApi",
f"{resource_prefix}/access-policies/<uuid:policy_id>/role-bindings",
),
member_bindings=register(
MemberBindingsApi,
f"RBAC{prefix}MemberBindingsApi",
f"{resource_prefix}/access-policies/<string:policy_id>/member-bindings",
),
workspace_matrix=register(
WorkspaceMatrixApi, f"RBACWorkspace{prefix}MatrixApi", f"{workspace_prefix}/access-policy"
),
workspace_role_bindings=register(
WorkspaceRoleBindingsApi,
f"RBACWorkspace{prefix}RoleBindingsApi",
f"{workspace_prefix}/access-policies/<uuid:policy_id>/role-bindings",
),
workspace_bindings=register(
WorkspaceBindingsApi,
f"RBACWorkspace{prefix}BindingsApi",
f"{workspace_prefix}/access-policies/<uuid:policy_id>/bindings",
),
workspace_member_bindings=register(
WorkspaceMemberBindingsApi,
f"RBACWorkspace{prefix}MemberBindingsApi",
f"{workspace_prefix}/access-policies/<uuid:policy_id>/member-bindings",
),
)
_RESOURCE_ACCESS_APIS = {spec.resource_type: _build_resource_access_apis(spec) for spec in _RESOURCE_ACCESS_ROUTES}
# ---------------------------------------------------------------------------
# Member ↔ role bindings (Settings > Members > Assign roles).
# ---------------------------------------------------------------------------
class _ReplaceMemberRolesRequest(BaseModel):
role_ids: list[str] = []
@field_validator("role_ids", mode="before")
@classmethod
def _coerce_role_ids(cls, value: Any) -> list[str]:
if value is None:
return []
return value
register_schema_models(console_ns, _ReplaceMemberRolesRequest)
@console_ns.route("/workspaces/current/rbac/members/<uuid:member_id>/rbac-roles")
class RBACMemberRolesApi(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[svc.MemberRolesResponse.__name__])
def get(self, member_id):
tenant_id, account_id = _current_ids()
return _dump(svc.RBACService.MemberRoles.get(tenant_id, account_id, str(member_id), session=db.session()))
@login_required
@console_ns.expect(console_ns.models[_ReplaceMemberRolesRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.MemberRolesResponse.__name__])
def put(self, member_id):
tenant_id, account_id = _current_ids()
request = _payload(_ReplaceMemberRolesRequest)
return _dump(
svc.RBACService.KnowledgeFSRoleMutations.replace_member_roles(
tenant_id,
account_id,
str(member_id),
role_ids=list(request.role_ids),
session=db.session(),
)
)
@console_ns.route("/workspaces/current/rbac/roles/<uuid:role_id>/members")
class ListMembersByRole(Resource):
@login_required
@console_ns.response(200, "Success", console_ns.models[_MembersInRoleList.__name__])
def get(self, role_id):
tenant_id, account_id = _current_ids()
return _dump(
svc.RBACService.Roles.list_members_by_role(tenant_id, role_id=role_id, options=_pagination_options())
)
api/controllers/console/workspace/rbac/__init__.py
+5
View File
@@ -0,0 +1,5 @@
"""Register workspace RBAC endpoints by use case."""
from controllers.console.workspace.rbac import members, policies, resources, roles
__all__ = ["members", "policies", "resources", "roles"]
api/controllers/console/workspace/rbac/members.py
+76
View File
@@ -0,0 +1,76 @@
"""Console RBAC members endpoints."""
from flask import request
from flask_restx import Resource
from werkzeug.exceptions import Forbidden
from controllers.common.errors import InvalidArgumentError
from controllers.common.rbac import rbac_language
from controllers.console import console_ns
from controllers.console.flask_admission import console_account_admission
from controllers.console.workspace.rbac.schemas import (
_ReplaceMemberRolesRequest,
)
from extensions.ext_application_services import application_services
from libs.helper import dump_response
from machinery.context import RequestContext
from services.account_errors import AccountNotFoundError
from services.errors.base import NoPermissionError
from services.errors.workspace import (
CannotOperateSelfError,
InvalidWorkspaceMemberRoleError,
MemberNotInTenantError,
RoleAlreadyAssignedError,
)
from services.rbac import contracts as dto
@console_ns.route("/workspaces/current/rbac/my-permissions")
class RBACMyPermissionsApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.MyPermissionsResponse.__name__])
def get(self, context: RequestContext):
return dump_response(
dto.MyPermissionsResponse,
application_services().rbac.members.permissions(
context.active_workspace_id,
context.account_id,
app_id=request.args.get("app_id") or None,
dataset_id=request.args.get("dataset_id") or None,
agent_id=request.args.get("agent_id") or None,
language=rbac_language(),
),
)
@console_ns.route("/workspaces/current/rbac/members/<uuid:member_id>/rbac-roles")
class RBACMemberRolesApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.MemberRolesResponse.__name__])
def get(self, context: RequestContext, member_id):
return dump_response(
dto.MemberRolesResponse,
application_services().rbac.members.get(context, str(member_id), language=rbac_language()),
)
@console_account_admission()
@console_ns.expect(console_ns.models[_ReplaceMemberRolesRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[dto.MemberRolesResponse.__name__])
def put(self, context: RequestContext, member_id):
request = _ReplaceMemberRolesRequest.model_validate(console_ns.payload or {})
try:
result = application_services().rbac.members.replace(
context, str(member_id), role_ids=list(request.role_ids), language=rbac_language()
)
except NoPermissionError as error:
raise Forbidden(str(error)) from error
except AccountNotFoundError as error:
raise InvalidArgumentError(description="Member not in tenant.") from error
except (
CannotOperateSelfError,
InvalidWorkspaceMemberRoleError,
MemberNotInTenantError,
RoleAlreadyAssignedError,
) as error:
raise InvalidArgumentError(description=str(error)) from error
return dump_response(dto.MemberRolesResponse, result)
api/controllers/console/workspace/rbac/policies.py
+114
View File
@@ -0,0 +1,114 @@
"""Console RBAC policies endpoints."""
from flask import request
from flask_restx import Resource
from controllers.common.fields import SimpleResultResponse
from controllers.common.rbac import RBACCheck, Workspace, rbac_language
from controllers.console import console_ns
from controllers.console.flask_admission import console_account_admission
from controllers.console.workspace.rbac.schemas import (
_AccessPolicyCreateRequest,
_AccessPolicyList,
_AccessPolicyUpdateRequest,
pagination_options,
)
from controllers.console.wraps import RBACPermission
from extensions.ext_application_services import application_services
from libs.helper import dump_response
from machinery.context import RequestContext
from services.rbac import contracts as dto
@console_ns.route("/workspaces/current/rbac/access-policies")
class RBACAccessPoliciesApi(Resource):
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(200, "Success", console_ns.models[_AccessPolicyList.__name__])
def get(self, context: RequestContext):
resource_type = request.args.get("resource_type") or None
return dump_response(
_AccessPolicyList,
application_services().rbac.policies.list(
context, resource_type=resource_type, options=pagination_options(), language=rbac_language()
),
)
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.expect(console_ns.models[_AccessPolicyCreateRequest.__name__])
@console_ns.response(201, "Policy created", console_ns.models[dto.AccessPolicy.__name__])
def post(self, context: RequestContext):
request = _AccessPolicyCreateRequest.model_validate(console_ns.payload or {})
policy = application_services().rbac.policies.create(
context,
dto.AccessPolicyCreate(
name=request.name,
resource_type=request.resource_type,
description=request.description,
permission_keys=list(request.permission_keys),
),
language=rbac_language(),
)
return (dump_response(dto.AccessPolicy, policy), 201)
@console_ns.route("/workspaces/current/rbac/access-policies/<uuid:policy_id>")
class RBACAccessPolicyItemApi(Resource):
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(200, "Success", console_ns.models[dto.AccessPolicy.__name__])
def get(self, context: RequestContext, policy_id):
return dump_response(
dto.AccessPolicy,
application_services().rbac.policies.get(context, str(policy_id), language=rbac_language()),
)
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.expect(console_ns.models[_AccessPolicyUpdateRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[dto.AccessPolicy.__name__])
def put(self, context: RequestContext, policy_id):
request = _AccessPolicyUpdateRequest.model_validate(console_ns.payload or {})
policy = application_services().rbac.policies.update(
context,
str(policy_id),
dto.AccessPolicyUpdate(
name=request.name, description=request.description, permission_keys=list(request.permission_keys)
),
language=rbac_language(),
)
return dump_response(dto.AccessPolicy, policy)
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
def delete(self, context: RequestContext, policy_id):
application_services().rbac.policies.delete(context, str(policy_id), language=rbac_language())
return dump_response(SimpleResultResponse, {"result": "success"})
@console_ns.route("/workspaces/current/rbac/access-policies/<uuid:policy_id>/copy")
class RBACAccessPolicyCopyApi(Resource):
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(201, "Policy copied", console_ns.models[dto.AccessPolicy.__name__])
def post(self, context: RequestContext, policy_id):
policy = application_services().rbac.policies.copy(context, str(policy_id), language=rbac_language())
return (dump_response(dto.AccessPolicy, policy), 201)
@console_ns.route("/workspaces/current/rbac/access-policy-bindings/<uuid:binding_id>/lock")
class RBACAccessPolicyBindingLockApi(Resource):
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(200, "Success", console_ns.models[dto.AccessPolicyBindingState.__name__])
def put(self, context: RequestContext, binding_id):
return dump_response(
dto.AccessPolicyBindingState,
application_services().rbac.policies.lock(context, str(binding_id), language=rbac_language()),
)
@console_ns.route("/workspaces/current/rbac/access-policy-bindings/<uuid:binding_id>/unlock")
class RBACAccessPolicyBindingUnlockApi(Resource):
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(200, "Success", console_ns.models[dto.AccessPolicyBindingState.__name__])
def put(self, context: RequestContext, binding_id):
return dump_response(
dto.AccessPolicyBindingState,
application_services().rbac.policies.unlock(context, str(binding_id), language=rbac_language()),
)
api/controllers/console/workspace/rbac/resources.py
+304
View File
@@ -0,0 +1,304 @@
"""Console RBAC resources endpoints."""
from collections.abc import Mapping
from dataclasses import dataclass
from flask_restx import Resource
from pydantic import BaseModel
from controllers.common.fields import SimpleResultResponse
from controllers.common.rbac import rbac_language
from controllers.common.schema import query_params_from_model
from controllers.console import console_ns
from controllers.console.flask_admission import console_account_admission
from controllers.console.workspace.rbac.schemas import (
_AccessControlLanguageQuery,
_DeleteMemberBindingsRequest,
_ReplaceBindingsRequest,
_ResourceAccessScopeRequest,
_ResourceUserAccessPoliciesQuery,
pagination_options,
)
from extensions.ext_application_services import application_services
from libs.helper import dump_response
from machinery.context import RequestContext
from services.rbac import contracts as dto
@dataclass(frozen=True)
class _ResourceAccessRoutes:
resource_type: dto.RBACResourceType
class_prefix: str
matrix_model: type[BaseModel]
@property
def url_segment(self) -> str:
return self.resource_type.route.segment
@property
def id_param(self) -> str:
return self.resource_type.route.id_param
@dataclass(frozen=True)
class _ResourceAccessApis:
catalog: type[Resource]
matrix: type[Resource]
whitelist: type[Resource]
whitelist_config: type[Resource]
user_access_policies: type[Resource]
user_access_policy_assignment: type[Resource]
role_bindings: type[Resource]
member_bindings: type[Resource]
workspace_matrix: type[Resource]
workspace_role_bindings: type[Resource]
workspace_bindings: type[Resource]
workspace_member_bindings: type[Resource]
_RESOURCE_ACCESS_ROUTES = (
_ResourceAccessRoutes(dto.RBACResourceType.APP, "App", dto.AppAccessMatrix),
_ResourceAccessRoutes(dto.RBACResourceType.DATASET, "Dataset", dto.DatasetAccessMatrix),
_ResourceAccessRoutes(dto.RBACResourceType.AGENT, "Agent", dto.AgentAccessMatrix),
)
def _build_resource_access_apis(spec: _ResourceAccessRoutes) -> _ResourceAccessApis:
id_param = spec.id_param
resource_prefix = f"/workspaces/current/rbac/{spec.url_segment}/<uuid:{id_param}>"
workspace_prefix = f"/workspaces/current/rbac/workspace/{spec.url_segment}"
def resource_id(path_params: Mapping[str, object]) -> str:
return str(path_params[id_param])
def register(resource: type[Resource], name: str, url: str) -> type[Resource]:
resource.__name__ = name
resource.__qualname__ = name
console_ns.route(url)(resource)
return resource
class CatalogApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.PermissionCatalogResponse.__name__])
def get(self, context: RequestContext):
return dump_response(
dto.PermissionCatalogResponse,
application_services().rbac.resources.catalog(context, spec.resource_type, language=rbac_language()),
)
class MatrixApi(Resource):
@console_account_admission()
@console_ns.doc(params=query_params_from_model(_AccessControlLanguageQuery))
@console_ns.response(200, "Success", console_ns.models[spec.matrix_model.__name__])
def get(self, context: RequestContext, **path_params):
result = application_services().rbac.resources.matrix(
context, spec.resource_type, resource_id(path_params), language=rbac_language()
)
return dump_response(spec.matrix_model, result)
class WhitelistApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.ResourceWhitelist.__name__])
def get(self, context: RequestContext, **path_params):
return dump_response(
dto.ResourceWhitelist,
application_services().rbac.resources.whitelist(
context, spec.resource_type, resource_id(path_params), language=rbac_language()
),
)
@console_account_admission()
@console_ns.expect(console_ns.models[_ResourceAccessScopeRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[dto.ResourceWhitelist.__name__])
def put(self, context: RequestContext, **path_params):
target_id = resource_id(path_params)
scope = _ResourceAccessScopeRequest.model_validate(console_ns.payload or {})
result = application_services().rbac.resources.replace_whitelist(
context,
spec.resource_type,
target_id,
dto.ReplaceMemberBindings(
automatic_include_workspace_members=scope.automatic_include_workspace_members
),
language=rbac_language(),
)
return dump_response(dto.ResourceWhitelist, result)
class WhitelistConfigApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.ResourceWhitelistConfig.__name__])
def get(self, context: RequestContext, **path_params):
return dump_response(
dto.ResourceWhitelistConfig,
application_services().rbac.resources.whitelist_config(
context, spec.resource_type, resource_id(path_params), language=rbac_language()
),
)
class UserAccessPoliciesApi(Resource):
@console_account_admission()
@console_ns.doc(params=query_params_from_model(_ResourceUserAccessPoliciesQuery))
@console_ns.response(200, "Success", console_ns.models[dto.ResourceUserAccessPoliciesResponse.__name__])
def get(self, context: RequestContext, **path_params):
target_id = resource_id(path_params)
options = pagination_options()
result = application_services().rbac.resources.user_access_policies(
context, spec.resource_type, target_id, options=options, language=rbac_language()
)
return dump_response(dto.ResourceUserAccessPoliciesResponse, result)
class UserAccessPolicyAssignmentApi(Resource):
@console_account_admission()
@console_ns.expect(console_ns.models[dto.ReplaceUserAccessPolicies.__name__])
@console_ns.response(200, "Success", console_ns.models[dto.ReplaceUserAccessPoliciesResponse.__name__])
def put(self, context: RequestContext, target_account_id, **path_params):
payload = dto.ReplaceUserAccessPolicies.model_validate(console_ns.payload or {})
return dump_response(
dto.ReplaceUserAccessPoliciesResponse,
application_services().rbac.resources.replace_user_access_policies(
context,
spec.resource_type,
resource_id(path_params),
str(target_account_id),
payload,
language=rbac_language(),
),
)
class RoleBindingsApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.RoleBindingsResponse.__name__])
def get(self, context: RequestContext, policy_id, **path_params):
return dump_response(
dto.RoleBindingsResponse,
application_services().rbac.resources.list_role_bindings(
context, spec.resource_type, resource_id(path_params), str(policy_id), language=rbac_language()
),
)
class MemberBindingsApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.MemberBindingsResponse.__name__])
def get(self, context: RequestContext, policy_id, **path_params):
return dump_response(
dto.MemberBindingsResponse,
application_services().rbac.resources.list_member_bindings(
context, spec.resource_type, resource_id(path_params), str(policy_id), language=rbac_language()
),
)
@console_account_admission()
@console_ns.expect(console_ns.models[_DeleteMemberBindingsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
def delete(self, context: RequestContext, policy_id, **path_params):
body = _DeleteMemberBindingsRequest.model_validate(console_ns.payload or {})
application_services().rbac.resources.delete_member_bindings(
context,
spec.resource_type,
resource_id(path_params),
str(policy_id),
dto.DeleteMemberBindings(account_ids=body.account_ids),
language=rbac_language(),
)
return dump_response(SimpleResultResponse, {"result": "success"})
class WorkspaceMatrixApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.WorkspaceAccessMatrix.__name__])
def get(self, context: RequestContext):
result = application_services().rbac.resources.workspace_matrix(
context, spec.resource_type, pagination_options(), language=rbac_language()
)
return dump_response(dto.WorkspaceAccessMatrix, result)
class WorkspaceRoleBindingsApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.RoleBindingsResponse.__name__])
def get(self, context: RequestContext, policy_id):
return dump_response(
dto.RoleBindingsResponse,
application_services().rbac.resources.workspace_role_bindings(
context, spec.resource_type, str(policy_id), language=rbac_language()
),
)
class WorkspaceBindingsApi(Resource):
@console_account_admission()
@console_ns.expect(console_ns.models[_ReplaceBindingsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[dto.AccessMatrixItem.__name__])
def put(self, context: RequestContext, policy_id):
body = _ReplaceBindingsRequest.model_validate(console_ns.payload or {})
return dump_response(
dto.AccessMatrixItem,
application_services().rbac.resources.replace_workspace_bindings(
context,
spec.resource_type,
str(policy_id),
dto.ReplaceBindings(role_ids=list(body.role_ids), account_ids=list(body.account_ids)),
language=rbac_language(),
),
)
class WorkspaceMemberBindingsApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.MemberBindingsResponse.__name__])
def get(self, context: RequestContext, policy_id):
return dump_response(
dto.MemberBindingsResponse,
application_services().rbac.resources.workspace_member_bindings(
context, spec.resource_type, str(policy_id), language=rbac_language()
),
)
prefix = spec.class_prefix
return _ResourceAccessApis(
catalog=register(
CatalogApi,
f"RBAC{prefix}CatalogApi",
f"/workspaces/current/rbac/role-permissions/catalog/{spec.resource_type.value}",
),
matrix=register(MatrixApi, f"RBAC{prefix}MatrixApi", f"{resource_prefix}/access-policy"),
whitelist=register(WhitelistApi, f"RBAC{prefix}WhitelistApi", f"{resource_prefix}/whitelist"),
whitelist_config=register(
WhitelistConfigApi, f"RBAC{prefix}WhitelistConfigApi", f"{resource_prefix}/whitelist_config"
),
user_access_policies=register(
UserAccessPoliciesApi, f"RBAC{prefix}UserAccessPoliciesApi", f"{resource_prefix}/user-access-policies"
),
user_access_policy_assignment=register(
UserAccessPolicyAssignmentApi,
f"RBAC{prefix}UserAccessPolicyAssignmentApi",
f"{resource_prefix}/users/<uuid:target_account_id>/access-policies",
),
role_bindings=register(
RoleBindingsApi,
f"RBAC{prefix}RoleBindingsApi",
f"{resource_prefix}/access-policies/<uuid:policy_id>/role-bindings",
),
member_bindings=register(
MemberBindingsApi,
f"RBAC{prefix}MemberBindingsApi",
f"{resource_prefix}/access-policies/<string:policy_id>/member-bindings",
),
workspace_matrix=register(
WorkspaceMatrixApi, f"RBACWorkspace{prefix}MatrixApi", f"{workspace_prefix}/access-policy"
),
workspace_role_bindings=register(
WorkspaceRoleBindingsApi,
f"RBACWorkspace{prefix}RoleBindingsApi",
f"{workspace_prefix}/access-policies/<uuid:policy_id>/role-bindings",
),
workspace_bindings=register(
WorkspaceBindingsApi,
f"RBACWorkspace{prefix}BindingsApi",
f"{workspace_prefix}/access-policies/<uuid:policy_id>/bindings",
),
workspace_member_bindings=register(
WorkspaceMemberBindingsApi,
f"RBACWorkspace{prefix}MemberBindingsApi",
f"{workspace_prefix}/access-policies/<uuid:policy_id>/member-bindings",
),
)
_RESOURCE_ACCESS_APIS = {spec.resource_type: _build_resource_access_apis(spec) for spec in _RESOURCE_ACCESS_ROUTES}
api/controllers/console/workspace/rbac/roles.py
+101
View File
@@ -0,0 +1,101 @@
"""Console RBAC roles endpoints."""
from flask import request
from flask_restx import Resource
from controllers.common.fields import SimpleResultResponse
from controllers.common.rbac import RBACCheck, Workspace, rbac_language
from controllers.console import console_ns
from controllers.console.flask_admission import console_account_admission
from controllers.console.workspace.rbac.schemas import (
CopyRoleParam,
_MembersInRoleList,
_RBACRoleList,
_RolesListQuery,
_RoleUpsertRequest,
pagination_options,
)
from controllers.console.wraps import RBACPermission
from extensions.ext_application_services import application_services
from libs.helper import dump_response
from machinery.context import RequestContext
from services.rbac import contracts as dto
@console_ns.route("/workspaces/current/rbac/role-permissions/catalog")
class RBACWorkspaceCatalogApi(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[dto.PermissionCatalogResponse.__name__])
def get(self, context: RequestContext):
return dump_response(
dto.PermissionCatalogResponse, application_services().rbac.roles.catalog(context, language=rbac_language())
)
@console_ns.route("/workspaces/current/rbac/roles")
class RBACRolesApi(Resource):
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(200, "Success", console_ns.models[_RBACRoleList.__name__])
def get(self, context: RequestContext):
req_data = _RolesListQuery.model_validate(request.args.to_dict(flat=True))
result = application_services().rbac.roles.list(
context, options=req_data.to_inner_options(), include_owner=req_data.include_owner, language=rbac_language()
)
return dump_response(_RBACRoleList, result)
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(201, "Role created", console_ns.models[dto.RBACRole.__name__])
def post(self, context: RequestContext):
request = _RoleUpsertRequest.model_validate(console_ns.payload or {})
role = application_services().rbac.roles.create(context, request.to_mutation(), language=rbac_language())
return (dump_response(dto.RBACRole, role), 201)
@console_ns.route("/workspaces/current/rbac/roles/<uuid:role_id>")
class RBACRoleItemApi(Resource):
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(200, "Success", console_ns.models[dto.RBACRole.__name__])
def get(self, context: RequestContext, role_id):
return dump_response(
dto.RBACRole, application_services().rbac.roles.get(context, str(role_id), language=rbac_language())
)
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(200, "Success", console_ns.models[dto.RBACRole.__name__])
def put(self, context: RequestContext, role_id):
request = _RoleUpsertRequest.model_validate(console_ns.payload or {})
role = application_services().rbac.roles.update(
context, str(role_id), request.to_mutation(), language=rbac_language()
)
return dump_response(dto.RBACRole, role)
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
def delete(self, context: RequestContext, role_id):
application_services().rbac.roles.delete(context, str(role_id), language=rbac_language())
return dump_response(SimpleResultResponse, {"result": "success"})
@console_ns.route("/workspaces/current/rbac/roles/<uuid:role_id>/copy")
class RBACRoleCopyApi(Resource):
@console_account_admission(rbac_checks=[RBACCheck(RBACPermission.WORKSPACE_ROLE_MANAGE, Workspace())])
@console_ns.response(201, "Role copied", console_ns.models[dto.RBACRole.__name__])
def post(self, context: RequestContext, role_id):
request = CopyRoleParam.model_validate(console_ns.payload or {})
role = application_services().rbac.roles.copy(
context, str(role_id), copy_member=request.copy_member, language=rbac_language()
)
return (dump_response(dto.RBACRole, role), 201)
@console_ns.route("/workspaces/current/rbac/roles/<uuid:role_id>/members")
class ListMembersByRole(Resource):
@console_account_admission()
@console_ns.response(200, "Success", console_ns.models[_MembersInRoleList.__name__])
def get(self, context: RequestContext, role_id):
return dump_response(
_MembersInRoleList,
application_services().rbac.roles.list_members_by_role(
context, role_id=str(role_id), options=pagination_options(), language=rbac_language()
),
)
api/controllers/console/workspace/rbac/schemas.py
+163
View File
@@ -0,0 +1,163 @@
"""HTTP payloads, pagination aliases and response schemas for workspace RBAC."""
from typing import Any, Literal
from flask import request
from pydantic import AliasChoices, BaseModel, ConfigDict, Field, field_validator
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console import console_ns
from services.rbac import contracts as dto
class _RBACRoleList(dto.Paginated[dto.RBACRole]):
pass
class _AccessPolicyList(dto.Paginated[dto.AccessPolicy]):
pass
class _MembersInRoleList(dto.Paginated[dto.MembersInRole]):
pass
register_response_schema_models(
console_ns,
dto.PermissionCatalogResponse,
dto.RBACRole,
_RBACRoleList,
_MembersInRoleList,
dto.AccessPolicy,
_AccessPolicyList,
dto.AccessPolicyBindingState,
dto.MyPermissionsResponse,
dto.AppAccessMatrix,
dto.DatasetAccessMatrix,
dto.AgentAccessMatrix,
dto.WorkspaceAccessMatrix,
dto.ResourceWhitelist,
dto.ResourceWhitelistConfig,
dto.ResourceUserAccessPoliciesResponse,
dto.ReplaceUserAccessPoliciesResponse,
dto.RoleBindingsResponse,
dto.MemberBindingsResponse,
dto.MemberRolesResponse,
dto.AccessMatrixItem,
)
class _PaginationQuery(BaseModel):
model_config = ConfigDict(extra="ignore")
page_number: int | None = Field(default=None, ge=1, validation_alias=AliasChoices("page", "page_number"))
results_per_page: int | None = Field(
default=None, ge=1, le=99999, validation_alias=AliasChoices("limit", "results_per_page")
)
reverse: bool | None = None
def to_inner_options(self) -> dto.ListOption:
return dto.ListOption.model_validate(self.model_dump())
class _RolesListQuery(_PaginationQuery):
include_owner: int = Field(default=0, ge=0, le=1)
class CopyRoleParam(BaseModel):
copy_member: bool = True
class _RoleUpsertRequest(BaseModel):
"""Accepts the payload sent by the Create/Edit Role dialog."""
name: str
description: str = ""
permission_keys: list[str] = []
def to_mutation(self) -> dto.RoleMutation:
return dto.RoleMutation(
name=self.name,
description=self.description,
permission_keys=list(self.permission_keys),
)
class _AccessPolicyCreateRequest(BaseModel):
name: str
resource_type: dto.RBACResourceType
description: str = ""
permission_keys: list[str] = []
class _AccessPolicyUpdateRequest(BaseModel):
name: str
description: str = ""
permission_keys: list[str] = []
register_schema_models(console_ns, _AccessPolicyCreateRequest, _AccessPolicyUpdateRequest)
class _ResourceAccessScopeRequest(BaseModel):
automatic_include_workspace_members: bool
class _ReplaceBindingsRequest(BaseModel):
role_ids: list[str] = Field(default_factory=list)
account_ids: list[str] = Field(default_factory=list)
@field_validator("role_ids", "account_ids", mode="before")
@classmethod
def _coerce_bindings(cls, value: Any) -> list[str]:
if value is None:
return []
return value
class _DeleteMemberBindingsRequest(BaseModel):
account_ids: list[str] = Field(default_factory=list)
@field_validator("account_ids", mode="before")
@classmethod
def _coerce_account_ids(cls, value: Any) -> list[str]:
if value is None:
return []
return value
class _AccessControlLanguageQuery(BaseModel):
language: Literal["en", "ja", "zh"] | None = Field(default=None, description="Localized policy label language")
class _ResourceUserAccessPoliciesQuery(_PaginationQuery):
language: Literal["en", "ja", "zh"] | None = Field(default=None, description="Localized policy label language")
register_schema_models(
console_ns,
_ResourceAccessScopeRequest,
_ReplaceBindingsRequest,
_DeleteMemberBindingsRequest,
_AccessControlLanguageQuery,
_ResourceUserAccessPoliciesQuery,
dto.ReplaceUserAccessPolicies,
)
class _ReplaceMemberRolesRequest(BaseModel):
role_ids: list[str] = []
@field_validator("role_ids", mode="before")
@classmethod
def _coerce_role_ids(cls, value: Any) -> list[str]:
if value is None:
return []
return value
register_schema_models(console_ns, _ReplaceMemberRolesRequest)
def pagination_options() -> dto.ListOption:
return _PaginationQuery.model_validate(request.args.to_dict(flat=True)).to_inner_options()
api/controllers/inner_api/agent/llm.py
+1 -1
View File
@@ -73,7 +73,7 @@ class AgentLLMInvokeApi(Resource):
status_code=400,
) from exc
def generate() -> Generator[str, None, None]:
def generate() -> Generator[str]:
try:
for chunk in service.invoke(prepared):
envelope = {"code": 0, "message": "", "data": chunk.model_dump(mode="json")}
api/controllers/openapi/_hints.py
+1 -1
View File
@@ -62,7 +62,7 @@ def _wanted_event(chunk: str, event: str) -> dict[str, Any] | None:
return parsed
def attach_stream_hints(events: Iterable[str], *, event: str, build: HintBuilder) -> Generator[str, None, None]:
def attach_stream_hints(events: Iterable[str], *, event: str, build: HintBuilder) -> Generator[str]:
"""Yield the source SSE chunks, adding a top-level `hints` list to every `event` that `build` hints.
`event: ping` chunks, other event kinds, events `build` returns nothing for, and events
api/controllers/openapi/app_run.py
+11 -7
View File
@@ -46,7 +46,7 @@ from controllers.openapi.auth.requirements import (
CheckSubject,
CheckWorkspaceMember,
)
from controllers.openapi.auth.subjects import AccountSubject, ExternalSsoSubject
from controllers.openapi.auth.subjects import AccountSubject, ExternalSsoSubject, ResourceAccessSubject
from controllers.openapi.human_input_form import with_form_hints
from controllers.service_api.app.error import (
AppUnavailableError,
@@ -91,7 +91,7 @@ _INVALID_RUN_INPUT: Final = "invalid run input"
@contextmanager
def _translate_service_errors() -> Generator[None, None, None]:
def _translate_service_errors() -> Generator[None]:
try:
yield
except WorkflowNotFoundError as ex:
@@ -133,7 +133,7 @@ def _translate_service_errors() -> Generator[None, None, None]:
_RUN_GUARDS: Final = (
CheckSubject(allowed=(AccountSubject, ExternalSsoSubject)),
CheckSubject(allowed=(AccountSubject, ExternalSsoSubject, ResourceAccessSubject)),
CheckAppApiEnabled(),
CheckWorkspaceMember(),
CheckScope(Scope.APPS_RUN),
@@ -191,7 +191,7 @@ class _ChatMessageEnd(MessageEndStreamResponse):
conversation_id: str
def with_reply_hints(events: Iterable[str], *, op: str, app_id: str) -> Generator[str, None, None]:
def with_reply_hints(events: Iterable[str], *, op: str, app_id: str) -> Generator[str]:
def build(event: Mapping[str, Any]) -> list[Hint]:
end = _ChatMessageEnd.model_validate(event)
return [
@@ -205,14 +205,14 @@ def with_reply_hints(events: Iterable[str], *, op: str, app_id: str) -> Generato
return attach_stream_hints(events, event=StreamEvent.MESSAGE_END.value, build=build)
HintLayer = Callable[[Iterable[str], str, str], Generator[str, None, None]]
HintLayer = Callable[[Iterable[str], str, str], Generator[str]]
def _reply_layer(events: Iterable[str], op: str, app_id: str) -> Generator[str, None, None]:
def _reply_layer(events: Iterable[str], op: str, app_id: str) -> Generator[str]:
return with_reply_hints(events, op=op, app_id=app_id)
def _form_layer(events: Iterable[str], op: str, app_id: str) -> Generator[str, None, None]:
def _form_layer(events: Iterable[str], op: str, app_id: str) -> Generator[str]:
return with_form_hints(events, app_id=app_id)
@@ -370,6 +370,10 @@ class AppRunTaskStopApi(Resource):
returns=(200, TaskStopResponse, "Task stopped"),
)
def post(self, ctx: Context, app_id: str, task_id: str):
if isinstance(ctx.subject, ResourceAccessSubject):
owner = redis_client.get(AppQueueManager._generate_task_belong_cache_key(task_id))
if owner != f"end-user-{ctx.end_user.id}".encode():
raise NotFound("Task not found")
AppQueueManager.set_stop_flag_no_user_check(task_id)
GraphEngineManager(redis_client).send_stop_command(task_id)
return TaskStopResponse(result="success")
api/controllers/openapi/apps.py
+13 -6
View File
@@ -33,7 +33,7 @@ from controllers.openapi.auth.requirements import (
CheckSubject,
CheckWorkspaceMember,
)
from controllers.openapi.auth.subjects import AccountSubject
from controllers.openapi.auth.subjects import AccountSubject, ResourceAccessSubject
from controllers.service_api.app.error import AppUnavailableError
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
from extensions.ext_application_services import application_services
@@ -115,7 +115,7 @@ class AppDescribeApi(Resource):
),
),
requirements=(
CheckSubject(allowed=(AccountSubject,)),
CheckSubject(allowed=(AccountSubject, ResourceAccessSubject)),
CheckAppApiEnabled(),
CheckWorkspaceMember(),
CheckScope(Scope.APPS_READ),
@@ -143,7 +143,7 @@ class AppListApi(Resource):
),
),
requirements=(
CheckSubject(allowed=(AccountSubject,)),
CheckSubject(allowed=(AccountSubject, ResourceAccessSubject)),
CheckScope(Scope.APPS_READ),
CheckWorkspaceMember(),
),
@@ -152,7 +152,7 @@ class AppListApi(Resource):
)
def get(self, ctx: Context, *, query: AppListQuery):
workspace_id = query.workspace_id
account_id = str(ctx.subject.account_id)
account_id = str(ctx.subject.account_id or ctx.subject.token_id)
empty = AppListResponse.build(page=query.page, limit=query.limit, total=0, items=[])
@@ -165,16 +165,23 @@ class AppListApi(Resource):
parsed_uuid = None
access_filter = (
resolve_app_access_filter(workspace_id, account_id, session=ctx.session)
if dify_config.RBAC_ENABLED
resolve_app_access_filter(workspace_id, account_id)
if dify_config.RBAC_ENABLED and isinstance(ctx.subject, AccountSubject)
else AppAccessFilter.unrestricted()
)
if ctx.resource_app_ids is not None:
access_filter = AppAccessFilter(set(ctx.resource_app_ids), can_manage_own_apps=False)
tenant_name: str | None = None
if parsed_uuid is not None:
app = application_services().apps.queries.get_visible_app_by_id(str(parsed_uuid), workspace_id)
if app is None or str(app.tenant_id) != workspace_id:
return empty
if ctx.resource_app_ids is not None and (
str(app.id) not in ctx.resource_app_ids or app.status != AppStatus.NORMAL
):
return empty
if not _is_listable(app):
return empty
# Apply RBAC visibility to the UUID fast-path the same way the service
api/controllers/openapi/auth/context.py
+1
View File
@@ -43,6 +43,7 @@ class Context:
view_args: Mapping[str, str]
request_id: str = field(default_factory=get_request_id)
trace_id: str | None = field(default_factory=get_trace_id)
resource_app_ids: frozenset[str] | None = field(default=None, init=False)
_app: App | None = field(default=None, init=False)
_workspace: Tenant | None = field(default=None, init=False)
_workspace_role: TenantAccountRole | None = field(default=None, init=False)
api/controllers/openapi/auth/pipelines.py
+6 -1
View File
@@ -19,8 +19,9 @@ from controllers.openapi.auth.requirements import (
Requirement,
ResolveCaller,
)
from controllers.openapi.auth.resource_access import CheckResourceAccess
from controllers.openapi.auth.spec import EndpointSpec
from controllers.openapi.auth.subjects import AccountSubject, ExternalSsoSubject, Subject
from controllers.openapi.auth.subjects import AccountSubject, ExternalSsoSubject, ResourceAccessSubject, Subject
from core.logging.context import get_request_id, get_trace_id
from enums import DeploymentEdition
from libs.oauth_bearer import AuthContext, reset_auth_ctx, set_auth_ctx
@@ -89,6 +90,10 @@ class AccountPipeline(Pipeline, serves=AccountSubject):
fixed = (_RequiresCurrentCatalog(), ResolveCaller())
class ResourceAccessPipeline(Pipeline, serves=ResourceAccessSubject):
fixed = (_RequiresCurrentCatalog(), CheckResourceAccess(), ResolveCaller())
class _RequiresEnterprise(Requirement):
"""A gate on the token kind, not on a route, so no endpoint declares it.
It runs after `authenticate` on purpose: a `dfoe_` string no row backs
api/controllers/openapi/auth/requirements.py
+3 -1
View File
@@ -24,7 +24,7 @@ from controllers.openapi._audit import emit_wrong_surface
from controllers.openapi._errors import MemberLicenseExceeded, MemberLimitExceeded
from controllers.openapi.auth.context import Context
from controllers.openapi.auth.loaders import load_app, load_caller, load_workspace, load_workspace_role
from controllers.openapi.auth.subjects import Subject
from controllers.openapi.auth.subjects import ResourceAccessSubject, Subject
from enums import DeploymentEdition
from extensions.ext_application_services import application_services
from models.account import TenantAccountRole
@@ -184,6 +184,8 @@ class CheckAppAccess(Requirement):
@override
def run(self, subject: Subject, ctx: Context, session: Session) -> None:
if isinstance(subject, ResourceAccessSubject):
return
if dify_config.DEPLOYMENT_EDITION != DeploymentEdition.ENTERPRISE:
return
access_mode = self._access_mode(load_app(ctx).id)
api/controllers/openapi/auth/resource_access.py
+65
View File
@@ -0,0 +1,65 @@
"""Adapt machine credential authentication and authorization to OpenAPI admission."""
import uuid
from typing import override
from flask import request
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden, Unauthorized
from controllers.common.resource_access_token_errors import resource_access_token_errors
from controllers.openapi.auth.context import Context
from controllers.openapi.auth.requirements import Rank, Requirement
from controllers.openapi.auth.subjects import Subject
from extensions.ext_application_services import application_services
from libs.oauth_bearer import AuthContext, TokenType, sha256_hex
from libs.rate_limit import enforce_bearer_rate_limit
from models.account import TenantStatus
from models.enums import AppStatus
from services.app_service import AppService
from services.auth.resource_access_token_contracts import ResourceAccessTokenInvalidError
def authenticate_resource_token(token: str) -> AuthContext:
enforce_bearer_rate_limit(sha256_hex(token))
try:
access = application_services().resource_access_tokens.authenticate(token)
except ResourceAccessTokenInvalidError as error:
raise Unauthorized("invalid_bearer") from error
return AuthContext(
subject_email=None,
subject_issuer=None,
account_id=None,
client_id=None,
token_id=uuid.UUID(access.token_id),
token_type=TokenType.RESOURCE_ACCESS,
expires_at=None,
)
class CheckResourceAccess(Requirement):
"""Load ORM context only after the service has authorized live bindings."""
rank = Rank.FIRST
@override
def run(self, subject: Subject, ctx: Context, session: Session) -> None:
app_id = ctx.view_args.get("app_id")
with resource_access_token_errors():
grant = application_services().resource_access_tokens.authorize_openapi(
token_id=str(subject.token_id),
workspace_id=ctx.view_args.get("workspace_id") or request.args.get("workspace_id"),
app_id=app_id,
)
# The current OpenAPI pipeline still needs ORM context. Reconstruct it
# within admission's session using the authorized owner chain.
tenant = application_services().workspaces.identity.get_workspace(grant.tenant_id)
if tenant is None or tenant.status != TenantStatus.NORMAL:
raise Forbidden("workspace unavailable")
ctx._workspace = tenant
ctx.resource_app_ids = grant.app_ids
if app_id:
app = AppService.get_app_in_workspace(tenant_id=grant.tenant_id, app_id=app_id, session=session)
if app is None or app.status != AppStatus.NORMAL or not app.enable_api:
raise Forbidden("resource_not_authorized")
ctx._app = app
api/controllers/openapi/auth/router.py
+4 -1
View File
@@ -11,12 +11,14 @@ from configs import dify_config
from controllers.openapi.auth.context import Context
from controllers.openapi.auth.pipelines import pipeline_for_subject
from controllers.openapi.auth.requirements import assert_license_valid
from controllers.openapi.auth.resource_access import authenticate_resource_token
from controllers.openapi.auth.spec import EndpointSpec
from controllers.openapi.auth.subjects import subject_from_auth
from core.db.session_factory import session_factory
from core.logging.context import get_trace_id
from enums import DeploymentEdition
from libs.oauth_bearer import InvalidBearerError, assert_bearer_feature_enabled, extract_bearer, get_authenticator
from services.auth.resource_access_token_contracts import is_resource_access_token
class AuthRouter:
@@ -45,11 +47,12 @@ class AuthRouter:
assert_bearer_feature_enabled()
token = extract_bearer(request)
is_resource_token = bool(token and is_resource_access_token(token))
if not token:
raise Unauthorized("bearer required")
try:
auth = get_authenticator().authenticate(token)
auth = authenticate_resource_token(token) if is_resource_token else get_authenticator().authenticate(token)
except InvalidBearerError:
# One answer for every rejection reason - unknown prefix, no live row,
# expired - so a caller cannot probe which one it hit. Same reasoning as
api/controllers/openapi/auth/subjects.py
+23
View File
@@ -146,5 +146,28 @@ class ExternalSsoSubject(Subject):
return account.id if account is not None else None
class ResourceAccessSubject(Subject):
subject_type = SubjectType.RESOURCE_ACCESS
caller_role = CreatorUserRole.END_USER
webapp_modes = frozenset[WebAppAccessMode]()
@override
def resolve_caller(self, ctx: Context, session: Session) -> EndUser:
return application_services().app_scoped_end_users.commands.get_or_create_end_user_by_type(
EndUserType.OPENAPI,
tenant_id=ctx.workspace.id,
app_id=ctx.app.id,
user_id=f"resource-token:{self.token_id}",
)
@override
def mounts_caller(self, ctx: Context) -> bool:
return route_has_app(ctx)
@override
def webapp_user_id(self, session: Session) -> str | None:
return None
def subject_from_auth(auth: AuthContext) -> Subject:
return _SUBJECT_CLASSES[auth.subject_type](auth)
api/controllers/openapi/human_input_form.py
+1 -1
View File
@@ -219,7 +219,7 @@ def form_hints(*, op: str, app_id: str, response: HumanInputRequiredResponse) ->
]
def with_form_hints(events: Iterable[str], *, app_id: str) -> Generator[str, None, None]:
def with_form_hints(events: Iterable[str], *, app_id: str) -> Generator[str]:
"""A run stream whose `human_input_required` events carry hints that target the submit route above."""
op = op_of(OpenApiWorkflowHumanInputFormSubmitApi.post)
api/controllers/openapi/workflow_events.py
+1 -1
View File
@@ -118,7 +118,7 @@ class OpenApiWorkflowEventsApi(Resource):
payload = response.model_dump(mode="json")
payload["event"] = response.event.value
def _generate_finished_events() -> Generator[str, None, None]:
def _generate_finished_events() -> Generator[str]:
yield f"data: {json.dumps(payload)}\n\n"
event_generator = _generate_finished_events
api/controllers/openapi/workspaces.py
+10 -3
View File
@@ -3,7 +3,8 @@
Bearer-authed counterparts to the cookie-authed /console/api/workspaces
endpoints. Account bearers (dfoa_) see every tenant they're a member of.
External SSO bearers (dfoe_) have no account_id and so see an empty list —
that matches /openapi/v1/account.
that matches /openapi/v1/account. Resource access bearers may list only their
own tenant, without inheriting any account membership or role.
Member management declares both authorization arms; ``RBAC_ENABLED`` picks one.
``GET /workspaces/<workspace_id>`` deliberately declares neither: it admits any
@@ -45,7 +46,7 @@ from controllers.openapi.auth.requirements import (
CheckWorkspaceMember,
CheckWorkspaceRole,
)
from controllers.openapi.auth.subjects import AccountSubject
from controllers.openapi.auth.subjects import AccountSubject, ResourceAccessSubject
from enums.account import TenantAccountRole
from extensions.ext_application_services import application_services
from services.account_errors import AccountNotFoundError, AccountRegisterError, SeatsLimitExceededError
@@ -79,11 +80,17 @@ class WorkspacesApi(Resource):
kind=Kind.LIST,
summary="List workspaces of the current account",
examples=(Example(title="List my workspaces, first page", input={"page": 1, "limit": 20}),),
requirements=(CheckSubject(allowed=(AccountSubject,)), CheckScope(Scope.WORKSPACE_READ)),
requirements=(CheckSubject(allowed=(AccountSubject, ResourceAccessSubject)), CheckScope(Scope.WORKSPACE_READ)),
query=WorkspaceListQuery,
returns=(200, WorkspaceListResponse, "Workspace list"),
)
def get(self, ctx: Context, *, query: WorkspaceListQuery):
if isinstance(ctx.subject, ResourceAccessSubject):
tenant = ctx.workspace
item = WorkspaceSummaryResponse(
id=str(tenant.id), name=tenant.name, role="", status=tenant.status, current=True
)
return WorkspaceListResponse.page_of([item], query=query)
rows = application_services().workspaces.management.list_memberships(str(ctx.subject.account_id))
return WorkspaceListResponse.page_of([_workspace_summary(row) for row in rows], query=query)
api/controllers/service_api/app/workflow_events.py
+1 -1
View File
@@ -128,7 +128,7 @@ class WorkflowEventsApi(Resource):
payload = response.model_dump(mode="json")
payload["event"] = response.event.value
def _generate_finished_events() -> Generator[str, None, None]:
def _generate_finished_events() -> Generator[str]:
yield f"data: {json.dumps(payload)}\n\n"
event_generator = _generate_finished_events
api/controllers/service_api/dataset/dataset.py
+2 -1
View File
@@ -56,6 +56,7 @@ from services.knowledge.entities.knowledge_entities import (
RetrievalModel,
SummaryIndexSetting,
)
from services.rbac import contracts as rbac_contracts
from services.tag_service import (
SaveTagPayload,
TagBindingCreatePayload,
@@ -560,7 +561,7 @@ class DatasetListApi(DatasetApiResource):
tenant_id,
current_user.id,
dataset.id,
enterprise_rbac_service.ReplaceMemberBindings(automatic_include_workspace_members=False),
rbac_contracts.ReplaceMemberBindings(automatic_include_workspace_members=False),
)
return _dump_service_dataset_detail(load_dataset_detail(dataset, session=session)), 200
api/controllers/service_api/dataset/rag_pipeline/rag_pipeline_workflow.py
+1 -1
View File
@@ -315,7 +315,7 @@ class PipelineRunApi(DatasetApiResource):
rag_pipeline_service = RagPipelineService(session)
pipeline = rag_pipeline_service.get_pipeline(tenant_id=tenant_id, dataset_id=dataset_id_str)
try:
response: dict[Any, Any] | Generator[str, Any, None] = PipelineGenerateService.generate(
response: dict[Any, Any] | Generator[str, Any] = PipelineGenerateService.generate(
generator=application_services().knowledge.pipeline_generator,
session=session,
pipeline=pipeline,
api/controllers/service_api/wraps.py
+69 -32
View File
@@ -15,6 +15,7 @@ from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden, NotFound, ServiceUnavailable, Unauthorized
from configs import dify_config
from controllers.common.resource_access_token_errors import resource_access_token_errors
from controllers.service_api.schema import (
USER_FETCH_FROM_ATTR,
USER_FORM_PARAM,
@@ -27,9 +28,11 @@ from extensions.ext_database import db
from libs.login import current_user
from models import Account, Tenant, TenantAccountJoin, TenantStatus
from models.dataset import Dataset
from models.model import ApiToken, App
from models.model import ApiToken
from repositories.knowledge import dataset_api_key_bindings
from services.api_token_service import ApiTokenCache, fetch_token_with_single_flight, record_token_usage
from services.app_service import AppService
from services.auth.resource_access_token_contracts import is_resource_access_token
from services.feature_service import FeatureService
from services.knowledge_rate_limit_service import KnowledgeRateLimitExceededError, enforce_knowledge_rate_limit
@@ -108,9 +111,21 @@ def validate_app_token[**P, R](
def decorator(view_func: Callable[P, R]) -> Callable[P, R]:
@wraps(view_func)
def decorated_view(*args: P.args, **kwargs: P.kwargs) -> R:
api_token = validate_and_get_api_token("app")
app_model = db.session.get(App, api_token.app_id)
auth_token = peek_service_api_bearer_token()
if auth_token and is_resource_access_token(auth_token):
with resource_access_token_errors():
grant = application_services().resource_access_tokens.resolve_app_for_service_api(
token=auth_token,
requested_app_id=request.headers.get("X-Dify-App-ID"),
)
app_model = AppService.get_app_in_workspace(
tenant_id=grant.tenant_id,
app_id=next(iter(grant.app_ids)),
session=db.session(),
)
else:
api_token = validate_and_get_api_token("app")
app_model = AppService.get_app_by_id(api_token.app_id, session=db.session())
if not app_model:
raise Forbidden("The app no longer exists.")
@@ -310,8 +325,6 @@ def validate_dataset_token[R](view: Callable[..., R]) -> Callable[..., R]:
@wraps(view)
def decorated(*args: object, **kwargs: object) -> R:
api_token = validate_and_get_api_token("dataset")
# Flask may pass URL path parameters positionally, so inspect both kwargs and args.
dataset_id = kwargs.get("dataset_id")
@@ -324,19 +337,30 @@ def validate_dataset_token[R](view: Callable[..., R]) -> Callable[..., R]:
except Exception:
logger.exception("Failed to parse dataset_id from positional args")
# Per-knowledge-base scoping is expressed by DatasetApiTokenBinding rows:
# no rows -> the key can reach every dataset in its tenant (default / back-compat)
# N rows -> the key is limited to exactly those resources
# Only legacy-dataset bindings count here: a key bound solely to KnowledgeFS spaces
# is restricted but has no reachable legacy dataset, so it is rejected. A bound key
# may only call endpoints carrying one of its dataset ids; endpoints without a
# dataset id (e.g. list/create datasets) are rejected. The scope is queried per
# request (not cached) so changes take effect immediately.
# db.session is Flask-SQLAlchemy's scoped_session proxy; cast so the plain-Session
# typed helper accepts it (runtime proxies every Session method through unchanged).
scope = dataset_api_key_bindings.get_key_scope(cast(Session, db.session), api_token.id)
if not scope.allows_dataset(str(dataset_id) if dataset_id else None):
raise Forbidden("The API key is not authorized to access this knowledge base.")
auth_token = peek_service_api_bearer_token()
if auth_token and is_resource_access_token(auth_token):
with resource_access_token_errors():
tenant_id = application_services().resource_access_tokens.resolve_tenant_for_dataset_service_api(
token=auth_token,
dataset_id=str(dataset_id) if dataset_id else None,
)
else:
api_token = validate_and_get_api_token("dataset")
tenant_id = api_token.tenant_id
# Per-knowledge-base scoping is expressed by DatasetApiTokenBinding rows:
# no rows -> the key can reach every dataset in its tenant (default / back-compat)
# N rows -> the key is limited to exactly those resources
# Only legacy-dataset bindings count here: a key bound solely to KnowledgeFS spaces
# is restricted but has no reachable legacy dataset, so it is rejected. A bound key
# may only call endpoints carrying one of its dataset ids; endpoints without a
# dataset id (e.g. list/create datasets) are rejected. The scope is queried per
# request (not cached) so changes take effect immediately.
# db.session is Flask-SQLAlchemy's scoped_session proxy; cast so the plain-Session
# typed helper accepts it (runtime proxies every Session method through unchanged).
scope = dataset_api_key_bindings.get_key_scope(cast(Session, db.session), api_token.id)
if not scope.allows_dataset(str(dataset_id) if dataset_id else None):
raise Forbidden("The API key is not authorized to access this knowledge base.")
if dataset_id:
dataset_id = str(dataset_id)
@@ -344,7 +368,7 @@ def validate_dataset_token[R](view: Callable[..., R]) -> Callable[..., R]:
select(Dataset)
.where(
Dataset.id == dataset_id,
Dataset.tenant_id == api_token.tenant_id,
Dataset.tenant_id == tenant_id,
)
.limit(1)
)
@@ -355,7 +379,7 @@ def validate_dataset_token[R](view: Callable[..., R]) -> Callable[..., R]:
tenant_account_join = db.session.execute(
select(Tenant, TenantAccountJoin).where(
Tenant.id == api_token.tenant_id,
Tenant.id == tenant_id,
TenantAccountJoin.tenant_id == Tenant.id,
TenantAccountJoin.role.in_(["owner"]),
Tenant.status == TenantStatus.NORMAL,
@@ -377,9 +401,9 @@ def validate_dataset_token[R](view: Callable[..., R]) -> Callable[..., R]:
if expects_bound_instance:
if not args:
raise TypeError("validate_dataset_token expected a bound resource instance.")
return view(args[0], api_token.tenant_id, *args[1:], **kwargs)
return view(args[0], tenant_id, *args[1:], **kwargs)
return view(api_token.tenant_id, *args, **kwargs)
return view(tenant_id, *args, **kwargs)
return decorated
@@ -395,15 +419,7 @@ def validate_and_get_api_token(scope: str | None = None):
The last_used_at field is updated asynchronously via Celery task
to avoid blocking the request.
"""
auth_header = request.headers.get("Authorization")
if auth_header is None or " " not in auth_header:
raise Unauthorized("Authorization header must be provided and start with 'Bearer'")
auth_scheme, auth_token = auth_header.split(None, 1)
auth_scheme = auth_scheme.lower()
if auth_scheme != "bearer":
raise Unauthorized("Authorization scheme must be 'Bearer'")
auth_token = extract_service_api_bearer_token()
# Try to get token from cache first
# Returns a CachedApiToken (plain Python object), not a SQLAlchemy model
@@ -419,6 +435,27 @@ def validate_and_get_api_token(scope: str | None = None):
return fetch_token_with_single_flight(auth_token, scope)
def extract_service_api_bearer_token() -> str:
auth_header = request.headers.get("Authorization")
if auth_header is None or " " not in auth_header:
raise Unauthorized("Authorization header must be provided and start with 'Bearer'")
auth_scheme, auth_token = auth_header.split(None, 1)
if auth_scheme.lower() != "bearer":
raise Unauthorized("Authorization scheme must be 'Bearer'")
return auth_token
def peek_service_api_bearer_token() -> str | None:
auth_header = request.headers.get("Authorization")
if auth_header is None or " " not in auth_header:
return None
auth_scheme, auth_token = auth_header.split(None, 1)
if auth_scheme.lower() != "bearer":
return None
return auth_token
class DatasetApiResource(Resource):
__apidoc__ = {"responses": DATASET_TOKEN_AUTH_RESPONSES}
api/controllers/web/workflow_events.py
+1 -1
View File
@@ -69,7 +69,7 @@ class WorkflowEventsApi(WebApiResource):
payload = response.model_dump(mode="json")
payload["event"] = response.event.value
def _generate_finished_events() -> Generator[str, None, None]:
def _generate_finished_events() -> Generator[str]:
yield f"data: {json.dumps(payload)}\n\n"
event_generator = _generate_finished_events
api/core/agent/fc_agent_runner.py
+2 -4
View File
@@ -100,9 +100,7 @@ class FunctionCallAgentRunner(BaseAgentRunner):
return prompt_message_contents
def run(
self, session: Session, message: Message, query: str, **kwargs: Any
) -> Generator[LLMResultChunk, None, None]:
def run(self, session: Session, message: Message, query: str, **kwargs: Any) -> Generator[LLMResultChunk]:
"""
Run FunctionCall agent application
"""
@@ -175,7 +173,7 @@ class FunctionCallAgentRunner(BaseAgentRunner):
"created_by": CreditUsageCreatedBy.APP,
}
chunks: Union[Generator[LLMResultChunk, None, None], LLMResult] = model_instance.invoke_llm(
chunks: Union[Generator[LLMResultChunk], LLMResult] = model_instance.invoke_llm(
prompt_messages=prompt_messages,
model_parameters=app_generate_entity.model_conf.parameters,
tools=prompt_messages_tools,
api/core/agent/output_parser/cot_output_parser.py
+2 -2
View File
@@ -10,8 +10,8 @@ from graphon.model_runtime.entities.llm_entities import LLMResultChunk
class CotAgentOutputParser:
@classmethod
def handle_react_stream_output(
cls, llm_response: Generator[LLMResultChunk, None, None], usage_dict: dict[str, Any]
) -> Generator[Union[str, AgentScratchpadUnit.Action], None, None]:
cls, llm_response: Generator[LLMResultChunk], usage_dict: dict[str, Any]
) -> Generator[Union[str, AgentScratchpadUnit.Action]]:
def parse_action(action) -> Union[str, AgentScratchpadUnit.Action]:
action_name = None
action_input = None
api/core/agent/strategy/base.py
+2 -2
View File
@@ -20,7 +20,7 @@ class BaseAgentStrategy(ABC):
app_id: str | None = None,
message_id: str | None = None,
credentials: InvokeCredentials | None = None,
) -> Generator[AgentInvokeMessage, None, None]:
) -> Generator[AgentInvokeMessage]:
"""
Invoke the agent strategy.
"""
@@ -41,5 +41,5 @@ class BaseAgentStrategy(ABC):
app_id: str | None = None,
message_id: str | None = None,
credentials: InvokeCredentials | None = None,
) -> Generator[AgentInvokeMessage, None, None]:
) -> Generator[AgentInvokeMessage]:
pass
api/core/agent/strategy/plugin.py
+1 -1
View File
@@ -44,7 +44,7 @@ class PluginAgentStrategy(BaseAgentStrategy):
app_id: str | None = None,
message_id: str | None = None,
credentials: InvokeCredentials | None = None,
) -> Generator[AgentInvokeMessage, None, None]:
) -> Generator[AgentInvokeMessage]:
"""
Invoke the agent strategy.
"""
api/core/app/apps/advanced_chat/app_generator.py
+7 -11
View File
@@ -103,7 +103,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
pause_state_config: PauseStateLayerConfig | None = None,
*,
session: Session,
) -> Generator[Mapping | str, None, None]: ...
) -> Generator[Mapping | str]: ...
@overload
def generate(
@@ -118,7 +118,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
pause_state_config: PauseStateLayerConfig | None = None,
*,
session: Session,
) -> Mapping[str, Any] | Generator[str | Mapping, None, None]: ...
) -> Mapping[str, Any] | Generator[str | Mapping]: ...
def generate(
self,
@@ -132,7 +132,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
pause_state_config: PauseStateLayerConfig | None = None,
*,
session: Session,
) -> Mapping[str, Any] | Generator[str | Mapping, None, None]:
) -> Mapping[str, Any] | Generator[str | Mapping]:
"""
Generate App response.
@@ -327,7 +327,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
streaming: bool = True,
*,
session: Session,
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], None, None]:
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any]]:
"""
Generate App response.
@@ -422,7 +422,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
streaming: bool = True,
*,
session: Session,
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], None, None]:
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any]]:
"""
Generate App response.
@@ -523,7 +523,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
graph_runtime_state: GraphRuntimeState | None = None,
graph_engine_layers: Sequence[GraphEngineLayer] = (),
response_stream_filter: ResponseStreamFilter | None = None,
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], None, None]:
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any]]:
"""
Generate App response.
@@ -760,11 +760,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
user: Account | EndUser,
draft_var_saver_factory: DraftVariableSaverFactory,
stream: bool = False,
) -> (
ChatbotAppBlockingResponse
| AdvancedChatPausedBlockingResponse
| Generator[ChatbotAppStreamResponse, None, None]
):
) -> ChatbotAppBlockingResponse | AdvancedChatPausedBlockingResponse | Generator[ChatbotAppStreamResponse]:
"""
Handle response.
:param application_generate_entity: application generate entity
api/core/app/apps/advanced_chat/generate_response_converter.py
+4 -4
View File
@@ -80,8 +80,8 @@ class AdvancedChatAppGenerateResponseConverter(
@classmethod
@override
def convert_stream_full_response(
cls, stream_response: Generator[AppStreamResponse, None, None]
) -> Generator[dict[str, Any] | str, Any, None]:
cls, stream_response: Generator[AppStreamResponse]
) -> Generator[dict[str, Any] | str, Any]:
"""
Convert stream full response.
:param stream_response: stream response
@@ -112,8 +112,8 @@ class AdvancedChatAppGenerateResponseConverter(
@classmethod
@override
def convert_stream_simple_response(
cls, stream_response: Generator[AppStreamResponse, None, None]
) -> Generator[dict[str, Any] | str, Any, None]:
cls, stream_response: Generator[AppStreamResponse]
) -> Generator[dict[str, Any] | str, Any]:
"""
Convert stream simple response.
:param stream_response: stream response
api/core/app/apps/advanced_chat/generate_task_pipeline.py
+33 -51
View File
@@ -222,7 +222,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
) -> Union[
ChatbotAppBlockingResponse,
AdvancedChatPausedBlockingResponse,
Generator[ChatbotAppStreamResponse, None, None],
Generator[ChatbotAppStreamResponse],
]:
"""
Process generate task pipeline.
@@ -242,7 +242,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
return self._to_blocking_response(generator)
def _to_blocking_response(
self, generator: Generator[StreamResponse, None, None]
self, generator: Generator[StreamResponse]
) -> Union[ChatbotAppBlockingResponse, AdvancedChatPausedBlockingResponse]:
"""
Process blocking response.
@@ -330,9 +330,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
),
)
def _to_stream_response(
self, generator: Generator[StreamResponse, None, None]
) -> Generator[ChatbotAppStreamResponse, Any, None]:
def _to_stream_response(self, generator: Generator[StreamResponse]) -> Generator[ChatbotAppStreamResponse, Any]:
"""
To stream response.
:return:
@@ -359,7 +357,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
def _wrapper_process_stream_response(
self, trace_manager: TraceQueueManager | None = None
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
tts_publisher = None
task_id = self._application_generate_entity.task_id
tenant_id = self._application_generate_entity.app_config.tenant_id
@@ -430,11 +428,11 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
if not self._workflow_run_id:
raise ValueError("workflow run not initialized.")
def _handle_ping_event(self, event: QueuePingEvent, **kwargs) -> Generator[PingStreamResponse, None, None]:
def _handle_ping_event(self, event: QueuePingEvent, **kwargs) -> Generator[PingStreamResponse]:
"""Handle ping events."""
yield self._base_task_pipeline.ping_stream_response()
def _handle_error_event(self, event: QueueErrorEvent, **kwargs) -> Generator[ErrorStreamResponse, None, None]:
def _handle_error_event(self, event: QueueErrorEvent, **kwargs) -> Generator[ErrorStreamResponse]:
"""Handle error events."""
with self._database_session() as session:
err = self._base_task_pipeline.handle_error(event=event, session=session, message_id=self._message_id)
@@ -444,7 +442,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
self,
event: QueueWorkflowStartedEvent,
**kwargs,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle workflow started events."""
runtime_state = self._resolve_graph_runtime_state()
run_id = self._extract_workflow_run_id(runtime_state)
@@ -462,7 +460,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
yield workflow_start_resp
def _handle_node_retry_event(self, event: QueueNodeRetryEvent, **kwargs) -> Generator[StreamResponse, None, None]:
def _handle_node_retry_event(self, event: QueueNodeRetryEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle node retry events."""
self._ensure_workflow_initialized()
@@ -474,9 +472,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
if node_retry_resp:
yield node_retry_resp
def _handle_node_started_event(
self, event: QueueNodeStartedEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
def _handle_node_started_event(self, event: QueueNodeStartedEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle node started events."""
self._ensure_workflow_initialized()
@@ -488,9 +484,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
if node_start_resp:
yield node_start_resp
def _handle_node_succeeded_event(
self, event: QueueNodeSucceededEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
def _handle_node_succeeded_event(self, event: QueueNodeSucceededEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle node succeeded events."""
# Record files if it's an answer node or end node
if event.node_type in [BuiltinNodeTypes.ANSWER, BuiltinNodeTypes.END, BuiltinNodeTypes.LLM]:
@@ -523,7 +517,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
self,
event: Union[QueueNodeFailedEvent, QueueNodeExceptionEvent],
**kwargs,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle various node failure events."""
node_finish_resp = self._workflow_response_converter.workflow_node_finish_to_stream_response(
event=event,
@@ -543,7 +537,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
tts_publisher: AppGeneratorTTSPublisher | None = None,
queue_message: Union[WorkflowQueueMessage, MessageQueueMessage] | None = None,
**kwargs,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle text chunk events."""
delta_text = event.text
if delta_text is None:
@@ -571,9 +565,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
answer=delta_text, message_id=self._message_id, from_variable_selector=event.from_variable_selector
)
def _handle_reasoning_chunk_event(
self, event: QueueReasoningChunkEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
def _handle_reasoning_chunk_event(self, event: QueueReasoningChunkEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle out-of-band reasoning chunk events.
Pure emit: reasoning is streamed on its own channel and never written to the
@@ -592,9 +584,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
),
)
def _handle_iteration_start_event(
self, event: QueueIterationStartEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
def _handle_iteration_start_event(self, event: QueueIterationStartEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle iteration start events."""
self._ensure_workflow_initialized()
@@ -605,9 +595,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
)
yield iter_start_resp
def _handle_iteration_next_event(
self, event: QueueIterationNextEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
def _handle_iteration_next_event(self, event: QueueIterationNextEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle iteration next events."""
self._ensure_workflow_initialized()
@@ -620,7 +608,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
def _handle_iteration_completed_event(
self, event: QueueIterationCompletedEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle iteration completed events."""
self._ensure_workflow_initialized()
@@ -631,7 +619,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
)
yield iter_finish_resp
def _handle_loop_start_event(self, event: QueueLoopStartEvent, **kwargs) -> Generator[StreamResponse, None, None]:
def _handle_loop_start_event(self, event: QueueLoopStartEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle loop start events."""
self._ensure_workflow_initialized()
@@ -642,7 +630,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
)
yield loop_start_resp
def _handle_loop_next_event(self, event: QueueLoopNextEvent, **kwargs) -> Generator[StreamResponse, None, None]:
def _handle_loop_next_event(self, event: QueueLoopNextEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle loop next events."""
self._ensure_workflow_initialized()
@@ -653,9 +641,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
)
yield loop_next_resp
def _handle_loop_completed_event(
self, event: QueueLoopCompletedEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
def _handle_loop_completed_event(self, event: QueueLoopCompletedEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle loop completed events."""
self._ensure_workflow_initialized()
@@ -672,7 +658,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
*,
trace_manager: TraceQueueManager | None = None,
**kwargs,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle workflow succeeded events."""
_ = trace_manager
self._ensure_workflow_initialized()
@@ -695,7 +681,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
*,
trace_manager: TraceQueueManager | None = None,
**kwargs,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle workflow partial success events."""
_ = trace_manager
self._ensure_workflow_initialized()
@@ -717,7 +703,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
self,
event: QueueWorkflowPausedEvent,
**kwargs,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle workflow paused events."""
validated_state = self._ensure_graph_runtime_initialized()
responses = self._workflow_response_converter.workflow_pause_to_stream_response(
@@ -749,7 +735,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
*,
trace_manager: TraceQueueManager | None = None,
**kwargs,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle workflow failed events."""
_ = trace_manager
self._ensure_workflow_initialized()
@@ -778,7 +764,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
graph_runtime_state: GraphRuntimeState | None = None,
trace_manager: TraceQueueManager | None = None,
**kwargs,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle stop events."""
_ = trace_manager
resolved_state = None
@@ -818,7 +804,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
*,
graph_runtime_state: GraphRuntimeState | None = None,
**kwargs,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle advanced chat message end events."""
resolved_state = self._ensure_graph_runtime_initialized(graph_runtime_state)
@@ -842,28 +828,24 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
def _handle_retriever_resources_event(
self, event: QueueRetrieverResourcesEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle retriever resources events."""
self._message_cycle_manager.handle_retriever_resources(event)
yield from ()
def _handle_annotation_reply_event(
self, event: QueueAnnotationReplyEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
def _handle_annotation_reply_event(self, event: QueueAnnotationReplyEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle annotation reply events."""
with self._database_session() as session:
self._message_cycle_manager.handle_annotation_reply(event, session)
yield from ()
def _handle_message_replace_event(
self, event: QueueMessageReplaceEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
def _handle_message_replace_event(self, event: QueueMessageReplaceEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle message replace events."""
yield self._message_cycle_manager.message_replace_to_stream_response(answer=event.text, reason=event.reason)
def _handle_human_input_form_filled_event(
self, event: QueueHumanInputFormFilledEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle human input form filled events."""
self._persist_human_input_extra_content(node_id=event.node_id)
yield self._workflow_response_converter.human_input_form_filled_to_stream_response(
@@ -872,7 +854,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
def _handle_human_input_form_timeout_event(
self, event: QueueHumanInputFormTimeoutEvent, **kwargs
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Handle human input form timeout events."""
yield self._workflow_response_converter.human_input_form_timeout_to_stream_response(
event=event, task_id=self._application_generate_entity.task_id
@@ -920,7 +902,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
return None
return form.id
def _handle_agent_log_event(self, event: QueueAgentLogEvent, **kwargs) -> Generator[StreamResponse, None, None]:
def _handle_agent_log_event(self, event: QueueAgentLogEvent, **kwargs) -> Generator[StreamResponse]:
"""Handle agent log events."""
yield self._workflow_response_converter.handle_agent_log(
task_id=self._application_generate_entity.task_id, event=event
@@ -971,7 +953,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
tts_publisher: AppGeneratorTTSPublisher | None = None,
trace_manager: TraceQueueManager | None = None,
queue_message: Union[WorkflowQueueMessage, MessageQueueMessage] | None = None,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""Dispatch events using elegant pattern matching."""
handlers = self._get_event_handlers()
event_type = type(event)
@@ -1009,7 +991,7 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
self,
tts_publisher: AppGeneratorTTSPublisher | None = None,
trace_manager: TraceQueueManager | None = None,
) -> Generator[StreamResponse, None, None]:
) -> Generator[StreamResponse]:
"""
Process stream response using elegant Fluent Python patterns.
Maintains exact same functionality as original 57-if-statement version.
api/core/app/apps/agent_app/app_generator.py
+30 -218
View File
@@ -33,7 +33,6 @@ from core.app.apps.agent_app.app_runner import AgentAppRunner
from core.app.apps.agent_app.errors import (
AgentAppGeneratorError,
AgentAppNotPublishedError,
AgentSessionSnapshotIncompatibleError,
)
from core.app.apps.agent_app.generate_response_converter import AgentAppGenerateResponseConverter
from core.app.apps.agent_app.runtime_request_builder import AgentAppRuntimeRequestBuilder
@@ -63,7 +62,6 @@ from models.agent import (
AgentConfigVersionKind,
AgentScope,
AgentStatus,
AgentWorkingResourceStatus,
AgentWorkspaceBinding,
AgentWorkspaceOwnerType,
)
@@ -85,7 +83,7 @@ class AgentAppGenerator(MessageBasedAppGenerator):
invoke_from: InvokeFrom,
session: Session,
streaming: bool = True,
) -> Mapping[str, Any] | Generator[Mapping | str, None, None]:
) -> Mapping[str, Any] | Generator[Mapping | str]:
if not streaming:
raise AgentAppGeneratorError("Agent App only supports streaming mode")
@@ -216,172 +214,6 @@ class AgentAppGenerator(MessageBasedAppGenerator):
)
return AgentAppGenerateResponseConverter.convert(response=response, invoke_from=invoke_from)
def resume_after_form_submission(
self,
*,
app_model: App,
user: Account | EndUser,
conversation_id: str,
form_id: str,
invoke_from: InvokeFrom,
session: Session,
) -> None:
"""Resume an Agent App conversation after a submitted ask_human HITL form.
ENG-635: triggered by a background task (not an HTTP request). Runs one
blocking turn with no user query; the runner threads the human's reply
into the agent run as deferred_tool_results and the assistant answer is
persisted to the conversation. Live streaming to a reconnected client is
out of scope here — the message is persisted and can be re-fetched.
"""
conversation = ConversationService.get_conversation(
app_model=app_model, conversation_id=conversation_id, user=user, session=session
)
draft_type, draft_id = self._resolve_resume_draft(
app_model=app_model,
conversation=conversation,
user=user,
form_id=form_id,
session=session,
)
agent, agent_config_id, agent_config_version_kind, agent_soul = self._resolve_agent(
app_model,
invoke_from=invoke_from,
draft_type=draft_type,
draft_id=draft_id,
user=user,
session=session,
conversation=conversation,
)
app_model_config = (
session.get(AppModelConfig, app_model.app_model_config_id) if app_model.app_model_config_id else None
)
annotation_reply = load_annotation_reply_config(session, app_model.id) if app_model_config else None
app_config = AgentAppConfigManager.get_app_config(
app_model=app_model,
agent_soul=agent_soul,
annotation_reply=annotation_reply,
app_model_config=app_model_config,
conversation=conversation,
)
model_conf = ModelConfigConverter.convert(app_config)
trace_manager = TraceQueueManager(app_model.id, user.id if isinstance(user, Account) else user.session_id)
# ENG-638: the agent backend requires the resume composition's layer
# names to match the suspended snapshot, which includes the per-turn
# user-prompt layer. So re-send the original user message (the paused
# turn's query); the continuation is driven by deferred_tool_results and
# the restored snapshot, not by re-processing this prompt. A blank prompt
# would drop the user-prompt layer and fail the snapshot match.
paused_message = session.scalar(
select(Message)
.where(Message.conversation_id == conversation.id, Message.query != "")
.order_by(Message.created_at.desc())
.limit(1)
)
resume_query = paused_message.query if paused_message and paused_message.query else "(resumed)"
application_generate_entity = AgentAppGenerateEntity(
task_id=str(uuid.uuid4()),
app_config=app_config,
model_conf=model_conf,
conversation_id=conversation.id,
# A resume carries no new user inputs; the human's answer is the
# submitted form, threaded in by the runner as deferred_tool_results.
# The query re-sends the paused turn's message (see above).
inputs={},
query=resume_query,
files=[],
parent_message_id=UUID_NIL,
user_id=user.id,
stream=False,
invoke_from=invoke_from,
extras={"auto_generate_conversation_name": False},
call_depth=0,
trace_manager=trace_manager,
agent_id=agent.id,
agent_config_snapshot_id=agent_config_id,
agent_config_version_kind=agent_config_version_kind,
agent_llm_gateway_enabled=True,
)
conversation, message = self._init_generate_records(
application_generate_entity,
conversation,
session=session,
)
queue_manager = MessageBasedAppQueueManager(
task_id=application_generate_entity.task_id,
user_id=application_generate_entity.user_id,
invoke_from=application_generate_entity.invoke_from,
conversation_id=conversation.id,
app_mode=conversation.mode,
message_id=message.id,
)
context = contextvars.copy_context()
worker_thread = threading.Thread(
target=self._generate_worker,
kwargs={
"flask_app": current_app._get_current_object(), # type: ignore
"context": context,
"application_generate_entity": application_generate_entity,
"queue_manager": queue_manager,
"conversation_id": conversation.id,
"message_id": message.id,
"user_from": UserFrom.ACCOUNT if isinstance(user, Account) else UserFrom.END_USER,
# Resume continues a paused agent run; skip input guards (see _generate_worker).
"is_resume": True,
},
)
worker_thread.start()
# Blocking: drive the chat task pipeline to persist the assistant answer.
self._handle_response(
application_generate_entity=application_generate_entity,
queue_manager=queue_manager,
conversation=conversation,
message=message,
user=user,
stream=False,
)
@staticmethod
def _resolve_resume_draft(
*,
app_model: App,
conversation: Any,
user: Account | EndUser,
form_id: str,
session: Session,
) -> tuple[str | None, str | None]:
if conversation.invoke_from != InvokeFrom.DEBUGGER:
return None, None
if not isinstance(user, Account):
return AgentConfigDraftType.DRAFT.value, None
build_draft = session.scalar(
select(AgentConfigDraft)
.join(
AgentWorkspaceBinding,
AgentWorkspaceBinding.id == AgentConfigDraft.agent_workspace_binding_id,
)
.where(
AgentConfigDraft.tenant_id == app_model.tenant_id,
AgentConfigDraft.draft_type == AgentConfigDraftType.DEBUG_BUILD,
AgentConfigDraft.account_id == user.id,
AgentWorkspaceBinding.tenant_id == app_model.tenant_id,
AgentWorkspaceBinding.status == AgentWorkingResourceStatus.ACTIVE,
AgentWorkspaceBinding.pending_form_id == form_id,
)
)
if build_draft is not None:
return AgentConfigDraftType.DEBUG_BUILD.value, build_draft.id
return AgentConfigDraftType.DRAFT.value, None
def _generate_worker(
self,
*,
@@ -392,7 +224,6 @@ class AgentAppGenerator(MessageBasedAppGenerator):
conversation_id: str,
message_id: str,
user_from: UserFrom,
is_resume: bool = False,
) -> None:
from libs.flask_utils import preserve_flask_contexts
@@ -402,46 +233,36 @@ class AgentAppGenerator(MessageBasedAppGenerator):
message = self._get_message(message_id)
app_config = application_generate_entity.app_config
if is_resume:
# ENG-638: a resume continues a paused agent run; the human's
# reply is threaded in by the runner as deferred_tool_results.
# The query is the replayed paused-turn message, kept only to
# match the suspended snapshot's layers — it is NOT new
# end-user input, so input guards must NOT run. Moderation or an
# annotation match on the replayed query would short-circuit the
# turn and drop the human reply, stranding the ask_human session.
query = application_generate_entity.query or ""
else:
# Apply app-level input guards (content moderation + annotation
# reply) before reaching the Agent backend, mirroring the EasyUI
# chat / agent-chat runners. These can short-circuit the turn.
with session_factory.get_session_maker().begin() as session:
app_model = session.get(App, app_config.app_id)
if app_model is None:
raise AgentAppGeneratorError("App not found")
handled, query, annotation_reply = self._run_input_guards(
session=session,
application_generate_entity=application_generate_entity,
app_model=app_model,
message=message,
queue_manager=queue_manager,
)
if annotation_reply:
from core.app.apps.agent_app.app_runner import publish_text_answer
from core.app.entities.queue_entities import QueueAnnotationReplyEvent
# Apply app-level input guards (content moderation + annotation
# reply) before reaching the Agent backend, mirroring the EasyUI
# chat / agent-chat runners. These can short-circuit the turn.
with session_factory.get_session_maker().begin() as session:
app_model = session.get(App, app_config.app_id)
if app_model is None:
raise AgentAppGeneratorError("App not found")
handled, query, annotation_reply = self._run_input_guards(
session=session,
application_generate_entity=application_generate_entity,
app_model=app_model,
message=message,
queue_manager=queue_manager,
)
if annotation_reply:
from core.app.apps.agent_app.app_runner import publish_text_answer
from core.app.entities.queue_entities import QueueAnnotationReplyEvent
queue_manager.publish(
QueueAnnotationReplyEvent(message_annotation_id=annotation_reply.id),
PublishFrom.APPLICATION_MANAGER,
)
publish_text_answer(
queue_manager=queue_manager,
model_name=application_generate_entity.model_conf.model,
answer=annotation_reply.content,
user_query=query,
)
if handled:
return
queue_manager.publish(
QueueAnnotationReplyEvent(message_annotation_id=annotation_reply.id),
PublishFrom.APPLICATION_MANAGER,
)
publish_text_answer(
queue_manager=queue_manager,
model_name=application_generate_entity.model_conf.model,
answer=annotation_reply.content,
user_query=query,
)
if handled:
return
dify_context = DifyRunContext(
tenant_id=app_config.tenant_id,
@@ -491,15 +312,6 @@ class AgentAppGenerator(MessageBasedAppGenerator):
)
except GenerateTaskStoppedError:
pass
except AgentSessionSnapshotIncompatibleError as error:
logger.info(
"Agent App session snapshot no longer matches the current composition",
extra={
"agent_id": application_generate_entity.agent_id,
"conversation_id": conversation_id,
},
)
queue_manager.publish_error(error, PublishFrom.APPLICATION_MANAGER)
except Exception as e:
logger.exception("Unknown Error in Agent App generate worker")
queue_manager.publish_error(e, PublishFrom.APPLICATION_MANAGER)
api/core/app/apps/agent_app/app_runner.py
+1 -146
View File
@@ -3,7 +3,7 @@
Unlike the legacy ``AgentChatAppRunner`` (which runs an in-process ReAct loop),
this runner delegates to the Agent backend, consumes the streamed event flow,
republishes the assistant answer through the existing EasyUI chat task
pipeline, and saves the latest Agenton snapshot on the persistent Binding.
pipeline, and saves the latest module state snapshot on the persistent Binding.
"""
from __future__ import annotations
@@ -15,13 +15,10 @@ from collections.abc import Mapping
from decimal import Decimal
from typing import Any, Literal
from dify_agent.layers.ask_human import AskHumanToolArgs
from dify_agent.protocol import DeferredToolResultsPayload
from pydantic import JsonValue
from clients.agent_backend import (
AgentBackendAgentMessageDeltaInternalEvent,
AgentBackendDeferredToolCallInternalEvent,
AgentBackendError,
AgentBackendInternalEventType,
AgentBackendRunCancelledInternalEvent,
@@ -53,9 +50,6 @@ from core.app.entities.queue_entities import (
QueueRetrieverResourcesEvent,
)
from core.app.knowledge_citations import knowledge_sources_from_tool_part
from core.repositories.human_input_repository import HumanInputFormRepository, HumanInputFormRepositoryImpl
from core.workflow.nodes.agent_v2.ask_human_hitl import AskHumanFormBuildError, create_ask_human_form
from core.workflow.nodes.agent_v2.ask_human_resume import build_deferred_tool_results, resolve_ask_human_form
from extensions.ext_database import db
from graphon.file import File
from graphon.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
@@ -677,8 +671,6 @@ class AgentAppRunner:
agent_config_version_kind=AgentConfigVersionKind(agent_config_version_kind),
build_draft_id=build_draft_id,
)
# ENG-638: if a prior turn paused on ask_human and the form is now answered,
# resume by threading the human's reply into this run as deferred_tool_results.
stored = self._session_store.load_or_create(scope)
runtime = self._build_runtime(
dify_context=dify_context,
@@ -694,7 +686,6 @@ class AgentAppRunner:
image_detail_config=image_detail_config,
idempotency_key=message_id,
stored=stored,
message_id=message_id,
)
create_response = self._agent_backend_client.create_run(runtime.request)
@@ -709,23 +700,6 @@ class AgentAppRunner:
binding_id=runtime.binding_id,
)
if isinstance(terminal, AgentBackendDeferredToolCallInternalEvent):
# ENG-635: the agent asked a human. End this turn with the question and
# a conversation-owned HITL form; a form submission resumes the run.
self._pause_for_ask_human(
terminal=terminal,
scope=scope,
dify_context=dify_context,
agent_soul=agent_soul,
conversation_id=conversation_id,
message_id=message_id,
model_name=model_name,
runtime=runtime,
queue_manager=queue_manager,
query=query,
)
return
terminal_usage = None
if isinstance(
terminal,
@@ -823,14 +797,8 @@ class AgentAppRunner:
image_detail_config: ImagePromptMessageContent.DETAIL | None,
idempotency_key: str,
stored: StoredAgentAppSession,
message_id: str | None,
) -> AgentAppRuntimeRequest:
session_snapshot = stored.session_snapshot
deferred_tool_results = (
self._resolve_pending_ask_human(stored=stored, dify_context=dify_context, message_id=message_id)
if message_id is not None
else None
)
return self._request_builder.build(
AgentAppRuntimeBuildContext(
dify_context=dify_context,
@@ -846,99 +814,9 @@ class AgentAppRunner:
binding_id=binding_id,
backend_binding_ref=backend_binding_ref,
session_snapshot=session_snapshot,
deferred_tool_results=deferred_tool_results,
)
)
def _pause_for_ask_human(
self,
*,
terminal: AgentBackendDeferredToolCallInternalEvent,
scope: AgentAppSessionScope,
dify_context: DifyRunContext,
agent_soul: AgentSoulConfig,
conversation_id: str,
message_id: str,
model_name: str,
runtime: AgentAppRuntimeRequest,
queue_manager: AppQueueManager,
query: str,
) -> None:
"""End the chat turn on a dify.ask_human call: create a conversation-owned
HITL form, persist the pause correlation, and surface the question."""
try:
created = create_ask_human_form(
deferred_tool_call=terminal.deferred_tool_call,
# Chat forms have no workflow node; key by the turn's message id.
node_id=message_id,
default_node_title="Agent",
contacts=agent_soul.human.contacts,
repository=self._build_form_repository(dify_context),
conversation_id=conversation_id,
)
except AskHumanFormBuildError as error:
raise AgentBackendError(f"Failed to build ask_human form for Agent App chat: {error}") from error
# Persist the snapshot + correlation so a form submission can start the
# second run with the human's answer (ENG-637/638 columns, conversation owner).
self._save_session(
scope=scope,
binding_id=runtime.binding_id,
snapshot=terminal.session_snapshot,
pending_form_id=created.form_id,
pending_tool_call_id=terminal.deferred_tool_call.tool_call_id,
)
# The structured form is delivered via the HITL surface(s); the chat turn
# ends by echoing the agent's question so the conversation reflects the ask.
self._publish_answer(
queue_manager=queue_manager,
model_name=model_name,
answer=self._ask_human_message(created.args),
query=query,
usage=_llm_usage_from_agent_backend(terminal.usage),
)
def _resolve_pending_ask_human(
self,
*,
stored: StoredAgentAppSession,
dify_context: DifyRunContext,
message_id: str,
) -> DeferredToolResultsPayload | None:
"""Build deferred_tool_results when a pending ask_human form is answered."""
if stored.pending_form_id is None or stored.pending_tool_call_id is None:
return None
outcome = resolve_ask_human_form(
form_id=stored.pending_form_id,
tenant_id=dify_context.tenant_id,
node_id=message_id,
)
if outcome is None or outcome.deferred_result is None:
# Form missing or still waiting — run a normal turn, no resume.
return None
return build_deferred_tool_results(
tool_call_id=stored.pending_tool_call_id,
result=outcome.deferred_result,
)
def _build_form_repository(self, dify_context: DifyRunContext) -> HumanInputFormRepository:
invoke_source = dify_context.invoke_from.value
return HumanInputFormRepositoryImpl(
tenant_id=dify_context.tenant_id,
app_id=dify_context.app_id,
workflow_execution_id=None,
invoke_source=invoke_source,
submission_actor_id=dify_context.user_id if invoke_source in {"debugger", "explore"} else None,
)
@staticmethod
def _ask_human_message(args: AskHumanToolArgs) -> str:
parts = [args.question]
if args.markdown:
parts.append(args.markdown)
return "\n\n".join(parts)
def _consume_stream(
self,
run_id: str,
@@ -1103,25 +981,6 @@ class AgentAppRunner:
exc_info=True,
)
def _publish_answer(
self,
*,
queue_manager: AppQueueManager,
model_name: str,
answer: str,
query: str | None,
usage: LLMUsage | None = None,
) -> None:
# MVP: emit the full answer as a single chunk + message-end. The chat
# task pipeline streams the chunk over SSE and persists the message.
publish_text_answer(
queue_manager=queue_manager,
model_name=model_name,
answer=answer,
user_query=query,
usage=usage,
)
def _publish_terminal_answer(
self,
*,
@@ -1180,16 +1039,12 @@ class AgentAppRunner:
scope: AgentAppSessionScope,
binding_id: str,
snapshot: Any,
pending_form_id: str | None = None,
pending_tool_call_id: str | None = None,
) -> bool:
try:
self._session_store.save_active_snapshot(
scope=scope,
binding_id=binding_id,
snapshot=snapshot,
pending_form_id=pending_form_id,
pending_tool_call_id=pending_tool_call_id,
)
return True
except Exception:
api/core/app/apps/agent_app/errors.py
-18
View File
@@ -1,24 +1,6 @@
from core.app.apps.exc import AppGenerateError
AGENT_SESSION_CONFIGURATION_CHANGED_ERROR_CODE = "agent_session_configuration_changed"
AGENT_SESSION_CONFIGURATION_CHANGED_MESSAGE = (
"The Agent configuration changed after this conversation started. Start a new conversation to continue."
)
class AgentAppGeneratorError(ValueError):
"""Raised when an Agent App turn cannot be set up."""
class AgentAppNotPublishedError(AgentAppGeneratorError):
"""Raised when a public Agent App runtime is requested before publish."""
class AgentSessionSnapshotIncompatibleError(AppGenerateError):
"""Raised when a retained session snapshot no longer matches the current composition."""
error_code = AGENT_SESSION_CONFIGURATION_CHANGED_ERROR_CODE
status_code = 409
def __init__(self) -> None:
super().__init__(AGENT_SESSION_CONFIGURATION_CHANGED_MESSAGE)
api/core/app/apps/agent_app/runtime_request_builder.py
+3 -29
View File
@@ -13,7 +13,6 @@ from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Literal, cast
from agenton.compositor import CompositorSessionSnapshot
from dify_agent.layers.execution_context import (
DifyExecutionContextInvokeFrom,
DifyExecutionContextLayerConfig,
@@ -25,8 +24,9 @@ from dify_agent.layers.user_prompt import (
DifyUserPromptFileType,
DifyUserPromptImageConfig,
)
from dify_agent.protocol import CreateRunRequest, DeferredToolResultsPayload
from dify_agent.protocol import CreateRunRequest
from dify_agent.protocol.knowledge_fs import KnowledgeFsError
from dify_agent.protocol.snapshot import SessionSnapshot
from clients.agent_backend import (
AgentBackendAgentAppRunInput,
@@ -48,7 +48,6 @@ from core.workflow.nodes.agent_v2.dify_tools_builder import (
)
from core.workflow.nodes.agent_v2.runtime_request_builder import (
append_runtime_warnings,
build_ask_human_layer_config,
build_config_aware_soul_mention_resolver,
build_config_layer_config,
build_shell_layer_config,
@@ -60,8 +59,6 @@ from models.agent_config_entities import AgentSoulConfig, AgentSoulToolsConfig
from models.provider_ids import ModelProviderID
from services.agent.prompt_mentions import expand_prompt_mentions
from .errors import AgentSessionSnapshotIncompatibleError
class AgentAppRuntimeRequestBuildError(ValueError):
"""Raised when Agent App state cannot be mapped to a valid run request."""
@@ -88,9 +85,7 @@ class AgentAppRuntimeBuildContext:
files: tuple[File, ...] = ()
image_detail_config: ImagePromptMessageContent.DETAIL | None = None
agent_config_version_kind: Literal["snapshot", "draft", "build_draft"] = "snapshot"
session_snapshot: CompositorSessionSnapshot | None = None
# ENG-638: set when resuming a chat turn after a submitted ask_human form.
deferred_tool_results: DeferredToolResultsPayload | None = None
session_snapshot: SessionSnapshot | None = None
@dataclass(frozen=True, slots=True)
@@ -212,16 +207,13 @@ class AgentAppRuntimeRequestBuilder:
core_tools=tool_layers.core_tools,
knowledge_fs=knowledge_config,
config_layer_config=config_layer_config,
ask_human_config=build_ask_human_layer_config(agent_soul),
include_shell=dify_config.AGENT_SHELL_ENABLED,
shell_config=build_shell_layer_config(agent_soul),
session_snapshot=context.session_snapshot,
deferred_tool_results=context.deferred_tool_results,
idempotency_key=context.idempotency_key,
metadata=metadata,
)
)
self._validate_session_snapshot_layers(request)
redacted = cast(dict[str, Any], redact_for_agent_backend_log(request))
return AgentAppRuntimeRequest(
request=request,
@@ -256,24 +248,6 @@ class AgentAppRuntimeRequestBuilder:
user_files.append(_build_user_download(file))
return user_files
@staticmethod
def _validate_session_snapshot_layers(request: CreateRunRequest) -> None:
"""Reject stale snapshots before they reach the Agent backend.
Draft rows are updated in place, so their IDs cannot prove that a
retained snapshot still belongs to the current composition. Agenton
requires the ordered layer names to match exactly; enforce the same
invariant at the API boundary and return a product-level error.
"""
snapshot = request.session_snapshot
if snapshot is None:
return
snapshot_layer_names = tuple(layer.name for layer in snapshot.layers)
composition_layer_names = tuple(layer.name for layer in request.composition.layers)
if snapshot_layer_names != composition_layer_names:
raise AgentSessionSnapshotIncompatibleError()
def _build_tool_layers(
self,
*,
api/core/app/apps/agent_app/session_store.py
+4 -16
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
from dataclasses import dataclass
from agenton.compositor import CompositorSessionSnapshot
from dify_agent.protocol.snapshot import SessionSnapshot
from sqlalchemy import select
from sqlalchemy.orm import Session
@@ -54,9 +54,7 @@ class StoredAgentAppSession:
binding_id: str
workspace_id: str
backend_binding_ref: str
session_snapshot: CompositorSessionSnapshot | None
pending_form_id: str | None = None
pending_tool_call_id: str | None = None
session_snapshot: SessionSnapshot | None
class AgentAppWorkspaceStore:
@@ -141,9 +139,7 @@ class AgentAppWorkspaceStore:
*,
scope: AgentAppSessionScope,
binding_id: str,
snapshot: CompositorSessionSnapshot | None,
pending_form_id: str | None = None,
pending_tool_call_id: str | None = None,
snapshot: SessionSnapshot | None,
) -> None:
if snapshot is None:
return
@@ -151,25 +147,17 @@ class AgentAppWorkspaceStore:
tenant_id=scope.tenant_id,
binding_id=binding_id,
session_snapshot=snapshot.model_dump_json(),
pending_form_id=pending_form_id,
pending_tool_call_id=pending_tool_call_id,
)
@staticmethod
def _stored(scope: AgentAppSessionScope, binding: AgentWorkspaceBinding) -> StoredAgentAppSession:
snapshot = (
CompositorSessionSnapshot.model_validate_json(binding.session_snapshot)
if binding.session_snapshot
else None
)
snapshot = SessionSnapshot.model_validate_json(binding.session_snapshot) if binding.session_snapshot else None
return StoredAgentAppSession(
scope=scope,
binding_id=binding.id,
workspace_id=binding.workspace_id,
backend_binding_ref=binding.backend_binding_ref,
session_snapshot=snapshot,
pending_form_id=binding.pending_form_id,
pending_tool_call_id=binding.pending_tool_call_id,
)
api/core/app/apps/agent_chat/app_generator.py
+3 -3
View File
@@ -57,7 +57,7 @@ class AgentChatAppGenerator(MessageBasedAppGenerator):
invoke_from: InvokeFrom,
streaming: Literal[True],
session: Session,
) -> Generator[Mapping | str, None, None]: ...
) -> Generator[Mapping | str]: ...
@overload
def generate(
@@ -69,7 +69,7 @@ class AgentChatAppGenerator(MessageBasedAppGenerator):
invoke_from: InvokeFrom,
streaming: bool,
session: Session,
) -> Mapping | Generator[Mapping | str, None, None]: ...
) -> Mapping | Generator[Mapping | str]: ...
def generate(
self,
@@ -80,7 +80,7 @@ class AgentChatAppGenerator(MessageBasedAppGenerator):
invoke_from: InvokeFrom,
streaming: bool = True,
session: Session,
) -> Mapping | Generator[Mapping | str, None, None]:
) -> Mapping | Generator[Mapping | str]:
"""
Generate App response.
api/core/app/apps/agent_chat/generate_response_converter.py
+4 -4
View File
@@ -58,8 +58,8 @@ class AgentChatAppGenerateResponseConverter(AppGenerateResponseConverter[Chatbot
@classmethod
@override
def convert_stream_full_response(
cls, stream_response: Generator[AppStreamResponse, None, None]
) -> Generator[dict[str, Any] | str, None, None]:
cls, stream_response: Generator[AppStreamResponse]
) -> Generator[dict[str, Any] | str]:
"""
Convert stream full response.
:param stream_response: stream response
@@ -90,8 +90,8 @@ class AgentChatAppGenerateResponseConverter(AppGenerateResponseConverter[Chatbot
@classmethod
@override
def convert_stream_simple_response(
cls, stream_response: Generator[AppStreamResponse, None, None]
) -> Generator[dict[str, Any] | str, None, None]:
cls, stream_response: Generator[AppStreamResponse]
) -> Generator[dict[str, Any] | str]:
"""
Convert stream simple response.
:param stream_response: stream response
api/core/app/apps/base_app_generate_response_converter.py
+8 -8
View File
@@ -23,14 +23,14 @@ class AppGenerateResponseConverter[TBlockingResponse: AppBlockingResponse](ABC):
@classmethod
def convert(
cls, response: Union[AppBlockingResponse, Generator[AppStreamResponse, Any, None]], invoke_from: InvokeFrom
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], Any, None]:
cls, response: Union[AppBlockingResponse, Generator[AppStreamResponse, Any]], invoke_from: InvokeFrom
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], Any]:
if invoke_from in {InvokeFrom.DEBUGGER, InvokeFrom.SERVICE_API, InvokeFrom.OPENAPI}:
if isinstance(response, AppBlockingResponse):
return cls.convert_blocking_full_response(cls._cast_blocking_response(response))
else:
def _generate_full_response() -> Generator[dict[str, Any] | str, Any, None]:
def _generate_full_response() -> Generator[dict[str, Any] | str, Any]:
yield from cls.convert_stream_full_response(response)
return _generate_full_response()
@@ -39,7 +39,7 @@ class AppGenerateResponseConverter[TBlockingResponse: AppBlockingResponse](ABC):
return cls.convert_blocking_simple_response(cls._cast_blocking_response(response))
else:
def _generate_simple_response() -> Generator[dict[str, Any] | str, Any, None]:
def _generate_simple_response() -> Generator[dict[str, Any] | str, Any]:
yield from cls.convert_stream_simple_response(response)
return _generate_simple_response()
@@ -57,15 +57,15 @@ class AppGenerateResponseConverter[TBlockingResponse: AppBlockingResponse](ABC):
@classmethod
@abstractmethod
def convert_stream_full_response(
cls, stream_response: Generator[AppStreamResponse, None, None]
) -> Generator[dict[str, Any] | str, None, None]:
cls, stream_response: Generator[AppStreamResponse]
) -> Generator[dict[str, Any] | str]:
raise NotImplementedError
@classmethod
@abstractmethod
def convert_stream_simple_response(
cls, stream_response: Generator[AppStreamResponse, None, None]
) -> Generator[dict[str, Any] | str, None, None]:
cls, stream_response: Generator[AppStreamResponse]
) -> Generator[dict[str, Any] | str]:
raise NotImplementedError
@classmethod
api/core/app/apps/base_app_generator.py
+3 -3
View File
@@ -94,9 +94,9 @@ class BaseAppGenerator:
@staticmethod
def _wrap_stream_with_worker_thread_join[ResponseT](
response_stream: Generator[ResponseT, None, None],
response_stream: Generator[ResponseT],
worker_thread: threading.Thread,
) -> Generator[ResponseT, None, None]:
) -> Generator[ResponseT]:
"""Keep the producer owned by the response stream until both finish."""
try:
yield from response_stream
@@ -310,7 +310,7 @@ class BaseAppGenerator:
return value
@classmethod
def convert_to_event_stream(cls, generator: Union[Mapping, Generator[Mapping | str, None, None]]):
def convert_to_event_stream(cls, generator: Union[Mapping, Generator[Mapping | str]]):
"""
Convert messages into event stream
"""
api/core/app/apps/base_app_runner.py
+2 -2
View File
@@ -214,7 +214,7 @@ class AppRunner:
def _handle_invoke_result(
self,
invoke_result: Union[LLMResult, Generator[Any, None, None]],
invoke_result: Union[LLMResult, Generator[Any]],
queue_manager: AppQueueManager,
stream: bool,
agent: bool = False,
@@ -275,7 +275,7 @@ class AppRunner:
def _handle_invoke_result_stream(
self,
invoke_result: Generator[LLMResultChunk, None, None],
invoke_result: Generator[LLMResultChunk],
queue_manager: AppQueueManager,
agent: bool,
message_id: str | None = None,
api/core/app/apps/chat/app_generator.py
+3 -3
View File
@@ -45,7 +45,7 @@ class ChatAppGenerator(MessageBasedAppGenerator):
streaming: Literal[True],
*,
session: Session,
) -> Generator[Mapping | str, None, None]: ...
) -> Generator[Mapping | str]: ...
@overload
def generate(
@@ -69,7 +69,7 @@ class ChatAppGenerator(MessageBasedAppGenerator):
streaming: bool,
*,
session: Session,
) -> Mapping[str, Any] | Generator[Mapping[str, Any] | str, None, None]: ...
) -> Mapping[str, Any] | Generator[Mapping[str, Any] | str]: ...
def generate(
self,
@@ -80,7 +80,7 @@ class ChatAppGenerator(MessageBasedAppGenerator):
streaming: bool = True,
*,
session: Session,
) -> Mapping[str, Any] | Generator[Mapping[str, Any] | str, None, None]:
) -> Mapping[str, Any] | Generator[Mapping[str, Any] | str]:
"""
Generate App response.
api/core/app/apps/chat/generate_response_converter.py
+4 -4
View File
@@ -58,8 +58,8 @@ class ChatAppGenerateResponseConverter(AppGenerateResponseConverter[ChatbotAppBl
@classmethod
@override
def convert_stream_full_response(
cls, stream_response: Generator[AppStreamResponse, None, None]
) -> Generator[dict[str, Any] | str, None, None]:
cls, stream_response: Generator[AppStreamResponse]
) -> Generator[dict[str, Any] | str]:
"""
Convert stream full response.
:param stream_response: stream response
@@ -90,8 +90,8 @@ class ChatAppGenerateResponseConverter(AppGenerateResponseConverter[ChatbotAppBl
@classmethod
@override
def convert_stream_simple_response(
cls, stream_response: Generator[AppStreamResponse, None, None]
) -> Generator[dict[str, Any] | str, None, None]:
cls, stream_response: Generator[AppStreamResponse]
) -> Generator[dict[str, Any] | str]:
"""
Convert stream simple response.
:param stream_response: stream response
api/core/app/apps/completion/app_generator.py
+4 -4
View File
@@ -46,7 +46,7 @@ class CompletionAppGenerator(MessageBasedAppGenerator):
streaming: Literal[True],
*,
session: Session,
) -> Generator[str | Mapping[str, Any], None, None]: ...
) -> Generator[str | Mapping[str, Any]]: ...
@overload
def generate(
@@ -70,7 +70,7 @@ class CompletionAppGenerator(MessageBasedAppGenerator):
streaming: bool = False,
*,
session: Session,
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], None, None]: ...
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any]]: ...
def generate(
self,
@@ -81,7 +81,7 @@ class CompletionAppGenerator(MessageBasedAppGenerator):
streaming: bool = True,
*,
session: Session,
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], None, None]:
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any]]:
"""
Generate App response.
@@ -280,7 +280,7 @@ class CompletionAppGenerator(MessageBasedAppGenerator):
stream: bool = True,
*,
session: Session,
) -> Mapping | Generator[Mapping | str, None, None]:
) -> Mapping | Generator[Mapping | str]:
"""
Generate App response.

Some files were not shown because too many files have changed in this diff Show More