opensource/dify
3
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-09-19 10:11:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

refactor(agents): simplify repository context (#39583)

Browse Source
This commit is contained in:
yyh
2026-07-26 06:13:37 +00:00
committed by GitHub
parent 1d34184999
commit fd2e26f8e6
33 changed files with 498 additions and 1665 deletions
Download Patch File Download Diff File Expand all files Collapse all files
cli/AGENTS.md
+15 -88
View File
@@ -1,99 +1,26 @@
# AGENTS.md — difyctl (TypeScript CLI)
TypeScript port of difyctl. Stack: custom CLI framework (`src/framework/`), Node 22+, ESM, ky for HTTP, Vitest, and Vite+ formatting and linting.
This package is the Node 22+, ESM TypeScript implementation of `difyctl`. Development also requires the Bun version pinned in `.bun-version`; command-tree generation and the `dev`, `test`, and `build` pre-scripts invoke it. Read [`ARD.md`] before adding a command or changing shared CLI infrastructure. Read `src/commands/AGENTS.md` for command-folder and registry rules.
> Architecture patterns, scaffolding recipe, printer chain, strategy pattern, testing conventions, anti-patterns: see **[`ARD.md`]**.
## Architecture Boundaries
## Code rules
- Every leaf command extends `DifyCommand`; command classes own framework parsing and delegate behavior to domain modules.
- Each command folder keeps its framework shell in `index.ts`. Extract behavior into sibling modules such as `run.ts` and `handlers.ts` when it needs an independently testable owner; those modules receive typed dependencies and do not import `src/framework/`.
- `src/http/` owns ky middleware and client construction; `src/api/` owns resource clients; `src/sys/io/` owns process streams and progress UI; `src/types/` remains a pure data and schema leaf.
- Preserve flags, output, and exit codes during refactors. Do not add dependencies or compatibility shims unless the task explicitly requires them.
- `ARD.md` owns CLI code structure. Keep wire behavior aligned with typed API clients and the real mock-server behavior tests.
- **Spaces, not tabs.**
- **Minimum comments.** Code speak for self. Comment only non-obvious WHY — hidden constraints, subtle invariants, bug-workaround notes. Never restate code. Never reference tasks, PRs, current callers.
- **No magic strings or numbers.** Enums or named constants for bounded value sets.
- **No long positional arg lists.** Use options objects.
- **No long if/switch ladders on discriminator.** Polymorphism, dispatch tables, or strategy pattern. Name concept, let implementations plug in.
- **No `any`. No `unknown` outside genuine wire boundaries** (HTTP body parse, env vars). Narrow types everywhere else.
- **Avoid `!` non-null assertions.** Narrow instead.
- **`readonly` on inputs not mutated.**
- **Discriminated unions** for variant data (SSE events, run outputs, error shapes), not optional-field bags.
- **No backwards-compat shims.** No re-exports of old names, no `// removed:` markers, no deprecation notes. Delete, update callers.
- **No new dependencies without explicit approval.**
- **No CLI behavior changes in refactor commit.** Same flags, same output, same exit codes.
- **Every leaf command extends `DifyCommand`.** Add `static agentGuide` string when command benefits from agent workflow docs — see `src/commands/AGENTS.md`.
## Commands
## Layering
Run package scripts from `cli/`:
| Layer | Path | Role |
| --------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| commands | `src/commands/` | Command class shells (extend `DifyCommand`). Only place framework imports run. |
| domain | `src/run/`, `src/get/`, etc. | Plain TS modules. Take typed deps via options. Testable without the framework. |
| api | `src/api/` | One typed client per resource. Each takes `KyInstance`. |
| http | `src/http/` | `createClient` + middleware (auth, retry, logging, error mapping). Only place ky runs. |
| io | `src/io/` | Streams + spinner. Fence between data-out and progress UI. |
| printers | `src/printers/` | `CompositePrintFlags` + `-o {json,yaml,name,wide,text}` matrix. |
| errors | `src/errors/` | `BaseError`, `ErrorCode` enum, `ExitCode` enum, dispatch table, `formatErrorForCli`. |
| guide | `src/commands/**/<cmd>/guide.ts` | Per-command agent guide string. Export `agentGuide`, assign `static agentGuide = agentGuide` in command class. Surfaced via `--help`. |
| cache | `src/cache/` | On-disk caches (app-info, etc.). |
| auth | `src/auth/` | Hosts file, token store, login flow. |
| config | `src/config/` | XDG dir resolution, config.yml load/save. |
| workspace | `src/workspace/` | Resolver: flag → env → bundle. |
| types | `src/types/` | Pure data + zod schemas for server contracts. No runtime imports outward. |
- Source CLI: `pnpm dev <command> [args...]`
- Tests: `pnpm test`
- Build: `pnpm build`
- Regenerate and verify the registry: `pnpm tree:gen` and `pnpm tree:check`
## Command Structure
Run the scoped static check from the repository root with `vp check cli`.
Scaffold recipe + checklist: see `ARD.md §New command scaffold`. Full folder convention (subcommands, guide.ts): see `src/commands/AGENTS.md`.
Layer rules:
- Commands thin shells. Use `this.authedCtx(opts)` for bearer context; delegate to domain function.
- Domain receives deps via options; never imports `src/framework/`.
- Only `src/http/client.ts` and `src/api/*` import ky at runtime; elsewhere use `import type { KyInstance }`.
- `process.*` lives in `src/io/`, `src/store/dir.ts`, `src/util/browser.ts`. Nowhere else.
- No circular imports. `types/` pure leaf.
## Dev commands
```sh
pnpm install # one-time
pnpm dev <command> [args...] # run CLI from source (no -- separator)
pnpm test # vitest
pnpm test:coverage # with coverage
pnpm -w check # repository-wide static check
pnpm -w check:fix # repository-wide static fixes
pnpm build # production bundle (vp pack)
pnpm tree:gen # regenerate src/commands/tree.ts (registry)
pnpm tree:check # verify tree.ts is up-to-date with the fs
```
Release binaries (5 platform targets, Bun-compiled) are produced by `pnpm build:bin` (called from `.github/workflows/cli-release.yml`).
## Tests
- Behavior tests run against real Hono mock at `test/fixtures/dify-mock/`. No `nock`, `msw`, or `fetchMock` — every test exercises real HTTP.
- Test files co-located: `foo.test.ts` next to `foo.ts`.
- The repository-wide static check and full test suite must be green before any commit.
## Spec docs (`docs/specs/`)
Behavior contracts. Living tree — amended in place, no version subfolders.
**Keep:** HTTP wire shape (req/resp JSON, headers, status codes), SQL DDL, Redis keys + TTL, state transitions, audit event names + payload, error/exit codes, rate-limit values, JWS/cookie envelope claims.
**Cut:** language type decls, internal helper sigs, decorator snippets, file-path tables, pseudocode mirroring code, "Open items"/"Handler walk"/"CI guard"/"Migration" sections, rationale (`Rejected:`/`Why X not Y`/`Historical note:`/product comparisons), release-pipeline lines, version-pinning (`in v1.0`, `post-v1.0`, milestone codes), frontmatter `date`/`status`/`author`.
**Test:** "rewrite in Rust tomorrow, does spec hold?" HTTP/SQL/Redis stays; type defs go.
**Rules:** behavior, not rationale. One topic per file; cross-refs = `auth.md §Storage`. Tables beat prose. Code wins on drift — update spec.
## Out of scope for unrelated work
Do not modify in passing:
- `test/fixtures/dify-mock/` public surface (endpoints, JSON shapes, status codes, scenario names) — that's the dify-api contract.
- `bin/`, `scripts/`, `Makefile`, `lint.config.ts`, `tsconfig*.json`, `package.json` (unless the change is required by the task).
## Commits
- One concern per commit. Style: `<type>(<scope>): <imperative subject>` lowercase. Body explains why if non-obvious.
- Never push, amend, force-push, or skip hooks (`--no-verify`) without explicit user approval.
Behavior tests use the real Hono server under `test/fixtures/dify-mock/`; do not replace it with `nock`, `msw`, or `fetchMock`. Keep tests colocated with their source files.
[`ARD.md`]: ARD.md
cli/ARD.md
+35 -79
View File
@@ -2,8 +2,6 @@
Onboarding ref for `dify/cli/` contributors. Cover canonical patterns, layer contracts, scaffolding recipe, dev workflow, anti-patterns. Read before adding command or touching shared infra.
Spec authority: [`docs/specs/`]. Specs own HTTP wire shape + server behavior; this file owns CLI code structure.
---
## Project layout
@@ -17,7 +15,7 @@ src/
config/ config.yml read/write
errors/ BaseError, ErrorCode, exit codes
http/ ky client factory + middleware
io/ IOStreams, spinner, printer chain
sys/io/ IOStreams, prompts, spinner, output rendering
limit/ --limit flag parsing
types/ shared TypeScript types
util/ small pure helpers
@@ -38,17 +36,17 @@ src/commands/<topic>/<verb>/
Examples: `get/app/`, `auth/devices/revoke/`, `describe/app/`.
**2. Mandatory files**
**2. Mandatory file**
| File | Responsibility |
| ---------- | --------------------------------------------------------------------------------------- |
| `index.ts` | `DifyCommand` subclass. Flag/arg declaration + `run()` wiring only. No business logic. |
| `run.ts` | Pure async function. Typed options + deps. Returns string. No `src/framework/` imports. |
| File | Responsibility |
| ---------- | ------------------------------------------------------------------------------------ |
| `index.ts` | `DifyCommand` subclass. Owns flag/arg parsing, framework output, and command wiring. |
**3. Optional files — add as needed**
| File | Purpose |
| ------------------ | ------------------------------------------------------------------ |
| `run.ts` | Typed behavior owner when logic merits independent tests or reuse |
| `handlers.ts` | Output types implementing `FormattedPrintable` or `TablePrintable` |
| `payload-shape.ts` | Response type narrowing/transformation |
| `run.test.ts` | Behavior tests against `run.ts` |
@@ -58,12 +56,12 @@ Examples: `get/app/`, `auth/devices/revoke/`, `describe/app/`.
- [ ] `index.ts` extends `DifyCommand`
- [ ] Authed command calls `this.authedCtx()`; non-authed skips
- [ ] No try/catch in `run()` — `DifyCommand.catch()` handles `BaseError`
- [ ] `run.ts` returns string; no direct stdout write
- [ ] `run.ts` no `src/framework/` imports
- [ ] Let the command boundary handle `BaseError`; catch only when the command owns recovery
- [ ] Keep framework parsing and output construction in `index.ts`
- [ ] When present, `run.ts` returns typed behavior data or owns explicit streaming/interactive I/O and does not import `src/framework/`
- [ ] HTTP client via factory dep, not direct
- [ ] `run.test.ts` written before impl (test-first)
- [ ] `pnpm tree:gen` run after adding command (updates `src/commands/tree.ts`)
- [ ] Add focused behavior tests when the command changes an observable contract
- [ ] `pnpm tree:gen` run after adding command (updates `src/commands/tree.generated.ts`)
- [ ] README command table updated by hand
---
@@ -74,27 +72,22 @@ All commands extend `DifyCommand`, not `Command`.
```typescript
export default class MyCommand extends DifyCommand {
async run(): Promise<void> {
async run(argv: string[]) {
const { args, flags } = this.parse(MyCommand, argv)
// Authed: authedCtx() sets outputFormat + builds context
const ctx = await this.authedCtx({ format: flags.output })
process.stdout.write(
await runMyThing(
{
// args
},
{ bundle: ctx.bundle, http: ctx.http, io: ctx.io },
),
const ctx = await this.authedCtx({ retryFlag: undefined, format: flags.output })
const result = await runMyThing(
{ id: args.id },
{ active: ctx.active, http: ctx.http, io: ctx.io },
)
return formatted({ format: flags.output, data: result.data })
}
}
```
**`authedCtx(opts)`** — wraps `buildAuthedContext`. Sets `this.outputFormat` as side effect. Required for any command needing bearer token.
**`authedCtx(opts)`** — wraps `buildAuthedContext` and returns the authenticated registry, account, HTTP, I/O, and optional cache dependencies. Pass the selected output format so authentication failures use the same serialization contract. Required for commands that need a bearer token.
**`catch(err)` override** — auto-handles `BaseError` with format-aware serialization. Never wrap `run()` in try/catch. Throw `BaseError`; base class catches.
The framework runner in `src/framework/run.ts` catches command errors, normalizes unknown failures, and serializes `BaseError` according to the selected output format. Catch inside a command only when that command owns a real recovery path.
---
@@ -103,8 +96,8 @@ export default class MyCommand extends DifyCommand {
Throw `BaseError`. Never throw raw `Error` for domain failures.
```typescript
import { BaseError } from '../../errors/base.js'
import { ErrorCode } from '../../errors/codes.js'
import { BaseError } from '@/errors/base'
import { ErrorCode } from '@/errors/codes'
throw new BaseError({
code: ErrorCode.UsageMissingArg,
@@ -113,7 +106,7 @@ throw new BaseError({
})
```
`ErrorCode` exhaustive const object — never use raw strings. `exitFor(code)` maps to exit codes auto. `DifyCommand.catch()` calls `formatErrorForCli` with `outputFormat` so JSON/YAML consumers get machine-readable error output.
`ErrorCode` is the exhaustive error-code object; do not scatter raw code strings. `exitFor(code)` maps it to a process exit code, and the framework runner calls `formatErrorForCli` so JSON/YAML consumers receive machine-readable errors.
| Exit | Meaning |
| ---- | ----------------------------------------- |
@@ -122,6 +115,7 @@ throw new BaseError({
| 2 | Usage error (bad flag, missing arg) |
| 4 | Auth error (not logged in, token expired) |
| 6 | Version/compat error |
| 7 | Rate limited |
New error code: add to `ErrorCode` + map to `ExitCode` in `codes.ts`. Never scatter exit codes inline.
@@ -172,7 +166,7 @@ Output rendering separated from data fetching via protocol objects.
- Data classes implement `TablePrintable` or `FormattedPrintable` from `src/framework/output`.
- Streaming commands implement `StreamPrinter` from `src/framework/stream`.
- `index.ts` wraps the result with `table({format, data})` or `formatted({format, data})` and returns it; the base class calls `stringifyOutput()`.
- `index.ts` wraps the result with `table({format, data})` or `formatted({format, data})` and returns it; `src/framework/run.ts` calls `stringifyOutput()`.
- Commands that write incrementally (streaming) write directly from the strategy via `deps.io.out.write(stringifyOutput(...))`.
```typescript
@@ -220,51 +214,15 @@ New mode = new class + one line in picker. Singletons avoid per-call allocation.
## HTTP clients
One file per resource under `src/api/`. Each exports class wrapping `KyInstance`.
Keep resource clients under `src/api/`. They receive the shared `HttpClient` and call generated oRPC operations through `createOpenApiClient(...)` when the OpenAPI contract covers the endpoint. Reuse generated request and response types instead of duplicating wire shapes.
```typescript
export class AppsClient {
private readonly http: KyInstance
constructor(http: KyInstance) {
this.http = http
}
async list(params: ListParams): Promise<ListResponse> {
/* ... */ throw new Error('elided')
}
async describe(id: string, workspaceId: string, fields: string[]): Promise<DescribeResponse> {
/* ... */ throw new Error('elided')
}
}
```
Inject via factory dep in `run.ts` for testability:
```typescript
type GetAppDeps = {
appsFactory?: (http: KyInstance) => AppsClient
}
// default: (h) => new AppsClient(h)
```
Never instantiate clients in `index.ts`.
Pass `HttpClient` into behavior owners. Add a client or factory dependency only when it owns a real substitution or lifecycle boundary; behavior tests normally exercise the real client stack against `test/fixtures/dify-mock/`. Keep client construction out of `index.ts` so the command remains a framework and output boundary.
---
## Testing
**Test-first.** Write failing test, run to confirm fail, then implement.
Tests live in `run.test.ts` alongside command. Test `run.ts` direct — never the `DifyCommand` class.
```typescript
const io = bufferStreams()
const result = await runGetApp(
{ format: 'json', appId: 'app-1' },
{ bundle, http: mockHttp, io, appsFactory: () => fakeClient },
)
expect(JSON.parse(result).data).toHaveLength(1)
```
Keep tests beside the owner as `*.test.ts`. When a command has a behavior module, test that public function directly for domain and protocol behavior. Test the command class or framework boundary when argument parsing, flags, help, output construction, or command wiring is the observable contract. Establish a failing case first when practical for behavior changes and bug fixes.
### dify-mock fixture server
@@ -306,14 +264,14 @@ expect(JSON.parse(out).workspaces).toHaveLength(2)
| `pnpm dev <cmd> [args]` | Run CLI from source during dev |
| `pnpm test` | Full vitest suite — run before every commit |
| `pnpm test:coverage` | Coverage report |
| `pnpm -w check` | Repository-wide static check |
| `pnpm -w check:fix` | Repository-wide static fixes |
| `vp check cli` | Scoped static check from the repository root |
| `vp check --fix cli` | Scoped static fixes from the repository root |
| `pnpm build` | Production bundle (`vp pack`) |
| `pnpm tree:gen` | Regenerate `src/commands/tree.ts` (registry) |
| `pnpm tree:check` | Verify `tree.ts` matches the filesystem |
| `pnpm tree:gen` | Regenerate `src/commands/tree.generated.ts` |
| `pnpm tree:check` | Verify the generated tree matches the commands |
| `pnpm build:bin` | Cross-compile standalone binaries via Bun (CI) |
**`pnpm tree:gen` rule:** run after adding, removing, renaming any command. The generated `tree.ts` is the runtime command registry — stale tree causes commands to be invisible at runtime. (Runs implicitly via `prebuild`/`predev`/`pretest`.)
**`pnpm tree:gen` rule:** run after adding, removing, or renaming any command. The generated `tree.generated.ts` is the runtime command registry; a stale tree makes commands invisible at runtime. It also runs through `prebuild`, `predev`, and `pretest`.
**README hand-maintained.** When adding a command, update the command table in `README.md` manually.
@@ -331,7 +289,7 @@ The repository runs Vite+ Oxlint as the primary code-quality linter, an explicit
| `unicorn/no-new-array` | Use `Array.from({ length: n })` not `new Array(n)` |
| `noUncheckedIndexedAccess` (tsc) | `arr[i]` is `T \| undefined`; guard before use |
Run `pnpm -w check:fix` for Oxlint, ESLint, TypeScript, and Oxfmt fixes and diagnostics.
Run `vp check --fix cli` from the repository root for scoped formatting, lint, and TypeScript fixes and diagnostics.
---
@@ -350,14 +308,12 @@ Run `pnpm -w check:fix` for Oxlint, ESLint, TypeScript, and Oxfmt fixes and diag
| Pattern | Do instead |
| -------------------------------------------------------------------- | -------------------------------------------------------------------------- |
| `if (format === 'json') { ... }` in `run.ts` | Printer handler per format |
| `try { ... } catch (e) { if (isBaseError(e)) ... }` in every command | Throw `BaseError`; `DifyCommand.catch()` handles |
| `try { ... } catch (e) { if (isBaseError(e)) ... }` in every command | Throw `BaseError`; `src/framework/run.ts` normalizes and formats it |
| Raw string error codes `'not_logged_in'` | `ErrorCode.NotLoggedIn` |
| `enabled: !isHuman` in `runWithSpinner` | Set `outputFormat` on `IOStreams`; spinner auto-detects |
| Long positional arg lists | Options struct |
| `Record<string, Strategy>` dispatch map | Named singletons + picker function |
| `src/framework/` import in `run.ts`, `api/`, or `auth/` | Framework imports belong in `index.ts`, `handlers.ts`, and strategies only |
| `buildAuthedContext(this, opts)` in command body | `this.authedCtx(opts)` |
| `console.log` in `src/` | Return string from `run.ts`; write in `index.ts` |
| `console.log` in `src/` | Return `CommandOutput` from the command or use owned I/O for streaming |
| New dependency without approval | Check first |
[`docs/specs/`]: docs/specs/
cli/src/commands/AGENTS.md
+6 -4
View File
@@ -13,7 +13,7 @@ src/commands/
<topic>/
<verb>/
index.ts ← command class (extends DifyCommand; the ONLY file the registry discovers)
run.ts ← business logic (not a command, invisible to the registry)
run.ts ← optional behavior owner (not a command, invisible to the registry)
handlers.ts ← helpers
guide.ts ← agent guide string (optional)
*.test.ts ← tests
@@ -23,7 +23,7 @@ src/commands/
<shared>.ts
```
The registry generator (`pnpm tree:gen` → `src/commands/tree.ts`) discovers
The registry generator (`pnpm tree:gen` → `src/commands/tree.generated.ts`) discovers
commands only via `**/index.+(js|cjs|mjs|ts)`. All other files in command
folders are invisible to the registry — add freely without glob exclusions.
Folders prefixed with `_` (e.g. `_shared/`, `_strategies/`) are excluded from
@@ -32,7 +32,7 @@ registry discovery and from coverage checks.
## Adding a new command
1. Create `src/commands/<topic>/<verb>/index.ts` extending `DifyCommand`.
1. Add business logic in sibling files (e.g. `run.ts`, `handlers.ts`).
1. Keep small owner-local behavior in `index.ts`; extract sibling modules such as `run.ts` or `handlers.ts` when logic needs independent tests, reuse, or a clearer owner.
1. Run `pnpm tree:gen` to regenerate the command tree (also runs implicitly via `prebuild`/`predev`/`pretest`).
1. Run `pnpm test` to verify coverage.
@@ -54,7 +54,9 @@ registry discovery and from coverage checks.
import { agentGuide } from './guide.js'
export default class MyCmd extends DifyCommand {
static agentGuide = agentGuide
override agentGuide(): string {
return agentGuide
}
}
```
1. The guide appears at the bottom of `difyctl <cmd> --help` automatically.