opensource/WeKnora
3
0
Fork 0
mirror of https://github.com/Tencent/WeKnora.git synced 2026-11-03 14:33:31 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,665 Commits 197 Branches 47 Tags
26fa43e2ccbb65ba5512a3ed6b60df01bf25e430
Commit Graph
12 Commits
Author SHA1 Message Date
nullkey 69fbbfc252 chore(cli): gofmt sweep 
`gofmt -w cli/` — trailing newlines, minor whitespace alignment. Caught
during PR-readiness review (`gofmt -l .` had been non-clean since the
em-dash → ASCII bulk replace earlier in the v0.4 series).
 2026-05-15 12:03:56 +08:00
nullkey f2e8e3f56c refactor(cli): drop aiclient package; align AGENTS.md with mainstream 
Survey of 10 mainstream CLIs (gh, lark, stripe, vercel, supabase, aws,
azure, gcloud, openai/codex, github-copilot-cli) showed env-gated
per-command --help blurbs are a Stripe-only pattern; gh uses env detect
for telemetry only, and lark relies on installed agent Skills + MCP.
Our cmd/mcp/serve already covers the dominant 2025/26 path, so
internal/aiclient/ (136 LOC + 38 callsites) is net maintenance burden
without precedent.

- Drop internal/aiclient/ entirely (annotations + detect + tests)
- Remove 38 SetAgentHelp callsites + agentAwareHelpFunc / SetHelpFunc
  wiring in cmd/root.go
- Migrate 4 command-level rules to standard Long help (visible to all,
  not env-gated): doc upload mode mutex, kb edit at-least-one,
  kb pin idempotent, search chunks channel mutex
- Rewrite AGENTS.md as a developer guide (gh-style 6 H2 / 167 lines):
  audience preamble + Build / Architecture / Command Structure /
  Testing / Code Style / Error Handling. Drops sections absent in
  surveyed projects (Commit & PR Conventions, Who Uses This CLI)
- Clean 14 internal doc refs (ADR-N, spec §X, v0.X) in source comments
  and docs that pointed at docs/superpowers/ — that directory is
  local-only / uncommitted, so refs are dead for outside readers
- Drop forward-looking "once v0.2 ships" from README
 2026-05-15 12:03:56 +08:00
nullkey e623e8208f refactor(cli): delete envelope infrastructure, errors to stderr 
Removes the entire envelope machinery now that every success path
emits bare JSON:

- cli/internal/format/envelope.go (Envelope, Success, Failure,
  SuccessWithRisk, WriteEnvelope, Meta, Notice, UpdateNotice,
  VersionSkewNotice, Risk, RiskLevel, ErrorBody) + tests.
- cli/internal/format/filter.go envelope-specific helpers
  (WriteEnvelopeFiltered, marshalEnvelope, applyFieldFilter,
  filterDataPayload, filterObjectData); the reusable
  filterArrayItems / filterObjectKeys / writeJQ stay for bare.go.
- cli/internal/cmdutil/exporter.go + tests (envelope-only).
- cli/internal/cmdutil/PrintErrorEnvelope + ToErrorBody +
  operationRiskOf + Error.OperationRisk field + OperationRisk struct.

Error path: all errors now go to stderr via cmdutil.PrintError in
`code: message\nhint: ...` form, regardless of --json. Stdout stays
empty (or holds the partial-success the command already wrote) so
downstream `--json | jq` pipelines never have to filter error shapes
out of the success stream. Typed exit codes (3 auth.* / 4
resource.not_found / 5 input.* / 6 server.rate_limited / 7 server.*
+ network.* / 10 input.confirmation_required) carry the failure
class for agents that branch on it.

Acceptance contract:
- envelope_test.go → wire_test.go (TestEnvelopeGolden → TestWireGolden).
- testdata/envelopes/ → testdata/wire/.
- Error-path cases assert the typed code substring on stderr.
- Orphan whoami.*.json goldens deleted.

AGENTS.md + README.md rewritten for the bare-data contract:
- Drop envelope schema section + dry-run rule.
- Document bare JSON on stdout + `code: msg\nhint: …` on stderr.
- ADR-3 reframed around bare data and why error separation matters
  for `--json | jq` pipelines.

WriteJSONFiltered short-circuits to WriteJSON when both filters are
empty (skip the marshal-buffer round-trip for the common case).

Final review pass:
- Fix wire-contract bug: `--json id,name` (space form) is broken by
  pflag's NoOptDefVal; AGENTS.md / README.md / SetAgentHelp + the
  field-discovery help text all switched to `--json=id,name`.
- Fix `weknora api --jq` silently ignored: api.go now routes through
  WriteJSONFiltered with jopts.JQ.
- AGENTS.md: drop the false claim that `auth logout` honors `-y`
  (logout is local-only with no ConfirmDestructive guard); list the
  actual destructive commands instead.
- Rewrite cli/acceptance/e2e/e2e_test.go for the bare-data wire shape
  (was still parsing `out["data"]` / `env["ok"]`).
- Add `JSONOptions.Emit(w, v)` helper; collapse ~33 repeated
  `format.WriteJSONFiltered(iostreams.IO.Out, X, jopts.Fields,
  jopts.JQ)` sites to `jopts.Emit(iostreams.IO.Out, X)` — drops the
  format import from 22 cmd/* files.
- Delete single-caller `cmdutil.MustRequireFlag`; inline as
  `_ = cmd.MarkFlagRequired(...)` everywhere.
- Add `_ = cmd.MarkFlagRequired("name")` to `kb create`; it was the
  only write command relying on runtime --name validation while
  `context add` already used the cobra-level mark.
- `context use`: register `--json` / `--jq` (was always emitting JSON
  unconditionally with no human path and no flag — diverged from
  every other write command); human mode now prints
  `✓ Switched context to X (was Y)`.
- Replace per-package `confirmPrompter` / `scriptedConfirm` /
  `errPrompter` test doubles with `testutil.ConfirmPrompter`.
- Rename `chatService` → `ChatService` (export to match siblings
  `ListService` / `ViewService`); rename `printUploadSuccess` →
  `renderUploadSuccess` (siblings use `render*`).
- `defaultHint(CodeResourceNotFound)`: drop the hardcoded
  "list available with `weknora kb list`" — misleading on agent /
  doc / session 404. Replaced with "verify the resource ID and try
  again".
- Strip stale `v0.2/v0.3` / "envelope" / "v0.0/v0.1 supports only"
  historical tags from production comments and a few test
  descriptions.
 2026-05-15 12:03:56 +08:00
nullkey cc8254f862 refactor(cli): drop --dry-run + introduce bare-JSON output path 
Two intertwined mainstream-alignment moves bundled because they share
the migration target (every command's --json path):

1. Drop --dry-run entirely. Survey of comparable API-wrapper CLIs
   (gh, aws, stripe, lark): none expose --dry-run. The mainstream that
   does (kubectl/git/helm/ansible) operates on declarative manifests
   or local state where the preview is materially different from the
   executed action. WeKnora's CLI just echoed the same parameters
   that would have gone on the wire — the preview added no real
   signal over `--help` + reading the call site. Removes:
   - root --dry-run persistent flag + cmdutil/dryrun.go
   - DryRun fields + EmitDryRun calls in 12 write commands
   - format.Envelope.DryRun field
   - 8 corresponding *_test.go cases
   - --dry-run mention from README.md and CHANGELOG.md
   - "dry_run":false from 16 golden envelopes

2. Migrate every --json output to bare data:
   - New format.WriteJSON / WriteJSONFiltered helpers
     (cli/internal/format/bare.go) share filterArrayItems /
     filterObjectKeys / writeJQ with the (still-live for now) envelope
     filter helpers.
   - Read commands (kb/doc/session list+view, search chunks/docs/
     sessions/kb, auth list/status, agent list/view, context list,
     doctor) emit bare arrays / objects on stdout.
   - Write commands (kb create/edit/delete/pin/empty, doc upload/
     upload_recursive/delete, session delete, auth login/logout/
     refresh/token, link/unlink, context add/use/remove, agent
     invoke, chat, api, version) emit bare result objects. Risk
     classification dropped — the resource + exit code already
     convey the action.

Per-command shape changes:
   list / search       → []T   (was {ok, data:{items:[…]}})
   view                → T     (was {ok, data:T, _meta:…})
   create / edit       → T
   delete / pin / etc. → {id, …action result…}
   doctor              → {summary, checks}
   api                 → {status, headers, body}

_meta dropped on the read path:
   pagination (page/page_size/total/has_more) — agents iterate with
   --all-pages or accept --limit (gh CLI parity);
   kb_id / context echo — caller already knows what it asked for.

Acceptance contract goldens regenerated for the new bare shape.
Error envelope on stdout (PrintErrorEnvelope) stays live for now —
the envelope-infra deletion lands in the next commit.
 2026-05-15 12:03:56 +08:00
nullkey bdc589e1c0 refactor(cli): --limit/--all-pages, Go 1.26, internal/agent → aiclient 
Cross-cutting cleanup that lands alongside the new feature surface:

- `--limit / -L` and `--all-pages` on every list command. Default
  --limit 30 (gh-parity); --all-pages drains every server page
  client-side, capped by --limit. Closes the audit finding that the
  old "1000 max per call" implicit cap was undiscoverable.
- `auth token` emits a TTY-only stderr advisory when stdout is a
  terminal (the credential just got displayed in scrollback) plus an
  api-key-mode rotation hint.
- Comment + doc discipline pass: drop external project name
  references from in-code comments (we reference them in design notes,
  not inline).
- Bump `go` directive to 1.26.0 and CI matrix to 1.26.x to align with
  the main module's go.mod.
- Rename `cli/internal/agent` → `cli/internal/aiclient` to
  disambiguate from the new `cli/cmd/agent` resource subtree. The
  package handles AI coding-agent env detection + per-command --help
  annotations; the new name reflects that more precisely.
 2026-05-15 12:03:56 +08:00
nullkey 9bb83b47fd feat(cli): mcp serve curated stdio MCP server 
`weknora mcp serve` — long-lived stdio MCP (Model Context Protocol)
transport that exposes a fixed, curated tool surface to MCP-aware
agents (Claude Desktop, Claude Code, custom MCP clients).

Curated tool set (readonly by default):
- whoami — active context + tenant
- search (hybrid retrieval against a KB)
- kb list / view
- doc list / view
- agent list / view / invoke
- session list / view

The list is intentionally narrow to the read + agent-invoke surface;
destructive verbs (`delete` / `empty` / `upload`) are gated behind
`--write`. Schema is built from each leaf cobra command's flags so
adding a new tool is a single registry entry plus a Service interface.

Includes the simplify post-review polish + a second simplify pass to
fold the resulting feedback (typed schemas, agent_help wording, unify
chat / agent invoke option names).
 2026-05-15 12:03:56 +08:00
nullkey 1b20b06f5e feat(cli): --json field-select, --jq, auth token, doc --from-url 
Output ergonomics:
- `--json` accepts a comma-separated field list (gh-parity); selects
  named keys from the per-command payload. Bare `--json` keeps the
  full shape.
- `--jq <expr>` evaluates a gojq expression over the JSON; pairs with
  `--json field-list` so projection runs before jq.
- `--version` is a global cobra flag in addition to the `version`
  subcommand; both render the same line.
- Per-command `--help` now renders the available JSON field list under
  "JSON fields available via `--json id,name,...`" (field-discovery
  parity with gh / kubectl `-o jsonpath`).

New commands:
- `auth token` — print the active context's credential to stdout for
  shell command substitution (`WEKNORA_TOKEN=$(weknora auth token)`).
  Default: raw secret, no trailing newline. `--json` emits
  `{token, mode, context}`.
- `doc upload --from-url <URL>` — ingest a remote URL via the SDK
  `CreateKnowledgeFromURL`. `--name` forwarded as `FileName` so the
  server's known-extension heuristic upgrades crawl-mode to
  file-download-mode where appropriate.

Includes the simplify post-review polish pass (field-filter unit
tests, --json/--jq compose check, agent_help copy fixes).
 2026-05-15 12:03:56 +08:00
nullkey 5adcedf170 refactor(cli): v0.3 cross-cutting cleanup 
Cross-cutting findings surfaced by the branch-completion review.

Perf bug:
- Factory.Client closure was not memoized. Factory.ResolveKB internally
  calls f.Client() to resolve --kb name → id, then the command's RunE
  calls f.Client() again. Two SDK clients, two keyring reads, two
  AuthRetryTransport allocations per name-resolved invocation, with
  *independent* token state (a refresh in one was invisible to the
  other). Switched to sync.Once like Secrets already does.

Silent bug bait:
- cmdutil.NormalizeHost docstring claimed CodeInputMissingFlag for the
  empty case; code returned CodeInputInvalidArgument. Aligned doc to
  code (present-but-empty is a bad value, not a missing flag).

Agent contract gaps:
- Five user-facing subcommands lacked SetAgentHelp: auth login /
  logout / list / status and chat. Added concise strings with error-
  code call-outs so agents can branch without parsing human strings.

Helper extraction (≥3 callers):
- text.KnowledgeDisplayName(fileName, title, id) — byte-identical
  formatter that was in both cmd/doc/list.go and cmd/search/docs.go.
  Takes fields directly so internal/text stays SDK-free.
- cmdutil.WrapHTTP(cause, fmt, args...) *Error — replaces the
  `Wrapf(ClassifyHTTPError(err), err, ...)` pattern across 24 SDK
  call sites. Sed-driven migration; off-pattern shapes in chat.go
  (used streamErr) and cmdutil/kb.go (in-package) hand-edited.
  Contract test gains a comment update: post-migration the dominant
  pattern is WrapHTTP which the AST scanner skips entirely (only
  NewError/Wrapf selectors inspected); ClassifyHTTPErrorOutputs()
  bridge still covers the dynamic codes those paths can yield.

UX consistency:
- cmd/doc/list.go --page-size help now reads "Items per page
  (1..1000)" matching cmd/session/list.go. The bounds validation
  already enforced 1..1000; the help text was the last drift.

Comment-discipline sweep:
- Deleted the WHAT-only "*Options captures `weknora ...` flag state"
  docstring across 23 files (context, kb, auth, doc, session, search,
  chat, doctor, link). Where the line carried a real WHY clause
  (kb/delete, doc/delete, session/delete, kb/edit), kept the WHY and
  dropped only the leading WHAT phrase.
- Stripped third-party project-name attribution from inline comments
  and one user-visible flag-help string across ~40 files in cli/cmd
  and cli/internal (plus 4 test-file comments). Removed phrases like
  "Mirrors `gh X`", "borrowed from lark-cli", "kubectl-style",
  "gcloud `--project`", "Stripe pattern", and the embedded GitHub
  URLs pointing at those projects. Behavioral descriptions and the
  WHY behind each comment are preserved; only the upstream-name
  attribution is gone. Inspiration / north-star references belong in
  cli/AGENTS.md (the design doc) and commit messages, not scattered
  through every file.

  Triggered by an audit round that surfaced several false / fragile
  parity claims (e.g. "Mirrors `gh repo edit`" — gh repo edit has no
  --name flag; "matches gcloud `--project` id-or-name" — gcloud's
  --project accepts ID only). Rather than fix them one by one, the
  whole category of in-comment external-project references was
  stripped uniformly.
 2026-05-14 10:57:17 +08:00
nullkey 41a98b5743 feat(cli): context CRUD 
New v0.3 P0 entry 3-4: kubectl-style context-management subtree using
gh's `<noun> <verb>` surface convention consistent with the rest of
this CLI.

- context list: tabwriter rendering + --json envelope; reads config.yaml
  only.
- context add <name> --host <url> [--user]: validates http(s) URL, first
  context auto-becomes current, rejects duplicates with did-you-mean.
- context remove <name>: best-effort keyring cleanup like `auth logout`.
  Removing the current context triggers exit-10 confirmation (lark-cli
  skill protocol) — subsequent commands would lose their default
  --context.

(`context use` predates v0.3; the subtree was previously use-only.)

Bugs caught and fixed inline by the post-commit reviewer round:
- auth login was accepting `http://` (empty host portion) because the
  old validateHost only checked the scheme. New cmdutil.NormalizeHost
  (shared by both login and context add) requires u.Host != "".
- context add's validateName claimed `..` was rejected but only denied
  / \\ space. Switched to positive allowlist [A-Za-z0-9._-] plus
  explicit ./../path-separator rejection.

Helper consolidation:
- cli/internal/cmdutil/host.go: NormalizeHost (trim, scheme, host
  non-empty) — both auth login and context add share it.
- cli/internal/format/dash.go: DashIfEmpty — promoted from copies in
  cmd/auth/list.go and cmd/context/list.go.
- recordingStore test stub dropped in favor of secrets.NewMemStore;
  contextKeyList test helper replaced by the existing contextKeys.

14 unit tests; 13 e2e branches verified.

Roadmap: 3-4.
 2026-05-14 10:57:17 +08:00
nullkey da9faa9e07 feat(cli): add agent-first affordance — envelope, exit-10, --dry-run 
Borrows the lark-cli agent-affordance model
(https://github.com/larksuite/cli/blob/main/AGENTS.md +
skills/lark-shared/SKILL.md) so weknora is designed to be agent-friendly:
error messages, output format, and flag design follow conventions agents
can rely on.

cli/AGENTS.md (operational reference for LLM agents invoking weknora):
  Public document covering envelope schema, exit-code protocol
  (0/1/2/10/130), stdout/stderr separation, and behavioral rules.
  Sensitive commands (\`context use\`, \`kb delete\`, \`doc delete\`, \`init\`)
  gain "AI agents:" paragraphs in their cobra Long descriptions so
  guidance shows in --help.

format.Envelope schema additions:
  Risk    per-operation classification (read / write / high-risk-write +
          action description), populated by write commands on both success
          and failure paths.
  Notice  system advisories (CLI update available, server-CLI version
          skew); type defined, emit sites land in v0.3.
  DryRun  marker for envelopes returned from --dry-run preview paths.

  RiskLevel constants realigned to lark's taxonomy: read / write /
  high-risk-write (was: read / mutating / destructive — not yet wired by
  any command).

  cmdutil.Error gains OperationRisk; PrintErrorEnvelope auto-attaches it
  to envelope.Risk so destructive failure paths surface uniformly.

Exit-10 confirmation protocol:
  New ErrorCode \`input.confirmation_required\` mapped to exit code 10 in
  cmdutil.ExitCode. ConfirmDestructive now returns this code (with
  OperationRisk attached) when stdout is non-TTY or --json was set, with
  -y/--yes absent. Previous behavior — silent proceed in non-TTY — was
  unsafe: scripts and agents could delete resources with no explicit
  approval. Three test cases re-pinned around the new contract.

  This is a wire-contract change for any caller who relied on silent
  proceed; v0.0/v0.1 had no destructive commands, so the blast radius is
  contained to v0.2 itself.

--dry-run global flag:
  cmd write paths (kb create/delete, doc upload/delete, api POST/PUT/PATCH/
  DELETE) check cmdutil.IsDryRun(cmd) and skip the SDK call, emitting an
  envelope with dry_run=true plus a Risk classification. Read commands
  ignore --dry-run by design (no side effect to preview). Human-mode
  prints \`[dry-run] would <action>\` to stdout.

Command discovery: agents introspect via the existing \`--help\` surface
(consistent with gh / kubectl / aws / gcloud / terraform — none of them
ship a CLI-tree self-description command). An earlier draft added a
\`weknora schema\` reflection command; dropped after a mainstream survey
found it has no stable analog (lark-cli's schema describes Lark API
methods, not its own CLI tree).

Tests: 27 cli packages pass at this commit. Added two new tests covering
envelope.risk and envelope._notice serialization.
 2026-05-12 13:20:42 +08:00
nullkey 9d2e740753 refactor(cli): align command surface with gh CLI conventions (ADR-3) 
Audited the v0.0~v0.2 21-command surface against gh / kubectl / cargo /
npm / git / docker / flyctl / vercel / supabase / brew. WeKnora was
cherry-picking from multiple heritages, producing an inconsistent feel:
the kb subtree mixed gh verbs (create / delete / list) with a kubectl
verb (get); confirmation flag duplicated --force (docker/kubectl) with
global -y/--yes (gh/vercel/npm); the --agent flag stretched Stripe's
telemetry-tag pattern into a behavior-mode switch that no mainstream
CLI does.

ADR-3 picks gh as the primary north star. Documented deviations remain
for project-link (vercel/cargo), chat (openai-cli), context (kubectl-
light), and doctor (brew/flutter). The decision and its deviations are
documented self-contained in cli/AGENTS.md.

Surface changes:

  - kb get → kb view (gh repo view convention); "get" kept as cobra Alias
    for v0.0/v0.1 callers — see https://cli.github.com/manual/gh_repo_view.

  - kb delete --force / doc delete --force removed in favor of the global
    -y/--yes persistent flag (gh repo delete --yes convention). One
    mechanism skips destructive prompts; ConfirmDestructive's parameter
    renamed `force` → `yes` to match.

  - --agent omnibus mode-switch removed. Stripe's DetectAIAgent (the
    cited inspiration) only tags User-Agent for telemetry, never flips
    behavior; gh / kubectl / aws / docker / flyctl all decline this kind
    of flag. The 7-env auto-detect list is reduced to the two entries
    Stripe also recognizes (CLAUDECODE, CURSOR_AGENT) — the other five
    had no agent-documented source. ApplyAgentSugar / ShouldUseAgentMode
    and the dead --no-interactive / --no-progress globals are deleted
    entirely.

  - DetectAIAgent and SetAgentHelp annotations are kept: env detection
    now only triggers AGENT-targeted help text rendering (no behavior
    change), matching Stripe's narrower scope.

Tests: 27 cli packages green (acceptance/contract still pins kb get
golden; the alias keeps it valid).
 2026-05-12 13:20:42 +08:00
nullkey a39805aa89 feat(cli): scaffold weknora CLI module foundation (PR-1) 
Empty cli/ Go module with Factory + Options skeleton that all v0.x commands
plug into.

Foundation packages (10 internal/):
- cmdutil: Factory(3 closure: Config/Client/Prompter) + Options + Errors
  with namespaced ErrorCode + typed predicates (IsAuthError / IsNotFound /
  IsTransient / IsAuthExpired) + Exporter + AddJSONFlags + ExitCode/PrintError
- iostreams: package singleton with TTY detection; ColorEnabled honors
  NO_COLOR / TERM=dumb on demand
- agent: mode + detect + annotations (Stripe pkg/useragent pattern;
  CLAUDECODE / CURSOR_AGENT / CODEX_* / AIDER_* / CONTINUE_* /
  OPENCODE_* / GEMINICODER env detection)
- build: ldflags-injected version/commit/date
- safepaths: separator-aware path-traversal protection
- config: yaml.v3 schema + atomic save 0600 (XDG-style)
- format: success/failure envelope (code/hint/request_id/risk/retryable/
  console_url) with typed RiskLevel
- prompt: Prompter interface + AgentPrompter (rejects in agent mode)
- secrets: file 0600 store with weknora:<context>:<key> namespace
- testutil: XDGTempDir(t) test helper

Smoke command: weknora version (-/--json). Build/test/vet matrix in
.github/workflows/cli.yml across linux x macos x windows x Go 1.24.

Factory.Client returns CodeLocalUnimplemented until PR-3 wires the SDK
adapter; PR-1 commands (version) don't need it.
 2026-05-07 23:19:46 +08:00