4 Commits

Author SHA1 Message Date
Tomás Barreiro 9c907af826 Send the Feature Flag Event when rolling out (#12325)
* Send the Feature Flag Event when rolling out

* Update apps/vscode-rollout/scripts/smoke-loader.mjs

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Saoud Rizwan <7799382+saoudrizwan@users.noreply.github.com>
2026-07-15 19:39:37 -07:00
Saoud Rizwan d618f8073a fix(vscode-rollout): align bundle versions and harden combined publish workflows (#12321)
* fix(vscode-rollout): align bundle versions in the stable AB workflow

Found by Max in local testing: the union manifest's version (what the
Marketplace and auto-update see) is the stitch input, but each bundle's
About tab and telemetry extension_version read that bundle's OWN
package.json — so the stable combined VSIX reported three different
versions (dispatch input / main's 4.0.0 / legacy's 4.0.8) depending on
where you looked. The nightly channel doesn't have this problem
(nightlify.mjs stamps one version into everything); this gives the stable
channel the identity-preserving equivalent: scripts/set-version.mjs stamps
the dispatch version into each checkout after install, before its build.

Also fixes a latent ab-package bug while restructuring the steps: the
next-bundle build never ran build:sdk, so the @cline/* workspace deps had
no dist and esbuild would fail on a fresh CI checkout (the workflow has
never run end-to-end — the publish environment gate blocked pre-merge
dispatches). Split install/build:sdk/align/build into separate steps,
mirroring the nightly workflow.

* fix(vscode-rollout): assert bundle sub-manifest versions in identity guardrails

Greptile round on #12321: the stable guardrail didn't assert version at
all. Went one further than the suggestion — both workflows' guardrails now
also assert each bundle sub-manifest's version (and name, for nightly)
matches the expected version, which is the check that actually regression-
guards the set-version.mjs/nightlify.mjs stamping (About tab + telemetry
extension_version read the sub-manifests, not the union). Expected version
routed through env rather than interpolated into the script body. Adds the
conventional paired test for set-version.mjs.

* fix(vscode-rollout): don't fail the nightly run when the tag push is rejected

First real combined publish (run 29454994164) published to both registries
successfully but the run went red at the last step: the default
GITHUB_TOKEN cannot create a ref whose commit modifies workflow files, and
HEAD was the #12253 squash merge which rewrote this very workflow. There
is no workflows permission grantable to the token, so this recurs any
night HEAD touched .github/workflows. The tag is bookkeeping — mark the
step continue-on-error so a successful publish isn't reported as a
failure. (Today's missing tag was pushed manually.)
2026-07-15 17:39:44 -07:00
Saoud Rizwan eb21ba583c fix(ci): restrict nightly publishing to main (#12322) 2026-07-15 15:35:09 -07:00
Saoud Rizwan f29c25395c feat(vscode-rollout): A/B loader and packaging for staged SDK extension rollout (#12253)
* feat(vscode-rollout): A/B loader and packaging for staged SDK extension rollout

Ship one marketplace VSIX containing a tiny loader plus two complete
extension bundles: next/ (SDK-based apps/vscode from main) and legacy/
(the legacy-extension branch). The loader picks one bundle per window
from a PostHog-flag-driven, sticky, one-way cohort assignment, activates
it with a Proxy-scoped ExtensionContext so each bundle resolves its
resources from its own subdirectory, and falls back to legacy (with
partial-registration cleanup and version pinning) if the next bundle
crashes during activation.

Includes the union-manifest generator with per-cohort when-clause
gating, the VSIX stitcher, a node-level loader smoke test, and the
ext-vscode-ab-package workflow that builds both refs and packages
(optionally publishes) the combined VSIX.

* fix(vscode-rollout): address rollout review feedback

* feat(vscode-rollout): versioned kill-switch, user-setting override, launch-cadence telemetry

Review follow-ups from #12253:

- Kill-switch is now scoped by version instead of boolean: the PostHog flag's
  payload carries {"maxKilledVersion": "x.y.z"} and the loader demotes only
  combined VSIXes <= that version, so killing a broken release never blocks
  the release that fixes it. Arming with no payload still demotes everything,
  and the old boolean memento format is normalized on read.

- cline.rollout.bundleOverride user setting (auto | next | legacy) as a
  manual escape hatch editable straight from settings.json: beats flags and
  the kill-switch in both directions, applies on window reload, reported as
  'override' on the activation event. Injected into the union manifest by
  gen-manifest so neither bundle has to know about it.

- parseRolloutFlags hardens flag typing: only a literal boolean true promotes
  (multivariate variants, numbers, junk fail safe), kill payloads are parsed
  defensively from /decide's JSON-string encoding.

- Activation events now carry ms_since_last_activation so the real window-
  reload cadence bounds how fast the rollout percentage gets dialed up.

- Walkthrough manifest invariant relaxed from byte-equality to structural
  equality (ids/media/completionEvents): the branches already diverge on one
  MCP step description, and since walkthrough markdown at the VSIX root comes
  from next regardless, hard-failing on copy tweaks bricked the release
  pipeline while protecting nothing. Copy divergence now warns and ships
  next's text.

* feat(vscode-rollout): identity-aware namespace, authoritative activation telemetry, nightly indicator

- Derive the setting section and sdkBundle context key from the packaged
  manifest name (cline.* for stable claude-dev, cline-nightly.* for the
  nightly identity, whose packaging rewrites the whole ID namespace);
  gen-manifest derives the same prefix for gates and the injected
  bundleOverride setting.
- Call the activated bundle's reportRolloutActivation export (merged on
  both branches) with attempted/actual/fallback — the authoritative
  extension.rollout.bundle_activated event, attributed via the bundle's
  variant-built telemetry. On crash fallback the LEGACY bundle reports it.
- Rename the loader's direct PostHog event to
  extension.rollout.loader_decision: it collided byte-for-byte with the
  bundles' event name under a different schema. It keeps the loader-side
  metadata (override, launch cadence, loader_version, extension_name) and
  gains double_failure for the both-bundles-dead case.
- Fix duplicate activation events on crash fallback: the recursive legacy
  activation no longer emits a second, contradictory fallback:false event.
- Nightly-only status bar indicator (Cline: Next / Cline: Legacy) so
  dogfooders can see which bundle a window is running.
- Union diverged engines to the newer requirement instead of hard-failing:
  main's VS Code engine (^1.101.0) has legitimately moved ahead of
  legacy-extension's (^1.84.0), which bricked every combined build.
- Smoke scenarios for all of the above.

* feat(vscode-rollout): publish the nightly as the combined A/B VSIX

Convert ext-vscode-publish-nightly.yml (cron + dispatch) from the
standalone SDK build to the combined loader + next + legacy package,
published as saoudrizwan.cline-nightly at <major>.<minor>.<unix-seconds>:

- scripts/nightlify.mjs reproduces publish-nightly.mjs's identity mutation
  (claude-dev -> cline-nightly, "cline. -> "cline-nightly., displayName,
  activity bar title) with the version as an explicit argument so ONE
  version reaches both bundle manifests and the union manifest. Runs after
  dependency install and before each bundle build.
- Both bundle builds get CLINE_ROLLOUT_VARIANT (next/legacy) in the nightly
  AND stable workflows — without it the merged rollout telemetry
  (extension_variant common prop + the authoritative bundle_activated
  capture) silently no-ops.
- dry-run dispatch input builds and uploads the installable .vsix without
  publishing or tagging; publish/tag steps are additionally gated to main,
  so the PR branch can be dispatched for pre-merge verification.
- Identity guardrails before packaging: nightly workflow asserts
  cline-nightly, the stable ab-package workflow asserts claude-dev.
- The nightly tag now records the legacy bundle sha in its message.
- README: nightly channel section (identity mapping, the two telemetry
  events and their owners, dry-run verification), and a note that the
  PostHog flags govern nightly only until the stable combined VSIX ships.

The single-bundle publish-nightly.mjs path remains for manual
feature-branch pre-release publishes; CI no longer invokes it.

* chore(vscode-rollout): harden nightly workflow gating

- Restore a job-level branch allowlist on the publish job (main + the
  rehearsal branch). Advisory defense-in-depth: the enforced gate is the
  PublishNightly environment's deployment-branch policy in repo settings,
  which must list the same branches; a dispatched branch runs its own copy
  of this file.
- Route the legacy-ref dispatch input through env instead of interpolating
  it into the run script body (script-injection hygiene; dispatch already
  requires write access).

* add otel vars to rollout build (#12316)

- Extension will not emit otel metrics to otel without these vars, so
adding those into the slow-rollout build workflow

Co-authored-by: Max Paulus 🥪 <max@cline.bot>

* fix(vscode-rollout): pass OTel env to the nightly legacy bundle build

Legacy's esbuild inlines OTEL_* at build time and its standalone publish
workflow passes them, so the combined nightly's legacy bundle was being
built with the OTel logs/metrics pipeline dead. Companion to #12316,
which fixes the same gap in ext-vscode-ab-package.yml (both bundles
there).

* feat(vscode-rollout): make the rollout two-way, remove the kill-switch

The one-way cohort + versioned kill-switch existed to avoid demoting users
whose SDK-bundle tasks aren't listed by legacy and whose rotated creds may
need a re-login. Decision: those are acceptable, temporary UX costs on an
emergency-only path — not worth a second flag and permanent mechanism
complexity (payload parsing, version scoping, killed-up-to cache format).

Now there is ONE knob: each background refresh caches exactly what
ext-sdk-bundle-rollout says for the next window. Dialing the percentage
down demotes; 0% pulls everyone back to legacy on their next reload.
Fail-safe direction preserved: only a literal boolean true promotes —
variant strings / numbers / a deleted flag all resolve to legacy; malformed
/decide responses leave the cache untouched. Local crash pinning (next
threw -> pin this version to legacy on this machine) is unchanged and
independent of the flag.

Removes KILLSWITCH_FLAG/KILLSWITCH_STATE_KEY/isVersionKilled/
normalizeKilledUpTo/compareVersions/nextCachedBundle; parseRolloutFlags
becomes parseRolloutAssignment returning the bundle to cache. Smoke
scenarios replaced with two-way promote/demote coverage.

---------

Co-authored-by: Max <maxpaulus43@gmail.com>
Co-authored-by: Max Paulus 🥪 <max@cline.bot>
2026-07-15 15:07:45 -07:00