improvement(tests+ci): phase 3 — shared-mock convergence completion and CI runner-minute cuts (#5875)

* chore(ci): cut redundant runner minutes — dedup promotion-PR test runs, companion-pr-check concurrency, right-size trivial jobs - ci.yml: new dedup-promotion gate skips the pull_request test-build on staging/main-headed promotion PRs only when the merge tree provably equals the head tree (empty base delta over the merge base) AND the push-event run at the same sha passed its test jobs (polled). Fail-open on any error/ timeout/failure, job-level skip only (skipped job reports Success); verified no required status checks are configured on main/staging rulesets. Measured 39 duplicate PR runs / 5.15 days (~227/mo) at ~7.1 min each on 8vcpu (~57 vcpu-min), probe costs ~9 vcpu-min worst case on 2vcpu. - companion-pr-check.yml: per-PR concurrency group with cancel-in-progress so superseded synchronize/edit runs stop; no paths filter (check depends on PR body + cross-repo state, not changed files). - detect-version and check-docs-changes: 4vcpu -> 2vcpu Blacksmith runners (pure shell / depth-2 checkout + path filter only). * improvement(testing): complete stateful shared mocks for env, urls, redis-config, environment-utils Shared mock infrastructure for vitest isolate:false convergence: - packages/testing/src/mocks/env.mock.ts: stateful envMock (live env proxy, setEnv/resetEnvMock, process.env fallback) - packages/testing/src/mocks/urls.mock.ts: complete urlsMock with real-behavior default impls + resetUrlsMock - packages/testing/src/mocks/redis-config.mock.ts: adds getRedisConnectionDefaults + resetRedisConfigMock - packages/testing/src/mocks/environment-utils.mock.ts: new environmentUtilsMock + fns + reset - contract tests: env.mock.test.ts, urls.mock.test.ts, redis-config.mock.test.ts, environment-utils.mock.test.ts - packages/testing/src/mocks/index.ts: barrel exports - apps/sim/vitest.setup.ts: global installs for env, urls, redis, environment/utils - real-module tests unmocked: lib/core/config/env.test.ts, lib/core/config/redis.test.ts, lib/core/utils/urls.test.ts, tools/index.test.ts (urls) - stubEnv/process.env fallout migrated to setEnv: lib/webhooks/providers/{revenuecat,rootly,instantly}.test.ts, app/api/auth/oauth2/authorize/route.test.ts * improvement(tests): drop redundant local mocks in executor/tools/providers and misc dirs (shared-worker readiness) * improvement(tests): drop redundant local mocks in app routes (shared-worker readiness) * improvement(tests): drop redundant local mocks in lib (shared-worker readiness) * fix(ci+testing): live base-tip recheck before dedup skip; prod-aware urls mock fallbacks - the dedup gate re-verifies merge-tree equivalence against the LIVE base tip at decision time, closing the window where the base branch gains real commits during the poll (frozen BASE_SHA check alone was stale) - the urls mock's getBaseUrl protocol prefix and getBaseDomain parse fallback now follow the shared isProd flag, mirroring the real module * fix(ci+testing): fail-closed nojobs fallback in dedup gate; TLS-aware redis defaults mock - the dedup gate no longer infers coverage from overall run conclusion when no 'Test and Build /' jobs match — a renamed or skipped test job now runs the tests instead of skipping them - the shared getRedisConnectionDefaults mock mirrors the real TLS resolution (rediss:// to a raw IP requires REDIS_TLS_SERVERNAME and yields tls.servername) * fix(ci): keep polling while nested test jobs have not appeared yet An in-progress push run lists its reusable-workflow jobs only after the caller starts; nojobs is now terminal (fail closed) only once the run has completed without them.
2026-09-24 15:45:35 +08:00 · 2026-07-22 19:04:17 -07:00
parent 9d8e14ce9f
commit dda602a367
269 changed files with 1825 additions and 1792 deletions
@@ -34,16 +34,144 @@ permissions:
  contents: read

 jobs:
+  # Promotion PRs (staging→main etc.) double-run the test suite: every commit
+  # on staging/main already gets a push-event run of the exact same test-build,
+  # and the pull_request "synchronize" run for the open release PR re-runs it on
+  # the same sha seconds later (~39 duplicate runs / 5 days measured Jul 2026).
+  # This gate skips the PR run's test-build ONLY when it can prove the identical
+  # work already passed elsewhere:
+  #   1. the PR base adds no file changes over the merge base with the head sha
+  #      (compare head...base has an empty diff), so the merge result's tree is
+  #      identical to the head tree the push run tested. Plain ancestry is not
+  #      enough of a check here: main's merge-only ruleset leaves merge commits
+  #      on main that staging lacks, so main...staging is permanently
+  #      "diverged" — but those merge commits carry no tree delta. A real
+  #      hotfix landed directly on the base makes the diff non-empty and we
+  #      run tests here;
+  #   2. the push-event CI run at the same head sha finished its test jobs with
+  #      conclusion success (polled, since push + PR runs start simultaneously).
+  # Fail-open by construction: any API error, timeout, missing run, or push-run
+  # failure leaves covered=false and the PR run tests normally, so the PR check
+  # is green only if tests passed either here or on the identical tree. Not a
+  # workflow-level filter on purpose — a job-level skip still reports a
+  # (successful) check context. NOTE: when test-build is skipped, its nested
+  # "Test and Build / ..." contexts are not created; verified 2026-07-22 that
+  # no required status checks are configured on main/staging (rulesets contain
+  # only pull_request/deletion/non_fast_forward). If required checks are ever
+  # added, require the caller "Test and Build" context, not the nested ones.
+  # dev is excluded: push runs on dev skip test-build, so dev-headed PRs have
+  # no push-run coverage to reuse.
+  dedup-promotion:
+    name: Dedup Promotion PR
+    runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
+    timeout-minutes: 15
+    if: >-
+      github.event_name == 'pull_request' &&
+      github.event.pull_request.head.repo.full_name == github.repository &&
+      contains(fromJSON('["main", "staging"]'), github.event.pull_request.head.ref)
+    permissions:
+      contents: read
+      actions: read
+    outputs:
+      covered: ${{ steps.probe.outputs.covered }}
+    steps:
+      - name: Probe for a passing push run at the same sha
+        id: probe
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REPO: ${{ github.repository }}
+          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+          BASE_REF: ${{ github.event.pull_request.base.ref }}
+        run: |
+          COVERED=false
+
+          # (1) Merge-tree equivalence: compare head...base diffs the merge
+          # base against the base tip. An empty diff means the base contributes
+          # nothing beyond what head already contains (merge commits only), so
+          # the PR merge tree equals the head tree the push run tested. Any
+          # error yields "unknown" and we run the tests.
+          BASE_DELTA="$(gh api "repos/${REPO}/compare/${HEAD_SHA}...${BASE_SHA}" --jq '.files | length' 2>/dev/null || echo unknown)"
+          if [ "$BASE_DELTA" != "0" ]; then
+            echo "Base tip changes ${BASE_DELTA} file(s) over the merge base; merge tree differs from head — running tests in this PR run."
+            echo "covered=false" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+
+          # (2) Poll the push-event CI run's test jobs (they start seconds after
+          # this run and take ~4 min). Any conclusion other than success, or
+          # deadline expiry, falls through to covered=false.
+          DEADLINE=$((SECONDS + 600))
+          while [ "$SECONDS" -lt "$DEADLINE" ]; do
+            RUN_JSON="$(gh api "repos/${REPO}/actions/workflows/ci.yml/runs?event=push&head_sha=${HEAD_SHA}&per_page=5" 2>/dev/null || echo '')"
+            RUN_ID="$(printf '%s' "$RUN_JSON" | jq -r '.workflow_runs[0].id // empty' 2>/dev/null || echo '')"
+            if [ -n "$RUN_ID" ]; then
+              # Jobs from the reusable test-build workflow are prefixed
+              # "Test and Build /". If that name ever changes, fall back to the
+              # overall run conclusion (stricter, still correct).
+              STATE="$(gh api "repos/${REPO}/actions/runs/${RUN_ID}/jobs?per_page=100" --jq '
+                [.jobs[] | select(.name | startswith("Test and Build /"))] as $t |
+                if ($t | length) == 0 then "nojobs"
+                elif all($t[]; .conclusion == "success") then "success"
+                elif any($t[]; .conclusion != null and .conclusion != "success") then "failed"
+                else "pending" end' 2>/dev/null || echo pending)"
+              if [ "$STATE" = "nojobs" ]; then
+                # Nested reusable-workflow jobs appear only after the caller
+                # starts, so an in-progress run with no "Test and Build /"
+                # jobs yet just needs another poll. Once the run has
+                # COMPLETED without them, fail closed: the job was renamed or
+                # tests were skipped — never infer coverage from the overall
+                # run conclusion. Update the prefix here on a rename.
+                RUN_STATUS="$(printf '%s' "$RUN_JSON" | jq -r '.workflow_runs[0].status // "unknown"' 2>/dev/null || echo unknown)"
+                if [ "$RUN_STATUS" = "completed" ]; then
+                  echo "Push run ${RUN_ID} completed with no 'Test and Build /' jobs — running tests in this PR run (update the prefix if the job was renamed)."
+                  break
+                fi
+                STATE="pending"
+              fi
+              if [ "$STATE" = "success" ]; then
+                # (3) Re-verify merge-tree equivalence against the LIVE base
+                # tip at decision time: the base branch may have gained real
+                # commits during the poll, in which case the frozen BASE_SHA
+                # check from step (1) is stale and the skip would be unsound.
+                # Any error yields "unknown" and we run the tests.
+                LIVE_DELTA="$(gh api "repos/${REPO}/compare/${HEAD_SHA}...heads/${BASE_REF}" --jq '.files | length' 2>/dev/null || echo unknown)"
+                if [ "$LIVE_DELTA" = "0" ]; then
+                  COVERED=true
+                  echo "Push run ${RUN_ID} passed its test jobs for ${HEAD_SHA} and the live base tip still adds no file changes — skipping duplicate test-build."
+                else
+                  echo "Base branch moved during the poll (live delta: ${LIVE_DELTA}) — running tests in this PR run."
+                fi
+                break
+              elif [ "$STATE" = "failed" ]; then
+                echo "Push run ${RUN_ID} did not pass (state: ${STATE}) — running tests in this PR run."
+                break
+              fi
+            fi
+            sleep 20
+          done
+
+          echo "covered=${COVERED}" >> "$GITHUB_OUTPUT"
+
  test-build:
    name: Test and Build
-    if: github.ref != 'refs/heads/dev' || github.event_name == 'pull_request'
+    needs: [dedup-promotion]
+    # !cancelled(): dedup-promotion is skipped on every non-promotion event and
+    # a skipped need would otherwise skip this job too. covered != 'true' is
+    # fail-open — empty (skipped/failed probe) means run the tests.
+    if: >-
+      !cancelled() &&
+      (github.ref != 'refs/heads/dev' || github.event_name == 'pull_request') &&
+      needs.dedup-promotion.outputs.covered != 'true'
    uses: ./.github/workflows/test-build.yml
    secrets: inherit

  # Detect if this is a version release commit (e.g., "v0.5.24: ...")
+  # Smallest runner on purpose: a few seconds of pure shell over the commit
+  # message, no checkout and no install.
  detect-version:
    name: Detect Version
-    runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
+    runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
    timeout-minutes: 5
    if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/staging' || github.ref == 'refs/heads/dev')
    outputs:
@@ -486,9 +614,11 @@ jobs:
          fi

  # Check if docs changed
+  # Smallest runner on purpose: a depth-2 checkout plus a path filter, no
+  # install and no build.
  check-docs-changes:
    name: Check Docs Changes
-    runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
+    runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
    timeout-minutes: 5
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    outputs:
@@ -22,6 +22,17 @@ on:
    branches: [staging, main]
  workflow_dispatch: {}

+# One live run per PR: a newer opened/edited/synchronize event supersedes the
+# previous run's work entirely (the sticky comment/label upsert is idempotent
+# and only the latest body matters), so cancel in-flight runs instead of
+# letting them race the new one. workflow_dispatch bulk scans get a unique
+# group via run_id and are never cancelled. No paths filter on purpose: the
+# check reads the PR body and cross-repo PR state, not changed files, so a
+# paths filter would both be semantically wrong and stop status refreshes.
+concurrency:
+  group: companion-pr-check-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
 permissions:
  pull-requests: write
  issues: write