Files
coder/.github/workflows/release.yaml
T
Bobby Ho d210b311dc ci(.github): retry build-tool downloads in Windows signing jobs (#27664)
## Problem

The Windows code-signing path downloads two build tools with bare `wget`
and no retry, in both `ci.yaml` (`build` job) and `release.yaml`
(`release` job):

- `rcodesign` from GitHub releases
- `jsign-6.0.jar` from GitHub releases

A single transient network failure on either fetch fails the whole job.
In `ci.yaml` that turns `main` red via the `required` aggregator; in
`release.yaml` it fails a release.

This has happened. `Install rcodesign` failed on **2026-02-25** (in the
since-deleted `build-dylib` job), **2026-03-04**, and **2026-04-30**.

## Root cause

Two parts, one structural and one local.

**Structural:** GitHub Actions has no per-step retry. This repo already
knows toolchain provisioning is network-flaky and has
`.github/scripts/retry.sh` (3 attempts, 2s/4s/8s backoff), applied in
roughly 20 places. But `retry.sh` is a shell wrapper, so it can only
wrap `run:` steps. These four downloads are `run:` steps that were
simply never wrapped.

**Local:** the failing step's body, under `set -euo pipefail`, is
exactly three commands:

```sh
wget -O /tmp/rcodesign.tar.gz https://github.com/indygreg/apple-platform-rs/releases/download/apple-codesign%2F0.22.0/...
sudo tar -xzf /tmp/rcodesign.tar.gz -C /usr/bin --strip-components=1 ...
rm /tmp/rcodesign.tar.gz
```

`tar` and `rm` operate on a file that was just written, so they are
deterministic. The only nondeterministic command in the step is the
network fetch, and a truncated download surfaces as a `tar` failure
whose cause is still the network.

### How we know

Enumerated failed runs through the GitHub Actions API and extracted, per
run, every failed job together with the names of its failed steps.

| Scan | Scope | Runs |
|---|---|---|
| `ci.yaml`, `main` | 2025-08-01 to 2026-07-29 | 934 |
| `ci.yaml`, all branches | most recent failures | 150 |
| `release.yaml` | all recorded failures | 22 |

The 934 is effectively the complete set; the API reports 923 failed
`main` runs over that period and the scans overlap slightly.

`Install rcodesign` appears **3 times on 3 separate dates**. Being
spread across dates rather than clustered, these behave as
**independent** events. That distinction is what selects the remedy, and
it is why this change is retry rather than removal.

For contrast, the `Setup Java` failures in the same jobs are **4
failures inside a single 90-minute window** on 2026-05-28, all from an
`api.azul.com` edge failure. That is a correlated outage, where every
attempt shares the same degraded dependency and retry provably cannot
help. **That defect is not addressed here** and needs a different fix;
see "Not addressed" below.

### Limits of the evidence

Stating these plainly so a reviewer can weigh them:

- **Cause is not directly confirmed.** Logs for all three `rcodesign`
failures are past GitHub's 90-day retention. The inference from the step
body above is strong but circumstantial.
- **Step-level attribution only reaches back about five months.** GitHub
prunes per-step detail from the jobs API while keeping job-level
conclusions. Probed directly: runs from 2026-03-01 onward return
populated `steps` arrays; runs from 2026-02-05 and earlier return empty
ones. So the true count over the full period could be higher; it cannot
be lower.
- **Impact is small.** This whole class of failure is 10 of 800
attributed non-`required` job failures, about **1.25%** of measured
`main` CI failure volume. This is not a significant reliability
improvement and should not be reviewed as one. The Postgres-backed Go
tests alone are over 40%.

## Solution

Wrap all four downloads in the existing retry helper:

```yaml
- ./.github/scripts/retry.sh -- wget -O /tmp/rcodesign.tar.gz https://...
```

Four lines changed, one per site: `ci.yaml:1287`, `ci.yaml:1321`,
`release.yaml:199`, `release.yaml:225`.

**How it works.** `retry.sh` runs the command, and on non-zero exit
sleeps 2s, 4s, then 8s before re-attempting, up to 3 attempts, then
fails with the original command in the error message. On success the
first time, behavior is unchanged.

**Why it works for these failures.** They are independent events, so
each attempt is a fresh trial with an independent chance of success. A
GitHub releases CDN blip on one run says nothing about the next 2
seconds. This is exactly the regime retry is for.

**Why retry rather than deletion.** These artifacts genuinely are not
present on the runner, so the network call is unavoidable. It can only
be made survivable. (Where a dependency *is* avoidable, deletion is the
better answer, which is the shape the `setup-java` fix will take.)

**Why `wget -O` is safe to retry.** `-O` truncates its output file on
each attempt, so a partial download from a failed attempt is overwritten
rather than appended to. No corruption path.

## Risks

Low, and worth naming precisely.

| Risk | Assessment |
|---|---|
| Behavior change on the success path | None. `retry.sh` execs the
command directly; a first-attempt success is identical to today. |
| A persistently broken URL now takes longer to fail | Yes, by up to 14s
of backoff, then it fails exactly as it does today. Negligible against a
job that takes tens of minutes. |
| `retry.sh` mangling `wget`'s own flags | `retry.sh` parses its own
options with `getopt`, so this was the main correctness concern.
Verified explicitly, both argument orders used in these workflows. See
Verification. |
| Relative path `./.github/scripts/retry.sh` resolving wrongly | These
steps set no `working-directory`, so cwd is the repo root. Deliberately
**excluded** the third `wget` at `release.yaml:713`
(`publish-homebrew`), which runs after `cd "$temp_dir"` where a
repo-relative path would break. |
| Retry masking a real regression | Bounded to 3 attempts over 14s. This
is not job-level auto-retry, which would hide regressions and is
explicitly not proposed. |

### Verification gap a reviewer should know about

**The changed steps do not run on PR CI.** `ci.yaml`'s `build` job is
gated on `github.ref == 'refs/heads/main' || startsWith(github.ref,
'refs/heads/release/')`, and `release.yaml` runs only on release. So
these four steps will execute for the first time on merge to `main`.
Verification below is therefore local plus static analysis, not a live
run of the modified steps.

## Verification

`retry.sh` argument passing, using a stub that prints what it received,
for both argument orders present in these workflows:

```
--- form A: -O before URL (rcodesign style) ---
argc=3
  arg1=[-O]
  arg2=[/tmp/rcodesign.tar.gz]
  arg3=[https://github.com/indygreg/apple-platform-rs/releases/download/apple-codesign%2F0.22.0/apple-codesign-0.22.0-x86_64-unknown-linux-musl.tar.gz]
--- form B: URL before -O (jsign style) ---
argc=3
  arg1=[https://github.com/ebourg/jsign/releases/download/6.0/jsign-6.0.jar]
  arg2=[-O]
  arg3=[/tmp/jsign-6.0.jar]
```

Order preserved and the `%2F` encoding in the rcodesign URL intact,
which was the specific failure mode to rule out.

`make lint/actions` (actionlint plus zizmor security audit):

```
✓ lint/actions/actionlint
No findings to report. Good job! (29 ignored, 102 suppressed)
```

`make pre-commit-light`:

```
✓ fmt/shfmt        ✓ lint/markdown    ✓ lint/actions/actionlint
✓ fmt/terraform    ✓ lint/shellcheck  ✓ lint/helm
✓ fmt/markdown     ✓ lint/bootstrap   ✓ lint/emdash
                   ✓ lint/migrations  ✓ lint/typos
                                      ✓ lint/mise-versions
✓ pre-commit-light passed (14s)
```

## Not addressed

Deliberately out of scope, listed so the remaining exposure is visible:

- **`actions/setup-java` with `distribution: "zulu"`** in both files.
This resolves a JDK from `api.azul.com` and downloads it from
`cdn.azul.com` on **every** run, confirmed from a successful `main`
build's log, because a `Java_Zulu_jdk` tool-cache lookup can never hit
the runner's cache. This is the correlated-outage defect from 2026-05-28
and retry cannot fix it.

The probe on this branch ([run
30492093096](https://github.com/coder/coder/actions/runs/30492093096))
has now answered what the fix should be. `depot-ubuntu-22.04-8` ships:

  ```
  RESULT: java found at /usr/bin/java
  OpenJDK Runtime Environment Temurin-11.0.31+11 (build 11.0.31+11)

  JAVA_HOME=/usr/lib/jvm/temurin-11-jdk-amd64
  JAVA_HOME_8_X64 / _11_X64 / _17_X64 / _21_X64 / _25_X64  (all present)
  tool cache: Java_Temurin-Hotspot_jdk
  ```

So the job downloads **Zulu 11.0.32+9 over two third-party hosts while
Temurin 11.0.31+11 is already on the runner's `PATH`**. The follow-up PR
will point `JAVA_HOME` at `$JAVA_HOME_11_X64` and drop the action, which
removes both Azul hosts while keeping the Java 11 pin rather than
inheriting whatever the image default becomes.
- **`storybook`'s `pnpm/action-setup`**, the last direct use in the repo
and the same unguarded `registry.npmjs.org` dependency originally
reported on the issue. Note `cache: true` does **not** mitigate it: per
the action's own `action.yml`, `cache` caches "the pnpm store
directory", not the pnpm binary.

## Scope

This PR is now a **single commit** (`673162b84e`) containing only the
four-line retry change.

A throwaway probe workflow briefly lived on this branch to answer the
JDK question above. It has served its purpose and the commit was
dropped, so nothing diagnostic remains here to review. Its result is
quoted in "Not addressed" and will be carried into the follow-up PR.

Refs coder/internal#929
2026-07-29 17:05:22 -07:00

846 lines
34 KiB
YAML

# GitHub release workflow.
name: Release
on:
workflow_dispatch:
inputs:
release_channel:
type: choice
description: Release channel
options:
- mainline
- stable
- rc
release_notes:
description: Release notes for the publishing the release. This is required to create a release.
dry_run:
description: Perform a dry-run release (devel). Note that ref must be an annotated tag when run without dry-run.
type: boolean
required: true
default: false
permissions:
contents: read
concurrency: ${{ github.workflow }}-${{ github.ref }}
env:
# Use `inputs` (vs `github.event.inputs`) to ensure that booleans are actual
# booleans, not strings.
# https://github.blog/changelog/2022-06-10-github-actions-inputs-unified-across-manual-and-reusable-workflows/
CODER_RELEASE: ${{ !inputs.dry_run }}
CODER_DRY_RUN: ${{ inputs.dry_run }}
CODER_RELEASE_CHANNEL: ${{ inputs.release_channel }}
CODER_RELEASE_NOTES: ${{ inputs.release_notes }}
jobs:
# Only allow maintainers/admins to release.
check-perms:
runs-on: ${{ github.repository_owner == 'coder' && 'depot-ubuntu-22.04-8' || 'ubuntu-latest' }}
steps:
- name: Allow only maintainers/admins
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const {data} = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: context.actor
});
const role = data.role_name || data.user?.role_name || data.permission;
const perms = data.user?.permissions || {};
core.info(`Actor ${context.actor} permission=${data.permission}, role_name=${role}`);
const allowed =
role === 'admin' ||
role === 'maintain' ||
perms.admin === true ||
perms.maintain === true;
if (!allowed) core.setFailed('Denied: requires maintain or admin');
release:
name: Build and publish
needs: [check-perms]
runs-on: ${{ github.repository_owner == 'coder' && 'depot-ubuntu-22.04-8' || 'ubuntu-latest' }}
permissions:
# Required to publish a release
contents: write
# Necessary to push docker images to ghcr.io.
packages: write
# Necessary for GCP authentication (https://github.com/google-github-actions/setup-gcloud#usage)
# Also necessary for keyless cosign (https://docs.sigstore.dev/cosign/signing/overview/)
# And for GitHub Actions attestation
id-token: write
# Required for GitHub Actions attestation
attestations: write
env:
# Necessary for Docker manifest
DOCKER_CLI_EXPERIMENTAL: "enabled"
outputs:
version: ${{ steps.version.outputs.version }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
# If the event that triggered the build was an annotated tag (which our
# tags are supposed to be), actions/checkout has a bug where the tag in
# question is only a lightweight tag and not a full annotated tag. This
# command seems to fix it.
# https://github.com/actions/checkout/issues/290
- name: Fetch git tags
run: git fetch --tags --force
- name: Print version
id: version
run: |
set -euo pipefail
version="$(./scripts/version.sh)"
echo "version=$version" >> "$GITHUB_OUTPUT"
# Speed up future version.sh calls.
echo "CODER_FORCE_VERSION=$version" >> "$GITHUB_ENV"
echo "$version"
# Verify that all expectations for a release are met.
- name: Verify release input
if: ${{ !inputs.dry_run }}
run: |
set -euo pipefail
if [[ "${GITHUB_REF}" != "refs/tags/v"* ]]; then
echo "Ref must be a semver tag when creating a release, did you use scripts/release.sh?"
exit 1
fi
# Derive the release branch from the version tag.
# Non-RC releases must be on a release/X.Y branch.
# RC tags are allowed on any branch (typically main).
version="$(./scripts/version.sh)"
# Strip any pre-release suffix first (e.g. 2.32.0-rc.0 -> 2.32.0)
base_version="${version%%-*}"
# Then strip patch to get major.minor (e.g. 2.32.0 -> 2.32)
release_branch="release/${base_version%.*}"
if [[ "$version" == *-rc.* ]]; then
echo "RC release detected, skipping release branch check (RC tags are cut from main)."
else
branch_contains_tag=$(git branch --remotes --contains "${GITHUB_REF}" --list "*/${release_branch}" --format='%(refname)')
if [[ -z "${branch_contains_tag}" ]]; then
echo "Ref tag must exist in a branch named ${release_branch} when creating a non-RC release, did you use scripts/release.sh?"
exit 1
fi
fi
if [[ -z "${CODER_RELEASE_NOTES}" ]]; then
echo "Release notes are required to create a release, did you use scripts/release.sh?"
exit 1
fi
echo "Release inputs verified:"
echo
echo "- Ref: ${GITHUB_REF}"
echo "- Version: ${version}"
echo "- Release channel: ${CODER_RELEASE_CHANNEL}"
echo "- Release branch: ${release_branch}"
echo "- Release notes: true"
- name: Create release notes file
run: |
set -euo pipefail
release_notes_file="$(mktemp -t release_notes.XXXXXX)"
echo "$CODER_RELEASE_NOTES" > "$release_notes_file"
echo CODER_RELEASE_NOTES_FILE="$release_notes_file" >> "$GITHUB_ENV"
- name: Show release notes
run: |
set -euo pipefail
cat "$CODER_RELEASE_NOTES_FILE"
- name: Docker Login
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up mise tools
uses: ./.github/actions/setup-mise
with:
install-args: "go node pnpm helm cosign syft"
- name: Install pnpm dependencies
uses: ./.github/actions/pnpm-install
- name: Install Go mise tools
run: ./.github/scripts/retry.sh -- mise install --locked go:github.com/tc-hib/go-winres go:github.com/goreleaser/nfpm/v2/cmd/nfpm
# Necessary for signing Windows binaries.
- name: Setup Java
uses: actions/setup-java@03ad4de0992f5dab5e18fcb136590ce7c4a0ac95 # v5.6.0
with:
distribution: "zulu"
java-version: "11.0"
- name: Install nsis and zstd
run: sudo apt-get install -y nsis zstd
- name: Install rcodesign
run: |
set -euo pipefail
./.github/scripts/retry.sh -- wget -O /tmp/rcodesign.tar.gz https://github.com/indygreg/apple-platform-rs/releases/download/apple-codesign%2F0.22.0/apple-codesign-0.22.0-x86_64-unknown-linux-musl.tar.gz
sudo tar -xzf /tmp/rcodesign.tar.gz \
-C /usr/bin \
--strip-components=1 \
apple-codesign-0.22.0-x86_64-unknown-linux-musl/rcodesign
rm /tmp/rcodesign.tar.gz
- name: Setup Apple Developer certificate and API key
run: |
set -euo pipefail
touch /tmp/{apple_cert.p12,apple_cert_password.txt,apple_apikey.p8}
chmod 600 /tmp/{apple_cert.p12,apple_cert_password.txt,apple_apikey.p8}
echo "$AC_CERTIFICATE_P12_BASE64" | base64 -d > /tmp/apple_cert.p12
echo "$AC_CERTIFICATE_PASSWORD" > /tmp/apple_cert_password.txt
echo "$AC_APIKEY_P8_BASE64" | base64 -d > /tmp/apple_apikey.p8
env:
AC_CERTIFICATE_P12_BASE64: ${{ secrets.AC_CERTIFICATE_P12_BASE64 }}
AC_CERTIFICATE_PASSWORD: ${{ secrets.AC_CERTIFICATE_PASSWORD }}
AC_APIKEY_P8_BASE64: ${{ secrets.AC_APIKEY_P8_BASE64 }}
- name: Setup Windows EV Signing Certificate
run: |
set -euo pipefail
touch /tmp/ev_cert.pem
chmod 600 /tmp/ev_cert.pem
echo "$EV_SIGNING_CERT" > /tmp/ev_cert.pem
./.github/scripts/retry.sh -- wget https://github.com/ebourg/jsign/releases/download/6.0/jsign-6.0.jar -O /tmp/jsign-6.0.jar
env:
EV_SIGNING_CERT: ${{ secrets.EV_SIGNING_CERT }}
- name: Test migrations from current ref to main
run: |
POSTGRES_VERSION=13 make test-migrations
# Setup GCloud for signing Windows binaries.
- name: Authenticate to Google Cloud
id: gcloud_auth
uses: google-github-actions/auth@7c6bc770dae815cd3e89ee6cdf493a5fab2cc093 # v3.0.0
with:
workload_identity_provider: ${{ vars.GCP_CODE_SIGNING_WORKLOAD_ID_PROVIDER }}
service_account: ${{ vars.GCP_CODE_SIGNING_SERVICE_ACCOUNT }}
token_format: "access_token"
- name: Setup GCloud SDK
uses: google-github-actions/setup-gcloud@aa5489c8933f4cc7a4f7d45035b3b1440c9c10db # v3.0.1
- name: Build binaries
run: |
set -euo pipefail
./.github/scripts/retry.sh -- go mod download
version="$(./scripts/version.sh)"
make gen/mark-fresh
make -j \
build/coder_"$version"_linux_{amd64,armv7,arm64}.{tar.gz,apk,deb,rpm} \
build/coder_"$version"_{darwin,windows}_{amd64,arm64}.zip \
build/coder_"$version"_windows_amd64_installer.exe \
build/coder_helm_"$version".tgz \
build/provisioner_helm_"$version".tgz \
build/ai-gateway_helm_"$version".tgz
env:
CODER_SIGN_WINDOWS: "1"
CODER_SIGN_DARWIN: "1"
CODER_SIGN_GPG: "1"
CODER_GPG_RELEASE_KEY_BASE64: ${{ secrets.GPG_RELEASE_KEY_BASE64 }}
CODER_WINDOWS_RESOURCES: "1"
AC_CERTIFICATE_FILE: /tmp/apple_cert.p12
AC_CERTIFICATE_PASSWORD_FILE: /tmp/apple_cert_password.txt
AC_APIKEY_ISSUER_ID: ${{ secrets.AC_APIKEY_ISSUER_ID }}
AC_APIKEY_ID: ${{ secrets.AC_APIKEY_ID }}
AC_APIKEY_FILE: /tmp/apple_apikey.p8
EV_KEY: ${{ secrets.EV_KEY }}
EV_KEYSTORE: ${{ secrets.EV_KEYSTORE }}
EV_TSA_URL: ${{ secrets.EV_TSA_URL }}
EV_CERTIFICATE_PATH: /tmp/ev_cert.pem
GCLOUD_ACCESS_TOKEN: ${{ steps.gcloud_auth.outputs.access_token }}
JSIGN_PATH: /tmp/jsign-6.0.jar
- name: Delete Apple Developer certificate and API key
run: rm -f /tmp/{apple_cert.p12,apple_cert_password.txt,apple_apikey.p8}
- name: Delete Windows EV Signing Cert
run: rm /tmp/ev_cert.pem
- name: Determine base image tag
id: image-base-tag
run: |
set -euo pipefail
if [[ "${CODER_RELEASE:-}" != *t* ]] || [[ "${CODER_DRY_RUN:-}" == *t* ]]; then
# Empty value means use the default and avoid building a fresh one.
echo "tag=" >> "$GITHUB_OUTPUT"
else
echo "tag=$(CODER_IMAGE_BASE=ghcr.io/coder/coder-base ./scripts/image_tag.sh)" >> "$GITHUB_OUTPUT"
fi
- name: Create empty base-build-context directory
if: steps.image-base-tag.outputs.tag != ''
run: mkdir base-build-context
- name: Install depot.dev CLI
if: steps.image-base-tag.outputs.tag != ''
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1
# This uses OIDC authentication, so no auth variables are required.
- name: Build base Docker image via depot.dev
id: build_base_image
if: steps.image-base-tag.outputs.tag != ''
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.18.0
with:
project: wl5hnrrkns
context: base-build-context
file: scripts/Dockerfile.base
platforms: linux/amd64,linux/arm64,linux/arm/v7
provenance: true
sbom: true
pull: true
no-cache: true
push: true
tags: |
${{ steps.image-base-tag.outputs.tag }}
- name: Verify that images are pushed properly
if: steps.image-base-tag.outputs.tag != ''
run: |
# retry 10 times with a 5 second delay as the images may not be
# available immediately
for i in {1..10}; do
rc=0
raw_manifests=$(docker buildx imagetools inspect --raw "${IMAGE_TAG}") || rc=$?
if [[ "$rc" -eq 0 ]]; then
break
fi
if [[ "$i" -eq 10 ]]; then
echo "Failed to pull manifests after 10 retries"
exit 1
fi
echo "Failed to pull manifests, retrying in 5 seconds"
sleep 5
done
manifests=$(
echo "$raw_manifests" | \
jq -r '.manifests[].platform | .os + "/" + .architecture + (if .variant then "/" + .variant else "" end)'
)
# Verify all 3 platforms are present.
set -euxo pipefail
echo "$manifests" | grep -q linux/amd64
echo "$manifests" | grep -q linux/arm64
echo "$manifests" | grep -q linux/arm/v7
env:
IMAGE_TAG: ${{ steps.image-base-tag.outputs.tag }}
- name: GitHub Attestation for Base Docker image
id: attest_base
if: ${{ !inputs.dry_run && steps.build_base_image.outputs.digest != '' }}
continue-on-error: true
uses: actions/attest@a1948c3f048ba23858d222213b7c278aabede763 # v4.1.1
with:
subject-name: ghcr.io/coder/coder-base
subject-digest: ${{ steps.build_base_image.outputs.digest }}
push-to-registry: true
- name: Build Linux Docker images
id: build_docker
run: |
set -euxo pipefail
# we can't build multi-arch if the images aren't pushed, so quit now
# if dry-running
if [[ "$CODER_RELEASE" != *t* ]]; then
echo Skipping multi-arch docker builds due to dry-run.
exit 0
fi
# build Docker images for each architecture
version="$(./scripts/version.sh)"
make build/coder_"$version"_linux_{amd64,arm64,armv7}.tag
# build and push multi-arch manifest, this depends on the other images
# being pushed so will automatically push them.
make push/build/coder_"$version"_linux.tag
multiarch_image="$(./scripts/image_tag.sh)"
echo "multiarch_image=${multiarch_image}" >> "$GITHUB_OUTPUT"
# For debugging, print all docker image tags
docker images
# if the current version is equal to the highest (according to semver)
# version in the repo, also create a multi-arch image as ":latest" and
# push it
if [[ "$(git tag | grep '^v' | grep -vE '(rc|dev|-|\+|\/)' | sort -r --version-sort | head -n1)" == "v$(./scripts/version.sh)" ]]; then
latest_target="$(./scripts/image_tag.sh --version latest)"
# shellcheck disable=SC2046
./scripts/build_docker_multiarch.sh \
--push \
--target "${latest_target}" \
$(cat build/coder_"$version"_linux_{amd64,arm64,armv7}.tag)
echo "created_latest_tag=true" >> "$GITHUB_OUTPUT"
echo "latest_target=${latest_target}" >> "$GITHUB_OUTPUT"
else
echo "created_latest_tag=false" >> "$GITHUB_OUTPUT"
fi
env:
CODER_BASE_IMAGE_TAG: ${{ steps.image-base-tag.outputs.tag }}
- name: SBOM Generation and Attestation
if: ${{ !inputs.dry_run }}
env:
COSIGN_EXPERIMENTAL: '1'
MULTIARCH_IMAGE: ${{ steps.build_docker.outputs.multiarch_image }}
VERSION: ${{ steps.version.outputs.version }}
CREATED_LATEST_TAG: ${{ steps.build_docker.outputs.created_latest_tag }}
run: |
set -euxo pipefail
# Generate SBOM for multi-arch image with version in filename
echo "Generating SBOM for multi-arch image: ${MULTIARCH_IMAGE}"
syft "${MULTIARCH_IMAGE}" -o spdx-json > "coder_${VERSION}_sbom.spdx.json"
echo "Attesting SBOM to multi-arch image: ${MULTIARCH_IMAGE}"
cosign clean --force=true "${MULTIARCH_IMAGE}"
cosign attest --type spdxjson \
--predicate "coder_${VERSION}_sbom.spdx.json" \
--yes \
"${MULTIARCH_IMAGE}"
# If latest tag was created, also attest it
if [[ "${CREATED_LATEST_TAG}" == "true" ]]; then
latest_tag="$(./scripts/image_tag.sh --version latest)"
echo "Generating SBOM for latest image: ${latest_tag}"
syft "${latest_tag}" -o spdx-json > coder_latest_sbom.spdx.json
echo "Attesting SBOM to latest image: ${latest_tag}"
cosign clean --force=true "${latest_tag}"
cosign attest --type spdxjson \
--predicate coder_latest_sbom.spdx.json \
--yes \
"${latest_tag}"
fi
- name: Resolve Docker image digests for attestation
id: docker_digests
if: ${{ !inputs.dry_run }}
continue-on-error: true
env:
MULTIARCH_IMAGE: ${{ steps.build_docker.outputs.multiarch_image }}
LATEST_TARGET: ${{ steps.build_docker.outputs.latest_target }}
run: |
set -euxo pipefail
if [[ -n "${MULTIARCH_IMAGE}" ]]; then
multiarch_digest=$(docker buildx imagetools inspect --raw "${MULTIARCH_IMAGE}" | sha256sum | awk '{print "sha256:"$1}')
echo "multiarch_digest=${multiarch_digest}" >> "$GITHUB_OUTPUT"
fi
if [[ -n "${LATEST_TARGET}" ]]; then
latest_digest=$(docker buildx imagetools inspect --raw "${LATEST_TARGET}" | sha256sum | awk '{print "sha256:"$1}')
echo "latest_digest=${latest_digest}" >> "$GITHUB_OUTPUT"
fi
- name: GitHub Attestation for Docker image
id: attest_main
if: ${{ !inputs.dry_run && steps.docker_digests.outputs.multiarch_digest != '' }}
continue-on-error: true
uses: actions/attest@a1948c3f048ba23858d222213b7c278aabede763 # v4.1.1
with:
subject-name: ghcr.io/coder/coder
subject-digest: ${{ steps.docker_digests.outputs.multiarch_digest }}
push-to-registry: true
- name: GitHub Attestation for "latest" Docker image
id: attest_latest
if: ${{ !inputs.dry_run && steps.docker_digests.outputs.latest_digest != '' }}
continue-on-error: true
uses: actions/attest@a1948c3f048ba23858d222213b7c278aabede763 # v4.1.1
with:
subject-name: ghcr.io/coder/coder
subject-digest: ${{ steps.docker_digests.outputs.latest_digest }}
push-to-registry: true
- name: GitHub Attestation for release binaries
id: attest_binaries
if: ${{ !inputs.dry_run }}
continue-on-error: true
uses: actions/attest@a1948c3f048ba23858d222213b7c278aabede763 # v4.1.1
with:
subject-path: |
./build/*.tar.gz
./build/*.zip
./build/*.deb
./build/*.rpm
./build/*.apk
./build/*_installer.exe
./build/*_helm_*.tgz
./build/provisioner_helm_*.tgz
# Report attestation failures but don't fail the workflow
- name: Check attestation status
if: ${{ !inputs.dry_run }}
run: | # zizmor: ignore[template-injection] We're just reading steps.attest_x.outcome here, no risk of injection
if [[ "${{ steps.attest_base.outcome }}" == "failure" && "${{ steps.attest_base.conclusion }}" != "skipped" ]]; then
echo "::warning::GitHub attestation for base image failed"
fi
if [[ "${{ steps.attest_main.outcome }}" == "failure" ]]; then
echo "::warning::GitHub attestation for main image failed"
fi
if [[ "${{ steps.attest_latest.outcome }}" == "failure" && "${{ steps.attest_latest.conclusion }}" != "skipped" ]]; then
echo "::warning::GitHub attestation for latest image failed"
fi
if [[ "${{ steps.attest_binaries.outcome }}" == "failure" && "${{ steps.attest_binaries.conclusion }}" != "skipped" ]]; then
echo "::warning::GitHub attestation for release binaries failed"
fi
- name: Generate offline docs
run: |
version="$(./scripts/version.sh)"
make -j build/coder_docs_"$version".tgz
- name: ls build
run: ls -lh build
- name: Publish Coder CLI binaries and detached signatures to GCS
if: ${{ !inputs.dry_run }}
run: |
set -euxo pipefail
version="$(./scripts/version.sh)"
# Source array of slim binaries
declare -A binaries
binaries["coder-darwin-amd64"]="coder-slim_${version}_darwin_amd64"
binaries["coder-darwin-arm64"]="coder-slim_${version}_darwin_arm64"
binaries["coder-linux-amd64"]="coder-slim_${version}_linux_amd64"
binaries["coder-linux-arm64"]="coder-slim_${version}_linux_arm64"
binaries["coder-linux-armv7"]="coder-slim_${version}_linux_armv7"
binaries["coder-windows-amd64.exe"]="coder-slim_${version}_windows_amd64.exe"
binaries["coder-windows-arm64.exe"]="coder-slim_${version}_windows_arm64.exe"
for cli_name in "${!binaries[@]}"; do
slim_binary="${binaries[$cli_name]}"
detached_signature="${slim_binary}.asc"
gcloud storage cp "./build/${slim_binary}" "gs://releases.coder.com/coder-cli/${version}/${cli_name}"
gcloud storage cp "./build/${detached_signature}" "gs://releases.coder.com/coder-cli/${version}/${cli_name}.asc"
done
- name: Publish release
run: |
set -euo pipefail
publish_args=()
if [[ $CODER_RELEASE_CHANNEL == "stable" ]]; then
publish_args+=(--stable)
fi
if [[ $CODER_RELEASE_CHANNEL == "rc" ]]; then
publish_args+=(--rc)
fi
if [[ $CODER_DRY_RUN == *t* ]]; then
publish_args+=(--dry-run)
fi
declare -p publish_args
# Build the list of files to publish
files=(
./build/*_installer.exe
./build/*.zip
./build/*.tar.gz
./build/*.tgz
./build/*.apk
./build/*.deb
./build/*.rpm
"./coder_${VERSION}_sbom.spdx.json"
)
# Only include the latest SBOM file if it was created
if [[ "${CREATED_LATEST_TAG}" == "true" ]]; then
files+=(./coder_latest_sbom.spdx.json)
fi
./scripts/release/publish.sh \
"${publish_args[@]}" \
--release-notes-file "$CODER_RELEASE_NOTES_FILE" \
"${files[@]}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CODER_GPG_RELEASE_KEY_BASE64: ${{ secrets.GPG_RELEASE_KEY_BASE64 }}
VERSION: ${{ steps.version.outputs.version }}
CREATED_LATEST_TAG: ${{ steps.build_docker.outputs.created_latest_tag }}
# Mark the Linear release as shipped.
- name: Extract Linear release version
if: ${{ !inputs.dry_run }}
id: linear_version
run: |
# Skip RC releases — they must not complete the Linear release.
if [[ "$VERSION" == *-rc* ]]; then
echo "RC release (${VERSION}), skipping Linear release completion."
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# Strip patch to get the Linear release version (e.g. 2.32.0 -> 2.32).
linear_version=$(echo "$VERSION" | cut -d. -f1,2)
echo "version=$linear_version" >> "$GITHUB_OUTPUT"
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "Completing Linear release ${linear_version}"
env:
VERSION: ${{ steps.version.outputs.version }}
- name: Complete Linear release
if: ${{ !inputs.dry_run && steps.linear_version.outputs.skip != 'true' }}
continue-on-error: true
uses: linear/linear-release-action@c0cb8354a362c24c6d3e0948f37fd66d07588e3f # v0.14.5
with:
access_key: ${{ secrets.LINEAR_ACCESS_KEY }}
command: complete
version: ${{ steps.linear_version.outputs.version }}
timeout: 300
- name: Authenticate to Google Cloud
uses: google-github-actions/auth@7c6bc770dae815cd3e89ee6cdf493a5fab2cc093 # v3.0.0
with:
workload_identity_provider: ${{ vars.GCP_WORKLOAD_ID_PROVIDER }}
service_account: ${{ vars.GCP_SERVICE_ACCOUNT }}
- name: Setup GCloud SDK
uses: google-github-actions/setup-gcloud@aa5489c8933f4cc7a4f7d45035b3b1440c9c10db # 3.0.1
- name: Publish Helm Chart
if: ${{ !inputs.dry_run }}
run: |
set -euo pipefail
version="$(./scripts/version.sh)"
mkdir -p build/helm
cp "build/coder_helm_${version}.tgz" build/helm
cp "build/provisioner_helm_${version}.tgz" build/helm
cp "build/ai-gateway_helm_${version}.tgz" build/helm
gsutil cp gs://helm.coder.com/v2/index.yaml build/helm/index.yaml
helm repo index build/helm --url https://helm.coder.com/v2 --merge build/helm/index.yaml
gsutil -h "Cache-Control:no-cache,max-age=0" cp "build/helm/coder_helm_${version}.tgz" gs://helm.coder.com/v2
gsutil -h "Cache-Control:no-cache,max-age=0" cp "build/helm/provisioner_helm_${version}.tgz" gs://helm.coder.com/v2
gsutil -h "Cache-Control:no-cache,max-age=0" cp "build/helm/ai-gateway_helm_${version}.tgz" gs://helm.coder.com/v2
gsutil -h "Cache-Control:no-cache,max-age=0" cp "build/helm/index.yaml" gs://helm.coder.com/v2
gsutil -h "Cache-Control:no-cache,max-age=0" cp "helm/artifacthub-repo.yml" gs://helm.coder.com/v2
helm push "build/coder_helm_${version}.tgz" oci://ghcr.io/coder/chart
helm push "build/provisioner_helm_${version}.tgz" oci://ghcr.io/coder/chart
helm push "build/ai-gateway_helm_${version}.tgz" oci://ghcr.io/coder/chart
- name: Upload artifacts to actions (if dry-run)
if: ${{ inputs.dry_run }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-artifacts
path: |
./build/*_installer.exe
./build/*.zip
./build/*.tar.gz
./build/*.tgz
./build/*.apk
./build/*.deb
./build/*.rpm
./coder_${{ steps.version.outputs.version }}_sbom.spdx.json
retention-days: 7
- name: Upload latest sbom artifact to actions (if dry-run)
if: inputs.dry_run && steps.build_docker.outputs.created_latest_tag == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: latest-sbom-artifact
path: ./coder_latest_sbom.spdx.json
retention-days: 7
- name: Send repository-dispatch event
if: ${{ !inputs.dry_run && inputs.release_channel != 'rc' }}
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.CDRCI_GITHUB_TOKEN }}
repository: coder/packages
event-type: coder-release
client-payload: '{"coder_version": "${{ steps.version.outputs.version }}", "release_channel": "${{ inputs.release_channel }}"}'
publish-homebrew:
name: Publish to Homebrew tap
runs-on: ubuntu-latest
needs: release
if: ${{ !inputs.dry_run && inputs.release_channel == 'mainline' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
- name: Update homebrew
env:
GH_REPO: coder/homebrew-coder
GH_TOKEN: ${{ secrets.CDRCI_GITHUB_TOKEN }}
VERSION: ${{ needs.release.outputs.version }}
run: |
# Keep version number around for reference, removing any potential leading v
coder_version="$(echo "${VERSION}" | tr -d v)"
set -euxo pipefail
# Setup Git
git config --global user.email "ci@coder.com"
git config --global user.name "Coder CI"
git config --global credential.helper "store"
temp_dir="$(mktemp -d)"
cd "$temp_dir"
# Download checksums
checksums_url="$(gh release view --repo coder/coder "v$coder_version" --json assets \
| jq -r ".assets | map(.url) | .[]" \
| grep -e ".checksums.txt\$")"
wget "$checksums_url" -O checksums.txt
# Get the SHAs
darwin_arm_sha="$(grep "darwin_arm64.zip" checksums.txt | awk '{ print $1 }')"
darwin_intel_sha="$(grep "darwin_amd64.zip" checksums.txt | awk '{ print $1 }')"
linux_sha="$(grep "linux_amd64.tar.gz" checksums.txt | awk '{ print $1 }')"
echo "macOS arm64: $darwin_arm_sha"
echo "macOS amd64: $darwin_intel_sha"
echo "Linux amd64: $linux_sha"
# Check out the homebrew repo
git clone "https://github.com/$GH_REPO" homebrew-coder
brew_branch="auto-release/$coder_version"
cd homebrew-coder
# Check if a PR already exists.
pr_count="$(gh pr list --search "head:$brew_branch" --json id,closed | jq -r ".[] | select(.closed == false) | .id" | wc -l)"
if [ "$pr_count" -gt 0 ]; then
echo "Bailing out as PR already exists" 2>&1
exit 0
fi
# Set up cdrci credentials for pushing to homebrew-coder
echo "https://x-access-token:$GH_TOKEN@github.com" >> ~/.git-credentials
# Update the formulae and push
git checkout -b "$brew_branch"
./scripts/update-v2.sh "$coder_version" "$darwin_arm_sha" "$darwin_intel_sha" "$linux_sha"
git add .
git commit -m "coder $coder_version"
git push -u origin -f "$brew_branch"
# Create PR
gh pr create \
-B master -H "$brew_branch" \
-t "coder $coder_version" \
-b "" \
-r "${GITHUB_ACTOR}" \
-a "${GITHUB_ACTOR}" \
-b "This automatic PR was triggered by the release of Coder v$coder_version"
publish-winget:
name: Publish to winget-pkgs
runs-on: windows-latest
needs: release
if: ${{ !inputs.dry_run && inputs.release_channel != 'rc' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
- name: Sync fork
run: gh repo sync cdrci/winget-pkgs -b master
env:
GH_TOKEN: ${{ secrets.CDRCI_GITHUB_TOKEN }}
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
# If the event that triggered the build was an annotated tag (which our
# tags are supposed to be), actions/checkout has a bug where the tag in
# question is only a lightweight tag and not a full annotated tag. This
# command seems to fix it.
# https://github.com/actions/checkout/issues/290
- name: Fetch git tags
run: git fetch --tags --force
- name: Install wingetcreate
run: |
Invoke-WebRequest https://aka.ms/wingetcreate/latest -OutFile wingetcreate.exe
- name: Submit updated manifest to winget-pkgs
run: |
# The package version is the same as the tag minus the leading "v".
# The version in this output already has the leading "v" removed but
# we do it again to be safe.
$version = $env:VERSION.Trim('v')
$release_assets = gh release view --repo coder/coder "v${version}" --json assets | `
ConvertFrom-Json
# Get the installer URLs from the release assets.
$amd64_installer_url = $release_assets.assets | `
Where-Object name -Match ".*_windows_amd64_installer.exe$" | `
Select -ExpandProperty url
$amd64_zip_url = $release_assets.assets | `
Where-Object name -Match ".*_windows_amd64.zip$" | `
Select -ExpandProperty url
$arm64_zip_url = $release_assets.assets | `
Where-Object name -Match ".*_windows_arm64.zip$" | `
Select -ExpandProperty url
echo "amd64 Installer URL: ${amd64_installer_url}"
echo "amd64 zip URL: ${amd64_zip_url}"
echo "arm64 zip URL: ${arm64_zip_url}"
echo "Package version: ${version}"
.\wingetcreate.exe update Coder.Coder `
--submit `
--version "${version}" `
--urls "${amd64_installer_url}" "${amd64_zip_url}" "${arm64_zip_url}"
env:
# For gh CLI:
GH_TOKEN: ${{ github.token }}
# For wingetcreate. We need a real token since we're pushing a commit
# to GitHub and then making a PR in a different repo.
# wingetcreate will read the token from the environment variable defined below.
# Reference: https://aka.ms/winget-create-token
WINGET_CREATE_GITHUB_TOKEN: ${{ secrets.CDRCI_GITHUB_TOKEN }}
VERSION: ${{ needs.release.outputs.version }}
- name: Comment on PR
run: |
# wait 30 seconds
Start-Sleep -Seconds 30.0
# Find the PR that wingetcreate just made.
$version = $env:VERSION.Trim('v')
$pr_list = gh pr list --repo microsoft/winget-pkgs --search "author:cdrci Coder.Coder version ${version}" --limit 1 --json number | `
ConvertFrom-Json
$pr_number = $pr_list[0].number
gh pr comment --repo microsoft/winget-pkgs "${pr_number}" --body "🤖 cc: @deansheather @matifali"
env:
# For gh CLI. We need a real token since we're commenting on a PR in a
# different repo.
GH_TOKEN: ${{ secrets.CDRCI_GITHUB_TOKEN }}
VERSION: ${{ needs.release.outputs.version }}