Files
sim/.github/workflows/helm.yml
T
Waleed 90f6708270 improvement(helm): hygiene pass — CI gating, strict values schema, ESO v1 default (#5939)
* improvement(helm): hygiene pass — CI gating, strict values schema, ESO v1 default, ci values

Closes the gaps from a best-practices audit of the chart (template-level
conformance was already clean: full label set, 82 unit tests, kubeconform-
valid renders):

- new Helm Chart workflow gates every chart change: helm lint, the 82
  helm-unittest cases, kubeconform validation of default + all-components
  renders (k8s 1.29 strict, CRD catalog), and a render of all 10 example
  values files
- helm/sim/ci/ values files (chart-testing convention) so the chart lints
  and templates cleanly out of the box with dummy secrets
- values.schema.json declares all 30 top-level keys (16 were invisible) and
  sets root additionalProperties: false, so top-level typos fail fast
- externalSecrets.apiVersion defaults to v1: current ESO releases removed
  the v1beta1 compatibility path in 2026, so the old default produced
  rejected manifests on new installs; NOTES/values comments updated
- wait-for-postgres init container gets requests/limits (the only container
  in the chart without them; broke ResourceQuota'd namespaces)
- drop the telemetry Prometheus scrape config for app/realtime — neither
  exposes /metrics, so it was dead config that also rendered a realtime
  target with realtime disabled
- chart 1.2.0 with README upgrade notes

* fix(helm): version-agnostic schema-error assertions in the secret-length suite

Helm v4 phrases schema rejections as 'minLength: got N, want 32' while
v3 says 'String length must be greater than or equal to 32'. The suite
grepped for 'minLength' only, so it passed on local helm v4 and failed on
CI's v3.16.4 — the enforcement itself works on both. Patterns now assert
the key name plus either wording. Caught by the new Helm Chart workflow
on its very first run.

* improvement(helm): kind install test + chart version-bump gate in CI

Benchmarked against the flagship OSS charts (ingress-nginx, argo-cd,
kube-prometheus-stack, grafana, bitnami, cert-manager): sim already exceeds
most of them on validation rigor (strict schema — 4 of 6 ship none; 82 unit
tests vs argo-cd's zero; kubeconform manifest validation none of them run),
but every top community chart repo actually installs the chart on a kind
cluster in chart CI — the one majority practice we lacked. Adds:

- install job: kind cluster, helm install with ci/default + a new
  small-footprint ci/kind-values.yaml overlay (default app requests of 4Gi
  can't schedule on a CI node), --wait, then the chart's helm test hook,
  with pod/event/log diagnostics on failure
- version-bump job (PR-only): fails when helm/sim/** changes without a
  Chart.yaml version increment — argo/kps/grafana all enforce this

* fix(helm): declare naming overrides in the strict schema; SHA-pin CI actions

- nameOverride/fullnameOverride are consumed by sim.name/sim.fullname but
  were never in values.yaml, so root additionalProperties: false rejected
  Helm's standard naming overrides — both now declared (a helper-wide sweep
  confirmed they were the only template-read keys missing), with a smoke
  regression test that installs under the strict schema and asserts the
  override lands in resource names
- CI supply-chain hardening: checkout/setup-helm/kind-action pinned to full
  commit SHAs (tag comments retained) and the kubeconform archive verified
  against its published sha256

* fix(helm): immutable unittest runner and fail-closed version gate in CI

- helm-unittest runs via the project's official docker image pinned by
  immutable sha256 digest instead of a plugin install from a mutable git tag
- the version-bump gate fetches the full base ref (a --depth=1 fetch could
  leave no merge base), computes the merge-base and diff outside the if so
  any git failure fails the job instead of falling into the skip branch

* fix(ci): run the helm-unittest container as the runner UID

The digest-pinned image runs as a non-root user that cannot write into the
runner-owned bind mount (it creates tests/__snapshot__, absent from the
checkout since empty dirs aren't tracked). Standard bind-mount pattern:
--user "$(id -u):$(id -g)" with HOME=/tmp for helm's cache.

* fix(helm): appVersion points at a real GHCR tag (v0.7.44)

The kind install job caught this on its first full run: the default image
tag (Chart.AppVersion 0.6.73) returns 404 on GHCR for all three images —
the registry's tags are v-prefixed — so an unpinned default install could
never pull. Updated appVersion to v0.7.44 (verified 200 for simstudio,
realtime, and migrations manifests), stale values comment refreshed, and
an upgrade note added. The CI kind values deliberately stay tag-free so
the job keeps exercising the true default path.
2026-07-24 14:25:46 -07:00

158 lines
6.6 KiB
YAML

name: Helm Chart
on:
push:
branches: [main, staging, dev]
paths:
- 'helm/sim/**'
- '.github/workflows/helm.yml'
pull_request:
branches: [main, staging, dev]
paths:
- 'helm/sim/**'
- '.github/workflows/helm.yml'
concurrency:
group: helm-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
chart:
name: Lint, test, and validate chart
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 15
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Set up Helm
uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4
with:
version: v3.16.4
- name: Helm lint
run: helm lint helm/sim --values helm/sim/ci/default-values.yaml
- name: Helm unit tests
run: |
# Official helm-unittest image, pinned by immutable digest (tag 3.17.3-0.8.2).
# Run as the runner's UID so the container can write into the bind
# mount (it creates tests/__snapshot__), with a writable HOME for helm.
docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp \
-v "$PWD/helm/sim:/apps" \
helmunittest/helm-unittest@sha256:b653db7d5665bc6cec677b15c5eaa1c0377c0de8ac4eb1df58b924478baa21e1 .
- name: Install kubeconform
run: |
curl -sSL -o /tmp/kubeconform.tar.gz \
https://github.com/yannh/kubeconform/releases/download/v0.6.7/kubeconform-linux-amd64.tar.gz
echo "95f14e87aa28c09d5941f11bd024c1d02fdc0303ccaa23f61cef67bc92619d73 /tmp/kubeconform.tar.gz" | sha256sum -c -
tar -xzf /tmp/kubeconform.tar.gz -C /tmp kubeconform
- name: Render and validate manifests (default configuration)
run: |
helm template sim helm/sim --namespace sim \
--values helm/sim/ci/default-values.yaml \
| /tmp/kubeconform -strict -summary \
-kubernetes-version 1.29.0 \
-schema-location default \
-schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json'
- name: Render and validate manifests (all components enabled)
run: |
helm template sim helm/sim --namespace sim \
--values helm/sim/ci/full-values.yaml \
| /tmp/kubeconform -strict -summary \
-kubernetes-version 1.29.0 \
-schema-location default \
-schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json'
- name: Render every example values file
run: |
set -euo pipefail
for f in helm/sim/examples/values-*.yaml; do
echo "--- $f"
# Examples intentionally omit secrets (their headers document the
# required --set flags), so supply the CI dummies alongside each.
helm template sim helm/sim --namespace sim \
--values "$f" \
--values helm/sim/ci/default-values.yaml \
--set copilot.postgresql.auth.password=ci-dummy-password \
--set copilot.server.env.AGENT_API_DB_ENCRYPTION_KEY=cicicicicicicicicicicicicicicicicicicicicicicicicicicicicicicici \
--set copilot.server.env.INTERNAL_API_SECRET=cicicicicicicicicicicicicicicicicicicicicicicicicicicicicicicici \
--set copilot.server.env.LICENSE_KEY=ci-dummy-license \
--set copilot.server.env.SIM_BASE_URL=https://ci.example.com \
--set copilot.server.env.SIM_AGENT_API_KEY=ci-dummy-agent-key \
--set copilot.server.env.REDIS_URL=redis://ci-redis:6379 \
--set copilot.server.env.OPENAI_API_KEY_1=ci-dummy-openai-key \
--set externalDatabase.password=ci-dummy-password > /dev/null
done
version-bump:
name: Chart version bumped
if: github.event_name == 'pull_request'
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 5
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
fetch-depth: 0
- name: Require a Chart.yaml version bump when chart content changes
run: |
set -euo pipefail
base="origin/${{ github.base_ref }}"
git fetch origin "${{ github.base_ref }}"
merge_base=$(git merge-base "$base" HEAD)
changed=$(git diff --name-only "$merge_base" HEAD)
if echo "$changed" | grep -q '^helm/sim/'; then
base_version=$(git show "$merge_base:helm/sim/Chart.yaml" | awk '/^version:/ {print $2}')
head_version=$(awk '/^version:/ {print $2}' helm/sim/Chart.yaml)
echo "base=$base_version head=$head_version"
if [ "$base_version" = "$head_version" ]; then
echo "::error::helm/sim/** changed but Chart.yaml version did not (still $head_version). Bump it per SemVer."
exit 1
fi
else
echo "No chart changes; skipping."
fi
install:
name: Install on kind and run helm test
needs: chart
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 25
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Set up Helm
uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4
with:
version: v3.16.4
- name: Create kind cluster
uses: helm/kind-action@ef37e7f390d99f746eb8b610417061a60e82a6cc # v1
with:
version: v0.24.0
- name: Install chart
run: |
helm install sim helm/sim \
--namespace sim --create-namespace \
--values helm/sim/ci/default-values.yaml \
--values helm/sim/ci/kind-values.yaml \
--wait --timeout 15m
- name: Diagnostics on failure
if: failure()
run: |
kubectl -n sim get pods -o wide || true
kubectl -n sim get events --sort-by=.lastTimestamp | tail -40 || true
kubectl -n sim describe pods | tail -100 || true
kubectl -n sim logs deploy/sim-app -c migrations --tail=50 || true
kubectl -n sim logs deploy/sim-app --tail=80 || true
- name: Run helm test
run: helm test sim --namespace sim --timeout 5m