From 6e5f02bfbebfa06832fb96e80dcab07be344bca4 Mon Sep 17 00:00:00 2001 From: Michael Suchacz <203725896+ibetitsmike@users.noreply.github.com> Date: Mon, 3 Aug 2026 19:49:22 +0200 Subject: [PATCH] fix: bound golangci-lint memory to stop lint OOM kills (#27637) ## Summary Reduce peak memory during linting so cold-cache runs do not exhaust memory on high-core development hosts or leave CI runners with little headroom. ## Problem `golangci-lint` v1 sizes its default concurrency from available CPUs, not the cgroup memory limit. On a 96-CPU development host capped at 32 GiB, a cold-cache full lint run peaked at about 33.9 GB and reached the cgroup limit. The CI workflow also used bare `make -j`, which allows every lint target to run concurrently. ## Fix Make the supported `make lint/go` path cap `golangci-lint` at the lower of detected CPUs and eight workers, preserving lower concurrency on smaller hosts. Preserve an inherited `GOMEMLIMIT`, use a configurable 8 GiB fallback when it is unset, and bound CI's parallel lint targets to the runner CPU count. In the same high-core cold-cache reproduction, peak memory for the full lint target fell to about 10.1 GB. Raw `golangci-lint run` invocations remain unchanged. A `golangci-lint` v2 migration remains separate because it does not size concurrency from the memory limit and requires broader configuration migration work. > Mux prepared and updated this PR on Mike's behalf. --- .github/workflows/ci.yaml | 3 ++- Makefile | 6 +++++- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 4985478dd2..8cb584cb06 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -368,7 +368,8 @@ jobs: run: helm version --short - name: make lint - run: make --output-sync=line -j lint + # Bound concurrent lint targets to keep peak memory below the runner limit. + run: make --output-sync=line -j"$(nproc)" lint - name: Save golangci-lint cache # Only the default branch is trusted to write the cache, so PR diff --git a/Makefile b/Makefile index 25e4a964b7..7bc0c594c0 100644 --- a/Makefile +++ b/Makefile @@ -745,8 +745,12 @@ lint/ts: site/node_modules/.installed pnpm lint .PHONY: lint/ts +# Cap cold-cache golangci-lint on high-core hosts to stay below memory limits. +GO_LINT_CONCURRENCY := $(shell n=$$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 1); echo $$(( n < 8 ? n : 8 ))) +GO_LINT_MEMLIMIT ?= 8GiB + lint/go: - golangci-lint run + GOMEMLIMIT="$${GOMEMLIMIT:-$(GO_LINT_MEMLIMIT)}" golangci-lint run --concurrency="$(GO_LINT_CONCURRENCY)" paralleltestctx -custom-funcs="testutil.Context,chatdTestContext" ./... go run ./scripts/intxcheck ./... .PHONY: lint/go