opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

build(Makefile): enable parallel make -j gen with correct dependency graph (#22612)

Browse Source 
`make gen` could not run with `-j` because inter-target dependency edges
were missing. Multiple recipes compile `coderd/rbac` (which includes
generated files like `object_gen.go`), and without explicit ordering,
parallel runs produced syntax errors from mid-write reads.

Three main changes:

**Dependency graph fixes** declare the compile-time chain through
`coderd/rbac` so that `object_gen.go` is written before anything that
imports it is compiled. The DB generation targets use a GNU Make 4.3+
grouped target (`&:`) so Make knows `generate.sh` co-produces
`querier.go`, `unique_constraint.go`, `dbmetrics`, and `dbauthz` in a
single invocation. `SKIP_DUMP_SQL=1` avoids re-entrant `make` inside
`generate.sh` when the Makefile already guarantees `dump.sql` is fresh.

**`scripts/atomicwrite` package** replaces `os.WriteFile` in all gen
scripts with a temp-file-in-same-dir + rename pattern, preventing
interrupted runs from leaving partial files.

**`.PRECIOUS` and shell atomic writes** protect git-tracked generated
files from Make's default delete-on-error behavior. Since these files
are committed, deletion is worse than staleness -- `git restore` is the
recovery path.

CI now runs `make -j --output-sync -B gen` (~32s, down from ~85s
serial).

| Scenario                          | Before             | After    |
|-----------------------------------|--------------------|----------|
| `make gen` (serial)               | 95s                | 95s      |
| `make -j gen` (parallel)          | race error         | **22s**  |
| CI `make -j --output-sync -B gen` | forced serial ~85s | **~32s** |
This commit is contained in:
Mathias Fredriksson
2026-03-05 11:58:10 +00:00
committed by GitHub
parent b0e10402c8
commit a6a8fd94d7
15 changed files with 218 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files
coderd/database/gen/dump/main.go
+2 -1
View File
@@ -11,6 +11,7 @@ import (
"github.com/coder/coder/v2/coderd/database/dbtestutil"
"github.com/coder/coder/v2/coderd/database/migrations"
"github.com/coder/coder/v2/scripts/atomicwrite"
)
var preamble = []byte("-- Code generated by 'make coderd/database/generate'. DO NOT EDIT.")
@@ -82,7 +83,7 @@ func main() {
if !ok {
panic("couldn't get caller path")
}
err = os.WriteFile(filepath.Join(mainPath, "..", "..", "..", "dump.sql"), append(preamble, dumpBytes...), 0o600)
err = atomicwrite.File(filepath.Join(mainPath, "..", "..", "..", "dump.sql"), append(preamble, dumpBytes...))
if err != nil {
err = xerrors.Errorf("write dump failed: %w", err)
panic(err)
coderd/database/generate.sh
+11 -3
View File
@@ -16,10 +16,15 @@ SCRIPT_DIR=$(dirname "${BASH_SOURCE[0]}")
echo generate 1>&2
# Dump the updated schema (use make to utilize caching).
make -C ../.. --no-print-directory coderd/database/dump.sql
if [[ "${SKIP_DUMP_SQL:-0}" != 1 ]]; then
make -C ../.. --no-print-directory coderd/database/dump.sql
fi
# The logic below depends on the exact version being correct :(
sqlc generate
tmpfile=$(mktemp "${TMPDIR:-/tmp}/queries.sql.go.XXXXXX")
trap 'rm -f "$tmpfile"' EXIT
first=true
files=$(find ./queries/ -type f -name "*.sql.go" | LC_ALL=C sort)
for fi in $files; do
@@ -33,14 +38,17 @@ SCRIPT_DIR=$(dirname "${BASH_SOURCE[0]}")
# Copy the header from the first file only, ignoring the source comment.
if $first; then
head -n 6 <"$fi" | grep -v "source" >queries.sql.go
head -n 6 <"$fi" | grep -v "source" >"$tmpfile"
first=false
fi
# Append the file past the imports section into queries.sql.go.
tail -n "+$cut" <"$fi" >>queries.sql.go
tail -n "+$cut" <"$fi" >>"$tmpfile"
done
# Atomically replace the target file.
mv "$tmpfile" queries.sql.go
# Move the files we want.
mv queries/querier.go .
mv queries/models.go .