Commit Graph
5 Commits
Author SHA1 Message Date
Garrett Delfosse a2e1ddb56f fix: validate FileSize in NewDataBuilder to prevent OOM DoS (#25710)
`NewDataBuilder` allocated `make([]byte, 0, req.FileSize)` using the
client-supplied `int64` with no upper-bound check. The DRPC 4 MiB wire
cap limits message size but not the integer value, so a crafted message
with `FileSize = 1<<40` forces a 1 TiB allocation, triggering an
unrecoverable `runtime.throw` that kills the entire `coderd` process.

Add a `MaxFileSize` constant (100 MiB, matching `HTTPFileMaxBytes` in
`coderd/files.go`) and reject negative or oversized `FileSize`, plus
negative or excessive `Chunks`, before the allocation.
`BytesToDataUpload` also returns an error for oversized data to preserve
the encode/decode round-trip contract. Fix a pre-existing reversed
subtraction in the `Add()` overflow error message.

Closes https://linear.app/codercom/issue/PLAT-231

<details>
<summary>Implementation details</summary>

- `provisionersdk/proto/dataupload.go`: New exported `MaxFileSize`
constant; validation in `NewDataBuilder` and `BytesToDataUpload`. Fixed
reversed subtraction in `Add()` error.
- `provisionersdk/proto/dataupload_test.go`: New
`TestNewDataBuilderValidation` with 7 subtests.
- Updated all 5 callers of `BytesToDataUpload` for new error return.
- Audited all `make([]byte, ...)` in provisioner paths; no other
client-supplied sizes.

</details>

> Generated by Coder Agents on behalf of @f0ssel
2026-05-27 14:30:11 -04:00
Steven Masley 60b3fd0783 chore!: send modules archive over the proto messages (#21398)
# What this does

Dynamic parameters caches the `./terraform/modules` directory for parameter usage. What this PR does is send over this archive to the provisioner when building workspaces.

This allow terraform to skip downloading modules from their registries, a step that takes seconds. 

<img width="1223" height="429" alt="Screenshot From 2025-12-29 12-57-52" src="https://github.com/user-attachments/assets/16066e0a-ac79-4296-819d-924f4b0418dc" />


# Wire protocol

The wire protocol reuses the same mechanism used to download the modules `provisoner -> coder`. It splits up large archives into multiple protobuf messages so larger archives can be sent under the message size limit.

# 🚨  Behavior Change (Breaking Change) 🚨 

**Before this PR** modules were downloaded on every workspace build. This means unpinned modules always fetched the latest version

**After this PR** modules are cached at template import time, and their versions are effectively pinned for all subsequent workspace builds.
2026-01-09 11:33:34 -06:00
Steven Masley d2044c2ee9 chore: update protobuf to reuse file request (#21447)
**This is just the protobuf changes for the PR https://github.com/coder/coder/pull/21398**

Moved `UploadFileRequest` from `provisionerd.proto` -> `provisioner.proto`.

Renamed to `FileUpload` because it is now bi-directional.

This **is backwards compatible**. I tested it to confirm the payloads are identical.  Types were just renamed and moved around.

```golang
func TestTypeUpgrade(t *testing.T) {
	t.Parallel()

	x := &proto2.UploadFileRequest{
		Type: &proto2.UploadFileRequest_ChunkPiece{
			ChunkPiece: &proto.ChunkPiece{
				Data:         []byte("Hello World!"),
				FullDataHash: []byte("Foobar"),
				PieceIndex:   42,
			},
		},
	}

	data, err := protobuf.Marshal(x)
	require.NoError(t, err)

	// Exactly the same output
	// EhgKDEhlbGxvIFdvcmxkIRIGRm9vYmFyGCo= on `main`
	// EhgKDEhlbGxvIFdvcmxkIRIGRm9vYmFyGCo= on this branch
	fmt.Println(base64.StdEncoding.EncodeToString(data))
}
```


# What this does

This allows provisioner daemons to download files from `coderd`'s `files` table. This is used to send over cached module files and prevent the need of downloading these modules on each workspace build.
2026-01-09 11:23:32 -06:00
Steven Masley 5a890c4aec test: fix TestUploadFileLargeModuleFiles flake, context to subtest (#18395)
Declared context outside the subtest t.Parallel.
2025-06-16 18:25:31 +00:00
Steven Masley c1341cccdd feat: use proto streams to increase maximum module files payload (#18268)
This PR implements protobuf streaming to handle large module files by:
1. **Streaming large payloads**: When module files exceed the 4MB limit,
they're streamed in chunks using a new UploadFile RPC method
2. **Database storage**: Streamed files are stored in the database and
referenced by hash for deduplication
3. **Backward compatibility**: Small module files continue using the
existing direct payload method
2025-06-13 12:46:26 -05:00