`AgentCoordinateeAuth.Authorize` validated every prefix in
`upd.Node.Addresses` (each must be a `/128` derived from the
authenticating agent's own UUID) but applied no equivalent check to
`upd.Node.AllowedIps`. Because `AllowedIPs` are installed verbatim into
the WireGuard peer config (`tailnet/configmaps.go`) and WireGuard
routing is driven by `AllowedIPs`, a malicious agent could advertise a
victim agent's `/128` and become an eligible route for that IP. With
`ServerTailnet` tunneling to many agents and routing by destination IP,
this could let an attacker intercept sessions intended for the victim
workspace.
This applies the same UUID-derivation validation to `AllowedIps` that
already guards `Addresses`, extracted into a shared
`authorizeNodePrefixes` helper. The check is the single chokepoint used
by both the in-memory coordinator (`tailnet/coordinator.go`) and the
Postgres coordinator (`enterprise/tailnet/connio.go`), so one fix covers
both. Legitimate agents are unaffected: an agent's `AllowedIPs` is a
clone of its `Addresses` (`tailnet/node.go`), which are already
UUID-derived `/128`s.
Fixes PLAT-264 (SEC-89): https://linear.app/codercom/issue/PLAT-264
<details>
<summary>Implementation notes and decision log</summary>
### Root cause
Asymmetric validation in `tailnet/tunnel.go`: `Addresses` were bound to
the agent's UUID, but `AllowedIps` were trusted as-is and propagated
into the WireGuard peer config, which drives routing.
### Why the fix is safe for legitimate agents
- `tailnet/node.go` builds the node with `AllowedIPs:
slices.Clone(u.addresses)`, identical to `Addresses`.
- `agent/agent.go` sets those addresses to
`TailscaleServicePrefix.PrefixFromUUID(agentID)` and
`CoderServicePrefix.PrefixFromUUID(agentID)` (both `/128`,
UUID-derived).
- The existing `Addresses` check already accepts exactly those prefixes
plus the legacy workspace agent IP, so identical validation of
`AllowedIPs` passes for real traffic and only rejects forged prefixes.
### Coverage: one method, both coordinators
`AgentCoordinateeAuth.Authorize` is the shared auth path. A failed
`Authorize` is wrapped as `AuthorizationError{Wrapped: err}` and closes
the agent's response stream.
### Tests
- `tailnet/tunnel_internal_test.go`: fast unit tests on `Authorize`
(valid AllowedIPs accepted; foreign `/128` rejected with
`InvalidNodeAddressError`; wrong-bits rejected with
`InvalidAddressBitsError`).
- `tailnet/coordinator_test.go`: in-memory coordinator closes the agent
stream on a forged `AllowedIp`.
- `enterprise/tailnet/pgcoord_test.go`: same regression for the Postgres
coordinator.
Verified the regression tests fail when the new `AllowedIps` check is
disabled, then pass with it enabled. Local validation: targeted tests
(in-memory, internal, and Postgres-backed enterprise), plus `make
pre-commit` (gen/fmt/lint/build) passing.
</details>
> Generated by Coder Agents on behalf of @f0ssel.
Drop start, lastWrite, and overwrites fields on connIO along with the
Stats() and Overwrites() methods. They have had no readers since
52901e121 which rewrote the PG coordinator's debug page to query the
database directly.
fixes: https://github.com/coder/internal/issues/1304
Subscribe to heartbeats synchronously on startup of PGCoordinator. This ensures tests that send heartbeats don't race with this subscription.
Fixes all our Go file imports to match the preferred spec that we've _mostly_ been using. For example:
```
import (
"context"
"time"
"github.com/prometheus/client_golang/prometheus"
"golang.org/x/xerrors"
"gopkg.in/natefinch/lumberjack.v2"
"cdr.dev/slog/v3"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/serpent"
)
```
3 groups: standard library, 3rd partly libs, Coder libs.
This PR makes the change across the codebase. The PR in the stack above modifies our formatting to maintain this state of affairs, and is a separate PR so it's possible to review that one in detail.
Upgrades to slog v3 which includes a small, but backward incompatible API change to the acceptible call arguments when logging. This change allows us to verify via compile time type checking that arguments are correct and won't cause a panic, as was possible in slog v1, which this replaces (v2 was tagged but never used in coder/coder).
It also updates dependencies that also use slog and were updated.
I've left the `aibridge` dependency as a commit SHA, under the assumption that the team there (cc @pawbana @dannykopping ) will tag and update the dependency soon and on their own schedule.
Other dependencies, I pushed new tags.
fixes https://github.com/coder/internal/issues/1045
Fixes a race condition in our PG Coordinator when a peer disconnects. We issue database queries to find the peer mappings (node structures for each peer connected via a tunnel), and then send these to the "mapper" that generates diffs and eventually writes the update to the websocket.
Before this change we erroneously used the querier's context for this update, which has the same lifetime as the coordinator itself. If the peer has disconnected, the mapper might not be reading from its channel, and this causes a deadlock in a querier worker. This also prevents us from doing any more work on the peer.
I also added some more debug logging that would have been helpful when tracking this down.
In preparation for adding the "member" permission level, which will also
be grouped by org ID, do a bit of a refactor to make room for it and the
existing "org" level to live in the same `map`
This PR improves the ruleguard rule for detecting `t.Fail` calls in goroutines. It picks up additional violations, of which are fixed in this PR.
See self-review for details.
The motivation for fixing this comes from a flake I fixed in https://github.com/coder/coder/pull/19599, where tests would fail from a `require` in an `Eventually`.
fixes https://github.com/coder/internal/issues/958
Logging was being done at error level, but most likely any errors are from simple races between an update triggered around the same time as a client disconnecting. Debug is fine for these.
Adds support to our coordinator implementations to send Error updates before disconnecting clients.
I was recently debugging a connection issue where the client was getting repeatedly disconnected from the Coordinator, but since we never send any error information it was really hard without server logs.
This PR aims to correct that, by sending a CoordinateResponse with `Error` set in cases where we disconnect a client without them asking us to.
It also logs the error whenever we get one in the client controller.
Updating golden files is an unnecessary extra step in addition to gen
that is easily overlooked, leading to the developer noticing the issue
in CI leading to lost developer time waiting for tests to complete.
- Adds `testutil.GoleakOptions` and consolidates existing options to
this location
- Pre-emptively adds required ignore for this Dependabot PR to pass CI
https://github.com/coder/coder/pull/16066
Refactors our use of `slogtest` to instantiate a "standard logger" across most of our tests. This standard logger incorporates https://github.com/coder/slog/pull/217 to also ignore database query canceled errors by default, which are a source of low-severity flakes.
Any test that has set non-default `slogtest.Options` is left alone. In particular, `coderdtest` defaults to ignoring all errors. We might consider revisiting that decision now that we have better tools to target the really common flaky Error logs on shutdown.
chore of #14729
Refactors the `ServerTailnet` to use `tailnet.Controller` so that we reuse logic around reconnection and handling control messages, instead of reimplementing. This unifies our "client" use of the tailscale API across CLI, coderd, and wsproxy.
This PR is the first in a series aimed at closing
[#15109](https://github.com/coder/coder/issues/15109).
### Changes
- **Template Database Creation:**
`dbtestutil.Open` now has the ability to create a template database if
none is provided via `DB_FROM`. The template database’s name is derived
from a hash of the migration files, ensuring that it can be reused
across tests and is automatically updated whenever migrations change.
- **Optimized Database Handling:**
Previously, `dbtestutil.Open` would spin up a new container for each
test when `DB_FROM` was unset. Now, it first checks for an active
PostgreSQL instance on `localhost:5432`. If none is found, it creates a
single container that remains available for subsequent tests,
eliminating repeated container startups.
These changes address the long individual test times (10+ seconds)
reported by some users, likely due to the time Docker took to start and
complete migrations.
Closes#14716Closes#14717
Adds a new user-scoped tailnet API endpoint (`api/v2/tailnet`) with a new RPC stream for receiving updates on workspaces owned by a specific user, as defined in #14716.
When a stream is started, the `WorkspaceUpdatesProvider` will begin listening on the user-scoped pubsub events implemented in #14964. When a relevant event type is seen (such as a workspace state transition), the provider will query the DB for all the workspaces (and agents) owned by the user. This gets compared against the result of the previous query to produce a set of workspace updates.
Workspace updates can be requested for any user ID, however only workspaces the authorised user is permitted to `ActionRead` will have their updates streamed.
Opening a tunnel to an agent requires that the user can perform `ActionSSH` against the workspace containing it.
re: #14715
This PR introduces the Coder service prefix: `fd60:627a:a42b::/48` and refactors our existing code as calling the Tailscale service prefix explicitly (rather than implicitly).
Removes the unused `Addresses` agent option. All clients today assume they can compute the Agent's IP address based on its UUID, so an agent started with a custom address would break things.
Drops support for v1 of the tailnet API, which was the original coordination protocol where we only sent node updates, never marked them lost or disconnected.
v2 of the tailnet API went GA for CLI clients in Coder 2.8.0, so clients older than that would stop working.
Removes the support for the Agent's "legacy IP" which was a hardcoded IP address all agents used to use, before we introduced "single tailnet". Single tailnet went GA in 2.7.0.
* fix: avoid deleting peers on graceful close
- Fixes an issue where a coordinator deletes all
its peers on shutdown. This can cause disconnects
whenever a coderd is redeployed.
Code that was in `/clock` has been moved to github.com/coder/quartz. This PR refactors our use of the clock library to point to the external Quartz repo.
* chore: create type for unique role names
Using `string` was confusing when something should be combined with
org context, and when not to. Naming this new name, "RoleIdentifier"
c.f. https://github.com/coder/coder/pull/13192#issuecomment-2097657692
We need to wait for PGCoordinator to finish its work before returning on `Close()`, so that we delete database state (best effort -- if this fails others will filter it out based on heartbeats).
Removes our pseudo rbac resources like `WorkspaceApplicationConnect` in favor of additional verbs like `ssh`. This is to make more intuitive permissions for building custom roles.
The source of truth is now `policy.go`
Just moved `rbac.Action` -> `policy.Action`. This is for the stacked PR to not have circular dependencies when doing autogen. Without this, the autogen can produce broken golang code, which prevents the autogen from compiling.
So just avoiding circular dependencies. Doing this in it's own PR to reduce LoC diffs in the primary PR, since this has 0 functional changes.
Instead of removing the mappings of unhealthy coordinators entirely,
mark them as lost instead. This prevents peers from disappearing from
other peers if a coordinator misses a heartbeat.
fixes#12923
Prevents Coordinate peer connections from generating spurious database queries like DeleteTailnetPeer when the coordinator is unhealthy.
It does this by checking the health of the querier before accepting a connection, rather than unconditionally accepting it only for it to get swatted down later.
Currently, importing `codersdk` just to interact with the API requires
importing tailscale, which causes builds to fail unless manually using
our fork.
This adds the ability for `TunnelAuth` to also authorize incoming wireguard node IPs, preventing agents from reporting anything other than their static IP generated from the agent ID.
Fixes#12141Fixes#11750
PGCoord shutdown was uncoordinated, so an update at an inopportune time during shutdown would be rejected because the coordinator row was already deleted.
This PR ensures that the PGCoord subcomponents that write updates are shut down before we take down the heartbeats, which is responsible for deleting the coordinator row.
Garrett Delfosse