47 Commits
Author SHA1 Message Date
Spike Curtis 8910ac715c feat: add tailnet v2 support to wsproxy coordinate endpoint (#11637)
wsproxy also needs to be updated to use tailnet v2 because the `tailnet.Conn` stores peers by ID, and the peerID was not being carried by the JSON protocol.  This adds a query param to the endpoint to conditionally switch to the new protocol.
2024-01-18 10:10:36 +04:00
Spike Curtis cae095fdb6 fix: stop logging errors on canceled cleanup queries (#11547)
Fixes flake seen here: https://github.com/coder/coder/actions/runs/7474259128/job/20340051975
2024-01-10 16:20:29 +04:00
Spike Curtis 64638b381d feat: promote PG Coordinator out of experimental (#11398)
Promotes PG Coordinator out of experimental to GA
2024-01-05 08:03:36 +04:00
Steven Masley dd05a6b13a chore: mockgen archived, moved to new location (#11415)
* chore: mockgen archived, moved to new location
2024-01-04 18:35:56 -06:00
Spike Curtis f2606a78dd fix: avoid converting nil node
fixes: #11276
2023-12-19 13:38:15 +04:00
Dean SheatherandSpike Curtis e46431078c feat: add AgentAPI using DRPC (#10811)
Co-authored-by: Spike Curtis <spike@coder.com>
2023-12-18 22:53:28 +10:00
Spike Curtis ad3fed72bc chore: rename Coordinator to CoordinatorV1 (#11222)
Renames the tailnet.Coordinator to represent both v1 and v2 APIs, so that we can use this interface for the main atomic pointer.

Part of #10532
2023-12-15 11:38:12 +04:00
Spike Curtis bf3b35b1e2 fix: stop logging context Canceled as error (#11177)
fixes #11166 and a related log that could have the same problem
2023-12-13 13:08:30 +04:00
Spike Curtis b34ecf1e9e fix: fix deadlock of mappingQuery on context canceled
Fixes #11078

replace bare channel send with SendCtx so that we properly shut down when context is canceled.
2023-12-07 17:19:18 +04:00
Spike Curtis 2c86d0bed0 feat: support v2 Tailnet API in AGPL coordinator (#11010)
Fixes #10529
2023-12-06 15:04:28 +04:00
Spike Curtis 812fb95273 fix: prevent connIO from panicking in race between Close and Enqueue (#10948)
Spotted during a code read.  ConnIO unlocks the mutex before attempting to write to the response channel, which could allow another goroutine to call Close() and close the channel, causing a panic.

Fix is to hold the mutex.  This won't cause a deadlock because the `select{}` has a `default` case, so we won't block even if the receiver isn't keeping up.
2023-12-01 10:23:29 +04:00
Spike Curtis 612e67a53b feat: add cleanup of lost tailnet peers and tunnels to PGCoordinator (#10939)
Adds the "lost" peer cleanup queries to PGCoordinator, including tests.
2023-12-01 10:13:29 +04:00
Spike Curtis 0cab6e7763 feat: support graceful disconnect in PGCoordinator (#10937)
Adds support for graceful disconnect to PGCoordinator.  When peers gracefully disconnect, they send a disconnect message.  This triggers the peer to be disconnected from all tunneled peers.

The Multi-Agent Client supports graceful disconnect, since it is in memory and we know that when it is closed, we really mean to disconnect.

The v1 agent and client Websocket connections do not support graceful disconnect, since the v1 protocol doesn't have this feature.  That means that if a v1 peer connects to a v2 peer, when the v1 peer's coordinator connection is closed, the v2 peer will
see it as "lost" since we don't know whether the v1 peer meant to disconnect, or it just lost connectivity to the coordinator.
2023-12-01 09:55:25 +04:00
Spike Curtis 52901e1219 feat: implement HTMLDebug for PGCoord with v2 API (#10914)
Implements HTMLDebug for the PGCoordinator with the new v2 API and related DB tables.
2023-11-28 22:37:20 +04:00
Spike Curtis 5c48cb4447 feat: modify PG Coordinator to work with new v2 Tailnet API (#10573)
re: #10528

Refactors PG Coordinator to work with the Tailnet v2 API, including wrappers for the existing v1 API.

The debug endpoint functions, but doesn't return sensible data, that will be in another stacked PR.
2023-11-20 14:31:04 +04:00
Spike Curtis fbabb43cbb fix: ignore spurious node updates while waiting for errors (#10175)
fixes #9921
2023-10-11 09:22:20 +04:00
Colin Adler 36f3151b71 fix(enterprise/tailnet): properly detect legacy agents (#10083) 2023-10-06 16:49:26 +00:00
Colin Adler 4da1223a80 fix: pass OnSubscribe to HA MultiAgent (#9947)
Fixes https://github.com/coder/coder/issues/9929
2023-09-29 13:37:17 -05:00
Colin Adler c900b5f8df feat: add single tailnet support to pgcoord (#9351) 2023-09-21 14:30:48 -05:00
Spike Curtis a415395e9e fix: stop dropping error log on context canceled after heartbeat (#9427)
Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-30 14:44:00 +04:00
Ammar Bandukwala 545a256b57 fix: correctly reject quota-violating builds (#9233)
Due to a logical error in CommitQuota, all workspace Stop->Start operations
were being accepted, regardless of the Quota limit. This issue only
appeared after #9201, so this was a minor regression in main for about
3 days. This PR adds a test to make sure this kind of bug doesn't recur.

To make the new test possible, we give the echo provisioner the ability
to simulate responses to specific transitions.
2023-08-22 02:55:39 +00:00
Kyle Carberry 22e781eced chore: add /v2 to import module path (#9072)
* chore: add /v2 to import module path

go mod requires semantic versioning with versions greater than 1.x

This was a mechanical update by running:
```
go install github.com/marwan-at-work/mod/cmd/mod@latest
mod upgrade
```

Migrate generated files to import /v2

* Fix gen
2023-08-18 18:55:43 +00:00
Spike Curtis 2f46f2315c fix: fix race in PGCoord at startup (#9144)
Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-18 09:53:03 +04:00
Spike Curtis c7a6d626b4 fix: make PGCoordinator close connections when unhealthy (#9125)
Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-17 09:36:47 +04:00
Colin Adler bc862fa493 chore: upgrade tailscale to v1.46.1 (#8913) 2023-08-09 19:50:26 +00:00
Colin Adler 0b4f333a6f chore: add http debug support to pgcoord (#8795) 2023-07-28 17:59:31 -05:00
Colin Adler dd2f79995b chore(tailnet): rewrite coordinator debug using html/template (#8752) 2023-07-26 22:54:21 +00:00
Colin Adler 6b92abebb9 fix(tailnet): track agent names for http debug (#8744) 2023-07-26 18:44:10 +00:00
Colin Adler 1cb39fc65d test: ignore more spurious pgcoord errors (#8628) 2023-07-20 19:55:25 +00:00
Colin Adler 00b9a3ce58 fix: prevent error log when pgcoord query is canceled (#8609) 2023-07-19 16:40:57 -05:00
Colin Adler 517fb19474 feat: add single tailnet support to moons (#8587) 2023-07-19 11:11:11 -05:00
Colin Adler c47b78c44b chore: replace wsconncache with a single tailnet (#8176) 2023-07-12 17:37:31 -05:00
Spike Curtis b4057bd74a feat: make pgCoordinator generally available (#8419)
* pgCoord to GA, fix tests

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix generation and coordinator delete RBAC

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix fakeQuerier -> FakeQuerier

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-07-12 13:35:29 +04:00
Spike Curtis c0a01ec81c fix: fix TestPGCoordinatorDual_Mainline flake (#8228)
* fix TestPGCoordinatorDual_Mainline flake

Signed-off-by: Spike Curtis <spike@coder.com>

* use slices.Contains instead of local function

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-06-28 11:37:45 +04:00
Spike Curtis 7943a5b85e fix PG coordinator context and RBAC subject (#8223)
Signed-off-by: Spike Curtis <spike@coder.com>
2023-06-27 10:14:31 +00:00
Spike Curtis 5d48122f12 fix: fix PG Coordinator to update when heartbeats (re)start (#8178)
* fix: fix PG Coordinator to update when heartbeats (re)start

Signed-off-by: Spike Curtis <spike@coder.com>

* rename resetExpiryTimer(WithLock)

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-06-23 10:38:58 +00:00
Spike Curtis ba9d038d42 feat: add periodic cleanup of PG Coordinator state (#8142)
* PG Coordinator cleans orphaned state

Signed-off-by: Spike Curtis <spike@coder.com>

* Don't need pubsub

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-06-23 13:23:28 +04:00
Spike Curtis cc17d2feea refactor: add postgres tailnet coordinator (#8044)
* postgres tailnet coordinator

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix db migration; tests

Signed-off-by: Spike Curtis <spike@coder.com>

* Add fixture, regenerate

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix fixtures

Signed-off-by: Spike Curtis <spike@coder.com>

* review comments, run clean gen

Signed-off-by: Spike Curtis <spike@coder.com>

* Rename waitForConn -> cleanupConn

Signed-off-by: Spike Curtis <spike@coder.com>

* code review updates

Signed-off-by: Spike Curtis <spike@coder.com>

* db migration order

Signed-off-by: Spike Curtis <spike@coder.com>

* fix log field name last_heartbeat

Signed-off-by: Spike Curtis <spike@coder.com>

* fix heartbeat_from log field

Signed-off-by: Spike Curtis <spike@coder.com>

* fix slog fields for linting

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-06-21 16:20:58 +04:00
Kyle Carberry e4b6f5695b chore: separate pubsub into a new package (#8017)
* chore: rename store to dbmock for consistency

* chore: remove redundant dbtype package

This wasn't necessary and forked how we do DB types.

* chore: separate pubsub into a new package

This didn't need to be in database and was bloating it.
2023-06-14 15:34:54 +00:00
Spike Curtis bd630113b2 fix: coordinator node update race (#7345)
* fix: coordinator node update race

Signed-off-by: Spike Curtis <spike@coder.com>

* Lint fixes, make core private

Signed-off-by: Spike Curtis <spike@coder.com>

* Don't log broken connections as errors

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-05-02 20:58:21 +04:00
Kyle Carberry f287889cd7 chore: move client pubsub for HA coordinator after register (#6487)
Similar fix that happened for the agent. See:
https://github.com/coder/coder/actions/runs/4356099388/jobs/7613634277
2023-03-07 16:46:54 +00:00
Kyle Carberry f19076cf06 chore: fix coordinator flake by moving pubsub below register (#6482)
After making the in-memory pubsub conform to the expectations of
PostgreSQL, this flake started appearing.

This fixes it because the agent socket is registered when a message
is received.
2023-03-07 09:38:18 -06:00
Kyle CarberryandColin Adler 026b1cd2a4 chore: update to go 1.20 (#5968)
Co-authored-by: Colin Adler <colin1adler@gmail.com>
2023-02-02 12:36:27 -06:00
Colin Adler cc694a55bc feat: add debug info to HA coordinator (#5883) 2023-01-26 16:32:38 -06:00
Colin Adler 1cd5f38cb0 feat: add debug server for tailnet coordinators (#5861)
Implements a Tailscale-like debug server for our in-memory coordinator. This should provide some visibility into why connections could be failing.
Resolves: https://github.com/coder/coder/issues/5845

![image](https://user-images.githubusercontent.com/6332295/214680832-2724d633-2d54-44d6-a7ce-5841e5824ee5.png)
2023-01-25 21:27:36 +00:00
Kyle Carberry c5afaffa7e fix: Tidy up closes for nicer output (#4605)
* fix: Tidy up closes for nicer output

There was a context canceled message that would appear
because of traces, and this was using the wrong close.

I don't think it was causing any specific problems, but
it could make a replica warning appear on restart.

* Fix migration and experimental
2022-10-17 18:36:23 -05:00
Kyle CarberryandColin Adler 2ba4a62a0d feat: Add high availability for multiple replicas (#4555)
* feat: HA tailnet coordinator

* fixup! feat: HA tailnet coordinator

* fixup! feat: HA tailnet coordinator

* remove printlns

* close all connections on coordinator

* impelement high availability feature

* fixup! impelement high availability feature

* fixup! impelement high availability feature

* fixup! impelement high availability feature

* fixup! impelement high availability feature

* Add replicas

* Add DERP meshing to arbitrary addresses

* Move packages to highavailability folder

* Move coordinator to high availability package

* Add flags for HA

* Rename to replicasync

* Denest packages for replicas

* Add test for multiple replicas

* Fix coordination test

* Add HA to the helm chart

* Rename function pointer

* Add warnings for HA

* Add the ability to block endpoints

* Add flag to disable P2P connections

* Wow, I made the tests pass

* Add replicas endpoint

* Ensure close kills replica

* Update sql

* Add database latency to high availability

* Pipe TLS to DERP mesh

* Fix DERP mesh with TLS

* Add tests for TLS

* Fix replica sync TLS

* Fix RootCA for replica meshing

* Remove ID from replicasync

* Fix getting certificates for meshing

* Remove excessive locking

* Fix linting

* Store mesh key in the database

* Fix replica key for tests

* Fix types gen

* Fix unlocking unlocked

* Fix race in tests

* Update enterprise/derpmesh/derpmesh.go

Co-authored-by: Colin Adler <colin1adler@gmail.com>

* Rename to syncReplicas

* Reuse http client

* Delete old replicas on a CRON

* Fix race condition in connection tests

* Fix linting

* Fix nil type

* Move pubsub to in-memory for twenty test

* Add comment for configuration tweaking

* Fix leak with transport

* Fix close leak in derpmesh

* Fix race when creating server

* Remove handler update

* Skip test on Windows

* Fix DERP mesh test

* Wrap HTTP handler replacement in mutex

* Fix error message for relay

* Fix API handler for normal tests

* Fix speedtest

* Fix replica resend

* Fix derpmesh send

* Ping async

* Increase wait time of template version jobd

* Fix race when closing replica sync

* Add name to client

* Log the derpmap being used

* Don't connect if DERP is empty

* Improve agent coordinator logging

* Fix lock in coordinator

* Fix relay addr

* Fix race when updating durations

* Fix client publish race

* Run pubsub loop in a queue

* Store agent nodes in order

* Fix coordinator locking

* Check for closed pipe

Co-authored-by: Colin Adler <colin1adler@gmail.com>
2022-10-17 13:43:30 +00:00