Some fetchers might generate a lot of errors which are aggregated and
sent to the `DiscoveryConfig.Status.ErrorMessage`.
This might cause gRPC clients to fail when listing DiscoveryConfigs with
a `grpc: received message larger than max (xyz vs. 4194304)` error.
Allowing 100KB for the error message should give the user enough context
on what's causing the errors.
* TLS routing connection upgrade using native websocket
* update ut in api
* lib/web/UT update
* fix typo, lint and race
* deal with subprotocol negotiation
* review comments round 1
* fix lint?
* add UT and address some other comments
* add env var to toggle mode
* fix lint
* Allow configuring number of parallel execution workers
* Remove an obsolete command
* Revert test: do not check parallel behavior
* Move worker count field into Assist config
* Replace waitgroup and semaphore with errgroup
* Adjust log message
* Switch ACEW to int32, add additional validation
An idle timeout of 6m keeps web api connections open too long on
Proxies that have any agents which do not have the changes from
#22832. Because the http.Server.IdleTimeout is set on all release
versions now, even when built with go.19, we experience this issue
because we no longer exercise the bug that was fixed in go1.20.
* ALPN connect test improvements
* fix typos
* remove extra period
* simplify error check
* fix func name typo
* add a few comments to clarify things
* fix lint
Sets `http.Server.IdleTimeout` and `http.Client.IdleConnTimeout`
on clients and servers which didn't have them set. A default of
360s was chosen to be on par with the default of an NLB without
being identical.
This was added as another safety measure to prevent leaking any
idle connections indefinitely as seen in #22757.
`apidefaults.DefaultDialTimeout` was also renamed to
`apidefaults.DefaultIOTimeout` to better reflect its usage.
* Make SessionTracker heartbeat loop more robust
`(*SessionTracker).UpdateExpirationLoop` terminated the first time
it got an error attempting to persist the updated expiry to Auth.
A momentary network outage could prevent the `types.SessionTracker`
from being updated again causing it to have the wrong TTL in the
backend. This could result in sessions sticking around longer than
needed or being assumed to have been terminated prior to the session
actually ending.
A `retryutils.Retry` has been added to allow for retrying to update
a `types.SessionTracker` with some backoff. The retry loop will continue
until the session is closed, the update is successful, or the retry
has occurred for longer than the `types.SessionTrackers` last known
expiry.
`apidefaults.SessionTrackerTTL` was also reduced from 1h to 30m to
reduce the amount of time a stale tracker may linger in the backend.
The added robustness to the update loop should alleviate the need for
the lengthy TTL.
* Add client side circuit breaker to auth clients
In order to apply back pressure we can utilize a circuit breaker that
monitors error responses from auth server. When tripped it will prevent
all outbound requests to auth for a period of time. This can also help
prevent a potential thundering heard when auth is in an unhealthy state.
By default the circuit breaker will only be tripped if 90% of the
requests made in the monitoring interval fail.
Some integration tests modify global "constants" to speed up test
execution (e.g. shortening polling intervals). This is occasionally
tripping the Go data race detector, so I have added explicit
serialisation to reading and writing these global settings.
These values are only ever changed in a test environment, and there
should be zero contention for them in a non-test environment.
* Use fake clock consistently in units tests.
* Split web session management into two interfaces and implement them separately for clear separation
* Split session management into New/Validate to make it aparent where the sessions are created and where existing sessions are managed. Remove ttlmap in favor of a simple map and handle expirations
explicitly.
Add web session management to gRPC server for the cache.
* Reintroduce web sessions APIs under a getter interface.
* Add SubKind to WatchKind for gRPC and add conversions from/to protobuf. Fix web sessions unit tests.
* lib/web: create/insert session context in ValidateSession if the session has not yet been added to session cache.
lib/cache: add event filter for web session in auth cache.
lib/auth: propagate web session subkind in gRPC event.
* Add implicit migrations for legacy web session key path for queries.
* Integrate web token in lib/web
* Add a bearer token when upserting a web session
* Fix tests. Use fake clock wherever possible.
* Converge session cache handling in lib/web
* Clean up and add doc comments where necessary
* Use correct form of sessions/tokens controller for ServerWithRoles. Use fake time in web tests
* Converge the web sessions/tokens handling in lib/auth to match the old behavior w.r.t access checking (e.g. implicit handling of the local user identity).
* Use cached reads and waiters only when necessary. Query sessions/tokens using best-effort - first looking in the cache and falling back to a proxy client
* Properly propagate events about deletes for values with subkind.
* Update to retrofit changes after recent teleport API refactorings
* Update comment on removing legacy code to move the deadline to 7.x
* Do not close the resources on the session when it expires - this beats the purpose of this PR.
Also avoid a race between closing the cached clients and an existing reference to the session by letting the session linger for longer before removing it.
* Move web session/token request structs to the api client proto package
* Only set HTTP fs on the web handler if the UI is enabled
* Properly tear down web session test by releasing resources at the end. Fix the web UI assets configuration by removing DisableUI and instead use the presence of assets (HTTP file system) as an indicator that the web UI has been enabled.
* Decrease the expired session cache clean up threshold to 2m. Only log the expiration error message for errors other than not found
* Add test for terminal disconnect when using two proxies in HA mode
David Boslee