This reverts commit 4f3aa9a3f2.
We're unable to build for 32-bit Linux due to
https://github.com/golang/go/issues/55152,
which looks like it will be fixed with Go 1.19.2 next week.
We'll re-evaluate with the next Go release and reintroduce this change
as soon as we can.
* Introduce proto types for ProvisionTokenV3
* Add methods to ProvisionTokenV3 to support ProvisionToken iface
* Start building v3 support into the client
* add support for mashalling and unmarshalling ProvisionTokenV3
* Start unit testing ProvisionTokenV3
* Remove oneof to support yaml marshal/unmarshal
* Client should try V3 methods and fallback to v2
* More tests
* Fix join tests
* Fix integration tests
* Switch integration tests to use v3 spec
* Switch iam tests to use ProvisionTokenV3
* Change ec2 join tests to use V3 tokens
* Fix events tests for V3 token
* support ProvisionTokenV3 within API client events handler
* Explicitly specify JoinMethod
* Tidy up final usage of NewProvisionTokenV2FromSpec in tests
* Improve proto docs on ProvisionTokenV3
* Fix bot join tests
* Clarify error message for invalid join method
* Adjust resource version comment
* Fix comments and return error rather than bool in V2() method
* Catch incompatible conversions case
* Include V2 ProvisionToken in tests and add appropriate DELETE IN notes
* Fix linter warnings/unit test failures
* Use nolint rather than lint:ignore
* Add more DELETE IN notes
* Run goimports on join_ec2_test.go
* Address PR comments from tim.
* Add more deprecation/delete in notices
* Improve godoc comments on checkAndSetDEfaults for provider config
* Simplify implementation by dropping client-ahead compatability
* Add some support for client-ahead but with conversion to v3
* Update code comments to include responsible party
* Rename `Role` to `RoleARN` in EC2 configuration for clarity
* Fix tests for Role -> RoleARN rename
* Move MustCreateProvisionToken out of API and into test packages
* Properly go imports files
* Add Yubikey PrivateKey implementation for use by Teleport clients.
- Add yubikey login logic, reusing previously stored private keys.
- Fix identity file decoding with PIV keys, which sign ecdsa certificates.
- Add libpcsclite-dev pre-req for building on linux.
- Remove unnecessary keys.Signer interface and move its functionality to keys.PrivateKey.
- Move retry and jitter utils to new api/utils/retryutils package.
* Add initial version of installer
* Resolve comments
- Use aws waiters when checking commands
- Use SSMRunRequest rather than passing instances
- General comments
* Resolve comments, (rebase) pass scriptname parameter
This resolves comments regarding running on multiple ec2 instances at
once by adding state to the instances cache to check if the instance
is known about and how far into installation it is
* Revert cache
* Dont cache on non discovery nodes
* Resolve some comments
* Move discovery out to its own serviec
* Add a `discovery_service` section
* Fix messed up conflict merge
* Make starting a standalone discovery agent work
* Resolve comments
* Resolve comments
- use a regular events.Emitter
- resolve a thousand typos :)
* Resolve comments
* resolve comments, fix a bad merge
* Fail when a non ec2 matcher type is configured
* fix lint-go
* Resolve comments
* Resolve comments, add initial test (currently broken)
* Fix log string so only 1 pair of [] are used
* Chunk instances for sending commands
* add 'isInitialized' to watchers
* Add test for chunked discovery, log output
* lints
* explicetly set matcher.Tags to "*":"*" if its unset
Coverts getting values in the `FnCache` from `FnCache.Get` to
`utils.FnCacheGet[T any]`. This allows us to have compile time
guarantees for the type being loaded, which allows us to remove
some of the manual type checking required of callers today.
Update metalinter, fix a few lint warnings and replace deprecated linters.
`deadcode`, `structcheck` and `varcheck` are abandoned and now replaced by [`unused`][1].
Since 1.19, `go fmt` reformats godocs according to https://go.dev/doc/comment. I've done a bulk-reformatting of the codebase to keep the linter happy. Backporting is mostly harmless (the exception being `lib/services/role_test.go`, that for some reason breaks the _old_ linter using the new format).
[1]: https://golangci-lint.run/usage/linters/
* Bump golangci-lint version
* Replace abandoned linters
* Fix bodyclose on lib/auth/github.com
* Fix bodyclose on lib/kube/proxy/streamproto/proto_test.go
* Fix bodyclose on lib/srv/alpnproxy/proxy_test.go
* Fix bodyclose on lib/web/conn_upgrade_test.go
* Silence staticcheck on lib/kube/proxy/forwarder_test.go
* Silence staticcheck on lib/utils/certs_test.go
* Address BuildNameToCertificate deprecation warnings
* Run `go fmt ./...`
* Run `go fmt ./...` on api/
* Ignore formatting in role_test.go
* Remove redundant initializers in lib/srv/uacc/
* Update e/
This PR introduces Auth Server CRUD API for managing the lifecycle of dynamic `kube_cluster` resources.
Currently, `kube_cluster` resources cannot be manipulated without using the API, nor are they being listened to. It will be part of future developments.
# Future work
- The `kubernetes_service` will have to watch these `kube_cluster` resources.
- The discovery service will have to create these resources.
Resource manipulation via `tctl` will be introduced later and is not part of the scope.
Part of #16135, #13376
Related to #12048
* Allow for multiple installer scripts instead of using a singleton
* Resolve comments
- require name be provided to installer
- MetaNameClusterInstallerScript -> defaults.InstallerScriptName
- Require VerbList for GetInstallers
* Resolve comments
- Include the default installer in GetInstallers output
- Update how the scripts are prefixed
* resolve conflicts
* make the linters happy
* Add an installer script resource type and HTTP endpoint
* Add default installer scripts
* Resolve comments
- run `shfmt` over the installer script
- add MustNewInstaller
- move default installer logic to GetInstaller
- only include PublicProxyAddr in server.Config
* Resolve comments
- move from auth_with_roles -> auth/grpcserver
- document `MustNewInstallerV1`
## What
First part of the Kubernetes [Discovery RFD](https://github.com/gravitational/teleport/pull/13376/) to introduce a Kubernetes server per cluster.
This PR introduces a separate Kubernetes server that uses the already introduced `KubernetesClusterV3`.
## Compatibility
In previous versions, Kubernetes Clusters were part of regular `ServerV2` resource and this refactoring deprecates the `ServerV2` usage but keeps them for compatibility with previous version.
Everything is backward compatible, so v10 kubernetes agents and trusted clusters can connect fine.
## Next steps
Once this is merged, a new PR will introduce dynamic registration for Kubernetes Clusters discovered through EKS Discovery.
* Embed auth.Cache in auth.Server
* Hit the backend during Auth initialization
* Bypass the cache when rotating CAs
* Services.UpsertTrustedCluster is different
* Bypass the cache in waitForTunnelConnections
* Fix infinite recursion
* More cache bypassing during init and rotations
* Rename Services to Uncached in auth.Server
* Further cleanups
* Don't start the auth cache immediately
* Go back to Services rather than Uncached
* Comments and a missing method
* Permit parallelization in cache init
Allow both the auth and proxy caches to fetch multiple resources
in parallel. All other services are still limited to a single
resource at a time to reduce network spikes when a cluster is
recovering.
Also adds some spans to cache fetch and collection apply to make
identifying cache init traces easier.
Adds a `trace.Tracer` to the `cache.Cache`so
that all operations can be traced. Also, updates
the `utils.FnCache` to link the `context.Context`
passed to the loadFn to any current spans. This
will allow any fetches made by the `FnCache` to
be associated with the call that lead to loading.
Note: There are a few methods in the `auth.Cache`
interface which do not take a `context.Context`.
For the time being all spans for these calls use
a `context.TODO` and will be updated in the future,
where the changes can be made on a per method basis.
Fixes total count errors for the web UI:
- Adds de-duplicating matches for `FakePaginate` func.
- This change does makes `FakePaginate` less efficient b/c
it will always run filter for every resource per fetch versus just
running it on the initial fetch to get the total count.
- Removes inaccurate de-dupping in the web api layer since we
do it from the back now
- Adds another criteria when de-duplicating `applications` by checking
for its `public_addr` as well as its name.
* Remove more deprecated code
In the case where we were using the PostalCode, StreetAddress,
and Province fields rather than ASN.1 extensions we could have
removed more code, but I opted for a phase approach where we stop
populating the fields in v10 (but still accept them), and then
stop reading them entirely in v11.
* Fix lints and remove more unused code
* Restore StreetAddress / PostalCode
We don't yet have dedicated extensions for these
* FnCache context improvements
Modifies the FnCache to be constructed with a context.Context
which is passed to all loadfn provided to Get. This removes
the burden placed on callers to ensure loadfn are not request
context scoped.
* Remove unused backend wrapper from Cache
* Remove double printShutdownStatus
* Fix readyz race condition
* Test coverage for the readyz.monitor fix
* Close listeners immediately in proxy.shutdown
* Use and handle net.ErrClosed correctly
This adapts utils.IsUseOfClosedNetworkError to check for net.ErrClosed
even inside trace.Aggregate errors, makes it so that we always return
something that would pass errors.Is(err, net.ErrClosed) when returning
from a (net.Listener).Accept(), and handles closed listeners within our
various Serve() loops so that we don't hit spurious backoff waits while
shutting down.
* Close listeners early and emitters late
* Test coverage for the proxy listener changes
* Revert some errors back to trace.ConnectionProblem
* Reduce PR scope to just the proxy, add comments
* Improve error logging.
The RemoteClusterTunnelManager was hanging on the call to
GetReverseTunnels because it did not take a context and thus did
not respect cancellation. During certain shutdown/restart scenarios
the auth server may not respond, which causes Sync to never return
and subsequently prevents Run from ever completing.
By requiring a context to be passed to GetReverseTunnels, we can
remove the hard coded context.TODO that was being sent to the
http client and fully respect cancellation.
* Prevent relative expiry from emitting more events than can be processed
This alters relative expiry in four ways:
1) Relative expiry is no longer exclusively run by the auth cache
2) Relative expiry no longer emits delete events
3) There is now a limit to the number of nodes removed per interval
4) Relative expiry now runs more frequently
We can remove the need to emit any fake delete events during relative
expiry by not running it exclusively in the auth cache. All caches
will now run relative expiry themselves, even the components that
don't watch for nodes - this is effectively a noop for them. To
prevent the individual caches from getting too far out of sync, the
expiry interval is set to a much smaller value and we limit the
number of nodes being deleted per interval.
This adds a test, `TestCacheWatchKindExistsInEvents`, that ensures
all `types.WatchKind` registered for all Cache components have a
corresponding `proto.Event` defined. By doing this we can ensure
that for a release there will be no `resource type X is not
suppported` errors causing the Cache to be unhealthy. Unfortunately
this does not prevent issues across releases which may have different
resources being emitted.
Introduce Database Certificate Authority. New CA is used by Database Access to sign database certificates making them independent from Host CA.
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Always use in-memory caches
This also cleans up now-useless fields and constants related to on-disk
caches.
* Remove the cache tombstone mechanism
As we're never reopening the same cache backend twice, this is no longer
useful.
* Warn if a cache directory exists on disk
We can't remove it automatically because we might be in the middle of an
upgrade with a old version of Teleport still running.
* Fix goroutine and memory leak in watchCertAuthorities
The CA Watcher was blocking both on writing to a channel when the watcher
was closed and on HTTP calls that had no request timeout or context passed
to cause cancellation.
All resourceWatcher implementations that had a bug which may cause them to block
on writing to a channel forever were fixed by selecting on the write and ctx.Done.
Adding context.Context to all Get/Put/Post/Delete methods on the auth HTTPClient to
force callers to propagate context. Prior all calls used context.TODO which
prevents requests from being properly cancelled.
Add context propagation to RotateCertAuthority, RotateExternalCertAuthority,
GetCertAuthority, GetCertAuthorities. This is needed to get the correct ctx
from the CertAtuhorityWatcher all the way down to the HTTPClient that makes
the call.
Closes#10648
* The web UI needed to show the total count of resources available after filter
* ListResources response returns as a struct to make adding extra fields easier
* Add support for windows desktop services proxying different desktops
* Add filter to GetWindowsDesktops, remove GetWindowsDesktop and GetWindowsDesktopByName
* Cache cleanup
* Fix cache deletes for Windows desktops
For deletes, the cache only gets the backend key, not the entire
resource. Do what database access does, which is to extract the
host ID from the path, and stuff it in the description field of
the resource header.
* Godoc cleanup
* Fix lint
* Address review comments
* Send error message if no desktop found
* Revert to x/net/websocket
This got converted to gorilla/websocket as part of moderated sessions.
We'll do a more intentional conversion post-release.
* fix lint
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Define sorters for resource Server, AppServer, and DbServer
* Add sorting to ListResources in caching and presence layer
* ListResources now returns nextKey set to the limit+1th item,
previously it returned a possible next key, where there
may or may not be more results.
This adds an rpc UpsertKubeServiceV2 to replace UpsertKubeService. Currently, kubernetes service does not have a keepalive heartbeat unlike app, db, and windows service. This brings functionality in line with the others. This would allow for future use of the keepalive to track connected agents via prometheus metrics.
* Dynamically resolve reverse tunnel address
The reverse tunnel address is currently a static string that is
retrieved from config and passed around for the duration of a
services lifetime. When the `tunnel_public_address` is changed
on the proxy and the proxy is then restarted, all established
reverse tunnels over the old address will fail indefinintely.
As a means to get around this, #8102 introduced a mechanism
that would cause nodes to restart if their connection to the
auth server was down for a period of time. While this did
allow the nodes to pickup the new address after the nodes
restarted it was meant to be a stop gap until a more robust
solution could be applid.
Instead of using a static address, the reverse tunnel address
is now resolved via a `reversetunnel.Resolver`. Anywhere that
previoulsy relied on the static proxy address now will fetch
the actual reverse tunnel address via the webclient by using
the Resolver. In addition this builds on the refactoring done
in #4290 to further simplify the reversetunnel package. Since
we no longer track multiple proxies, all the left over bits
that did so have been removed to accomodate using a dynamic
reverse tunnel address.
Fix flaky unit tests
Addresses issues causing failures in TestCache_Backoff, TestTeleportProcess_reconnectToAuth
and TestResourceWatcher_Backoff. By utilizing FakeClock.BlockUntil tests ensure that the clock
will not be advanced until retry.After has been called. Move retry duration channels to config in order to allow them to be buffered by tests.
Lisa Kim