Adds a new `GetCertAuthorities` unary RPC to
`teleport.v1.trust.TrustService` and deprecates the equivalent
HTTP endpoint. All associated interfaces have been updated to remove
the marshal options as well.
This PR removes the `types.KindKubeService` objects that were
deprecated in Teleport 11.
Teleport 12 removed the double heartbeat and for Teleport 13 we can
safely drop the support of the old hearbeat based on `types.Server`
objects.
Clients older than Teleport 12 connecting to a Teleport 13 server will
fail since they cannot heartbeat using the old mechanism.
The HTTP client created by `auth.NewClient` has an idle timeout of
30s and each instance periodically gets cert authorities in
`TeleportProcess.syncRotationStateCycle`. If there has been no
activity when the ticker expires then the request will result in a
brand new connection to Auth to be established. This can cause a
spikes in open file descriptors on the Proxy when there are a large
number of instances connected via a tunnel.
To prevent periodic dialing when retrieving cert authorities the
endpoint has been migrated to the new Trust gRPC service. For now
only the get endpoint has been migrated but the rest of the CRUD
operations for cert authorities should be migrated from the HTTP
api to the gRPC service as well.
* Add UiConfig option to proxy service config
* Add uiConfig to GRV_CONFIG window object and use in terminal constructor
* Update configuration reference docs
* Remove extra structs and change property to scrollbackLines
* Update web/packages/teleport/src/config.ts
Co-authored-by: Jeff Pihach <hatched@users.noreply.github.com>
* Use a common struct and apply in applyProxyConfig
* Remove webclient dependency for fileconf
* Add missing doc comments
* Set individual properties of cfg.Proxy.UI
* Remove unneeded comment
* Add nolint rule
* Fix nolint
* Add test for UIConfig
* Cleanup tests
* Fix TestUIConfig context
* Allow dynamic config for web ui (#22097)
* Update protofiles
---------
Co-authored-by: Jeff Pihach <hatched@users.noreply.github.com>
* Create a generic local backend service.
Many of the local backend services have a very similar pattern in terms of how
objects are retrieved and stored. This PR introduces a generic backend which
implements backend retrieval and storage of objects and migrates the SAML
IdP service provider service to it. More services could easily be migrated
to this generic service implementation.
* Feedback, add in generic tests.
* Tim's feedback, added in a test for updating a resource that doesn't exist.
* Tim's feedback.
* Apply suggestions from code review
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Edoardo's feedback.
* Added in UpdateAndSwap.
* Edoardo's feedback, Tim's feedback.
* Use a default page size if the page limit is not specified.
* More of Edoardo's feedback.
* Propagate error instead of panic.
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* SAML IdP sessions added to the API and cache.
SAML IdP sessions have now been added to the API and to the cache.
* Update lib/auth/sessions.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Revert changes to types.proto.
* Fix missing session ID, misnamed variable, gRPC update.
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Add SAML IdP service providers to the cache and CLI.
SAML IdP service providers have been added to the cache and are now manageable
via the CLI. Some alterations to role permissions have occurred to ensure that
the system and users can manage these.
* Tiago's feedback.
* Update tool/tctl/common/collection.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Tim's feedback.
* Update lib/cache/collections.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
Update the version we should check against: v12 (the current in master)
Update the list of resources to watch for RemoteOldProxy.
The list should be equal to the RemoteProxy as seen in `branch/v11`
As soon as we cut v12, we should update this again in master because
we'll be preparing v13.
When modifying the `types.WatchKind` of `ForRemoteProxy` a series of
steps need to be performed to prevent bricking remote cluster caches.
First, the `cfg.Watches` of `ForOldRemoteProxy` must be replaced
with the current `cfg.Watches` of `ForRemoteProxy`. Next, the
version used by `lib/reversetunnel/srv.go` to determine whether
to use `ForRemoteProxy` or `ForOldRemoteProxy` must be updated
to be the release in which the new resource(s) will exist in.
Once both of these are done, `ForRemoteProxy` may be updated.
Comments are added to `ForRemoteProxy`, `ForOldRemoteProxy`, and
`createRemoteAccessPoint` to help prevent backward incompatible
changes like https://github.com/gravitational/teleport/issues/17211
and https://github.com/gravitational/teleport/issues/19907 from
occurring again in future releases.
n the context of Teleport Discover, we must be able to known if there's any DatabaseService available to proxy a given Database resource.
If there's none available, we will offer a script for the user to run and install a DatabaseService which proxies the desired Database resource.
By DatabaseService, we mean the process that Teleport binary manages when the `teleport.yaml` config has the following section:
```yaml
db_service:
enabled: "yes"
```
To accomplish this we are creating a new resource: DatabaseService.
The UI will fetch all DatabaseServices and check if there's any ResourceMatcher that matches the DatabaseLabels.
Previous PRs created the DatabaseService resource and its CRUD methods.
This PR adds an heartbeat for DatabaseServices similar to what we have for Databases.
There's also a new command to list DatabaseServices using `tctl`:
```
$ tctl get db_service --format text
Name Resource Matchers
------------------------------------ --------------------------------------
a6065ee9-d5ee-4555-8d47-94a78625277b (Labels: <all databases>)
d4e13f2b-0a55-4e0a-b363-bacfb1a11294 (Labels: env=[prod],aws-tag=[xyz abc])
```
Test using Teleport Cloud
```
dinis@lenix ~/p/cloudagents> tctl get db_services
kind: db_service
metadata:
expires: "2022-12-21T18:05:10Z"
id: 1671645310983808522
name: 2a28d394-900c-42ea-a120-eed918e4526b
spec:
resources:
- labels:
aws-tag:
- xyz
- abc
env: prod
version: v1
dinis@lenix ~/p/cloudagents> tctl status
Cluster marcoacme.cloud.gravitational.io
Version 12.0.0-dev
```
Part of #19032
Related #19363#19469
* Improve and unify cache logging
Re-init the cache errors now provide the error via `WithError` instead
of passing the stringify error into the log message. This allows for
the stack trace of the log to show up in the logs to help determine
which resource caused the error.
Logging in the various collection resources was all converted to
follow the same pattern to ensure consistency.
The `Logger` field of the cache was also moved to a named field instead
of being an embedded field.
With the arrival of Teleport 12, we stop supporting v10.x.x clients, and we no longer require the `types.KindKubernetesService` legacy heartbeat.
This change removes the legacy heartbeat from Kubernetes Service but keeps the legacy Auth Server CRUD methods and heartbeat support to maintain compatibility with Teleport 11 Kubernetes Service - Teleport 11 still heartbeats the legacy type.
We postponed the related `DELETES` to Teleport 13.
This PR presents a watcher for automatic `kube_cluster` discovery for Azure AKS clusters. Given a user with access to the Azure cloud, the auto-discovery service will scan the cloud and register all clusters available in AKS .
Once the discovery service creates a `kube_cluster` in Auth Server, the Kubernetes Service will start serving it. The credentials used to access the cluster depend on the different AKS clusters configurations:
# Authentication
## Local Accounts
If the AKS cluster auth is based on local accounts created during the provisioning phase of the cluster, the agent will use the [`aks:ListClusterUserCredentials`](https://learn.microsoft.com/en-us/rest/api/aks/managed-clusters/list-cluster-user-credentials?tabs=HTTP) endpoint.
This endpoint returns a `kubeconfig` fully populated with user credentials that Teleport can use to access the cluster.
## AZ Active Directory
When AZ active directory integration is enabled, Azure allows login with AD users. Azure forces the login to happen with dynamic short-lived user tokens. These tokens are generated by calling `credentials.GetToken` with a fixed Scope: `6dae42f8-4368-4678-94ff-3960e28e3630` and with the cluster's `tenant_id`. The token contains the user details as well as `group_ids` to match with authorization rules.
```go
// getAzureToken generates an authentication token for clusters with AD enabled.
func (a *aKSClient) getAzureToken(ctx context.Context, tentantID string, clientCfg *rest.Config) (time.Time, error) {
const (
azureManagedClusterScope = "6dae42f8-4368-4678-94ff-3960e28e3630"
)
cred, err := a.azIdentity(&azidentity.DefaultAzureCredentialOptions{
TenantID: tentantID,
})
if err != nil {
return time.Time{}, trace.Wrap(ConvertResponseError(err))
}
cliAccessToken, err := cred.GetToken(ctx, policy.TokenRequestOptions{
// azureManagedClusterScope is a fixed scope that identifies azure AKS managed clusters.
Scopes: []string{azureManagedClusterScope},
},
)
if err != nil {
return time.Time{}, trace.Wrap(ConvertResponseError(err))
}
// reset the old exec provider credentials
clientCfg.ExecProvider = nil
clientCfg.BearerToken = cliAccessToken.Token
return cliAccessToken.ExpiresOn, nil
}
```
# Authorization
## Local Accounts
The [`aks:ListClusterUserCredentials`](https://learn.microsoft.com/en-us/rest/api/aks/managed-clusters/list-cluster-user-credentials?tabs=HTTP) endpoint returns credentials with enough permissions for Teleport to enroll the cluster.
## AZ AD
### Azure RBAC
When Azure RBAC mode is enabled, the cluster authorization is based on rules specified in the Azure Identity permissions.
The AZ group associated with the AZ identity the Teleport Process is running has to define the following permissions:
```json
{
"Name": "AKS Teleport Discovery Permissions",
"Description": "Required permissions for Teleport auto-discovery.",
"Actions": [],
"NotActions": [],
"DataActions": [
"Microsoft.ContainerService/managedClusters/pods/read",
"Microsoft.ContainerService/managedClusters/users/impersonate/action",
"Microsoft.ContainerService/managedClusters/groups/impersonate/action",
"Microsoft.ContainerService/managedClusters/serviceaccounts/impersonate/action",
"Microsoft.ContainerService/managedClusters/authorization.k8s.io/selfsubjectaccessreviews/write",
"Microsoft.ContainerService/managedClusters/authorization.k8s.io/selfsubjectrulesreviews/write",
],
"NotDataActions": [],
"assignableScopes": [
"/subscriptions/{subscription_id}"
]
}
```
If correctly specified, the Azure authentication service automatically grants access to any cluster within `subscription_id`
without any other definition. On the other hand, if it's incorrectly configured, an error is triggered but Teleport cannot gain access to the cluster.
### Kubernetes RBAC
If AZ RBAC integration is disabled, the authorization to the cluster is processed by Kubernetes RBAC. This is done by matching the Az Identity principals (`group_ids`) with `Role`, `ClusterRole` objects that live in the AKS cluster. This mode requires that the `ClusterRole` and `ClusterRoleBinding` must exist and must be well configured for each cluster to enroll.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: teleport-role
rules:
- apiGroups:
- ""
resources:
- users
- groups
- serviceaccounts
verbs:
- impersonate
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- "authorization.k8s.io"
resources:
- selfsubjectaccessreviews
- selfsubjectrulesreviews
verbs:
- create
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: teleport-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: teleport-role
subjects:
- kind: Group
name: {group_name}
apiGroup: rbac.authorization.k8s.io
```
#### `ClusterRole` and `ClusterRoleBinding` configured
If cluster operators or previous Teleport run has configured access to the cluster, no further action is required since Teleport already has access to the cluster.
#### Cluster `aks:ListClusterAdminCredentials` returns valid credentials
If the Teleport process has access to [`aks:ListClusterAdminCredentials`](https://learn.microsoft.com/en-us/rest/api/aks/managed-clusters/list-cluster-admin-credentials?tabs=HTTP) and the endpoint returns valid cluster admin credentials, Teleport will automatically create the `ClusterRole` and `ClusterRoleBinding` objects in the cluster configured to the `group_id` that is listed in the access token. In order to extract the `group_id` from the token, Teleport parses the JWT claims and extracts the first element.
If the object creation was successful, Teleport can access it, otherwise, it will use the `aks:BeginRunCommand` method to try to configure access to itself.
#### Cluster `aks:BeginRunCommand` returns valid credentials
When we reach this mode, Teleport tries to run a `kubectl` command against the cluster to configure the `ClusterRole` and `ClusterRoleBinding`. `aks:BeginRunCommand` allows any user with access to that endpoint to run arbitrary commands in the cluster (commands cannot be validated). Teleport will use it as the last resource to configure the access to itself.
If the command failed, Teleport cannot grant access to the cluster and an error is returned.
# UX
Currently, to discover AKS resources created and to have them dynamically served by the `kubernetes_service`one can define the following configuration.
```yaml
discovery_service:
enabled: true
azure:
- subscriptions: ["*"]
types: ["aks"]
regions: ["*"]
tags:
'*': '*'
kubernetes_service:
enabled: true
resources:
labels:
'*': '*'
```
# Future work
- Support AWS dynamic authentication
Part of #16135, #13376
Related to #12048, #16276, #16281
V10 Leaf clusters when connecting to a V11 Root cluster failed to initialize because they do not have `kube_server` resources defined.
This PR bumps the OLD remote proxy to versions <11 and removes the expected `kube_server` watcher.
It also removes `types.KindInstaller`, `types.KindDatabase`, and `types.KindApp` watchers from Remote Proxies since they do not affect remote reverse tunnels tracked from leaf clusters.
Fix incompatibility with pre/post v10.2.1 versions as the `types.KindInstaller` feature did not exist in older versions.
Fixes#17211Fixes#17219
This reverts commit 4f3aa9a3f2.
We're unable to build for 32-bit Linux due to
https://github.com/golang/go/issues/55152,
which looks like it will be fixed with Go 1.19.2 next week.
We'll re-evaluate with the next Go release and reintroduce this change
as soon as we can.
* Add Yubikey PrivateKey implementation for use by Teleport clients.
- Add yubikey login logic, reusing previously stored private keys.
- Fix identity file decoding with PIV keys, which sign ecdsa certificates.
- Add libpcsclite-dev pre-req for building on linux.
- Remove unnecessary keys.Signer interface and move its functionality to keys.PrivateKey.
- Move retry and jitter utils to new api/utils/retryutils package.
* Add initial version of installer
* Resolve comments
- Use aws waiters when checking commands
- Use SSMRunRequest rather than passing instances
- General comments
* Resolve comments, (rebase) pass scriptname parameter
This resolves comments regarding running on multiple ec2 instances at
once by adding state to the instances cache to check if the instance
is known about and how far into installation it is
* Revert cache
* Dont cache on non discovery nodes
* Resolve some comments
* Move discovery out to its own serviec
* Add a `discovery_service` section
* Fix messed up conflict merge
* Make starting a standalone discovery agent work
* Resolve comments
* Resolve comments
- use a regular events.Emitter
- resolve a thousand typos :)
* Resolve comments
* resolve comments, fix a bad merge
* Fail when a non ec2 matcher type is configured
* fix lint-go
* Resolve comments
* Resolve comments, add initial test (currently broken)
* Fix log string so only 1 pair of [] are used
* Chunk instances for sending commands
* add 'isInitialized' to watchers
* Add test for chunked discovery, log output
* lints
* explicetly set matcher.Tags to "*":"*" if its unset
Coverts getting values in the `FnCache` from `FnCache.Get` to
`utils.FnCacheGet[T any]`. This allows us to have compile time
guarantees for the type being loaded, which allows us to remove
some of the manual type checking required of callers today.
Update metalinter, fix a few lint warnings and replace deprecated linters.
`deadcode`, `structcheck` and `varcheck` are abandoned and now replaced by [`unused`][1].
Since 1.19, `go fmt` reformats godocs according to https://go.dev/doc/comment. I've done a bulk-reformatting of the codebase to keep the linter happy. Backporting is mostly harmless (the exception being `lib/services/role_test.go`, that for some reason breaks the _old_ linter using the new format).
[1]: https://golangci-lint.run/usage/linters/
* Bump golangci-lint version
* Replace abandoned linters
* Fix bodyclose on lib/auth/github.com
* Fix bodyclose on lib/kube/proxy/streamproto/proto_test.go
* Fix bodyclose on lib/srv/alpnproxy/proxy_test.go
* Fix bodyclose on lib/web/conn_upgrade_test.go
* Silence staticcheck on lib/kube/proxy/forwarder_test.go
* Silence staticcheck on lib/utils/certs_test.go
* Address BuildNameToCertificate deprecation warnings
* Run `go fmt ./...`
* Run `go fmt ./...` on api/
* Ignore formatting in role_test.go
* Remove redundant initializers in lib/srv/uacc/
* Update e/
This PR introduces Auth Server CRUD API for managing the lifecycle of dynamic `kube_cluster` resources.
Currently, `kube_cluster` resources cannot be manipulated without using the API, nor are they being listened to. It will be part of future developments.
# Future work
- The `kubernetes_service` will have to watch these `kube_cluster` resources.
- The discovery service will have to create these resources.
Resource manipulation via `tctl` will be introduced later and is not part of the scope.
Part of #16135, #13376
Related to #12048
* Allow for multiple installer scripts instead of using a singleton
* Resolve comments
- require name be provided to installer
- MetaNameClusterInstallerScript -> defaults.InstallerScriptName
- Require VerbList for GetInstallers
* Resolve comments
- Include the default installer in GetInstallers output
- Update how the scripts are prefixed
* resolve conflicts
* make the linters happy
* Add an installer script resource type and HTTP endpoint
* Add default installer scripts
* Resolve comments
- run `shfmt` over the installer script
- add MustNewInstaller
- move default installer logic to GetInstaller
- only include PublicProxyAddr in server.Config
* Resolve comments
- move from auth_with_roles -> auth/grpcserver
- document `MustNewInstallerV1`
## What
First part of the Kubernetes [Discovery RFD](https://github.com/gravitational/teleport/pull/13376/) to introduce a Kubernetes server per cluster.
This PR introduces a separate Kubernetes server that uses the already introduced `KubernetesClusterV3`.
## Compatibility
In previous versions, Kubernetes Clusters were part of regular `ServerV2` resource and this refactoring deprecates the `ServerV2` usage but keeps them for compatibility with previous version.
Everything is backward compatible, so v10 kubernetes agents and trusted clusters can connect fine.
## Next steps
Once this is merged, a new PR will introduce dynamic registration for Kubernetes Clusters discovered through EKS Discovery.
* Embed auth.Cache in auth.Server
* Hit the backend during Auth initialization
* Bypass the cache when rotating CAs
* Services.UpsertTrustedCluster is different
* Bypass the cache in waitForTunnelConnections
* Fix infinite recursion
* More cache bypassing during init and rotations
* Rename Services to Uncached in auth.Server
* Further cleanups
* Don't start the auth cache immediately
* Go back to Services rather than Uncached
* Comments and a missing method
* Permit parallelization in cache init
Allow both the auth and proxy caches to fetch multiple resources
in parallel. All other services are still limited to a single
resource at a time to reduce network spikes when a cluster is
recovering.
Also adds some spans to cache fetch and collection apply to make
identifying cache init traces easier.
Adds a `trace.Tracer` to the `cache.Cache`so
that all operations can be traced. Also, updates
the `utils.FnCache` to link the `context.Context`
passed to the loadFn to any current spans. This
will allow any fetches made by the `FnCache` to
be associated with the call that lead to loading.
Note: There are a few methods in the `auth.Cache`
interface which do not take a `context.Context`.
For the time being all spans for these calls use
a `context.TODO` and will be updated in the future,
where the changes can be made on a per method basis.
* FnCache context improvements
Modifies the FnCache to be constructed with a context.Context
which is passed to all loadfn provided to Get. This removes
the burden placed on callers to ensure loadfn are not request
context scoped.
* Remove unused backend wrapper from Cache
* Remove double printShutdownStatus
* Fix readyz race condition
* Test coverage for the readyz.monitor fix
* Close listeners immediately in proxy.shutdown
* Use and handle net.ErrClosed correctly
This adapts utils.IsUseOfClosedNetworkError to check for net.ErrClosed
even inside trace.Aggregate errors, makes it so that we always return
something that would pass errors.Is(err, net.ErrClosed) when returning
from a (net.Listener).Accept(), and handles closed listeners within our
various Serve() loops so that we don't hit spurious backoff waits while
shutting down.
* Close listeners early and emitters late
* Test coverage for the proxy listener changes
* Revert some errors back to trace.ConnectionProblem
* Reduce PR scope to just the proxy, add comments
* Improve error logging.
The RemoteClusterTunnelManager was hanging on the call to
GetReverseTunnels because it did not take a context and thus did
not respect cancellation. During certain shutdown/restart scenarios
the auth server may not respond, which causes Sync to never return
and subsequently prevents Run from ever completing.
By requiring a context to be passed to GetReverseTunnels, we can
remove the hard coded context.TODO that was being sent to the
http client and fully respect cancellation.
* Prevent relative expiry from emitting more events than can be processed
This alters relative expiry in four ways:
1) Relative expiry is no longer exclusively run by the auth cache
2) Relative expiry no longer emits delete events
3) There is now a limit to the number of nodes removed per interval
4) Relative expiry now runs more frequently
We can remove the need to emit any fake delete events during relative
expiry by not running it exclusively in the auth cache. All caches
will now run relative expiry themselves, even the components that
don't watch for nodes - this is effectively a noop for them. To
prevent the individual caches from getting too far out of sync, the
expiry interval is set to a much smaller value and we limit the
number of nodes being deleted per interval.
* Always use in-memory caches
This also cleans up now-useless fields and constants related to on-disk
caches.
* Remove the cache tombstone mechanism
As we're never reopening the same cache backend twice, this is no longer
useful.
* Warn if a cache directory exists on disk
We can't remove it automatically because we might be in the middle of an
upgrade with a old version of Teleport still running.
* Fix goroutine and memory leak in watchCertAuthorities
The CA Watcher was blocking both on writing to a channel when the watcher
was closed and on HTTP calls that had no request timeout or context passed
to cause cancellation.
All resourceWatcher implementations that had a bug which may cause them to block
on writing to a channel forever were fixed by selecting on the write and ctx.Done.
Adding context.Context to all Get/Put/Post/Delete methods on the auth HTTPClient to
force callers to propagate context. Prior all calls used context.TODO which
prevents requests from being properly cancelled.
Add context propagation to RotateCertAuthority, RotateExternalCertAuthority,
GetCertAuthority, GetCertAuthorities. This is needed to get the correct ctx
from the CertAtuhorityWatcher all the way down to the HTTPClient that makes
the call.
Closes#10648
* The web UI needed to show the total count of resources available after filter
* ListResources response returns as a struct to make adding extra fields easier
rosstimothy