* plumb context to keystore methods
Contexts are more relevant now with the GCP KMS keystore which makes
http requests, this commit makes sure they are passed everywhere they
are needed, with only a few `context.TODO`s sprinkled in.
* Embed auth.Cache in auth.Server
* Hit the backend during Auth initialization
* Bypass the cache when rotating CAs
* Services.UpsertTrustedCluster is different
* Bypass the cache in waitForTunnelConnections
* Fix infinite recursion
* More cache bypassing during init and rotations
* Rename Services to Uncached in auth.Server
* Further cleanups
* Don't start the auth cache immediately
* Go back to Services rather than Uncached
* Comments and a missing method
* Fix goroutine and memory leak in watchCertAuthorities
The CA Watcher was blocking both on writing to a channel when the watcher
was closed and on HTTP calls that had no request timeout or context passed
to cause cancellation.
All resourceWatcher implementations that had a bug which may cause them to block
on writing to a channel forever were fixed by selecting on the write and ctx.Done.
Adding context.Context to all Get/Put/Post/Delete methods on the auth HTTPClient to
force callers to propagate context. Prior all calls used context.TODO which
prevents requests from being properly cancelled.
Add context propagation to RotateCertAuthority, RotateExternalCertAuthority,
GetCertAuthority, GetCertAuthorities. This is needed to get the correct ctx
from the CertAtuhorityWatcher all the way down to the HTTPClient that makes
the call.
Closes#10648
* hsm: migrate CA storage schema
Migrate types.CertAuthorityV2 schema according to
https://github.com/gravitational/teleport/blob/master/rfd/0025-hsm.md#backend-storage
Includes proto changes, types.CertAuthority wrapper changes and data
migration.
Note that we keep and update the old fields for backwards-compatibility.
If a cluster is upgraded to v7 and then downgraded back to v6,
everything should keep working.
* Address review feedback
In `auth.Context`, the `Identity` field used to contain the original
caller identity and `User` field contained the mapped local user. These
are different, if the request comes from a remote trusted cluster.
Lots of code assumed that `auth.Context.Identity` contained the local
identity and used roles/traits from there.
To prevent this confusion, populate `auth.Context.Identity` with the
*mapped* identity, and add `auth.Context.UnmappedIdentity` for callers
that actually need it.
One caller that needs `UnmappedIdentity` is the k8s proxy. It uses that
identity to generate an ephemeral user cert. Using the local mapped
identity in that case would make the downstream server (e.g.
kubernetes_service) to treat it like a real local user, which doesn't
exist in the backend and causes trouble.
`ProcessKubeCSR` endpoint on the auth server was also updated to
understand the unmapped remote identities.
Co-authored-by: Andrew Lytvynov <andrew@goteleport.com>
Implements RFD #7https://github.com/gravitational/teleport/blob/master/rfd/0007-rbac-oss.md
OSS users can use roles. Some FedRamp related role options
are limited to enterprise.
All users are migrated to a new role "ossuser".
This role is a limited access role downgrading all users
from OSS role "admin".
All trusted clusters are mapped to "ossuser" as well.
Github connector maps teams to generated roles.
For transition period, format `tctl users add alice` works
alongside with `tctl users add alice --roles=admin`, but prints
a warning.
Auth server has to trust the proxy with all k8s CSR fields. Usage is the
only field it can enforce, to somewhat limit the blast radius of
compromising this cert flow.
When a proxy forwards a k8s request to another proxy in a trusted leaf
cluster, it dynamically generates a new client key/cert to impersonate
the user. This key/cert is presented to the leaf proxy as if a user was
directly making a request to it.
Auth server, when processing the CSR, would load user identity from the
backend. If this user had temporary role grants via workflow API, they
would not be loaded from the backend.
This means that if a user accesses k8s through:
```
kubectl -> root proxy -> leaf proxy -> k8s
```
the second hop would drop their temporary role grants.
To fix this, preserve full user identity from the original client cert
presented to root proxy, as encoded in CSR Subject. The auth server
implicitly trusts the CSR Subject that the proxy presents.
This also requires fixing the Subject encoding on the proxy side, which
was inconsistent with how auth server does it.
One downside here is: a compromised proxy can mint k8s client certs for
known users with arbitrary roles.
This commit fixes#3369, refs #3374
It adds support for kuberenetes_users section in roles,
allowing Teleport proxy to impersonate user identities.
It also extends variable interpolation syntax by adding
suffix and prefix to variables and function `email.local`:
Example:
```yaml
kind: role
version: v3
metadata:
name: admin
spec:
allow:
# extract email local part from the email claim
logins: ['{{email.local(external.email)}}']
# impersonate a kubernetes user with IAM prefix
kubernetes_users: ['IAM#{{external.email}}']
# the deny section uses the identical format as the 'allow' section.
# the deny rules always override allow rules.
deny: {}
```
Some notes on email.local behavior:
* This is the only function supported in the template variables for now
* In case if the email.local will encounter invalid email address,
it will interpolate to empty value, will be removed from resulting
output.
Changes in impersonation behavior:
* By default, if no kubernetes_users is set, which is a majority of cases,
user will impersonate themselves, which is the backwards-compatible behavior.
* As long as at least one `kubernetes_users` is set, the forwarder will start
limiting the list of users allowed by the client to impersonate.
* If the users' role set does not include actual user name, it will be rejected,
otherwise there will be no way to exclude the user from the list).
* If the `kuberentes_users` role set includes only one user
(quite frequently that's the real intent), teleport will default to it,
otherwise it will refuse to select.
This will enable the use case when `kubernetes_users` has just one field to
link the user identity with the IAM role, for example `IAM#{{external.email}}`
* Previous versions of the forwarding proxy were denying all external
impersonation headers, this commit allows 'Impesrsonate-User' and
'Impersonate-Group' header values that are allowed by role set.
* Previous versions of the forwarding proxy ignored 'Deny' section of the roles
when applied to impersonation, this commit fixes that - roles with deny
kubernetes_users and kubernetes_groups section will not allow
impersonation of those users and groups.
If an attacker can force a username change at an IdP, upon second login,
the services.User object of the original user can be updated with new
roles and traits. If these new roles and traits differ, the original
user can have their privileges raised (or lowered).
To mitigate this, encode roles and traits within the certificate and use
these when fetching roles to make RBAC decisions. If roles and traits are
not encoded within an certificate (for example for old style SSH
certificates then fallback to using the services.User object and log a
warning.
* Support resource-based bootstrapping for backend.
Outside of static configuration, most of the persistent state of an
auth server exists as a collection of resources, stored in its
backend. The resource API also forms the basis of Teleport's more
advanced dynamic configuration options.
This commit extends the usefulness of the resource API by adding
the ability to bootstrap backend state with a set of previously
exported resources. This allows the resource API to serve as a
rudimentary backup/migration tool.
Notes: This features is a work in progress, and very easy to misuse;
while it will prevent you from overwriting the state of an existing
auth server, it won't stop you from bootstrapping into a wildly
misconfigured state. In general, resource-based bootstrapping is
not a complete solution for backup or migration.
* update e-ref
This commit expands the usage of the caching layer
for auth server API:
* Introduces in-memory cache that is used to serve all
Auth server API requests. This is done to achieve scalability
on 10K+ node clusters, where each node fetches certificate authorities,
roles, users and join tokens. It is not possible to scale
DynamoDB backend or other backends on 10K reads per seconds
on a single shard or partition. The solution is to introduce
an in-memory cache of the backend state that is always used
for reads.
* In-memory cache has been expanded to support all resources
required by the auth server.
* Experimental `tctl top` command has been introduced to display
common single node metrics.
Replace SQLite Memory Backend with BTree
SQLite in memory backend was suffering from
high tail latencies under load (up to 8 seconds
in 99.9%-ile on load configurations).
This commit replaces the SQLite memory caching
backend with in-memory BTree backend that
brought down tail latencies to 2 seconds (99.9%-ile)
and brought overall performance improvement.
This commit switches Teleport proxy to use impersonation
API instead of the CSR API.
This allows Teleport to work on EKS clusters, GKE and all
other CNCF compabitble clusters.
This commit updates helm chart RBAC as well.
It introduces extra configuration flag to proxy_service
configuration parameter:
```yaml
proxy_service:
# kubeconfig_file is used for scenarios
# when Teleport Proxy is deployed outside
# of the kubernetes cluster
kubeconfig_file: /path/to/kube/config
```
It deprecates similar flag in auth_service:
```yaml
auth_service:
# DEPRECATED. THIS FLAG IS IGNORED
kubeconfig_file: /path/to/kube/config
```
* Fetch groups for GSuite SSO. (#2456)
Fixes#2455
This commit adds support for fetching
groups for GSuite SSO logins via
OIDC connector interface.
If OIDC connector has a special scope:
`https://www.googleapis.com/auth/admin.directory.group.readonly`
teleport will fetch user's group membership and populate
groups claim.
* Pass kubernetes groups to the remote cluster. (#2484)
This commit allows remote cluster to reference
the kubernetes groups coming from the roles
of the main cluster in the trusted clusters
configuration.
For example, main cluster can have a user
with a role 'main' and kubernetes groups:
kube_groups: ['system:masters']
and SSH logins:
logins: ['root']
Remote cluster can choose to map
this 'main' cluster to it's own:
'remote-admin' cluster in the trusted cluster config:
role_map:
- remote: 'main'
local: 'remote-admin'
The role 'remote-admin' of the remote cluster
can now be templated to use the main cluster role main
logins and kubernetes_groups using variables:
logins: ['{{internal.logins}}']
kubernetes_groups: ['{{internal.kubernetes_groups}}']
This is possible because teleport now encodes
both values in X509 certificate metadata
and remote cluster passes these values as 'internal' traits
to the template engine.
Fixes#1986
When deployed outside of the kubernetes cluster,
teleport now reads all configuration from kubernetes
config file, supplied via parameter.
Auth server then passes information about
target api server back to the proxy.
This commit moves proxy kubernetes configuration
to a separate nested block to provide more fine
grained settings:
```yaml
auth:
kubernetes_ca_cert_path: /tmp/custom-ca
proxy:
enabled: yes
kubernetes:
enabled: yes
public_addr: [custom.example.com:port]
api_addr: kuberentes.example.com:443
listen_addr: localhost:3026
```
1. Kubernetes config section is explicitly enabled
and disabled. It is disabled by default.
2. Public address in kubernetes section
is propagated to tsh profile
The other part of the commit updates Ping
endpoint to send proxy configuration back to
the client, including kubernetes public address
and ssh listen address.
Clients updates profile accordingly to configuration
received from the proxy.
This issue updates #1986.
This is intial, experimental implementation that will
be updated with tests and edge cases prior to production 2.7.0 release.
Teleport proxy adds support for Kubernetes API protocol.
Auth server uses Kubernetes API to receive certificates
issued by Kubernetes CA.
Proxy intercepts and forwards API requests to the Kubernetes
API server and captures live session traffic, making
recordings available in the audit log.
Tsh login now updates kubeconfig configuration to use
Teleport as a proxy server.
Nic Klaassen