Many of our tests (db package, I'm looking at you) generate many RSA keys. This has two main side effects; makes our tests slow and flaky as CPU usage spikes in random moments when the tests are run in parallel.
This change pre-generates RSA keys at the beginning of each test module and reuse them in randomized order to reduce the situation that one key has been used multiple times in one test.
I had to move a few files to avoid circular dependencies.
* Add Yubikey PrivateKey implementation for use by Teleport clients.
- Add yubikey login logic, reusing previously stored private keys.
- Fix identity file decoding with PIV keys, which sign ecdsa certificates.
- Add libpcsclite-dev pre-req for building on linux.
- Remove unnecessary keys.Signer interface and move its functionality to keys.PrivateKey.
- Move retry and jitter utils to new api/utils/retryutils package.
* Optionally provide ca_pin as a file path
* Fix lint errors
* Test CA pin reads from file and from string
* Simplify ApplyCAPins
* Update lib/service/cfg.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update lib/utils/utils_test.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Move length check into ApplyCAPins
* Rename token.go to config.go
* Add test for statictoken parsing
* Refactor token parsing test
* Error on empty ca_pin file
* Warn if config value is a path to an empty file
* Update lib/config/fileconf_test.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Log empty file warning instead of printing
* Allow multiple entries in files referenced by config values
* Update static token parsing test
* Test reading multiple tokens/pins from a file referenced by a config
* Update lib/config/fileconf.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
This commit introduces GRPC API for streaming sessions.
It adds structured events and sync streaming
that avoids storing events on disk.
You can find design in rfd/0002-streaming.md RFD.
This commit fixes#3369, refs #3374
It adds support for kuberenetes_users section in roles,
allowing Teleport proxy to impersonate user identities.
It also extends variable interpolation syntax by adding
suffix and prefix to variables and function `email.local`:
Example:
```yaml
kind: role
version: v3
metadata:
name: admin
spec:
allow:
# extract email local part from the email claim
logins: ['{{email.local(external.email)}}']
# impersonate a kubernetes user with IAM prefix
kubernetes_users: ['IAM#{{external.email}}']
# the deny section uses the identical format as the 'allow' section.
# the deny rules always override allow rules.
deny: {}
```
Some notes on email.local behavior:
* This is the only function supported in the template variables for now
* In case if the email.local will encounter invalid email address,
it will interpolate to empty value, will be removed from resulting
output.
Changes in impersonation behavior:
* By default, if no kubernetes_users is set, which is a majority of cases,
user will impersonate themselves, which is the backwards-compatible behavior.
* As long as at least one `kubernetes_users` is set, the forwarder will start
limiting the list of users allowed by the client to impersonate.
* If the users' role set does not include actual user name, it will be rejected,
otherwise there will be no way to exclude the user from the list).
* If the `kuberentes_users` role set includes only one user
(quite frequently that's the real intent), teleport will default to it,
otherwise it will refuse to select.
This will enable the use case when `kubernetes_users` has just one field to
link the user identity with the IAM role, for example `IAM#{{external.email}}`
* Previous versions of the forwarding proxy were denying all external
impersonation headers, this commit allows 'Impesrsonate-User' and
'Impersonate-Group' header values that are allowed by role set.
* Previous versions of the forwarding proxy ignored 'Deny' section of the roles
when applied to impersonation, this commit fixes that - roles with deny
kubernetes_users and kubernetes_groups section will not allow
impersonation of those users and groups.
Fixes#2648
Teleport does not support SAML identity provider
initiated logins, this commit gives a better
error message to the user instructing them
what to do.
This commit adds `--bind-addr` flag to tsh login
and TELEPORT_LOGIN_BIND_ADDR environment variable
to set up login bind address for SSO redirect flows.
Usage examples:
```
tsh login --bind-addr=localhost:3333
tsh login --bind-addr=:3333
tsh login --bind-addr=[::1]:3333
TELEPORT_LOGIN_BIND_ADDR=localhost:7777 tsh login
```
Refactor redirect flow and fix URLs for --bind-addr
This commit expands the usage of the caching layer
for auth server API:
* Introduces in-memory cache that is used to serve all
Auth server API requests. This is done to achieve scalability
on 10K+ node clusters, where each node fetches certificate authorities,
roles, users and join tokens. It is not possible to scale
DynamoDB backend or other backends on 10K reads per seconds
on a single shard or partition. The solution is to introduce
an in-memory cache of the backend state that is always used
for reads.
* In-memory cache has been expanded to support all resources
required by the auth server.
* Experimental `tctl top` command has been introduced to display
common single node metrics.
Replace SQLite Memory Backend with BTree
SQLite in memory backend was suffering from
high tail latencies under load (up to 8 seconds
in 99.9%-ile on load configurations).
This commit replaces the SQLite memory caching
backend with in-memory BTree backend that
brought down tail latencies to 2 seconds (99.9%-ile)
and brought overall performance improvement.
This commit fixes an issue when tctl
output in YAML was not compatible with tctl create.
Before:
```
$ tctl get users
- User: value
- User: value
```
After:
```
$ tctl get users
User: value
---
User: value
```
This commit adds two extensions to template variables
in roles and adds support for regular expressions
and group captures in role mapping of trusted clusters.
1. Roles node_labels can expand variables from traits:
allow:
node_labels:
'{{external.key}}': '{{external.val}}'
deny:
node_labels:
'{{external.key}}': '{{external.val}}'
If traits variable is not found, label key pair in allow or
deny rule will be set to empty key or value, so if 'external.val'
trait is missing, the resulting role will not match
allow or deny rule:
allow:
node_labels:
'': 'val'
deny:
node_labels:
'': 'val'
Same thing will happen for missing value:
allow:
node_labels:
'key': ''
deny:
node_labels:
'key': ''
2. Trusted cluster role mapping can now
support advanced expressions:
a. Glob values will math any string, including
empty one
role_map:
- remote: 'cluster-*'
local: [clusteradmin]
a. Regular expression syntax is supported:
Syntax: https://github.com/google/re2/wiki/Syntax
Brackets can be used as a capture group and referred
to with expand variable:
role_map:
- remote: '^clusteradmin-(.*)$'
local: [unprivileged-$1]
Will map incoming role 'clusteradmin-account-1' to 'guest-account-1'.
3. Same regular expression syntax is supported for SAML and OIDC
mappings:
a. Glob matches of values instead of static matches:
claims_to_roles:
- {claim: "roles", value: "gravitational/*", roles: ["clusteradmin"]}
b. Regexp matches with subgroup expands:
attributes_to_roles:
- {name: "roles", value: "^gravitational/(.*)$", roles: ["cluster-$1"]}
This commit fixes#1803, fixes#1889
* Adds support for public_addr for Proxy and Auth
* Parameter advertise_ip now supports host:port format
* Fixes incorrect output for tctl get proxies
* Fixes duplicate output of some error messages.
Updates #1755
Design
------
This commit adds support for pluggable events and
sessions recordings and adds several plugins.
In case if external sessions recording storage
is used, nodes or proxies depending on configuration
store the session recordings locally and
then upload the recordings in the background.
Non-print session events are always sent to the
remote auth server as usual.
In case if remote events storage is used, auth
servers download recordings from it during playbacks.
DynamoDB event backend
----------------------
Transient DynamoDB backend is added for events
storage. Events are stored with default TTL of 1 year.
External lambda functions should be used
to forward events from DynamoDB.
Parameter audit_table_name in storage section
turns on dynamodb backend.
The table will be auto created.
S3 sessions backend
-------------------
If audit_sessions_uri is specified to s3://bucket-name
node or proxy depending on recording mode
will start uploading the recorded sessions
to the bucket.
If the bucket does not exist, teleport will
attempt to create a bucket with versioning and encryption
turned on by default.
Teleport will turn on bucket-side encryption for the tarballs
using aws:kms key.
File sessions backend
---------------------
If audit_sessions_uri is specified to file:///folder
teleport will start writing tarballs to this folder instead
of sending records to the file server.
This is helpful for plugin writers who can use fuse or NFS
mounted storage to handle the data.
Working dynamic configuration.
Fixes#1663, #1665
This commit fixes a couple of issues with tsh and tctl:
* tsh now check the version of the server and prints error
if tsh is newer version than the server.
* tctl did not work properly when yaml file contained
multiple resources in a list
This commit:
* Improves on the previous commit on better name resolution by the proxy
(and fixes a bug introduced by the previous commit)
* Removes 'host_login' from tsh client profile. Closes#729
Teleport has always worked on OSX, but not all tests were OSX
compatible:
- some wouuld hang
- others would fail
This commit fixes that. `make test` passes on OSX.
auth server. This is needed when you configure cluster from scratch and
all nodes including auth server spin up simultaneously.
* Add tctl tools to generate keys and certificates
+ Command "tctl authorities gen" generates public and private keypair.
+ Command "tctl authorities gencert" generates public and private keypair signed
by existng private key
+ Command "tctl authorities export" was modified to be able to export exisitng private
CA keys to local storage
All of these commands are hidden by default.
section "static configuration"
* Add ability to configure teleport from environment variable
Environment variable TELEPORT_CONFIG can contain base64 encoded
YAML file config file of the standard file format, so teleport will use it on start
* Add special secrets section to the config file
Section "secrets" was updated to support pre-configured trusted CA keys and pre-generated keys
* Add special rts hidden section to add support for provisioning
Three changes:
1. Pings are randomized. Note `utils.RandomizedDuration()` function.
2. Sleeps are twice as long between failures.
3. Made tests less vulnerable to sleep times (also sped them up by about
three seconds)
+ minor refactoring, moved "magic constants" into a proper
constants in `defaults` module.
