This code is not caught by linters because it's exported and they assume
there's some external users.
Since teleport is relatively self-contained, we can tell for sure
whether something is called or not.
Fixed findings:
```
lib/sshutils/server_test.go:163:2: SA4006: this value of `clt` is never used (staticcheck)
clt, err := ssh.Dial("tcp", srv.Addr(), &cc)
^
lib/sshutils/server_test.go:91:3: SA5001: should check returned error before deferring ch.Close() (staticcheck)
defer ch.Close()
^
lib/shell/shell_test.go:33:2: SA4006: this value of `shell` is never used (staticcheck)
shell, err = GetLoginShell("non-existent-user")
^
lib/cgroup/cgroup_test.go:111:2: SA9003: empty branch (staticcheck)
if err != nil {
^
lib/cgroup/cgroup_test.go:119:2: SA5001: should check returned error before deferring service.Close() (staticcheck)
defer service.Close()
^
lib/client/keystore_test.go:138:2: SA4006: this value of `keyCopy` is never used (staticcheck)
keyCopy, err = s.store.GetKey("host.a", "bob")
^
lib/client/api.go:1604:3: SA4004: the surrounding loop is unconditionally terminated (staticcheck)
return makeProxyClient(sshClient, m), nil
^
lib/backend/test/suite.go:156:2: SA4006: this value of `err` is never used (staticcheck)
result, err = s.B.GetRange(ctx, prefix("/prefix/c/c1"), backend.RangeEnd(prefix("/prefix/c/cz")), backend.NoLimit)
^
lib/utils/timeout_test.go:84:2: SA1019: t.Dial is deprecated: Use DialContext instead, which allows the transport to cancel dials as soon as they are no longer needed. If both are set, DialContext takes priority. (staticcheck)
t.Dial = func(network string, addr string) (net.Conn, error) {
^
lib/utils/websocketwriter.go:83:3: SA4006: this value of `err` is never used (staticcheck)
utf8, err = w.encoder.String(string(data))
^
lib/utils/loadbalancer_test.go:134:2: SA4006: this value of `out` is never used (staticcheck)
out, err = Roundtrip(frontend.String())
^
lib/utils/loadbalancer_test.go:209:2: SA4006: this value of `out` is never used (staticcheck)
out, err = RoundtripWithConn(conn)
^
lib/srv/forward/sshserver.go:582:3: SA4004: the surrounding loop is unconditionally terminated (staticcheck)
return
^
lib/service/service.go:347:4: SA4006: this value of `err` is never used (staticcheck)
i, err = auth.GenerateIdentity(process.localAuth, id, principals, dnsNames)
^
lib/service/signals.go:60:3: SA1016: syscall.SIGKILL cannot be trapped (did you mean syscall.SIGTERM?) (staticcheck)
syscall.SIGKILL, // fast shutdown
^
lib/config/configuration_test.go:184:2: SA4006: this value of `conf` is never used (staticcheck)
conf, err = ReadFromFile(s.configFileBadContent)
^
lib/config/configuration.go:129:2: SA5001: should check returned error before deferring reader.Close() (staticcheck)
defer reader.Close()
^
lib/kube/kubeconfig/kubeconfig_test.go:227:2: SA4006: this value of `err` is never used (staticcheck)
tlsCert, err := ca.GenerateCertificate(tlsca.CertificateRequest{
^
lib/srv/sess.go:720:3: SA4006: this value of `err` is never used (staticcheck)
result, err := s.term.Wait()
^
lib/multiplexer/multiplexer_test.go:169:11: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
_, err = fmt.Fprintf(conn, proxyLine.String())
^
lib/multiplexer/multiplexer_test.go:221:11: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
_, err = fmt.Fprintf(conn, proxyLine.String())
^
```
All changes should be noop, except for
`integration/integration_test.go`.
The integration test was ignoring `recordingMode` test case parameter
and always used `RecordAtNode`. When switching to `recordingMode`, test
cases with `RecordAtProxy` fail with a confusing error about missing
user agent. Filed https://github.com/gravitational/teleport/issues/3606
to track that separately and unblock enabling `structcheck` linter.
`SyncBuffer` has a goroutine running `io.Copy` to read from the
underlying pipe. Close stops the pipe, but doesn't wait for the last
chunk of data to be written by `io.Copy` to the buffer.
Both `Bytes` and `String` assume that the buffer received no further
writes after `Close`.
Add explicit synchronization between `io.Copy` goroutine and `Close`.
Spring cleaning!
A very mechanical cleanup using several linters (unused, deadcode,
structcheck). Build and tests still pass so no behavior should be
affected.
This commit fixes#3369, refs #3374
It adds support for kuberenetes_users section in roles,
allowing Teleport proxy to impersonate user identities.
It also extends variable interpolation syntax by adding
suffix and prefix to variables and function `email.local`:
Example:
```yaml
kind: role
version: v3
metadata:
name: admin
spec:
allow:
# extract email local part from the email claim
logins: ['{{email.local(external.email)}}']
# impersonate a kubernetes user with IAM prefix
kubernetes_users: ['IAM#{{external.email}}']
# the deny section uses the identical format as the 'allow' section.
# the deny rules always override allow rules.
deny: {}
```
Some notes on email.local behavior:
* This is the only function supported in the template variables for now
* In case if the email.local will encounter invalid email address,
it will interpolate to empty value, will be removed from resulting
output.
Changes in impersonation behavior:
* By default, if no kubernetes_users is set, which is a majority of cases,
user will impersonate themselves, which is the backwards-compatible behavior.
* As long as at least one `kubernetes_users` is set, the forwarder will start
limiting the list of users allowed by the client to impersonate.
* If the users' role set does not include actual user name, it will be rejected,
otherwise there will be no way to exclude the user from the list).
* If the `kuberentes_users` role set includes only one user
(quite frequently that's the real intent), teleport will default to it,
otherwise it will refuse to select.
This will enable the use case when `kubernetes_users` has just one field to
link the user identity with the IAM role, for example `IAM#{{external.email}}`
* Previous versions of the forwarding proxy were denying all external
impersonation headers, this commit allows 'Impesrsonate-User' and
'Impersonate-Group' header values that are allowed by role set.
* Previous versions of the forwarding proxy ignored 'Deny' section of the roles
when applied to impersonation, this commit fixes that - roles with deny
kubernetes_users and kubernetes_groups section will not allow
impersonation of those users and groups.
Added package cgroup to orchestrate cgroups. Only support for cgroup2
was added to utilize because cgroup2 cgroups have unique IDs that can be
used correlated with BPF events.
Added bpf package that contains three BPF programs: execsnoop,
opensnoop, and tcpconnect. The bpf package starts and stops these
programs as well correlating their output with Teleport sessions
and emitting them to the audit log.
Added support for Teleport to re-exec itself before launching a shell.
This allows Teleport to start a child process, capture it's PID, place
the PID in a cgroup, and then continue to process. Once the process is
continued it can be tracked by it's cgroup ID.
Reduced the total number of connections to a host so Teleport does not
quickly exhaust all file descriptors. Exhausting all file descriptors
happens very quickly when disk events are emitted to the audit log which
are emitted at a very high rate.
Added tarballs for exec sessions. Updated session.start and session.end
events with additional metadata. Updated the format of session tarballs
to include enhanced events.
Added file configuration for enhanced session recording. Added code to
startup enhanced session recording and pass package to SSH nodes.
When the remote address is an IP address, we cannot directly convert
byte array to string. Otherwise, we will see hex strings like
"\xac\x11\x01\xc8:443".
See more details about the bug in:
https://community.gravitational.com/t/ssh-dynamic-port-forwarding-not-working/432
The above reported issue is resolved after applying the patch.
Drop TLS_RSA_WITH_AES_128_GCM_SHA{256,384} from default ciphersuites
due to being banned by HTTP2 which breaks GRPC clients. For more
information see: https://tools.ietf.org/html/rfc7540#appendix-A.
This has a nice side effect of Teleport now only supporting ciphersuites
that support PFS.
Note that both ciphersuites can still be be added back in file
configuration.
Validate all incoming events (and archives) to ensure that the server ID
within the event matches the x509 identity of the connected host. This
check makes sure nodes can only submit events for themselves.
In addition, make sure session recordings to disk or S3 can not be
overwritten.
Fixes#2648
Teleport does not support SAML identity provider
initiated logins, this commit gives a better
error message to the user instructing them
what to do.
Update utils.CertChecker to only check key and certificate algorithms
when in FIPS mode. Otherwise accept keys and certificates generated with
any algorithm.
This commit implements #2543
In SSH terms ProxyJump is a shortcut for SSH client
connecting the proxy/jumphost and requesting .port forwarding to the
target node.
This commit adds support for direct-tcpip request support
in teleport proxy service that is an alias to the existing proxy
subsystem and reuses most of the code.
This commit also adds support to "route to cluster" metadata
encoded in SSH certificate making it possible to have client
SSH certificates to include the metadata that will cause the proxy
to route the client requests to a specific cluster.
`tsh ssh -J proxy:port ` is supported in a limited way:
Only one jump host is supported (-J supports chaining
that teleport does not utilise) and tsh will return with error
in case of two jumphosts: -J a,b will not work.
In case if `tsh ssh -J user@proxy` is used, it overrides
the SSH proxy coming from the tsh profile and port-forwarding
is used instead of the existing teleport proxy subsystem
This commit implements #2872.
Similarly to file://, the scheme `stdout://` could be used complimentary
to the existing external scheme to logs audit logs:
to stdout:
```yaml
audit_events_uri: ['dynamodb://events', 'stdout://',]
```
Just like `file://` scheme it is only possible to use 'stdout://'
scheme when external events and session uploader are defined,
so all audit upload and search features of teleport could work.
* Improve help text and error messages for tctl rm, fixes#2594
* Change 'kind' to 'type' for consistency
* Changed examples from role/admin to connector/github
* Added link to Teleport Enterprise
* Update e ref
This commit refactor discovery protocol
to make it less dependent on the database and
scale better on large numbers of tunnels.
Reverse tunnel is now always sending
back the list of all proxies registered in the
cluster in the form of discovery requests.
Before this commit, reverse tunnel server was comparing
existing TunnelConnection with the Proxies
and sending back the list of proxies that were not
discovered.
This required nodes to register tunnel connections
in the database and servers poll the connections.
On 10K clusters this is not scalable. Instead,
the change assumes that there is not a lot of
proxies so it's OK to send the information about
them back to all connected agents.
Agent pools can make up their own mind about what to
do with the information - they can ignore
the request as long as they observe all agents
connected to the requested proxies.
At the same time, to avoid using too much traffic,
reverse tunnel server only sends the discovery requests
after the first agent heartbeat and in case if
proxy list changes. To make it possible reverse tunnel
sets up a watch on the proxies.
Added "--fips" flag to "teleport start" command which can start
Enterprise in FedRAMP/FIPS 140-2 mode.
In FIPS mode, Teleport configures the TLS and SSH servers with FIPS
compliant cryptographic algorithms. In FIPS mode, if non-compliant
algorithms are chosen, Teleport will fail to start. In addition,
Teleport checks if the binary was compiled against an approved
cryptographic module (BoringCrypto) and fails to start if it was not.
If a client, like tsh, tries to use non-FIPS encryption, like NaCl,
those requests are also rejected.
Previous Login code path was calling itself recursively.
In some cases that lead to endless loops with browser
opening up forever.
This commit factors out retry logic to RetryWithLogin
decorator handler that is used by CLI explicitly.
Client code became better as a result as there are no
hidden side effects.
This commit adds `--bind-addr` flag to tsh login
and TELEPORT_LOGIN_BIND_ADDR environment variable
to set up login bind address for SSO redirect flows.
Usage examples:
```
tsh login --bind-addr=localhost:3333
tsh login --bind-addr=:3333
tsh login --bind-addr=[::1]:3333
TELEPORT_LOGIN_BIND_ADDR=localhost:7777 tsh login
```
Refactor redirect flow and fix URLs for --bind-addr
This commit expands the usage of the caching layer
for auth server API:
* Introduces in-memory cache that is used to serve all
Auth server API requests. This is done to achieve scalability
on 10K+ node clusters, where each node fetches certificate authorities,
roles, users and join tokens. It is not possible to scale
DynamoDB backend or other backends on 10K reads per seconds
on a single shard or partition. The solution is to introduce
an in-memory cache of the backend state that is always used
for reads.
* In-memory cache has been expanded to support all resources
required by the auth server.
* Experimental `tctl top` command has been introduced to display
common single node metrics.
Replace SQLite Memory Backend with BTree
SQLite in memory backend was suffering from
high tail latencies under load (up to 8 seconds
in 99.9%-ile on load configurations).
This commit replaces the SQLite memory caching
backend with in-memory BTree backend that
brought down tail latencies to 2 seconds (99.9%-ile)
and brought overall performance improvement.
This commit hex encodes trusted cluster names
in target addresses for kubernetes SNI proxy.
For example, assuming public address of Teleport
Kubernetes proxy is main.example.com, and trusted
cluster is remote.example.com, resulting target
address added to kubeconfig will look like
k72656d6f74652e6578616d706c652e636f6d0a.main.example.com
And Teleport Proxy's DNS Name will include wildcard:
'*.main.example.com' in addition to 'main.example.com'
Note that no dots are in the SNI address thanks to hex encoding.
This will allow administrators to avoid manually updating
list of public_addr sections every time the trusted cluster and use
the wildcard DNS name.
The following addr:
remote.example.com.main.example.com would not have matched
*.main.example.com per DNS wildcard spec.
Added utils.CertChecker that wraps a ssh.CertChecker. The new
certificate checker first checks if the certificate is a valid
certificate for Teleport. At the moment that is 2048-bit RSA then calls
the underlying certificate checker to perform the requested validation.
Improved error messaging when Certificate Authority (CA) pin does not
match provided pin. Fixed typo in abbreviation of "Subject Public Key
Info" from SKPI to SPKI.
Created *utils.TrackingConn that wraps the server side net.Conn and is
used to track how much data is transmitted and received over the
net.Conn. At the close of a connection (close of a *srv.ServerContext)
the total data transmitted and received is emitted to the Audit Log.
This commit fixes an issue when tctl
output in YAML was not compatible with tctl create.
Before:
```
$ tctl get users
- User: value
- User: value
```
After:
```
$ tctl get users
User: value
---
User: value
```
Whenever teleport was started with -d flag,
output target was always overwritten to stderr.
This commit makes sure that target supplied
in the configuration is not overwritten
even in case when -d flag is set.
This commit introduces several key changes to
Teleport backend and API infrastructure
in order to achieve scalability improvements
on 10K+ node deployments.
Events and plain keyspace
--------------------------
New backend interface supports events,
pagination and range queries
and moves away from buckets to
plain keyspace, what better aligns
with DynamoDB and Etcd featuring similar
interfaces.
All backend implementations are
exposing Events API, allowing
multiple subscribers to consume the same
event stream and avoid polling database.
Replacing BoltDB, Dir with SQLite
-------------------------------
BoltDB backend does not support
having two processes access the database at the
same time. This prevented Teleport
using BoltDB backend to be live reloaded.
SQLite supports reads/writes by multiple
processes and makes Dir backend obsolete
as SQLite is more efficient on larger collections,
supports transactions and can detect data
corruption.
Teleport automatically migrates data from
Bolt and Dir backends into SQLite.
GRPC API and protobuf resources
-------------------------------
GRPC API has been introduced for
the auth server. The auth server now serves both GRPC
and JSON-HTTP API on the same TLS socket and uses
the same client certificate authentication.
All future API methods should use GRPC and HTTP-JSON
API is considered obsolete.
In addition to that some resources like
Server and CertificateAuthority are now
generated from protobuf service specifications in
a way that is fully backward compatible with
original JSON spec and schema, so the same resource
can be encoded and decoded from JSON, YAML
and protobuf.
All models should be refactored
into new proto specification over time.
Streaming presence service
--------------------------
In order to cut bandwidth, nodes
are sending full updates only when changes
to labels or spec have occured, otherwise
new light-weight GRPC keep alive updates are sent
over to the presence service, reducing
bandwidth usage on multi-node deployments.
In addition to that nodes are no longer polling
auth server for certificate authority rotation
updates, instead they subscribe to event updates
to detect updates as soon as they happen.
This is a new API, so the errors are inevitable,
that's why polling is still done, but
on a way slower rate.
Andrew Lytvynov