This code is not caught by linters because it's exported and they assume
there's some external users.
Since teleport is relatively self-contained, we can tell for sure
whether something is called or not.
* Add UpdateUser rpc to proto
* Differentiate between create and update in github,oidc,saml
* Edit updated_by event field to be more generic (used with contexts to capture user modifying records)
* Update security issue by removing secrets from user when update/upsert/create (forrest)
* Update createUser in resource_command and require force for updates
Fixed issues:
```
lib/web/apiserver_test.go:499:2: SA4006: this value of `re` is never used (staticcheck)
re, err = pack.clt.Get(context.Background(), pack.clt.Endpoint("webapi", "sites"), url.Values{})
^
lib/web/apiserver_test.go:590:2: SA4006: this value of `re` is never used (staticcheck)
re, err = newPack.clt.Get(context.Background(), pack.clt.Endpoint("webapi", "sites"), url.Values{})
^
lib/web/apiserver_test.go:598:2: SA4006: this value of `re` is never used (staticcheck)
re, err = oldClt.Get(context.Background(), pack.clt.Endpoint("webapi", "sites"), url.Values{})
^
lib/web/apiserver_test.go:608:2: SA4006: this value of `re` is never used (staticcheck)
re, err = newPack.clt.Get(context.Background(), pack.clt.Endpoint("webapi", "sites"), url.Values{})
^
lib/web/apiserver_test.go:1106:2: SA4006: this value of `err` is never used (staticcheck)
loginReq, err := json.Marshal(createSessionReq{
^
lib/web/apiserver_test.go:1120:2: SA4006: this value of `err` is never used (staticcheck)
re, err := clt.Client.RoundTrip(func() (*http.Response, error) {
^
lib/web/apiserver_test.go:1148:2: SA4006: this value of `re` is never used (staticcheck)
re, err = clt.Get(context.Background(), clt.Endpoint("webapi", "sites"), url.Values{})
^
lib/web/apiserver_test.go:1154:2: SA4006: this value of `re` is never used (staticcheck)
re, err = clt.Get(context.Background(), clt.Endpoint("webapi", "sites"), url.Values{})
^
lib/web/apiserver_test.go:1192:2: SA4006: this value of `err` is never used (staticcheck)
data, err := json.Marshal(auth.ChangePasswordWithTokenRequest{
^
lib/web/apiserver_test.go:1206:2: SA4006: this value of `err` is never used (staticcheck)
re, err = clt.Client.RoundTrip(func() (*http.Response, error) {
^
lib/web/apiserver_test.go:1246:2: SA4006: this value of `err` is never used (staticcheck)
data, err := json.Marshal(auth.ChangePasswordWithTokenRequest{
^
lib/web/apiserver_test.go:1260:2: SA4006: this value of `err` is never used (staticcheck)
re, err = clt.Client.RoundTrip(func() (*http.Response, error) {
^
lib/web/static_test.go:93:2: SA4006: this value of `n` is never used (staticcheck)
n, err = f.Seek(-50, io.SeekEnd)
^
lib/web/static_test.go:100:2: SA4006: this value of `n` is never used (staticcheck)
n, err = f.Seek(-50, io.SeekCurrent)
^
```
All changes should be noop, except for
`integration/integration_test.go`.
The integration test was ignoring `recordingMode` test case parameter
and always used `RecordAtNode`. When switching to `recordingMode`, test
cases with `RecordAtProxy` fail with a confusing error about missing
user agent. Filed https://github.com/gravitational/teleport/issues/3606
to track that separately and unblock enabling `structcheck` linter.
* Define a CreateUser event, code, and its related const
* Define CreateUser rpc in auth/proto file
* Define CreateUser between layers
* Replace UpsertUser with CreateUser in tctl and in unit test
* Add monorepo
* Add reset/passwd capability for local users (#3287)
* Add UserTokens to allow password resets
* Pass context down through ChangePasswordWithToken
* Rename UserToken to ResetPasswordToken
* Add auto formatting for proto files
* Add common Marshaller interfaces to reset password token
* Allow enterprise "tctl" reuse OSS user methods (#3344)
* Pass localAuthEnabled flag to UI (#3412)
* Added LocalAuthEnabled prop to WebConfigAuthSetting struct in webconfig.go
* Added LocalAuthEnabled state as part of webCfg in apiserver.go
* update e-refs
* Fix a regression bug after merge
* Update tctl CLI output msgs (#3442)
* Use local user client when resolving user roles
* Update webapps ref
* Add and retrieve fields from Cluster struct (#3476)
* Set Teleport versions for node, auth, proxy init heartbeat
* Add and retrieve fields NodeCount, PublicURL, AuthVersion from Clusters
* Remove debug logging to avoid log pollution when getting public_addr of proxy
* Create helper func GuessProxyHost to get the public_addr of a proxy host
* Refactor newResetPasswordToken to use GuessProxyHost and remove publicUrl func
* Remove webapps submodule
* Add webassets submodule
* Replace webapps sub-module reference with webassets
* Update webassets path in Makefile
* Update webassets
1b11b26 Simplify and clean up Makefile (#62) https://github.com/gravitational/webapps/commit/1b11b26
* Retrieve cluster details for user context (#3515)
* Let GuessProxyHost also return proxy's version
* Unit test GuessProxyHostAndVersion & GetClusterDetails
* Update webassets
4dfef4e Fix build pipeline (#66) https://github.com/gravitational/webapps/commit/4dfef4e
* Update e-ref
* Update webassets
0647568 Fix OSS redirects https://github.com/gravitational/webapps/commit/0647568
* update e-ref
* Update webassets
e0f4189 Address security audit warnings Updates "minimist" package which is used by 7y old "optimist". https://github.com/gravitational/webapps/commit/e0f4189
* Add new attr to Session struct (#3574)
* Add fields ServerHostname and ServerAddr
* Set these fields on newSession
* Ensure webassets submodule during build
* Update e-ref
* Ensure webassets before running unit-tests
* Update E-ref
Co-authored-by: Lisa Kim <lisa@gravitational.com>
Co-authored-by: Pierre Beaucamp <pierre@gravitational.com>
Co-authored-by: Jenkins <jenkins@gravitational.io>
Spring cleaning!
A very mechanical cleanup using several linters (unused, deadcode,
structcheck). Build and tests still pass so no behavior should be
affected.
* Add temp. check for mismatch proxy/auth version
* Follow standard for making code temporary
* Rename confusing variable names
* Add rolling version and modify message
Added support for reexec during port forwarding. This allows Teleport
nodes to run PAM code before port forwarding requests. This makes any
memory leaks in PAM code less dangerous as well as bringing port
forwarding logic in-line with execution requests (exec or shell).
Added package cgroup to orchestrate cgroups. Only support for cgroup2
was added to utilize because cgroup2 cgroups have unique IDs that can be
used correlated with BPF events.
Added bpf package that contains three BPF programs: execsnoop,
opensnoop, and tcpconnect. The bpf package starts and stops these
programs as well correlating their output with Teleport sessions
and emitting them to the audit log.
Added support for Teleport to re-exec itself before launching a shell.
This allows Teleport to start a child process, capture it's PID, place
the PID in a cgroup, and then continue to process. Once the process is
continued it can be tracked by it's cgroup ID.
Reduced the total number of connections to a host so Teleport does not
quickly exhaust all file descriptors. Exhausting all file descriptors
happens very quickly when disk events are emitted to the audit log which
are emitted at a very high rate.
Added tarballs for exec sessions. Updated session.start and session.end
events with additional metadata. Updated the format of session tarballs
to include enhanced events.
Added file configuration for enhanced session recording. Added code to
startup enhanced session recording and pass package to SSH nodes.
This commit fixes Web UI in FIPS mode when local_auth is false
and removes two ciphers banned by HTTP2 rfc spec:
https://tools.ietf.org/html/rfc7540#appendix-A
and used by FIPS, causing Teleport GRPC to fail.
If an attacker can force a username change at an IdP, upon second login,
the services.User object of the original user can be updated with new
roles and traits. If these new roles and traits differ, the original
user can have their privileges raised (or lowered).
To mitigate this, encode roles and traits within the certificate and use
these when fetching roles to make RBAC decisions. If roles and traits are
not encoded within an certificate (for example for old style SSH
certificates then fallback to using the services.User object and log a
warning.
* Support resource-based bootstrapping for backend.
Outside of static configuration, most of the persistent state of an
auth server exists as a collection of resources, stored in its
backend. The resource API also forms the basis of Teleport's more
advanced dynamic configuration options.
This commit extends the usefulness of the resource API by adding
the ability to bootstrap backend state with a set of previously
exported resources. This allows the resource API to serve as a
rudimentary backup/migration tool.
Notes: This features is a work in progress, and very easy to misuse;
while it will prevent you from overwriting the state of an existing
auth server, it won't stop you from bootstrapping into a wildly
misconfigured state. In general, resource-based bootstrapping is
not a complete solution for backup or migration.
* update e-ref
Fixes#2648
Teleport does not support SAML identity provider
initiated logins, this commit gives a better
error message to the user instructing them
what to do.
Invalid or inaccessible OIDC connectors were preventing
all other logins to work, as they were grabbing mutex
and making a network call.
Besides, the error was not informative as the user
saw "web headers timeout" error in the browser.
This fix makes sure that:
* Connector errors are isolated to the connector name
and other flows are not blocked
* Error is presented to user in a reasonable way and
provides instructions
* Administrator can see the error in the audit logs.
Added "--fips" flag to "teleport start" command which can start
Enterprise in FedRAMP/FIPS 140-2 mode.
In FIPS mode, Teleport configures the TLS and SSH servers with FIPS
compliant cryptographic algorithms. In FIPS mode, if non-compliant
algorithms are chosen, Teleport will fail to start. In addition,
Teleport checks if the binary was compiled against an approved
cryptographic module (BoringCrypto) and fails to start if it was not.
If a client, like tsh, tries to use non-FIPS encryption, like NaCl,
those requests are also rejected.
In case of IOT (whenever teleport nodes are
connecting to the proxy), there is no need
to create ReverseTunnel objects in the backend,
as there is always one reverse tunnel per node.
This commit removes the logic that created
reverse tunnel object in the backed in IOT cases
and refactors some other parts of the code.
Whenever many IOT style nodes are connecting
back to the web proxy server, they all
call /find endpoint to discover the configuration.
This new endpoint is designed to be fast and not
hit the database.
In addition to that every proxy reverse tunnel
connection handler was fetching auth servers and
this commit adds caching for the auth servers
on the proxy side.
Updated services.ReverseTunnel to support type (proxy or node). For
proxy types, which represent trusted cluster connections, when a
services.ReverseTunnel is created, it's created on the remote side with
name /reverseTunnels/example.com. For node types, services.ReverseTunnel
is created on the main side as /reverseTunnels/{nodeUUID}.clusterName.
Updated services.TunnelConn to support type (proxy or node). For proxy
types, which represent trusted cluster connections, tunnel connections
are created on the main side under
/tunnelConnections/remote.example.com/{proxyUUID}-remote.example.com.
For nodes, tunnel connections are created on the main side under
/tunnelConnections/example.com/{proxyUUID}-example.com. This allows
searching for tunnel connections by cluster then allows easily creating
a set of proxies that are missing matching services.TunnelConn.
The reverse tunnel server has been updated to handle heartbeats from
proxies as well as nodes. Proxy heartbeat behavior has not changed.
Heartbeats from nodes now add remote connections to the matching local
site. In addition, the reverse tunnel server now proxies connection to
the Auth Server for requests that are already authenticated (a second
authentication to the Auth Server is required).
For registration, nodes try and connect to the Auth Server to fetch host
credentials. Upon failure, nodes now try and fallback to fetching host
credentials from the web proxy.
To establish a connection to an Auth Server, nodes first try and connect
directly, and if the connection fails, fallback to obtaining a
connection to the Auth Server through the reverse tunnel. If a
connection is established directly, node startup behavior has not
changed. If a node establishes a connection through the reverse tunnel,
it creates an AgentPool that attempts to dial back to the cluster and
establish a reverse tunnel.
When nodes heartbeat, they also heartbeat if they are connected directly
to the cluster or through a reverse tunnel. For nodes that are connected
through a reverse tunnel, the proxy subsystem now directs the reverse
tunnel server to establish a connection through the reverse tunnel
instead of directly.
When sending discovery requests, the domain field has been replaced with
tunnelID. The tunnelID field is either the cluster name (same as before)
for proxies, or {nodeUUID}.example.com for nodes.
This commit expands the usage of the caching layer
for auth server API:
* Introduces in-memory cache that is used to serve all
Auth server API requests. This is done to achieve scalability
on 10K+ node clusters, where each node fetches certificate authorities,
roles, users and join tokens. It is not possible to scale
DynamoDB backend or other backends on 10K reads per seconds
on a single shard or partition. The solution is to introduce
an in-memory cache of the backend state that is always used
for reads.
* In-memory cache has been expanded to support all resources
required by the auth server.
* Experimental `tctl top` command has been introduced to display
common single node metrics.
Replace SQLite Memory Backend with BTree
SQLite in memory backend was suffering from
high tail latencies under load (up to 8 seconds
in 99.9%-ile on load configurations).
This commit replaces the SQLite memory caching
backend with in-memory BTree backend that
brought down tail latencies to 2 seconds (99.9%-ile)
and brought overall performance improvement.
Added github.com/gravitational/teleport/lib/secret package to replace
github.com/mailgun/lemma to move from NaCl to AES-GCM. NaCl is still
supported for legacy clients.
Moved expiry field from spec to metadata for services.Users and updated
expiry check to prefer metadata and fallback to spec if not found. Added
test coverage.
This commit introduces several key changes to
Teleport backend and API infrastructure
in order to achieve scalability improvements
on 10K+ node deployments.
Events and plain keyspace
--------------------------
New backend interface supports events,
pagination and range queries
and moves away from buckets to
plain keyspace, what better aligns
with DynamoDB and Etcd featuring similar
interfaces.
All backend implementations are
exposing Events API, allowing
multiple subscribers to consume the same
event stream and avoid polling database.
Replacing BoltDB, Dir with SQLite
-------------------------------
BoltDB backend does not support
having two processes access the database at the
same time. This prevented Teleport
using BoltDB backend to be live reloaded.
SQLite supports reads/writes by multiple
processes and makes Dir backend obsolete
as SQLite is more efficient on larger collections,
supports transactions and can detect data
corruption.
Teleport automatically migrates data from
Bolt and Dir backends into SQLite.
GRPC API and protobuf resources
-------------------------------
GRPC API has been introduced for
the auth server. The auth server now serves both GRPC
and JSON-HTTP API on the same TLS socket and uses
the same client certificate authentication.
All future API methods should use GRPC and HTTP-JSON
API is considered obsolete.
In addition to that some resources like
Server and CertificateAuthority are now
generated from protobuf service specifications in
a way that is fully backward compatible with
original JSON spec and schema, so the same resource
can be encoded and decoded from JSON, YAML
and protobuf.
All models should be refactored
into new proto specification over time.
Streaming presence service
--------------------------
In order to cut bandwidth, nodes
are sending full updates only when changes
to labels or spec have occured, otherwise
new light-weight GRPC keep alive updates are sent
over to the presence service, reducing
bandwidth usage on multi-node deployments.
In addition to that nodes are no longer polling
auth server for certificate authority rotation
updates, instead they subscribe to event updates
to detect updates as soon as they happen.
This is a new API, so the errors are inevitable,
that's why polling is still done, but
on a way slower rate.
This commit improves performance of teleport with
hundreds of connected trusted clusters.
TLS handshake protocol expects server to send a
list of trusted certificate authorities to the client
and client must present certificate signed by those.
With Teleport current implementation, every remote cluster
client is signed by local certificate and is not cross
signed.
Auth server now expects clients to announce the
remote cluster they are connecting from using SNI.
Auth server will send only certificate authorities
of the cluster announced via SNI.
Alternative idea is to cross sign the certificate
of the client of the remote cluster. We will explore
this idea in the next releases.
This commit also removes unnecessary reads
from the database to check the remote server status
that slows down user interface and other clients.
This is done at the expense of proxies showing
servers as offline in case if this individual
proxy does not have the connection, although
it's a small UI price to pay for not reading
the database, as proxy will eventually
get the connection thanks to the discovery
protocol.
Andrew Lytvynov