* feat: Extend `AWSRole` struct w/ `RequiresRequest`
- Differentiate between already-granted vs. requestable AWSRoles when
including requestable resources in `clusterUnifiedResourcesGet`
req
* feat: Implement Resource Constraints support for Proxy, Auth
* fixup: Fix tests
* test: Add tests for resourceaccessid-related helpers
* fixup: Address code review feedback
* test: Update identity/events tests for new `ResourceAccessID` fields
* fixup: Expand/improve godocs/comments for sentinel ResourceID handling
* fixup: Address code review feedback
* feat: Conv ResourceID->ResourceAccessID at cert decoding
- Convert ResourceIDs present on certs to ResourceAccessIDs at decoding
time, rather than at each callsite. Update tests/usages of tlsca/sshca
Identities to reflect this.
In TestKube/Join, we previously used os.Pipe[1] to create stdin for the
test. However, os.File instances are implicitly closed when garbage
collected via a runtime finalizer inserted in newFile[2].
Under CPU-constrained conditions, the garbage collector can run
mid-test and detect that participantStdinW is no longer referenced.
This triggers the finalizer, which closes the writer file descriptor,
causing the reader to receive io.EOF.
Since Kubernetes terminates exec sessions when stdin closes, the EOF
can arrive before the client receives the server payload, causing the
test to fail intermittently.
Replacing os.Pipe with io.Pipe resolves this issue, as io.Pipe does
not use file descriptors and is not subject to finalizer-based cleanup.
[1] https://github.com/golang/go/blob/1b291b70dff51732415da5b68debe323704d8e8d/src/os/pipe_unix.go#L27
[2] https://github.com/golang/go/blob/1b291b70dff51732415da5b68debe323704d8e8d/src/os/file_unix.go#L224
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
* [cfg] expose auth reconnect retry parameters in configuration
The new YAML config is as follows:
```
reconnect_backoff:
max_period: "90s"
min_period: "9s" # When unset max_period / 10
step: "18s" # When unset max_period / 5
```
Note that currently this only exposes the existing linear backoff, but
the naming here is generic enough that we could consider other policies if needed.
Changelog: Added a new global configuration section `reconnect_backoff` allowing users to configure the backoff behaviour for instances attempting auth connections.
* better naming
* fix ratios for defaults
* fix default config test
* clarify instance type
* Initialize default Username/HostLogin only in tsh
* Move `Username()` from `api.go` to `tsh.go`
* Remove wrong `Profile.SiteName` default
* Remove resetting `SiteName`
Not sure why it was needed. Perhaps to clear the default that we just removed? But even if add the default back and remove this fix, everything works.
* Gracefully handle missing SSH/TLS certs
* Remove unused `TeleportClient.LoadKeyForClusterWithReissue`
* Revert "Move `Username()` from `api.go` to `tsh.go`"
This reverts commit f7ff0ff6d1.
* Revert "Initialize default Username/HostLogin only in tsh"
This reverts commit ed38babf5e.
* When any of SSH/TLS cert is missing, return partial profile
* Only log non-nil errors
* Revert "Remove wrong `Profile.SiteName` default"
* Revert "Remove resetting `SiteName`"
This reverts commit f54ab3f22a.
* Set `SiteName` when adding cluster
* Improve comments
* Add test
* Fix test
* Add myself to TODO
* Add test for logging out with missing SSH cert
* Lint
The test was reliant on entering \a as input to a
session which signaled to sleep for 1s before writing
the data. This made the test undeterministic because
the 1s delays might not have been long enough for the
second party to have joined and start consuming the
session data.
This changes the test to validate that the appropriate
number of parties are in the session before proceeding
to the next step of the test. All \a have been removed
from terminal input as we now know for certain that the
session contains the appropriate parties before any
input is entered.
Closes https://github.com/gravitational/teleport/issues/25982.
Closes https://github.com/gravitational/teleport/issues/13699.
Several integration tests make use of the function to
know that a session was created, a SessionTracker was
created in the backend, and to retrieve the session id.
The function polled the backend for session trackers
until the context deadline set in individual tests
was exceeded. In an effort to reduce flakiness and
make the API easier for callers the helper has been
rewritten. It now leverages a require.Eventually
internally, has a hardcoded limit of 30s instead of
allowing each test to define their own, and validates
the number of expected participants in a session.
Closes https://github.com/gravitational/teleport/issues/17224.
Closes https://github.com/gravitational/teleport/issues/36959.
The test was reliant on all the output of a session being
replicated to all parties before a 5s timer closed the session.
To make the test more resiliant it was rewritten such that
it validates the expected output from one of the parties output,
then exits the session, then when all parties have been closed,
it examines the entire output of the other parties to validate
their output.
Fixes https://github.com/gravitational/teleport/issues/55044
The tests establish trust between two clusters, connect via SSH
to a host in the leaf, tear down trust, reestablish trust, and
then attempt another SSH connection. The flakiness stemmed from
tearing down and and reestablishing trust too quickly without
first properly verifying that the leaf cluster disappeared from
the root before reestablishing trust. While the test did make
use of `helpers.WaitForClusters`, it is satisfied if tunnels
for clusters were healthy within the last 10s. But this could
be true of a stale cluster that was not yet cleaned up. To
remedy this the test was updated to first wait for the reverse
tunnel for the leaf cluster to be completely gone before
reestablishing trust and proceeding.
Closes https://github.com/gravitational/teleport/issues/60836.
Closes https://github.com/gravitational/teleport/issues/48823.
The changes here are meant to reduce resources consumed by
integration tests and eliminate the footgun of connecting to
a host before its been populated in the appropriate caches.
`(integrationTestSuite) defaultServiceConfig` now disables more services
by default. Host user creation, the Web UI and Web Service, and the
Database Proxy are all now turned off by default. The tests that require
them can opt into turning them on.
`(TeleInstance) Start` now blocks until the processes SSH server is present
in caches before returning. Most tests were already calling this manually
after Start, however, not all of them were. This is frequently forgotten
in new tests and causes flakes if client connections are attempted as
soon as Start returns. There are still cases where calling WaitForNodeCount
directly applies - tests that spin up additional nodes or tests that create
nodes in a leaf cluster.
The datadir in this test consists of TempDir
and typically suffix which exceeds the
BSD socket limits causing the test to fail on MacOS.
This commit disables the debug service in the failing tests,
for testing the debug service specifically it should be
enabled in the test itself.
This test previously passed in CI, relying
on slower machines and fails locally on both
Mac and in buildbox on macbook pro m3 with:
```
cluster site-A is offline
```
Adding the wait resolves the issue.
* Include expiration time in `LoggedInUser`
This will allow the profile watcher to detect when the user relogged.
* Display expiration time in UI
* Add `ClearStaleClusterClients` RPC
* Implement `ClearStaleClusterClients`
* Clear stale clients when profile changes
* Improve session expiration component
* Move refresh button back to top
* `ClearCachedStaleClientsForRoot` -> `ClearStaleCachedClientsForRoot`
* `unchanged` -> `stale`
* Make "closing stale clients" a subtest
* Add `clientcache` test
* Remove `getProfile` error wrapping
* Improve comment
* Convert story to controls
* Generalize PrepareToReceiveSessionID.
* Initialize session ID in the connection context and update it from node current-session-id request.
* Add session-id-query-v2@goteleport.com request and ensure new session ID is correctly set in proxy recording mode during the channel request.
* Replace PrepareToReceiveSessionID with simpler in-place logic.
* Don't emit session events or tracker when proxy forwarding to a Teleport Node.
* Fix missing session tracker for outdated Teleport Node.
* Remove extra major version grace period.
* Update integration test.
* Cleanup current session ID handling and fix failing tests.
* Fix tests.
* Address comments.
* Restructure currentSessionID handling.
* Set newSessionID in test server context.
* Fix integration test.
* Fix AuditOn integration test.
* Address comment on channel close.
* Track session on forwarding node.
* Fix web shutdown.
* Fix nil pointer dereference in test.
* Fix test flake.
* Fix nil pointer in test.
* Fix test flake.
* Update lib/srv/ctx.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Forwarding Node accepts client connection after receiving preparing session ID from node. This way, the forwarder can reject client connections if there is an issue preparing the session ID (impossible join sessions).
* Remove check for session.data event which may not be emitted in time for the test.
* Address comments.
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Unify behavior of the uploaders
- Changes the behavior of uploaders to allow uploading pending
summaries. It also makes all uploaders consistently fail to overwrite
existing recordings and final summaries.
- Upgrades the gofakes3 library to support the If-None-Match headers.
- Copies the MemBuffer utility from the enterprise code.
* Fix a deadlock in gcsessions.Handler
The deadlock occurred when the upload passed a certain number of parts,
in which case the errors channel buffer got saturated and the object
deletion goroutines started piling up, ultimately leading to locking the
main loop that iterated through the object slice when the semaphore
channel got its buffer filled up too.
This deadlock prevented running the GCS uploader integration tests.
* review
* Another review round
- Move the pending session downloading to the caller
- Add a missing MemBuffer test
- Tweaks to the memory uploader (required by the Enterprise change)
* Build fix
* review
* Fix lint and integration tests
* Delete the pending summaries when uploading final, use versioning on s3
* Some comments
* license
* Reduce TestKube/Exec flakiness
The test was not explicitly providing the target Kubernetes
cluster. As a result of this the test harness would try to
infer the Kubernetes cluster by examining the inventory. This
could potentially happen fast enough that no Kubernetes clusters
exist in the inventory yet. As a result the empty field in the
identity could lead to the `Kubernetes cluster "" not found`
error reported in https://github.com/gravitational/teleport/issues/59462.
Closes https://github.com/gravitational/teleport/issues/59462.
* Extract alpine-webserver:v1 building from GHA
The logic to construct an image used by Kubernetes integration
tests has been moved to standalone make targets in the fixtures
folder to facilitate easier local testing. The GHA workflows
have been updated to call the new targets directly.
To reduce the number of connections established to MongoDB servers,
Teleport Database Service instances will now cache connection pools to
MongoDB servers and reuse the connections when handling connections from
the same Teleport client.
The ability to alter the auth server clock existed
to allow tests to inject a fake clock. While this sounds
great it introduces opportunity for data races if other
components are already consuming the existing clock. For
example:
```
==================
WARNING: DATA RACE
Write at 0x00c003014720 by goroutine 168:
github.com/gravitational/teleport/lib/auth.(*Server).SetClock()
/__w/teleport.e/teleport.e/lib/auth/auth.go:2308 +0x88
github.com/gravitational/teleport/lib/client_test.newStandaloneTeleport()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:629 +0x11ad
github.com/gravitational/teleport/lib/client_test.newStandaloneTeleport()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:621 +0x1092
github.com/gravitational/teleport/lib/client_test.TestTeleportClient_Login_local.func21()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:290 +0x14e
testing.tRunner()
/opt/go/src/testing/testing.go:1934 +0x21c
testing.(*T).Run.gowrap1()
/opt/go/src/testing/testing.go:1997 +0x44
Previous read at 0x00c003014720 by goroutine 9018:
github.com/gravitational/teleport/lib/auth.(*Server).GenerateHostCerts()
/__w/teleport.e/teleport.e/lib/auth/auth.go:5127 +0x59d
github.com/gravitational/teleport/lib/auth.GenerateIdentity()
/__w/teleport.e/teleport.e/lib/auth/init.go:1606 +0x450
github.com/gravitational/teleport/lib/service.(*TeleportProcess).reRegister()
/__w/teleport.e/teleport.e/lib/service/connect.go:450 +0x25a
github.com/gravitational/teleport/lib/service.(*TeleportProcess).rotate()
/__w/teleport.e/teleport.e/lib/service/connect.go:1160 +0xaa7
github.com/gravitational/teleport/lib/service.(*TeleportProcess).syncServiceRotationState()
/__w/teleport.e/teleport.e/lib/service/connect.go:1017 +0x20b
github.com/gravitational/teleport/lib/service.(*TeleportProcess).syncRotationState()
/__w/teleport.e/teleport.e/lib/service/connect.go:996 +0x1c9
github.com/gravitational/teleport/lib/service.(*TeleportProcess).syncRotationStateAndBroadcast()
/__w/teleport.e/teleport.e/lib/service/connect.go:962 +0x4e
github.com/gravitational/teleport/lib/service.(*TeleportProcess).syncRotationStateCycle()
/__w/teleport.e/teleport.e/lib/service/connect.go:906 +0xcb
github.com/gravitational/teleport/lib/service.(*TeleportProcess).periodicSyncRotationState()
/__w/teleport.e/teleport.e/lib/service/connect.go:870 +0x5f3
github.com/gravitational/teleport/lib/service.(*TeleportProcess).periodicSyncRotationState-fm()
<autogenerated>:1 +0x33
github.com/gravitational/teleport/lib/service.(*LocalService).Serve()
/__w/teleport.e/teleport.e/lib/service/supervisor.go:605 +0x35
github.com/gravitational/teleport/lib/service.(*LocalSupervisor).serve.func1()
/__w/teleport.e/teleport.e/lib/service/supervisor.go:328 +0x4a2
Goroutine 168 (running) created at:
testing.(*T).Run()
/opt/go/src/testing/testing.go:1997 +0x9d2
github.com/gravitational/teleport/lib/client_test.TestTeleportClient_Login_local()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:285 +0x691
testing.tRunner()
/opt/go/src/testing/testing.go:1934 +0x21c
testing.(*T).Run.gowrap1()
/opt/go/src/testing/testing.go:1997 +0x44
Goroutine 9018 (running) created at:
github.com/gravitational/teleport/lib/service.(*LocalSupervisor).serve()
/__w/teleport.e/teleport.e/lib/service/supervisor.go:317 +0x10a
github.com/gravitational/teleport/lib/service.(*LocalSupervisor).Start()
/__w/teleport.e/teleport.e/lib/service/supervisor.go:360 +0x2f6
github.com/gravitational/teleport/lib/client_test.startAndWait()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:804 +0x91
github.com/gravitational/teleport/lib/client_test.startAndWait()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:802 +0x45
github.com/gravitational/teleport/lib/client_test.newStandaloneTeleport()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:621 +0x1092
github.com/gravitational/teleport/lib/client_test.TestTeleportClient_Login_local.func21()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:290 +0x14e
testing.tRunner()
/opt/go/src/testing/testing.go:1934 +0x21c
testing.(*T).Run.gowrap1()
/opt/go/src/testing/testing.go:1997 +0x44
==================
```
The setter is also redundant - Auth already exposes a way
to be constructed with a custom clock. In order to get rid
of the races once and for all, the clock setter was removed and
all tests have been updated to plumb the clock through on
construction.
* [api] Add ListSemaphores
This commits deperecates GetSemaphores, which should be replaced by a paginated counterpart:
ListSemaphores
Note that GetSemaphores is also used as a point getter when both Kind and Name are
given. This commit only focuses on pagination and as such takes the existing pattern
and adds paging support without adding a point getter for consistency with other RPCs.
Ranging is not implemented on client side due to the compound key used.
Towards: gravitational/teleport.e/issues/6759
Changelog: Add paginated API ListSemaphores, deprecate GetSemaphores
* nil safe getters for filter, simplify match
* only use new list for non point gets
This fixes a topology monitoring connection leak.
The leak was triggered by an error during server selection or the
initial auth handshake with the MongoDB server.
* Remove `ClusterRemove` RPC, make logging out idempotent
* Move calling `removeKubeConfig` and `maybeRemoveAppUpdatesManagingCluster` to main process
The main process should not depend on the renderer to clean up its own resources.
* Remove cleaning up kube dir
* Lint
* Add OS and ARCH info for client tools managed updates
* Optimized initial tsh/tctl installation in tests to perform it once and re-use
* Add reading binary headers in order to identify OS and ARCH if it is not defined in configuration file.
* Disable race test for integration tests of client tools managed updates and run them on windows platform
* Fix tests where tool name for Windows have extension
* Fix test with break event of windows
* Revert the disabling of the -race flag for MU tests, as recompilation takes more time than running the tests
Fixes https://github.com/gravitational/teleport/issues/60055 and
https://github.com/gravitational/teleport/issues/53267.
Errors emitted when access is denied or hosts are missing from
the inventory have been updated to be clearer to end users.
The current error when connecting to a host that does not exist
```bash
$ tsh ssh tim@fake-host
ERROR: failed connecting to host fake-host:0: failed to receive cluster details response
failed to dial target host
direct dialing to nodes not found in inventory is not supported
```
has been updated to remove ancillary details that have zero meaning to anyone
other than a Teleport developer and better clarify the host does not exist.
```
$ tsh ssh tim@fake-host
ERROR: failed connecting to host fake-host: target host fake-host is offline or does not exist
```
The access denied error
```
$ tsh ssh tim@server01 uptime
ERROR: access denied to tim connecting to server01:0
```
has had a relatively minor and possibly unnoticed
change to omit the port from the dial target.
```
$ tsh ssh tim@server01 uptime
ERROR: access denied to tim connecting to server01
```
fix: use net.SplitHostPort instead of strings.Cut
* fix: prevent `crypto/ssh` to block on dialing
`crypto/ssh` dialing doesn't support timeouts or context cancellation
propagation by default. Although `ssh.ClientConfig` has a field
`Timeout`, this field is only used when establishing a new TCP
connection but it's never used for ssh handshake.
This means that if the target server never replies and no intermediary
closes the connection, the connecton will be kept alive forever causing
Teleport dialer to deadlock and never recover.
Although teleport `ssh` package has `NewClientConnWithDeadline`, not all
every Teleport implementation of `net.Conn` supports read deadlines,
which means we can have cases where the system also deadlocks when using
`NewClientConnWithDeadline` and a timeout.
This PR tries to solve it by closing the underlying connection when the
minimum time between the context's deadline, the timeout and if none set
30s.
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
* Apply suggestion from @espadolini
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* fix code
* rename methods to correctly describe the behavior
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
* handle code review comment
---------
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
rosstimothy