* Use serverUID for scp target
Revert unneeded hostname change
Remove comment
Get name
* Change name to serverId
* Fix lint errors
* Add comment
* Fix comment and signature format
A new `api/client/proxy/Client` has been added to interact with
the SSH and gRPC servers that the Proxy serves on its SSH port.
The client will first try connecting to the gRPC server and if
that fails it will fall back to the SSH server.
Much of the SSH functionality mimics the existing behavior of the
`ProxyClient` in `lib/client`. This is the first part of phasing
out that client in favor of the new client. There will be a follow
up PR that migrates `lib/client` to make use of the new client instead.
Part of #19812
* Fix TestDiagnoseSSHConnection to be less strict on the exact unknown user error message, which may vary between OS's.
* Decouple/refactor SkipLocalAuth, UseKeyPrincipal, and static auth methods.
* Resolve comments.
* * Add tests for new methods
* Update proxy ssh principal test to catch regression
* Bump predicate error in switch case priority
Adds a new `GetCertAuthorities` unary RPC to
`teleport.v1.trust.TrustService` and deprecates the equivalent
HTTP endpoint. All associated interfaces have been updated to remove
the marshal options as well.
* * Add proper context handling to auth.AuthenticateUser.
* Move PublicKey field to AuthenticateUserRequest where it can be used
for actual authentication.
* Use a simple switch statement in /webapi/ssh/certs logic to switch
between password, otp, and eventually headless login.
* Add Headless flow to /webapi/ssh/certs login enpdoint.
* Add 3 minute callback timeout.
Sets `http.Server.IdleTimeout` and `http.Client.IdleConnTimeout`
on clients and servers which didn't have them set. A default of
360s was chosen to be on par with the default of an NLB without
being identical.
This was added as another safety measure to prevent leaking any
idle connections indefinitely as seen in #22757.
`apidefaults.DefaultDialTimeout` was also renamed to
`apidefaults.DefaultIOTimeout` to better reflect its usage.
The `http.Client.Transport` created in `newWebClient` wraps a
chain of `http.RoundTripper` over an underlying `http.Transport`.
Since we cannot guarantee that each `http.RoundTipper` has a
`CloseIdleConnections` method the usage of
`defer clt.CloseIdleConnections()` does not guarantee that the http
connections created during Find/Ping/etc are closed.
To prevent leaking connections implementations of `http.RoundTripper`
have added a `CloseIdleConnections` method added that forwards the
request on to the wrapped interface. The `otelhttp.Transport` does
not implement this method either so care has been taken to wrap it
in a `http.RoundTripper` which will call the root transport in
`enforceCloseIdleConnections`. An upstream issue has been filed
with otelhttp: https://github.com/open-telemetry/opentelemetry-go-contrib/issues/3543
to get the method added to their `Transport` implementation.
This wasn't noticed prior to upgrading to go1.20 becuase prior to
[this](https://github.com/golang/go/commit/4e7e7ae1406c70d9cc0809ec11105a55a60a0b70)
commit the `ReadHeaderTimeout` set on the Proxy web api http.Server
would cause the connection to appear idle and get terminated by the
server.
`TestWebClientClosesIdleConnections` was added to capture the leak
in connections as reported in #22757 and prevent any regressions.
The HTTP client created by `auth.NewClient` has an idle timeout of
30s and each instance periodically gets cert authorities in
`TeleportProcess.syncRotationStateCycle`. If there has been no
activity when the ticker expires then the request will result in a
brand new connection to Auth to be established. This can cause a
spikes in open file descriptors on the Proxy when there are a large
number of instances connected via a tunnel.
To prevent periodic dialing when retrieving cert authorities the
endpoint has been migrated to the new Trust gRPC service. For now
only the get endpoint has been migrated but the rest of the CRUD
operations for cert authorities should be migrated from the HTTP
api to the gRPC service as well.
* Move configuration from lib/service to lib/service/servicecfg
The new servicecfg package will hold only configuration for services.
This will allow other packages (like tctl and tsh) to depend on
servicecfg without pulling in all of lib/service (which has a number
of platform-specific details).
This is the first step towards being able to build tctl for Windows.
* Move PAM and BPF config into servicecfg
This breaks a compile-time dependency on BPF/PAM for tctl.
* Improve error messages for tsh login connectivity and ssh port connection errors
* spell fix
* lint fix
* lint fix
* suggestion applied
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Moved message to helper function for ssh proxy connection error
* format function
* Include error message on non-specified port login
* Give error message on attempted proxy hosts
* style coding
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* change to string construction
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* bytes buffer update
* lint fix
* Update message
* Update lib/client/api.go
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Update lib/client/api.go
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Update tool/tsh/tsh.go
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
---------
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
While loading a key from storage, Teleport client transverses the
storage directory to read the secrets and reads each file one by one.
This has huge impacts on `tsh kube credentials` because when calling
`kubectl` in parallel, the disk operations become overloaded and the
operation time increases exponentially.
This PR removes the transverse when reading credentials for Kubernetes
clusters and adds a quick read operation that reads the following files:
- `$TSH_HOME/current_profile`
- `$TSH_HOME/$profile.yaml`
- `$TSH_HOME/keys/$PROXY/$USER-kube/$TELEPORT_CLUSTER/$KUBE_CLUSTER-x509.pem`
- `$TSH_HOME/keys/$PROXY/$USER`
| - | P=1 | P=10 | P=40 | P=80 | P=120 |
| --- | --- | --- | --- | --- | --- |
| old direct | 9.90s user 3.14s system 64% cpu 20.284 total | 11.08s user 3.60s system 496% cpu 2.955 total | 12.46s user 4.36s system 820% cpu 2.050 total | 12.81s user 4.76s system 811% cpu 2.164 total | 13.02s user 5.07s system 861% cpu 2.100 total|
| old teleport | 18.45s user 7.80s system 51% cpu 51.085 total | 21.58s user 10.03s system 434% cpu 7.270 total | 22.33s user 10.52s system 178% cpu 18.402 total | 22.04s user 10.83s system 96% cpu 33.977 total | 22.57s user 11.45s system 55% cpu 1:01.34 total|
| new direct | 9.51s user 2.90s system 146% cpu 8.460 total | 13.75s user 4.95s system 718% cpu 2.603 total | 12.21s user 4.43s system 832% cpu 1.999 total | 11.72s user 4.50s system 839% cpu 1.933 total | 12.06s user 4.62s system 830% cpu 2.009 total |
| new teleport | 16.10s user 5.97s system 42% cpu 52.104 total | 21.62s user 9.01s system 381% cpu 8.023 total | 23.22s user 10.11s system 540% cpu 6.162 total | 21.64s user 9.81s system 603% cpu 5.208 total | 20.70s user 9.13s system 602% cpu 4.948 total |
---
| - | P=1 | P=10 | P=40 | P=80 | P=120 |
| --- | --- | --- | --- | --- | --- |
| old no tsh kube credential | 8.61s user 2.99s system 31% cpu 36.336 total | 9.57s user 3.39s system 283% cpu 4.565 total | 10.22s user 4.03s system 553% cpu 2.574 total | 9.32s user 3.82s system 559% cpu 2.351 total | 11.49s user 4.85s system 624% cpu 2.617 total
| new no tsh kube credential | 8.27s user 2.94s system 24% cpu 45.041 total | 10.85s user 4.31s system 195% cpu 7.767 total | 11.32s user 4.64s system 469% cpu 3.401 total | 10.83s user 4.51s system 553% cpu 2.773 total | 10.90s user 4.84s system 537% cpu 2.930 total |
---
| instances | old total | old avg | **new total** | **total avg** |
| --- | --- | --- | --- | --- |
| 1 | 0.145 | 0.145 | 0.074 | 0.074 |
| 10 | 0.430 | 0.043 | 0.146 | 0.0146 |
| 20 | 0.740 | 0.037 | 0.284 | 0.0142 |
| 30 | 1.108 | 0.037 | 0.417 | 0.0139 |
| 40 | 1.483 | 0.037 | 0.580 | 0.0145 |
| 60 | 13.604 | 0.228 | 0.825 | 0.0137 |
| 80 | 30.236 | 0.378 | 1.143 | 0.0142 |
| 106 | 46.681 | 0.440 | 1.413 | 0.0133 |
Fixes#19313
Co-authored-by: Brian Joerger <bjoerger@goteleport.com>
Fixes#19293
The error message "missing parameter hostname" wasn't very clear what hostname
was missing. Fail faster further up the stack and print a better error message
explaining what is happening.
This applies some checks aroung argument parsing while also adding some
belt-and-suspenders validation into `NewClientStoreFromIdentityFile`
* Change formatting for "/version" endpoint
This PR changes the formatting for "/version" endpoint since Kubernetes
clients do not expect a JSON response on that endpoint.
This PR returns the error message directly into the body without any
formatting so that `kubectl` is able to present the desired error.
```
kubectl version --short
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
Client Version: v1.25.4
Kustomize Version: v4.5.7
Error from server (InternalError): an error on the server ("this user cannot request kubernetes access, has no assigned groups or users") has prevented the request from succeeding
```
Fixes#20900
* change message wording
* add validation when generating kube creds
* fix typo
This commit adds a user message ("Detected security key tap") that is
printed at the earliest opportunity after `tsh` succesfully detects a
security key tap.
This solves a problem users have when there is significant latency in
their connection to their teleport proxy, where there may be zero
visible feedback to the user for upwards of 8 seconds after they tap
their security key.
This often causes users to think that the tap was not detected or
failed, and they attempt to tap again.
This can cause the user to accidentally paste a "yubisneeze" into their
terminal, which is not great from a security perspective.
As an API choice here I added a `TouchAcknowledger` returned by they
`PromptTouch` of the `LoginPrompt` interface.
It may seem like a bit of a weird API, but I did it this way so that all
callers of `PromptTouch` are forced to explicitly acknowledge the touch
or explicitly ignore that return value, they cannot simply forget to ack.
This applies to WebAuthn and Passwordless logins from `tsh`, no changes
are made to Connect.
Previously doing an `ls -R` would list all resources per profile in
parallel. This had little benefit when one profile needed to list
multiple clusters as those were still done serially. To mitigate this
things are broken up into two separate steps:
1) Connect to all clusters the user has credentials for in parallel
2) List resources for each cluster in parallel
This ensures that by the time a client is established for all available
clusters that resource listing can happen in parallel regardless of
profile. All operations are still restricted to an upper bound, however
those limits were also bumped to help performance.
* Adding in a new SAML IdP certificate authority.
A new SAML IdP certificate authority has been added. It has been added to
initialization, the cache, and the certificate can be rotated.
* Update lib/services/authority.go
Co-authored-by: Sam Freiberg <sfreiberg@goteleport.com>
* Update api/types/trust.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Only validate private key if it's present.
---------
Co-authored-by: Sam Freiberg <sfreiberg@goteleport.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Our current SFTP implementation uses a single channel to transfer data. Connections with a high latency cause very slow transfer. github.com/pkg/sftp allows transferring data using concurrent streams, which seems to solve the problem. In order to utilize the feature, the remote file must implement the WriteTo() or ReadFrom() and Stat() methods. I've moved the progress bar to relay the data from a local file, as we can still call Read() or Write() on it to update the progress.
This PR doesn't address the high CPU usage by github.com/pkg/sftp on low latency links.
Closes#20579
* Cleanup
* Fix fileTransferProgress
* Simplify interfaces
* Move helper to utils package
Add comments
* Added comments
* Change canceler implementation to return correct values from Read/Write
* Fix infinite recursive call in cancelWriter.
* Save private key policy in profile.
* Fix tsh status/logout with yubikey unplugged; fix cluster alert on tsh status with hardware_key_touch.
* Set private key policy from ping response.
* Skip cluster alerts on tsh status.
* Set SkipLocalAuth on identity file use; Simplify logic for SkipLocalAuth.
* Fix TrustedCertsFromCACerts to work with known hosts without proxy host set in host name; Fix HostKeyCallback for keys.
* Add regression test where known hosts are not written with proxy host.
* Fix client key default host key callback.
* Return empty list, not not found error, when there are no trusted certs.
* Restore option to filter trusted host keys in host key callback.
* Update lib/client/identityfile/identity_test.go
Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
---------
Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
* Expose device verification methods in lib/devicetrust/authz
* Set device fields in auth.Context.GetAccessState
* Enable role-based device verification for SSH
* Enable role-based device verification for DBs
* Add TODO for better SSH error messages
* Do not override device error on lib/kube/proxy
* Run gci
Since #16964, `tsh login` defaults to passwordless if it finds suitable platform
credentials - it is safer and more convenient than other authentication methods.
Unfortunately, it is a bit too eager in doing that.
This change adds the following prerequisites for the switch, in addition to
existing conditions:
1. `--auth` must not be specified
2. `--mfa-mode` must be either `auto` or `platform`
(1) avoids defaulting when `--auth=local` is present, as well as for other
custom/future values. (If `--auth=passwordless` is passed, we'll do passwordless
anyway.)
(2) includes both an explicit platform attachment test (which is a bit more
conservative in face of changes), as well as checking for `--mfa-mode=otp`.
#20429 and #20322
User certs for desktop access are only valid for a few minutes
and are never written to disk. This can make it difficult to
troubleshoot cert validity.
This commit adds support for generating Windows user certificates
which can be exported to the Windows environment for validation.
Note: at this time, we only write the certificate and not the
corresponding private key, making it impossible to use the generated
cert for any real purpose.
* WIP adding OpenSSH CA
* fix creating OpenSSH CA for tests
* fix a test name and tctl usage flag
* fix 'tctl status' not displaying openssh CA sometimes
* add OpenSSH CA to TestInit_bootstrap, remove TLS key pair
* add OpenSSH CA to a few more places
* make help text of 'tctl auth rotate' more helpful and accurate
* fix web cert export test
* tbot doesn't need to know about OpenSSH CA
* address feedback
* fix comment formatting
This commit optimizes `PostJSONWithFallback` by not requiring it to
parse the endpoint (to determine if the host is loopback) if HTTP
fallback is not allowed in the first place.
Michael