* Vendor gravitational/trace/trail in api
Pulling in the trail package directly in api will allow the trace
module to shed the grpc-go dependency. This needs to land prior
to https://github.com/gravitational/trace/pull/112 being included
in a new version of trace.
There should be no noticable change in the api depdency tree since
it already depends on grpc-go. Some additional items from the
trace/internal package were also vendored within trail as needed.
Additionally, some of the public api of trail that was not being
consumed has been made private.
* fix: appease linters
* Cache yubikey objects.
* Cache PIV connections to share across the program execution.
* Do not release the connection until `sign` returns
* Do not ignore errors
* Perform a "warm up" call to YubiKey
* Fix tests
* Use a specific interface to check if the key can be "warmed up"
* Allow abandoning `signer.Sign` call when context is canceled
* Make sure that the cached key is valid for the given private key policy
The reason for adding this check was failing `invalid key policies` test.
* Make `hardwareKeyWarmer` private
* Force callers to release connection
* Improve comments
* Fix lint
* Improve `connect` comment
* Fix race condition
* Simplify `release` logic
* Trigger license/cla
---------
Co-authored-by: joerger <bjoerger@goteleport.com>
* Skeleton out SSHProxyService
* Skeleton out actual service implementation
* Simple unit tests for yaml/validation
* Add main config file unit test
* Crudely copy in the connection handle code
* Copy in basic setup code
* Add rudimentary "dial cycler"
* Tidy up handling of listener closure
* Add basic prometheus metrics
* Add ssh proxy command connect
* Fix crashes
* Make session resumption enabled by default
* Use `utils.ProxyConn`
* last few changes before i convert to socks5
* Rename socket
* Add support for loading proxy templates
* Tidy up logging
* rename to multiplexer
* Fix up a few bits
* Generate artifacts on initial start
* Support specifying command and subcommand
* Add identity generation and renewal to ssh multiplexer
* Tidy up code post-identity generation
* Tidy up logging for connection multiplexing/proxying
* Americanize spellings
* Info( -> InfoContext(
* stash
* Replace low-level cycler with higher-level but broken cycler
* Remove reference to grpcClientConnInterfaceCloser
* minor tweaks and fixes to the cycler
* Fix lone tab in the ssh_config template
* Better metrics
* Fix ineff assign
* Simplify config for custom muxcommand
* Start wrtiign test
* Fix broken fdpass
* Potentially flaky but working test
* Switch to NULL delimited
* Tidy up tests
* Remove commented out code
* Drain buffer after the conn resumption has been enabled
* Change socket name to `v1.sock`
* Switch away from JSON
* Switch to ProxyCommand style config for overrides
* Switch to `filepath`
* Update lib/tbot/service_ssh_multiplexer.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Update lib/tbot/ssh/ssh.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Update lib/tbot/service_ssh_multiplexer.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Absolute paths
* Remove unused dep
* Update lib/tbot/service_ssh_multiplexer.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Update lib/tbot/ssh/ssh.go
Co-authored-by: Tim Buckley <tim@goteleport.com>
* Drain buffer as part of proxying goroutine
* Fix missing godoc
* Missing godoc
* Add test for connection cycler
* Update lib/tbot/service_ssh_multiplexer.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Simpler buffer draining
* Fix race in test
* More generous time outs
---------
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
Co-authored-by: Tim Buckley <tim@goteleport.com>
The new subcommand is meant to be a replacement for tbot proxy ssh
which does the ssh proxying entirely within tbot to reduce resource
consumption. The old mechanism spawned a tsh process to perform the
proxying which leads to increased CPU and memory consumption - which
can be quite problematic when used as an OpenSSH ProxyCommand for
an ansible playbook.
The new command also deviates in behavior from tsh proxy ssh in that,
for the moment, it requires users to explicitly dictate whether
tls routing, alpn connection upgrades instead of guessing by making a
call to webapi/ping. This again turns out to be quite resource
intensive when tbot is used as an OpenSSH ProxyCommand for an ansible
playbook.
In the future tbot can look this information up out of band and store
it in the proper location so that tbot ssh-proxy-command can consume
it directly without users needing to provide it. The behavior as is
provides a path toward the future while also allowing users to gain
the performance improvements now.
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
Updates the proxy.Client to allow loading specific tls.Config for
individual clusters. This prevents issues when trying to access
leaf resources via the root cluster if WithAllCAs is not set.
Takes inspiration from https://github.com/gravitational/teleport.e/pull/1944
which addressed similar EOF error showing up when driving streams
in tests.
From https://pkg.go.dev/google.golang.org/grpc#ClientStream.SendMsg:
> // SendMsg is generally called by generated code. On error, SendMsg aborts
> // the stream. If the error was generated by the client, the status is
> // returned directly; otherwise, io.EOF is returned and the status of
> // the stream may be discovered using RecvMsg.
Prefer the stdlib version of the experimental package.
Includes a depguard update to prevent future regressions,
as well as an e-ref update to appease depguard.
Generated with:
$ gofmt -w -r '"golang.org/x/exp/slices" -> "slices"' . && \
make fix-imports
The new `tsh latency` command allows visualizing latency to
resources in real time. Currently, supported is limited to SSH
instances, but may be extended to anything that implements a
`latency.Pinger` to support other protocols in the future.
* Remove `Dialer(client.Config)` from Credential interface
* Re-add support for address-less configuration
* Only use profile address if none are explicitly provided
* Try all methods with credential provided address
* Fall back to address from credential
* Remove log message
* Adjust log message
* Spell explicitly correctly
* Formalize CredentialsWithDefaultAddrs interface
* GetDefaultAddrs -> DefaultAddrs
* Move gRPC error intercetpors to api/utils/grpc/interceptors.
* Use error interceptors in api client and mock server.
* Apply suggestions from CR.
* Unwrap FromGRPC errors in middleware.
* Use gRPC auth service in tests instead of external example service.
* It's gRPC!!!
* Fix unit test.
* Add error interceptor to proxy client.
* Fix merge conflict.
In preparation for v14 we can remove support for SSH connections
to the Proxy from the `api/proxy.Client` since all supported instances
should all be serving the TransportService.
* Use web address when appropriate for a jump hosts
Determines whether the jump host provided via `tsh ssh -J` is belongs
to the Proxy SSH or Web server to ensure when using jump hosts that
connections are established directly on the target cluster.
Closes#25178
* Modify tsh tests to capture issues with jump hosts
Alters the root and leaf cluster and node names used by tsh tests
so that the root cluster is named `root` instead of `localhost` and
sets a unique `NodeName` for each cluster instead of reusing
`localnode` for both. This was masking problems in jump hosts tests
by connecting to the node in the root cluster instead of the leaf
cluster.
Some additional changes to tsh tests were made as a result of
changing the cluster and node names.
* fix proxy client tests
* update TestList to login once
* ignore TestList in flaky test detector
* ALPN connect test improvements
* fix typos
* remove extra period
* simplify error check
* moving things over
* tsh dials
* reverse tunnel
* fix auth connect
* move ping
* add ssh support
* add HTTP client support
* Move ALPN dialer, ALPN conn upgrade, Ping conn to api
* beatify
* add test
* beautify round 2
* fix timeout
* Implement alpn-ping upgrade for reversetunnel and ssh
* clean up
* fix proxy test
* minor refactor
* remove WebProxyAddr
* require IsALPNConnUpgradeRequiredFunc
* add tlsRoutingWithConnUpgradeConnect
* fix lint
* simplify
* remove debug log and change unknown upgrade type to 404
* Force new proxy client to use web proxy when TLS routing is enabled
* Convert tsh ssh to use the proxy transport service instead of ssh
In an effort to reduce latency establishing sessions `tsh ssh` is
migrating away from connecting to the Proxy via SSH in favor of
using gRPC. The SSH handshakes with the Proxy increase latency in
situations where the distance between geolocations of the client
and Proxy are large. TLS handshakes used by the gRPC service have
proven to reduce latency by ~20% in the same scenario.
A new `lib/client.ClusterClient` has been introduced that should
be used instead of `lib/client.ProxyClient` to connect to a Teleport
cluster. Most of the functionality within the `ClusterClient` was
a direct copy from the `ProxyClient`.
The `lib/client.TeleportClient` now has a `ConnectToCluster` method
which will connect to both the Proxy and Auth service via the
`api/client.ProxyClient` which first attempts to use gRPC and reverts
back to SSH to preserve backwards compatability. The `ClusterClient`
should be passed around and reused instead of following the
established pattern of `tc.ConnectToProxy` followed by a
`proxy.ConnectToCluster` to get an `auth.ClientI`.
Additionally some of the `agentless` package was refactored to reduce
dependencies and allow it to work with connections to the Proxy that
originated via gRPC instead of SSH.
Changes to the integration tests are mostly to accomodate IP Pinning
and ensure that it works for both connections established via SSH
and gPRC.
This is the final PR needed to complete #19812.
* fix typos and unify span attributes
* pass node name to ConnectToNode
* simplify jump host resetting
Instead of relying on users to provide the cluster name, the client
now determines the cluster name by inspecting the certificate
presented by the Proxy during the TLS or SSH handshake. This is
required when connecting to a Proxy via a jump host since the
name of the cluster may not match the currently logged in cluster.
This is achieved by leveraging a custom `credentials.TransportCredentials`
when connecting via gRPC and a custom `ssh.HostKeyCallback` when
connecting SSH.
The interface smuggling within `stream.ReadWriter.Close` now checks
for `io.Closer` instead of `grpc.ClientStream` to allow clients
that need to perform additoinal closing logic besides sending a
`CloseSend` message to do so. All existing client implementations
of `stream.Source` now implement `io.Closer` so they are still
cleaned up properly.
The multiplexed ssh streams are now tied together to prevent attempts
to send a message at the same time. In addition, the client ssh
streams have been updated to only send a single `CloseSend` on which
ever protocol is terminated first.
A new `api/client/proxy/Client` has been added to interact with
the SSH and gRPC servers that the Proxy serves on its SSH port.
The client will first try connecting to the gRPC server and if
that fails it will fall back to the SSH server.
Much of the SSH functionality mimics the existing behavior of the
`ProxyClient` in `lib/client`. This is the first part of phasing
out that client in favor of the new client. There will be a follow
up PR that migrates `lib/client` to make use of the new client instead.
Part of #19812
The `http.Client.Transport` created in `newWebClient` wraps a
chain of `http.RoundTripper` over an underlying `http.Transport`.
Since we cannot guarantee that each `http.RoundTipper` has a
`CloseIdleConnections` method the usage of
`defer clt.CloseIdleConnections()` does not guarantee that the http
connections created during Find/Ping/etc are closed.
To prevent leaking connections implementations of `http.RoundTripper`
have added a `CloseIdleConnections` method added that forwards the
request on to the wrapped interface. The `otelhttp.Transport` does
not implement this method either so care has been taken to wrap it
in a `http.RoundTripper` which will call the root transport in
`enforceCloseIdleConnections`. An upstream issue has been filed
with otelhttp: https://github.com/open-telemetry/opentelemetry-go-contrib/issues/3543
to get the method added to their `Transport` implementation.
This wasn't noticed prior to upgrading to go1.20 becuase prior to
[this](https://github.com/golang/go/commit/4e7e7ae1406c70d9cc0809ec11105a55a60a0b70)
commit the `ReadHeaderTimeout` set on the Proxy web api http.Server
would cause the connection to appear idle and get terminated by the
server.
`TestWebClientClosesIdleConnections` was added to capture the leak
in connections as reported in #22757 and prevent any regressions.
Changes the name to better align with the naming used by the rest
of the feature and to prevent ambiguity as the term proxy is quite
overloaded. This has not landed in a release yet so there are no
backward compatability concerns as nothing serves or consumes this
api yet.
Provides a client that can be used to connect to and interact with
the transport service in `lib/srv/transport`. The client abstracts
the fact that a gRPC stream is being used for the `net.Conn` created
by `DialCluster` and `DialHost`.
This also moves `lib/utils/grpc/stream` to `api/utils/grpc/stream`
so that the client and server can make use of the same stream
abstractions.
Part of #19812
Before this commit, the `tsh` HTTP requests that had the extra headers
were those that did not use `roundtrip`.
This commit leverages `http.RoundTripper.RoundTrip` to ensure that all
requests have the the extra headers.
* Enable nolintlint linter
* Fix nolint comments in the api package
* Fix RDP client comment
* Address review comment
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Allow unused for nolintlint linter
* Remove redundant casting
* Add comment on why allowed unused is enabled
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
This change updates NO_PROXY handling to allow blocking specific host:port combinations, rather than just the host. It also adds a special case for downgrading requests to plain HTTP when --insecure is true and the request goes through a plain HTTP proxy at localhost (i.e. HTTP_PROXY=http://localhost).
rosstimothy