* Ensure ssh connection rejection errors are returned
#18302 altered the connection flow which changed the communication
via ssh. Prior to the change an ssh channel was opened first, then
global requests were sent. After the change a global request is sent
prior to opening a channel. This caused users to no longer receive a
human readable rejection message a la: `ssh: rejected: administratively
prohibited (too many concurrent ssh connections for user`, instead only
an `EOF` was returned.
To ensure that the correct message is provided to users two changes are
made, although only the client side change is strictly required.
1) the tracing ssh client now returns any rejected errors cached from
opening the tracing channel
2) the ssh server now processes global requests during it's wait period
waiting for a channel to reject and send the replies to any requests
with said rejection
`TestLockInForce` was updated to include attempting to send a global
request so both scenarios are covered as order should not matter.
* Forward traces from the web UI
Adds a `/webapi/traces` endpoint to the proxy web handler
to allow receiving traces from the UI so that they can
be forwarded to the configured exporter in the proxy_service.
To accomodate traces coming in via websockets the tracing
handler is updated to pull tracing context first from the
standard headers and fallback to retrieving it from a query
parameter as web sockets from the UI cannot alter headers.
Additionally update `web.Terminal` to propagate the tracing
context properly and instruments some of the functions
to ensure spans from the UI are properly correlated to spans
on the backend.
We were inconsistent throughout the codebase and would sometimes
use the slices package and other times use our own equivalents
in api/.
This removes our versions in favor of the golang.org/x package that
does the same, which has the added benefit of reducing the surface
area of the public API module.
Note: despite existing uses of the slices package, for some reason
it didn't show up in go.mod or go.sum. Fixed that too.
* Make SessionTracker heartbeat loop more robust
`(*SessionTracker).UpdateExpirationLoop` terminated the first time
it got an error attempting to persist the updated expiry to Auth.
A momentary network outage could prevent the `types.SessionTracker`
from being updated again causing it to have the wrong TTL in the
backend. This could result in sessions sticking around longer than
needed or being assumed to have been terminated prior to the session
actually ending.
A `retryutils.Retry` has been added to allow for retrying to update
a `types.SessionTracker` with some backoff. The retry loop will continue
until the session is closed, the update is successful, or the retry
has occurred for longer than the `types.SessionTrackers` last known
expiry.
`apidefaults.SessionTrackerTTL` was also reduced from 1h to 30m to
reduce the amount of time a stale tracker may linger in the backend.
The added robustness to the update loop should alleviate the need for
the lengthy TTL.
Wire the Device Trust client into existing clients, a necessity for implementing
device-related commands.
A few user-friendly enum conversions are added as well, as these are useful to
write said commands.
gravitational/teleport.e#514
Add verbs specific to Device Trust ("create_enroll_token" and "enroll") and
change the editor preset to have full device administration powers.
I'll be conservative here and not retrofit permissions into editor, a fact to be
called out in our to-be-written user-facing documentation.
gravitational/teleport.e#514
* added Github auth creator
This will allow for multiple implementations of Github
auth connectors.
* work to allow multiple github auth connector implementations
* auth works, need to fix upserts
* fix upserting, add comments
* fix imports from merge
* removed now unused error
* point e submodule to master branch
* fix unit test
* use correct capitalization of 'GitHub'
* move Github auth connector conversions out of api module
* fix linter warning
This PR presents a watcher for automatic `kube_cluster` discovery for GCP GKE clusters. Given an identity with access to the GCP cloud, the auto-discovery service will scan the cloud and register all clusters available in Kubernetes Engine.
Once the discovery service creates a `kube_cluster` on the Auth Server, the Kubernetes Service will start serving it. The credentials used to access the cluster are short-lived and generated through Google OAuth2 associated with the GCP Service Account configured for the Kubernetes Service.
GCP's Service Account must have the following role def attached:
```yaml
description: 'GKE Auto-Discovery'
includedPermissions:
- container.clusters.impersonate
- container.clusters.get
- container.clusters.list
- container.pods.get
- container.selfSubjectAccessReviews.create
- container.selfSubjectRulesReviews.create
name: projects/{projectID}/roles/GKEKubernetesAutoDisc
stage: GA
title: GKEKubernetesAutoDisc
```
Part of #16135, #13376
Related to #12048, #16276, #16281, #16633, #14991
* feat: add GCP KMS support for Teleport CA key material
This commit implements support for GCP KMS as a backend for CA
operations in Teleport.
This is able to take advantage of much of the infrastucture that we have
already created for HSM support, and simply appears as a new backend for
the private key material.
The necessary configuration parameters include only the name of the KMS
keyring to use, and the protection level (which can be HSM or SOFTWARE).
These are configured in the teleport.yaml directly, in a new section
under the existing `ca_key_params` used for HSM configuration.
The GCP credentials are expected to be provided to the Teleport auth
server via the
[Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc).
This means that it "just works" if the auth server is running on a GCP
compute instance with the correct attached role, and you can run tests
locally by authenticating with `gcloud auth login`.
This does not support Teleport Cloud, as our current HSM support does
not, because the Auth server needs the configuration and the access to
the GCP account. That would be a larger effort probably requiring a new
Teleport service.
* Add auth_connector api resources
- Add the go types in `operator/apis`
- Add the OIDCConnector, SAMLConnector and GithubConnector resources to `protoc-gen-crd`
- Add `wrappers.StringValues` support to `protoc-gen-crd`
- Update crdgen test fixtures
* Regenerate CRD manifests
* Introduce the teleport reconciler abstraction
* operator: Add auth_connector resources support
- Reconcile OIDC, SAML and Github authconnectors
- Make the oidc, saml and github interfaces implement `ResourceWithOrigin`
* operator: Add standard tests for auth connectors
This Commit introduces a set of default tests that can be reused for
each controller based on `TeleportResourceReconciler`.
* Bump CRD manifests to go 1.19
* operator: make Makefile OSX friendly
* Add newKubeResource tests and fix godocs
* Detect if HTTP_PROXY/HTTPS_PROXY is specified in `tsh proxy ssh`
* Support HTTPS_PROXY in `tsh proxy ssh`
* Ensure we tidy up connections on failure
* Lintering
This commit adds two new endpoints:
Create a database with
```
POST .../databases
{
"name": "<databasename>",
"protocol": "<protocol>",
"uri": "<uri>",
"labels": [{"name":"<name>", "value": "<value>"}...]
}
```
Update a database's `ca_cert` (the only updatable field for now)
```
PUT .../databases/:database
{
"ca_cert": "------BEGIN CERTIFICATE-----\n <certificate> --END..."
}
```
Also update the active sessions test to operate based on the presence
of session trackers rather than actually starting a session. This
allows us to test additional session types and is more deterministic.
For app sessions, use the name from the types.Application, since
the value in the cert is only present for CLI sessions.
For database sessions, use the database service name, which maps
to the Teleport database service the user is connecting to, since
the actual "logical" database name is not required.
Closes#17126Closes#17871
* Ensure invalid tunnel agent connections get closed
Connections from reverse tunnel agents were being marked
as invalid by the proxy under certain conditions but would
ultimately never be closed. This could lead to scenarios where
the agent thought things were fine but the proxy considered
that agent unhealthy and unroutable.
Pruning of invalid connections used to occur when a proxy
tried to retrieve a connection for that tunnel. This also
further muddied the point in time at which the proxy could
close a connection as it never explicitly stopped tracking
the connection and closed it at the same time.
To remedy this, connections are explicitly closed by the proxy
and removed from the mapping to stop tracking immediately. In order
to prevent a connection that is servicing an active connection
from being closed the proxy now tracks which connections have
sessions. Closing does not occur when there are any active
sessions to prevent them from being force terminated.
When the proxy receives a heartbeat from an agent it now restores
the connection to a valid state. In the event that too many heart
beats have been missed for an agent, the proxy will now terminate
the connection, again only if it is not serving any sessions.
Fixes#15911
In the context of Discover, we want to ease the installation process of Teleport.
Our current focus is Databases.
Adding a database resource requires at least one Database service/agent running with matching labels for that resource.
The focus of this PR is to ease the installation of those agents.
Those service/agents are currently installed in a multi step flow:
- install Teleport on the host (select OS/arch/distribution format)
- configure the service by running `teleport configure db ...`
- start the application (possibly configuring the systemd unit file.
As you can see, this is a lot of steps.
This PR adds a new installation script: install-database.sh
This script is actually the same used for installing Server Nodes or App Service/Agents but now can also install database service/agents.
So, how does the new flow look like?
After adding the Database info (type, hostname, port and labels), and if there's no agent available, a new step appears: add a database agent.
The user provides a list of labels they want the agent to monitor.
Those labels are stored as SuggestedAgentMatcherLabels within the token which is generate after confirming the labels.
The user then gets an URL which they can use to install the agent.
This URL contains the token and when invoked creates the install-database.sh script containing the `db_service` configuration.
The field `db_service.resources.0.labels` contains the previously defined labels.
This script is almost the same that exists today for installing Server Nodes and App services/agents, it only changes the `teleport.yaml` configuration.
When the user run this command, the UI gets notified and can then proceed to the next step.
**Demo**
```bash
root@marcoacme-node-001:/shared# install-database.sh
2022-10-20 16:56:27 UTC [teleport-installer] TELEPORT_VERSION: 11.0.0-rc.1
2022-10-20 16:56:27 UTC [teleport-installer] TARGET_HOSTNAME: 127.0.0.1.nip.io
2022-10-20 16:56:27 UTC [teleport-installer] TARGET_PORT: 3080
2022-10-20 16:56:27 UTC [teleport-installer] JOIN_TOKEN: 6417b719b68d979ac01d3039f4246ce3
2022-10-20 16:56:27 UTC [teleport-installer] CA_PIN_HASHES: sha256:9fa22af321476074164f40aa901e291df92f3dbeea1b1f641bc5bf47efd00174
2022-10-20 16:56:27 UTC [teleport-installer] Checking TCP connectivity to Teleport server (127.0.0.1.nip.io:3080)
2022-10-20 16:56:27 UTC [teleport-installer] Couldn't find nc, telnet or /dev/tcp to do a connection test
2022-10-20 16:56:27 UTC [teleport-installer] Going to blindly continue without testing connectivity
2022-10-20 16:56:27 UTC [teleport-installer] Detected host: linux-gnu, using Teleport binary type linux
2022-10-20 16:56:27 UTC [teleport-installer] Detected arch: x86_64, using Teleport arch amd64
2022-10-20 16:56:27 UTC [teleport-installer] Detected distro type: debian
2022-10-20 16:56:27 UTC [teleport-installer] Using Teleport distribution: deb
2022-10-20 16:56:27 UTC [teleport-installer] Created temp dir /tmp/teleport-PWn6zEcG9P
2022-10-20 16:56:27 UTC [teleport-installer] Downloading Teleport deb release 11.0.0-rc.1
2022-10-20 16:56:27 UTC [teleport-installer] Running curl -fsSL --retry 5 --retry-delay 5 https://get.gravitational.com/teleport_11.0.0-rc.1_amd64.deb
2022-10-20 16:56:27 UTC [teleport-installer] Downloading to /tmp/teleport-PWn6zEcG9P/teleport_11.0.0-rc.1_amd64.deb
2022-10-20 16:56:50 UTC [teleport-installer] Downloaded file size: 119220326 bytes
2022-10-20 16:56:50 UTC [teleport-installer] Will use sha256sum to validate the checksum of the downloaded file
2022-10-20 16:56:52 UTC [teleport-installer] The downloaded file's checksum validated correctly
2022-10-20 16:56:52 UTC [teleport-installer] Using dpkg to install /tmp/teleport-PWn6zEcG9P/teleport_11.0.0-rc.1_amd64.deb
Selecting previously unselected package teleport.
(Reading database ... 8182 files and directories currently installed.)
Preparing to unpack .../teleport_11.0.0-rc.1_amd64.deb ...
Unpacking teleport (11.0.0-rc.1) ...
Setting up teleport (11.0.0-rc.1) ...
2022-10-20 16:56:54 UTC [teleport-installer] Found: Teleport v11.0.0-rc.1 git:v11.0.0-rc.1-0-g6950d74 go1.19.2
2022-10-20 16:56:54 UTC [teleport-installer] Writing Teleport database service config to /etc/teleport.yaml
2022-10-20 16:56:54 UTC [teleport-installer] Host does not appear to be using systemd
2022-10-20 16:56:54 UTC [teleport-installer] ---> This host is not running systemd, so Teleport cannot be started automatically when it exits.
2022-10-20 16:56:54 UTC [teleport-installer] ---> Please investigate an alternative way to keep Teleport running.
2022-10-20 16:56:54 UTC [teleport-installer] ---> You can find information in our documentation: https://goteleport.com/docs/
2022-10-20 16:56:54 UTC [teleport-installer] ---> For now, Teleport will be started in the foreground - you can press Ctrl+C to exit.
Run this command to start Teleport in future:
/usr/local/bin/teleport start --config=/etc/teleport.yaml
------------------------------------------------------------------------
| IMPORTANT: TELEPORT WILL STOP RUNNING AFTER YOU CLOSE THIS TERMINAL! |
| YOU MUST CONFIGURE A SERVICE MANAGER TO MAKE IT RUN ON STARTUP! |
------------------------------------------------------------------------
2022-10-20 16:56:54 UTC [teleport-installer] Starting Teleport in the foreground
2022-10-20T16:56:54Z INFO Starting Teleport v11.0.0-rc.1 common/teleport.go:435
2022-10-20T16:56:54Z [PROC:1] INFO Generating new host UUID: 51b404d7-e501-43f1-83dc-8298c1fcab2b. pid:379.1 service/service.go:874
2022-10-20T16:56:54Z [PROC:1] INFO Joining the cluster with a secure token. pid:379.1 service/connect.go:582
2022-10-20T16:56:54Z [PROC:1] INFO Joining the cluster with a secure token. pid:379.1 service/connect.go:582
2022-10-20T16:56:55Z [AUTH] INFO Attempting registration via proxy server. auth/register.go:239
2022-10-20T16:56:55Z [AUTH] INFO Attempting registration via proxy server. auth/register.go:239
...
root@marcoacme-node-001:/shared# cat /etc/teleport.yaml
version: v3
teleport:
nodename: marcoacme-node-001
auth_token: 6417b719b68d979ac01d3039f4246ce3
ca_pin:
- sha256:9fa22af321476074164f40aa901e291df92f3dbeea1b1f641bc5bf47efd00174
proxy_server: 127.0.0.1.nip.io:3080
log:
output: stderr
severity: INFO
auth_service:
enabled: no
ssh_service:
enabled: no
proxy_service:
enabled: no
db_service:
enabled: "yes"
resources:
- labels:
dev: prod
```
**Tech Details**
Request to create token with `suggested_agent_matcher_labels`:
```shell
curl 'https://127.0.0.1.nip.io:3080/v1/webapi/token' \
-H 'accept: application/json' \
-H 'content-type: application/json; charset=utf-8' \
--data-raw '
{
"roles":["Db"],
"join_method":"token",
"allow":[],
"suggested_agent_matcher_labels": {"dev":"prod"}
}
'
```
Closes#17446
* protobuf update
* Update proto to use dynamodb request event specific to app-access
We will include a similar event for dynamodb via database-access.
We split the events so that app and database access events are not coupled.
This way we do not have to include optional database/app metadata in one event too.
* Update protos
* Update oneof
* Move AppMetaData up with the other metadata and add a 'target' field
* Remove operation plane
* Add API types for CircleCI joining
* Add validation for CircleCI configuration
* Add JoinMethodCircleCI across codebase
* Add token validator and token source for circleci
* Update join methods RFD
* Add CircleCI token source to register.go
* Add serverside support for circleci joining
* Add test for TokenSource
* Add success case for token validation test
* Add expired test case
* Add test case for token from another org
* Test RegisterWithToken for CircleCI
* Refactor GitHub RegisterUsingToken tests
* Refactor CircleCI RegisterUsingToken tests
* Add tests for ProvisionTokenSpecV2
* Appease linters
* Go Imports files
* Fix failing test for msising IDtoken
* Update lib/auth/join_circleci.go
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Move test server cleanup closer to initialisation
* Fix weird import reordering
* Include unexpected type in error message
* Simplify boolean algebra :)
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
- When user registers or resets, when running into
policy errors, we don't send back an error, but instead
a 200 (to indicate user has successfully registered/resetted)
and a flag to determine if policy was enabled
- Send back policy configurations for cluster config,
this will allow the web UI login page to just display
redirection messages without login attempts
- For role configured policy, login attempts will be
required and a specific error will be returned so that
the UI can then better redirect the user
* Bump libc from 0.2.135 to 0.2.136
* Bump k8s.io/apiserver from 0.25.2 to 0.25.3
* Bump github.com/aws/aws-sdk-go-v2/feature/ec2/imds
* Bump github.com/aws/aws-sdk-go-v2/service/sts from 1.16.19 to 1.17.1
* Bump github.com/golang-jwt/jwt/v4 from 4.2.0 to 4.4.2
* Bump go.opentelemetry.io/otel/sdk from 1.11.0 to 1.11.1
* Bump github.com/aws/aws-sdk-go-v2/credentials from 1.12.21 to 1.12.23
* Bump github.com/aws/aws-sdk-go-v2/service/ec2 from 1.63.1 to 1.63.3
* Bump go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
* Bump github.com/stretchr/testify from 1.8.0 to 1.8.1 in /api
* Bump go.opentelemetry.io/otel/sdk from 1.11.0 to 1.11.1 in /api
* Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
* Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
* Bump github.com/aws/aws-sdk-go from 1.44.117 to 1.44.123
clients close the stream and any connection when they are done with it
servers close the stream after a 1 minute timeout
proxy JoinService gRPC server closes idle connections with no RPC calls after 10 seconds
`types.Metadata` was not properly copied when removing credentials and the internal map was not deep copied.
The race condition happened when service labels were set which caused the watcher and heartbeat functions to be reading and manipulating static labels.
STeve (Xin) Huang