This eliminates all remaining references to AWS SDK v1 in the
lib/srv/db/... and lib/srv/discovery/... packages.
Most of the changes are simply to remove dead code.
This eliminates the SigningService in favor of a free function
SignRequest, which eliminates AWS credential loading and simplifies the
AWS utils package.
In the process of doing this refactor, the AWS app handler has been
changed to use AWS SDK v2 to load its credentials as well.
Only DynamoDB and AWS MongoDB Atlas depended on GetAWSSession, and the
migration for these packages was trivial.
Since this was the last AWS method in lib/cloud/clients, the vast majority
of the changes are to remove dead code.
* GCP Server AutoDiscover: only log when the script fails
When running Auto Discover for GCP VMs, teleport always logs the output
in debug level.
Now, it only logs the stdout/stderr when the installation fails.
It also adds some context on which VMs it failed and the log level is
now Error.
* Log once per RunCommand invocation
This PR fixes a behavior problem that results in GKE discovery to
completely fail when the client misses permissions in some of the
discovered projects.
If the client misses list permissions for projectID 1 but has the
required permissions for projectID 2, the discovery service should
continue with the projectID 2 discovery and skip the failing one.
Fixes#48101
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
* Vendor gravitational/trace/trail in api
Pulling in the trail package directly in api will allow the trace
module to shed the grpc-go dependency. This needs to land prior
to https://github.com/gravitational/trace/pull/112 being included
in a new version of trace.
There should be no noticable change in the api depdency tree since
it already depends on grpc-go. Some additional items from the
trace/internal package were also vendored within trail as needed.
Additionally, some of the public api of trail that was not being
consumed has been made private.
* fix: appease linters
* Protobuf and configuration for Access Graph Azure Discovery
* Adding the Azure sync module functions along with new cloud client functionality
* Forgot to decouple role definitions fetching function from the fetcher
* Moving reconciliation to the upstream azure sync PR
* Moving reconciliation test to the upstream azure sync PR
* Updating go.sum
* Fixing rebase after protobuf gen
* Nolinting until upstream PRs
* Updating to use existing msgraph client
* Adding protection around nil values
* PR feedback
* Updating principal fetching to incorporate metadata from principal subtypes
* Updating opts to not leak URL parameters
* Conformant package name
* Using variadic options
* PR feedback
* Removing memberOf expansion
* Expanding memberships by calling memberOf on each user
* Also returning expanded principals for improved readability
* Removing ptrToList
* PR feedback
* Rebase go.sum stuff
* Go mod tidy
* Linting
* Linting
* Collecting errors from fetching memberships and using a WithContext error group
* Fixing go.mod
* Update lib/msgraph/paginated.go
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
* PR feedback
* e ref update
* e ref update
* Fixing method
* Fetching group members from groups rather than memberships of each principal
* Linting
---------
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
Calling the AWS API `ecs:ListServices` with a non-existent ECS Cluster
name will return a 400 w/ ClusterNotFoundException.
The existing code was not handling that error and a raw error was
returned.
This PR changes the logic to ensure that case is handled and that the
ListDeployedDatabaseServices returns an empty list.
An alternative would be to call the ListClusters beforehand, but that
would increase the number of API calls we do to external services.
This replaces awsconfig.WithIntegrationCredentialProvider option with the
awsconfig.WithOIDCIntegrationClient option.
This solves a chicken/egg problem with AWS config loading - callers no
longer need to load AWS config (to create a credential provider) to load
AWS config.
The OIDCIntegrationClient interface is also much simpler to implement.
This also adds default option overrides when creating an awsconfig.Cache.
For now, this is used to add an OIDCIntegrationClient when creating the
cache so that dependent callers don't have to.
* Add a Cache for caching credentials, similar to SDK v1 session cache.
* Add a Provider interface that provides aws.Config
* Simplified role chaining options
Unlike our SDK v1 session cache, the SDK v2 implementation in this PR
does not include region as a cache key.
There are regional AWS STS endpoints for lower latency calls, but the
lowest latency path is to just grab credentials from the cache if we
already have them - the region they were originally taken from doesn't
matter.
* fix: access graph fails to fetch s3 bucket details
With SDK V1, AWS get bucket details such as policies, acls, status...
didn't require the specification of the target s3 bucket location. With
the recent changes to support newer versions of the AWS SDK, the get
bucket details started to fail with the following error:
```
BucketRegionError: incorrect region, the bucket is not in 'ap-south-1' region at endpoint '', bucket is in 'eu-central-1' region
status code: 301, request id: QS5C24H12ZV3VNM4, host id: mzVDk4010MPTCFdxyE/XwERX9W35MSge85PG+h5Jvwyvi7MhxdXLaysb2PTZCMY9r1ngBi6Gv6g=, failed to fetch bucket "cyz" acls polic
```
This PR uses `HeadBucket` to retrieve the location of the s3 bucket and
then uses the bucket location client to retrieve the s3 bucket details.
Fixes#50757
* fix tests
* handle review comments
The Kubernetes service occasionally fails to forward requests to EKS clusters or retrieve the cluster schema due to AWS rejecting the request with an "expired token" error.
EKS access tokens are generated using STS presigned URLs, which include details such as the cluster, backend credentials, and assumed roles. By default, these tokens are valid for 15 minutes, and the Kubernetes service refreshes them every $(15 - 1) / 2 = 7\text{ }minutes$.
However, our cloud SDK caches the underlying `aws.Session`, particularly those with assumed roles, for 15 minutes.
This leads to a scenario where the token is refreshed a second time at approximately 14 minutes, close to the token's 15-minute validity. If the underlying credentials expire before the next token refresh, given that they were reused from the previous query and cached since then, it results in the Kubernetes Service considering the token valid (since it is a Base64-encoded presigned URL without knowledge about the credentials), but AWS EKS cluster rejects the request, treating the credentials as expired.
This PR adds an option to disable cache for EKS STS token signing which results in creating a session per EKS cluster sign process.
Bellow one can find the error message EKS returns.
```
2024-12-09T17:00:15Z ERRO [KUBERNETE] Failed to update cluster schema error:[
ERROR REPORT:
Original Error: *errors.StatusError the server has asked for the client to provide credentials
Stack Trace:
github.com/gravitational/teleport/lib/kube/proxy/scheme.go:140 github.com/gravitational/teleport/lib/kube/proxy.newClusterSchemaBuilder
github.com/gravitational/teleport/lib/kube/proxy/cluster_details.go:193 github.com/gravitational/teleport/lib/kube/proxy.newClusterDetails.func1
runtime/asm_amd64.s:1695 runtime.goexit
User Message: the server has asked for the client to provide credentials] pid:7.1 start_time:2024-12-09T17:00:15Z proxy/cluster_details.go:210
2024-12-09T17:00:24Z ERRO [KUBERNETE] Failed to update cluster schema error:[
ERROR REPORT:
Original Error: *errors.StatusError the server has asked for the client to provide credentials
Stack Trace:
github.com/gravitational/teleport/lib/kube/proxy/scheme.go:140 github.com/gravitational/teleport/lib/kube/proxy.newClusterSchemaBuilder
github.com/gravitational/teleport/lib/kube/proxy/cluster_details.go:193 github.com/gravitational/teleport/lib/kube/proxy.newClusterDetails.func1
runtime/asm_amd64.s:1695 runtime.goexit
User Message: the server has asked for the client to provide credentials] pid:7.1 start_time:2024-12-09T17:00:24Z proxy/cluster_details.go:210
```
Changelog: Fixes an intermittent EKS authentication failure when dealing with EKS auto-discovery.
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
This updates GetByVMID, used by azure joining, to filter by id for
each page of vms instead of filtering after accumulating _all_ vms
in memory first. While this does reduce memory consumption, the listing
process is still suboptimal in that _all_ instances in a subscription
are retrieved in response to _every_ join request.
Replaces the use of go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws
with the tracing support now provided directly by the aws sdk.
See https://github.com/aws/aws-sdk-go-v2/discussions/2810 for more
details.
Some additional aws clients were also updated to include tracing
support.
* [gcp] support project discovery
This PR extends Teleport discovery service to be able to support find all GKE and VM servers in every project a user has access to.
```
discovery_service:
enabled: true
discovery_group: "test"
gcp:
- types: ["gke"]
locations: ["*"]
project_ids: ["*"]
tags:
'*': '*'
```
* simplify docs by adding examples
* feat: adds a new 'policy-preset' flag to 'integration configure awsoidc-idp' command
* adapt to new awsactions
* output formatting and golden test
* run ui prettier, fix getAwsOidcConfigureIdpScriptUrl test
* comments cleanup, mend TestBuildAWSOIDCIdPConfigureScript test
* address review:
- use AwsOidcPolicyPreset in UrlAwsOidcConfigureIdp
- use NoSuchEntityException instead of EntityAlreadyExistsException in PutRolePolicy mock
- fix typo
* use empty string as unspecified value
* add optional field comment
* Merge branch 'master' of github.com:gravitational/teleport into sshah/aws-oidc-preset-policy
* Revert "Merge branch 'master' of github.com:gravitational/teleport into sshah/aws-oidc-preset-policy"
This reverts commit ad2a0904e9.
* integration configure commands prompt for changes
This applies to the following "teleport integration configure" subcommands:
* deployservice-iam
* eice-iam
* ec2-ssm-iam
* aws-app-access-iam
* eks-iam
* access-graph
* listdatabases-iam
These commands will now describe the actions they will take, the
resources that will be created/configured, and will by default prompt
the user to confirm the changes.
Confirmation prompt can be skipped by passing the new --confirm flag.
The confirmation prompt is enabled by default because the most common
case, if not the only case, where a user runs these commands is via an
opaque "bash -c $(curl ...)" script generated by a web UI enrollment
wizard.
* address feedback
* fix test
* pluralize only when there are multiple actions
* remove leading newline from func body
* remove extra newline in ec2 iam configure request
* improve --confirm flag description
* fix test
* remove extraneous else
* dont display escaped ssm doc for op plan
Applies to the integration command that the web UI discover flows tell
users to run in AWS CloudShell to setup the AWS OIDC identity provider:
teleport integration configure awsoidc-idp
The command describes itself, its actions, and the desired state after
it runs. It then prompts the user (by default) to confirm the action
plan before proceeding.
The confirmation prompt can be overridden with cli flag --confirm
if desired.
The IAM role it configures is no longer required to have the "ownership"
tags that teleport applies if it's created by teleport, since the user is
now prompted for confirmation before making changes.
This allows a user to configure an existing IAM role without tagging the
role for configuration by teleport.
The command will still attempt to ensure the IAM role it configures has
teleport tags, but failing to do so is only a warning.
* migrate aws configurator to sdk v2
* rename package imports with lowercase
* export iam client interface
* mock loading aws config
* remove underscore from test name
* remove loop var reassignment
* add ListTags permissions for memorydb/elasticache managed users, which
is needed to find users managed by Teleport
* remove obsolete boundary policy
* deprecate the --boundary flag
* replace dynamic iam policies with static permissions
* Auto Discover GCP GCE: add project id label
This PR adds a new label when auto configuring the teleport.yaml for a
VM in GCP.
This new label has the project id.
We already had the project id, but it was in a protected/internal only
label and so the users couldn't see it in the WebUI.
* Update api/types/constants.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update api/types/constants.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update lib/srv/server/installer/autodiscover_test.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* fix var name and remove godoc from not implemented methods
---------
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Add ownership tags to AWS OIDC EKS Enrollment
This PR adds the ownership tags to the temporary EKS Access Entry
created during the EKS Enrollment.
There might happen that the user already completed the set up once, and
so they won't be asked to run the configuration script (using oneoff +
cloudshell) again. If there's a access denied, the Access Entry is
created without the tags.
With or without the tags, the Access Entry is removed after the
enrollment process.
* log message asking users to add eks:TagResource action
* fix indentation in docs
* fix new tests
* Add periodic import of database objects from supported databases.
A single implementation is included: for Postgres, along with necessary changes.
Also refactors SQLAdminClient interface to accept cert expiry instead of pulling that information from `tlsca.Identity`.
* fix logging, imports
* add missing license
* review feedback
- several minor changes
- apply import rules for all registered object fetchers using wrapper type
- refactor DAC to use object fetcher
- adjust tests
* add comment and missing log call
* support config overrides through unstable env vars, including disabling the service entirely
* refactoring and minor changes
- be more specific about required API for ObjectFetcherConfig
- measure scan time
- bump default scan interval
- call StopImporter from stopDatabase to fix issue with dynamic db resource updates
- add comments
* Refactor `connectAsAdmin`, add `withDefaultDatabase()` method for clarity of implementation.
* Reduce surface area for fetcher config
* make the not-implemented case less noisy by skipping the error
* lint fix
* decrease verbosity of reporting for unsupported configurations
* make `reason` field private
* move `unregisterObjectFetcher` to tests
* avoid exceedingly long lines with log calls
* correct formatting leftover from slog conversion
* tighten logic around disabled importer
if the scan interval is zero or less, the importer becomes disabled, which avoids panic from interval.New().
add some tests and comments for clarity.
* drop unused method
If the ECS service-linked role managed by AWS does not exist yet, then
when the OIDC integration calls ecs:CreateCluster it will fail.
Adding this permission will allow the API call to automatically
create the service-linked role on first use of ECS.
rosstimothy