When the AWS Account ID is provided, we must check that it is a valid
one.
AWS Account ID is a well documented field: 12-digit string
This check is meant to prevent typos when creating Databases mainly from
the Web UI in the context of Teleport Discover.
* Add helper func to check for AWS unrecognized engine error
* Retry RDS queries with individual filters if engine name is unrecognized
* godoc
* pass engine names instead of filters, to make retry logic simpler
* Update mocks to check/apply RDS engine filters
* Update watcher tests
* Add watcher test for unrecognized RDS engines
* Update lib/srv/db/cloud/mocks.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
Co-authored-by: Marek Smoliński <marek@goteleport.com>
We were inconsistent throughout the codebase and would sometimes
use the slices package and other times use our own equivalents
in api/.
This removes our versions in favor of the golang.org/x package that
does the same, which has the added benefit of reducing the surface
area of the public API module.
Note: despite existing uses of the slices package, for some reason
it didn't show up in go.mod or go.sum. Fixed that too.
This PR presents a watcher for automatic `kube_cluster` discovery for GCP GKE clusters. Given an identity with access to the GCP cloud, the auto-discovery service will scan the cloud and register all clusters available in Kubernetes Engine.
Once the discovery service creates a `kube_cluster` on the Auth Server, the Kubernetes Service will start serving it. The credentials used to access the cluster are short-lived and generated through Google OAuth2 associated with the GCP Service Account configured for the Kubernetes Service.
GCP's Service Account must have the following role def attached:
```yaml
description: 'GKE Auto-Discovery'
includedPermissions:
- container.clusters.impersonate
- container.clusters.get
- container.clusters.list
- container.pods.get
- container.selfSubjectAccessReviews.create
- container.selfSubjectRulesReviews.create
name: projects/{projectID}/roles/GKEKubernetesAutoDisc
stage: GA
title: GKEKubernetesAutoDisc
```
Part of #16135, #13376
Related to #12048, #16276, #16281, #16633, #14991
* Add Yubikey PrivateKey implementation for use by Teleport clients.
- Add yubikey login logic, reusing previously stored private keys.
- Fix identity file decoding with PIV keys, which sign ecdsa certificates.
- Add libpcsclite-dev pre-req for building on linux.
- Remove unnecessary keys.Signer interface and move its functionality to keys.PrivateKey.
- Move retry and jitter utils to new api/utils/retryutils package.
Coverts getting values in the `FnCache` from `FnCache.Get` to
`utils.FnCacheGet[T any]`. This allows us to have compile time
guarantees for the type being loaded, which allows us to remove
some of the manual type checking required of callers today.
Update metalinter, fix a few lint warnings and replace deprecated linters.
`deadcode`, `structcheck` and `varcheck` are abandoned and now replaced by [`unused`][1].
Since 1.19, `go fmt` reformats godocs according to https://go.dev/doc/comment. I've done a bulk-reformatting of the codebase to keep the linter happy. Backporting is mostly harmless (the exception being `lib/services/role_test.go`, that for some reason breaks the _old_ linter using the new format).
[1]: https://golangci-lint.run/usage/linters/
* Bump golangci-lint version
* Replace abandoned linters
* Fix bodyclose on lib/auth/github.com
* Fix bodyclose on lib/kube/proxy/streamproto/proto_test.go
* Fix bodyclose on lib/srv/alpnproxy/proxy_test.go
* Fix bodyclose on lib/web/conn_upgrade_test.go
* Silence staticcheck on lib/kube/proxy/forwarder_test.go
* Silence staticcheck on lib/utils/certs_test.go
* Address BuildNameToCertificate deprecation warnings
* Run `go fmt ./...`
* Run `go fmt ./...` on api/
* Ignore formatting in role_test.go
* Remove redundant initializers in lib/srv/uacc/
* Update e/
* Add Azure auto-discovery configuration fields
* Init databases if azure matchers are in config
* Use AzureMatchers in db service
* Use all azure subscriptions/resource groups if omitted in matcher
* Add azure config tests
* Go mod tidy to update dependencies
* Add azure response error conversion
* Check for azure access denied and give a helpful error message
* Add azure subscriptions api
* Add azure mysql/postgresql api and wrappers
* Test generic db server for azure
* Make server properties its own type
* Convert server types manually instead of via json
* Move server list method selection logic out of api client
* Update azure db server tests
* Fixup merge
* Update comments
* Update more comments and remove junk code
* Move all azure api into lib/cloud/azure
* Update state and version checks
* Convert Azure DB Server into database
* Add mutex to subscription client for caching, just in case
* Test database conversion from azure db server
* Refactor common code
* Add azure matchers to watch config
* Remove unused imports
* Use common max pages in memorydb users
* Add azure clients to cloud clients
* Add azure fetchers to watcher
* Skip fetcher NotFound errors
* Add azure watcher tests
* Fix test comment
* Add wildcard region matching
* Remove redundant import
* Update lib/srv/db/cloud/watchers/azure.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Move db server listing logic into separate func
* Refactor some db conversion code
* Use local var to cache sub IDs
* Rename azure subscription client
* Rename azure subscription client and remove sub ID caching
* Move azure cloud clients into separate embedded interface
* Update watcher test for 'NotFound' handling
* Embed azureClients struct
* Update lib/cloud/azure/db_server_test.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Update lib/cloud/azure/subscriptions_test.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Update lib/cloud/azure/db_server_test.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Update lib/cloud/azure/db_server_test.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Update lib/cloud/azure/db_server_test.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Rename azure subscription client and remove sub ID caching
* Add reference links for azure db ports
* Move indirect dep into group
* Wrap all converted azure response errors
* Remove unreachable panic
* Godoc DBServer
* Remove maxPages arg to azure client funcs
* Gofmt
* Spacing between copyright and package
* import order
* Update lib/srv/db/cloud/watchers/azure.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Fix import order
* Refactor region matching
* Fix usage of azure clients to remove max pages arg
* Query azure subscriptions in fetcher.Get if matching wildcard subscription
* Fix azure matcher construction
* Test azure fetcher discovers new subscriptions
* Remove unused test helper func
* Set azure database resource id metadata
* Log errors when fetching from azure subscription fails
* Use more readable protocol in azure db description
* Get azure mysql engine version from labels
* Update lib/srv/db/cloud/watchers/watcher_test.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Update lib/srv/db/cloud/watchers/watcher_test.go
Co-authored-by: Marek Smoliński <marek@goteleport.com>
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Add an AWS EC2 instance fetcher
* Resolve comments
- Remove leftover VPCID stuff
- Document public functions
- Return a notfound when there are no nodes fetched instead of nil nil
* Resolve comments
* Fix fileconf_test.go
Allow users to secure GCP Cloud SQL instances by setting "Allow only SSL
connections", which enforces client certificate authentication.
This implementation does not require any configuration changes for Teleport
users. Teleport will detect whether client certificate authentication is
required and handle either case automatically.
Client certificates are ephemeral. They are created for every connection by
calling the GCP Cloud SQL API's GenerateEphemeralCert function. Certificates
are only created when the destination Cloud SQL instance is configured to
require client certificate authentication. The configuration is detected by
requesting instance settings from the GCP Cloud SQL API on every connection
attempt.
A special case was implemented for MySQL. MySQL servers in GCP Cloud SQL do not
trust the ephemeral certificate's CA but GCP Cloud Proxy does. To work around
this issue, the implementation will connect to the MySQL Cloud Proxy port using
a TLS dialer instead of the default MySQL port when client certificate
authentication is required.
The common.CloudClients interface and implementation now return an interface
(GCPSQLAdminClient) from the GetGCPSQLAdminClient function instead of the GCP
client's sqladmin.Service. Returning an interface simplified calling code and
allowed for the client to be mocked for testing.
Existing GCP Cloud SQL tests are configured to not require client certificate
authentication by default. A new test named TestGCPRequireSSL was created to
simulate client certificate authentication for both Postgres and MySQL. This
required some minor changes to the test server code.
A new ConnectWithDialer function was added to the
github.com/gravitational/go-mysql fork. This function is available upstream in
v1.4.0 but other changes upstream resulted in a number of errors and a panic
processing network packets. So instead of upgrading, the dialer function was
copied to the Teleport fork and a custom version was created instead:
v1.1.1-teleport.1.
Marco André Dinis