Sets `http.Server.IdleTimeout` and `http.Client.IdleConnTimeout`
on clients and servers which didn't have them set. A default of
360s was chosen to be on par with the default of an NLB without
being identical.
This was added as another safety measure to prevent leaking any
idle connections indefinitely as seen in #22757.
`apidefaults.DefaultDialTimeout` was also renamed to
`apidefaults.DefaultIOTimeout` to better reflect its usage.
`btree.BTree` has been a backward compatability wrapper over a
`btree.BTreeG` since `github.com/google/btree@v1.1.0`. This converts
all usages to the new generic variant.
* Create a generic local backend service.
Many of the local backend services have a very similar pattern in terms of how
objects are retrieved and stored. This PR introduces a generic backend which
implements backend retrieval and storage of objects and migrates the SAML
IdP service provider service to it. More services could easily be migrated
to this generic service implementation.
* Feedback, add in generic tests.
* Tim's feedback, added in a test for updating a resource that doesn't exist.
* Tim's feedback.
* Apply suggestions from code review
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Edoardo's feedback.
* Added in UpdateAndSwap.
* Edoardo's feedback, Tim's feedback.
* Use a default page size if the page limit is not specified.
* More of Edoardo's feedback.
* Propagate error instead of panic.
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
When performing a KeepAlive the key was first retrieved from
etcd to verify that it existed before updating the item. However,
a Put operation with the `ignore_value` set will return a not
found error if the key does not exist
(https://etcd.io/docs/v3.4/learning/api/#put). So we can safely
remove the get and rely on etcd to prevent updating a key that
does not exist.
Also converts `concurrentqueue.Queue` to be generic over the given
Input and Output of the worker function.
The reqs member of the mock Firestore Server used in the `TestDeleteDocuments`
test was occasionally causing a data race. Accesses to this field have now
been wrapped in a lock.
Since #16636 etcd leases have been bucketed, yet when watching events
we still determine the ttl for each particular item in `fromEvent`
by retrieving the ttl from etcd. To reduce the number of ttl requests
sent to etcd we now cache the ttl per lease id.
Closes#21380
* Use `ctx` rather than `context.TODO()` where available
* Further propagation of context
* Use `context.Background()` instead of `context.TODO()` where appropriate in tests
* Further remove `context.TODO` from tests
* Appease linter on parameter order
This PR adds a `post-delete` hook that cleans the state secret that each Pod creates when it receives the identity from the Teleport Auth Server.
Fixes#20035Fixes#18854
If all etcd nodes are unreachable, Teleport exits with the following
message:
```
ERROR: initialization failed
context deadline exceeded
```
With this commit we get instead:
```
ERROR: timed out dialing etcd endpoints: [https://127.0.0.1:2381https://127.0.0.1:12381]
initialization failed
context deadline exceeded
```
Note that if only some of the etcd nodes are unreachable, the etcd
client already provides a useful error message:
```
ERROR: initialization failed
failed to dial endpoint https://127.0.0.1:12381 with maintenance client: context deadline exceeded
```
We were inconsistent throughout the codebase and would sometimes
use the slices package and other times use our own equivalents
in api/.
This removes our versions in favor of the golang.org/x package that
does the same, which has the added benefit of reducing the surface
area of the public API module.
Note: despite existing uses of the slices package, for some reason
it didn't show up in go.mod or go.sum. Fixed that too.
* Revert "Azure AD authentication for the Postgres backend (#15757)"
This reverts commit 33c6d82dc3.
* Revert "SQL Backend (#11048)"
This reverts commit 06fef2abf1.
* Remove Postgres backend from the docs
* Remove the Postgres backend from the testplan
From the etcd docs:
> To obtain lower latencies and higher throughput for read requests
> clients can configure a request’s consistency mode to serializable,
> which may access stale data with respect to quorum, but removes the
> performance penalty of linearized accesses' reliance on live consensus.
By leveraging serializable mode it is entirely possible to retrieve
stale data. This means that a Create followed by a Get could result
in a NotFound error, which results in issues like #13276. While this
improves latency it breaks the consistency guarantees required by
Teleport.
To ensure that our consistency requirements are met all etcd operations
are converted to use the default linearized mode by removing the
`WithSerializable` option.
Closes#13276.
* Add Yubikey PrivateKey implementation for use by Teleport clients.
- Add yubikey login logic, reusing previously stored private keys.
- Fix identity file decoding with PIV keys, which sign ecdsa certificates.
- Add libpcsclite-dev pre-req for building on linux.
- Remove unnecessary keys.Signer interface and move its functionality to keys.PrivateKey.
- Move retry and jitter utils to new api/utils/retryutils package.
Update metalinter, fix a few lint warnings and replace deprecated linters.
`deadcode`, `structcheck` and `varcheck` are abandoned and now replaced by [`unused`][1].
Since 1.19, `go fmt` reformats godocs according to https://go.dev/doc/comment. I've done a bulk-reformatting of the codebase to keep the linter happy. Backporting is mostly harmless (the exception being `lib/services/role_test.go`, that for some reason breaks the _old_ linter using the new format).
[1]: https://golangci-lint.run/usage/linters/
* Bump golangci-lint version
* Replace abandoned linters
* Fix bodyclose on lib/auth/github.com
* Fix bodyclose on lib/kube/proxy/streamproto/proto_test.go
* Fix bodyclose on lib/srv/alpnproxy/proxy_test.go
* Fix bodyclose on lib/web/conn_upgrade_test.go
* Silence staticcheck on lib/kube/proxy/forwarder_test.go
* Silence staticcheck on lib/utils/certs_test.go
* Address BuildNameToCertificate deprecation warnings
* Run `go fmt ./...`
* Run `go fmt ./...` on api/
* Ignore formatting in role_test.go
* Remove redundant initializers in lib/srv/uacc/
* Update e/
* Add Username to sqlbk and don't leak connConfigs
* Azure AD authentication for sqlbk/Postgres
* Add a Postgres Config test
* Cache Azure tokens, document azureBeforeConnect
* Move the config test to sqlbk
* go mod tidy
* go get azcore azidentity
Large clusters which experience a lot of churn can have a backend
populated with expired items yet to be deleted due to dynamo ttl
limitations. Increasing the limit as a stop gap for now until a
better solution can be put in place.
## What
First part of the Kubernetes [Discovery RFD](https://github.com/gravitational/teleport/pull/13376/) to introduce a Kubernetes server per cluster.
This PR introduces a separate Kubernetes server that uses the already introduced `KubernetesClusterV3`.
## Compatibility
In previous versions, Kubernetes Clusters were part of regular `ServerV2` resource and this refactoring deprecates the `ServerV2` usage but keeps them for compatibility with previous version.
Everything is backward compatible, so v10 kubernetes agents and trusted clusters can connect fine.
## Next steps
Once this is merged, a new PR will introduce dynamic registration for Kubernetes Clusters discovered through EKS Discovery.
This MR introduces Kubernetes Secret storage for Teleport Kubernetes Agent state.
When running Teleport Agent inside a Kubernetes cluster where TELEPORT_REPLICA_NAME and KUBE_NAMESPACE exist, the agent will store identities and states in a Kubernetes Secret instead of local storage.
Otherwise, if the above environment variables are not present or not in Kubernetes, the agent persists state in SQLite.
Notes:
Fresh installs will enable the Kubernetes Secret Storage by default.
If PV storage is configured, the agent will store its identity in a Secret and use the PV for non-identity objects.
If the operator is upgrading from an existing release that didn't support Secret storage, the Helm chart will convert the objects to enable Kube Secret Storage. If the current object is a Deployment, the Helm chart installs a new Statefulset side by side and will trigger a Helm hook that is responsible for deleting the Deployment once Statefulset pods are healthy. Otherwise, if upgrading from a Statefulset the agent will read the credentials stored in PV and move them into Kube Secret.
Implementation of #12958
Related to #5585
* Add dynamodb metrics
Adds promethues counters for dynamo api requests, requests failed,
and latency of requests. Each counter is labeled with the table type,
either backend or events, and the operation being requested.
* configure golangci-lint misspell to check for anglicized spellings
* Americanize spellings
* fix aws constant value with british spelling 🇬🇧
* update api types with americanized spellings
* use american spellings .cloudbuild/scripts
Super minor, but in this case we log that we're initializing the backend
_prior_ to cfg.CheckAndSetDefaults which means that if we are indeed
using the default the following will be printed:
> {"caller":"dynamo/dynamodbbk.go:206","component":"dynamodb","level":"info","message":"Initializing
backend. Table: \"example-table\", poll streams every
0s.","timestamp":"2022-06-01T23:49:49Z"}
Which is misleading as it's not polling the stream every "0s", it's
polling every 1s as per backend.DefaultPollStreamPeriod
Signed-off-by: Michael McAllister <michael.mcallister@goteleport.com>
Adds a `trace.Tracer` to the `backend.Reporter`
wrapper so that all `bakend.Backend` implementations
can be traced. Further instrumentation of each specific
backend will be added at a later date to see how long
each sql query, or call to dynamo/etcd took within
each backend operation.
#12241
Always set the user field of the database connection string to the same
user defined in the CommonName field of the client certificate.
The pgx library always includes a user field in the connection string
and defaults to $USER. The database connection will fail when $USER is
not the same as the value defined in the CommonName field of the client
certificate.
Storage paths returned by Key do not contain a trailing slash, which will
cause range matching to match on paths that contain the original path as
a prefix. By using ExactKey, range matching will only match sub-paths.
Fixes#5188.
Firestore only allows 500 writes per commit. Prior to this change
DeleteRange or purgeExpiredDocuments could result in deleting
more than 500 documents and would result in an error. To comply with
the limits both DeletRange and purgeExpiredDocuments now call
deleteDocuments which properly handles limiting a batch to 500 writes
and commiting multiple times until all documents have been removed.
Fixes#12007
rosstimothy