Adds `UpsertCertAuthority` to the `teleport.v1.trust.TrustService`
and deprecates the HTTP api. The `services.Trust` interface was
also updated to include a `context.Context` in both
`CreateCertAuthority` and `UpsertCertAuthority.
To ensure backwards compatibility the `auth.Client` will first
attempt to use the gRPC client and if that fails with a not
implemented error it will fallback to using HTTP.
Part of #6394
* Add CreatedBy and CreatedOn fields to locks.
* Move the setting of the CreatedBy and CreatedOn for locks to the server instead of the client.
* Add tests for the tctl resource lock import and for the tctl lock create command.
* Use clockwork for time in lock creation tests.
* Make fake clock optional and use the golang 0 date.
* Change CreatedOn to CreatedAt.
* Allow a custom time to be passed through as a fake clock.
* Actually accept a fake clock with useFakeClock.
* Pass Clock through as a configuration value when initializing Teleport for the tests.
* Test cleanup
* Set CreatedBy and CreatedAt values in test mocks.
* Fix failing test from enabling clock passthrough to auth.
Sets `http.Server.IdleTimeout` and `http.Client.IdleConnTimeout`
on clients and servers which didn't have them set. A default of
360s was chosen to be on par with the default of an NLB without
being identical.
This was added as another safety measure to prevent leaking any
idle connections indefinitely as seen in #22757.
`apidefaults.DefaultDialTimeout` was also renamed to
`apidefaults.DefaultIOTimeout` to better reflect its usage.
* Emit new `AgentMetadataEvent`
Part of https://github.com/gravitational/cloud/issues/3550.
This commits adds a new `UpstreamInventoryAgentMetadata` that is sent
from a Teleport agent to an auth server.
Once received, it is transformed into an `AgentMetadataEvent` and sent
to PreHog.
Most `UpstreamInventoryAgentMetadata` fields are intentionally kept as
empty in this PR. Follow up PRs will be opened with the mechanisms
required to fill them (as described in #21337).
* server_id -> host_id
* compute OS and host architecture
* Compute OS version and container runtime
* Close stream if agent receives an agent metadata message
* cmd -> exec ; file -> read
* implement fetchOSVersion for linux
* Remove unused import
* Add note about `agentMetadataCh`
* Allow commands with args
* Remove parseFun abstraction
* fetch glibc version
* fetch container orchestrator
* Fix lint
* Fix TODO
* Add note about glibc version
* cmd -> command
* fetch cloud environment
* fetch install methods
* GLibCVersion -> GlibcVersion
* Use `http.NewRequestWithContext`
* Add missing comment
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* GLibCVersion -> GlibcVersion
* Fix lint
* Fix helm unit tests
* Add missing comments
* 5 second timeout on http requests
* Spawn gorountine that fetches metadata on each new stream
* Use `defaults.HTTPClient()`
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Add missing import
* Handle error
* Revert "Spawn gorountine that fetches metadata on each new stream"
This reverts commit fe4f2790be.
* Send agent metadata to auth server once per stream
* Improve note about agentMetadataDone
* Don't process command output & file content on the agent
Since agents cannot be trusted, regex validation and sanitization should
happen in PreHog anyways. So this commit removes such logic in favor of
moving it to PreHog.
* Fix lint
* Trim space
* Move handling of `AgentMetadataEvent` to `handleControlStream`
* Use cached hello message
* Move metadata files to lib/inventory/metadata
* make sending of agent metadata more self contained
* Minimize diff
* Send all system roles to PreHog
* Remove unused import
* Add parsing of command output / file content back
* Usage reporter refactor
* Usage reporter refactor
* Add missing handling of inventory agent metadata msg
* Fix ICS usage reporter
* Improve comments
* Add cached `metadata.Get*` methods
* Use systemctl status instead of is-active
* Add `Metadata` struct
* return pointer in `FetchMetadata`
* Pass context to `GetMetadata`
* metadataFetchConfig -> fetchConfig
* GetMetadata -> Get
* Add note about `Get` result
* Ensure install methods are non-nil
* Exit `metadata.Get` if context is closed
* Replace sync.Once with atomic.Bool.Swap
* Initialize channel
* Fix lint
* Fix lint
* Make `metadata.Get` return an error instead of bool
* Allow multiple true/false values for env vars
* Use `strings.Cut`
* Use /etc/os-release ID instead of NAME
* Improve `autoEmitMetadata`
* Use `gnu_get_libc_version`
* Ubuntu -> ubuntu
* Use GOARCH
* gofmt
* Move import C up
* Variables may include quotes
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Default values for ID and VERSION_ID
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Blank lines are permitted
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Anonymize host id
---------
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
When the SAML IdP CA was added, it was not added to `checkResourceConsistency`
during `auth.Init`. This made bootstrapping a fresh server from a cluster state
impossible from versions of Teleport that support the SAML IdP CA, as the
`auth.Init` process couldn't recognize the CA.
With this change, the SAML IdP CA is now handled properly by
`checkResourceConsistency`, which makes bootstrapping a cluster from a cluster
state possible again.
Closes#22662
* services.usagereporter to lib/usagereporter/teleport
* teleterm/daemon/usagereporter to usagereporter/daemon
* usagereporter/daemon type aliases
* Move UsageAnonymizable newtypes in a separate file
* DiscardUsageReporter should be passed by value
* Type renames in usagereporter/teleport
* Move audit event to usage event conversion to lib/usagereporter
* package import renames
* Don't break the e build for now
* move web usage report events to lib/usagereporter
* forgot license blurb
* lib/usagereporter/discover.go to types_discover.go
* Add RBAC labels for Databas Services access
This PR adds a new set of Labels that controls access to the
DatabaseService resource.
We changed the `RoleSet.checkAccess` to allow for checks in Resources
without Labels.
However adding complexity to this function is something that can bit us
in the future.
So, this PR reverts that change and adds a new set of labels.
Those labels will control whether the RoleSet has access to
DatabaseServices.
Given that this is a new condition that we add to the default `editor`
role, we had to create a migration/backfill for the roles that don't
have them.
This allow us to migrate the roles created previously and add this new
allow condition.
* remove preset for remote proxy and database labels
* Remove db_service_labels wildcard from editor preset role
* Fix early exit logic
* SAML IdP service provider APIs
The SAML IdP service provider's associated APIs have been implemented. This
will support the use of Teleport as a SAML IdP by allowing Teleport users to
add service providers through the API.
* Correct permissions.
* Update lib/auth/grpcserver_test.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* WIP adding OpenSSH CA
* fix creating OpenSSH CA for tests
* fix a test name and tctl usage flag
* fix 'tctl status' not displaying openssh CA sometimes
* add OpenSSH CA to TestInit_bootstrap, remove TLS key pair
* add OpenSSH CA to a few more places
* make help text of 'tctl auth rotate' more helpful and accurate
* fix web cert export test
* tbot doesn't need to know about OpenSSH CA
* address feedback
* fix comment formatting
This commit adds a new Prometheus gauge `teleport_migrations` that
tracks for each migration if it is active (1) or not (0).
This gauge is then leveraged in `tctl top` to show a set of active
migrations.
* DatabaseService resource: client and server CRUD
In the context of Teleport Discover, we must be able to known if there's
any DatabaseService available to proxy a given Database resource.
If there's none available, we will offer a script for the user to run
and install a DatabaseService which proxies the desired Database
resource.
By DatabaseService, we mean the process that Teleport binary manages
when the `teleport.yaml` config has the following section:
```yaml
db_service:
enabled: "yes"
```
To accomplish this we are creating a new resource: DatabaseService.
The UI will fetch all DatabaseServices and check if there's any
ResourceMatcher that matches the DatabaseLabels.
A previous PR created the DatabaseService resource.
This PR creates all the necessary CRUD operations:
- Add Service to manage DatabaseServices resources in the backend.
- Add GetAll, Upsert, Delete and DeleteAll operations to Client.
- Add DatabaseService support to ListResources.
- Add WebAPI endpoint to list DatabaseServices using ListResources.
The next PR will add the heartbeat mechanism to the DatabaseService
process.
* [draft] Add a new usage reporter
This adds a new usage reporter service to the auth server. It's
disabled by default in OSS and can only be turned on via startup hook
in Cloud / Enterprise. In OSS, the audit log wrapper is never
configured and any usage events are sent to a no-op discard reporter.
Usage events are defined in prehog and can be sent to the new
UsageReporter Service on the auth server. An audit event wrapper is
used to capture certain events that are otherwise difficult to hook.
Events are anonymized before submission, then held in a non-blocking
queue for batching and submission purposes.
* Remove dead code
* Add SubmitUsageEvent RPC to Auth.
This adds a new SubmitUsageEvent RPC to the Auth API that external
clients (e.g. the UI) can use to submit usage events externally.
* Slight refactor for unit testing
* Add Prometheus metrics and add initial working prehog submitter
* Add more metrics, tweak prehog client, and add unit tests
* Further tweak http transport settings based on Teleport defaults
* Add missing metrics
* Fix goimports
* Add new UI usage events
* Update e ref
* Add prehog directly for now. Improve logging.
* update prehog
* Add new prehog events; use username from request identity
* add HTTP server for user events
* Add username back to pre-onboard events
* unauthenticated user events
* Fix userevent build error
* Use event-provided username where appropriate
* Move barebones prehog reqs to lib/prehog and generate here.
Also, use prod tunable values.
* Fix license lints
* De-flake tests by adding unfortunate amounts of synchronization.
* Add missing license header
* Misc PR cleanup for review
* Update lib/events/usageevents/usageevents.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Address a batch of review comments
Adds `anonymizer.AnonymizeString` and parent loggers
* Update e ref
* Clean up comments
* Remove onboard prefix from recovery code event
* Address another batch of feedback
* Use defaults.HTTPClient()
* Remove a noisy log message
* Demote noisy log message to debug
* Temporarily revert e ref for merge
Co-authored-by: Michelle Bergquist <michelle.bergquist@goteleport.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* plumb context to keystore methods
Contexts are more relevant now with the GCP KMS keystore which makes
http requests, this commit makes sure they are passed everywhere they
are needed, with only a few `context.TODO`s sprinkled in.
* feat: add GCP KMS support for Teleport CA key material
This commit implements support for GCP KMS as a backend for CA
operations in Teleport.
This is able to take advantage of much of the infrastucture that we have
already created for HSM support, and simply appears as a new backend for
the private key material.
The necessary configuration parameters include only the name of the KMS
keyring to use, and the protection level (which can be HSM or SOFTWARE).
These are configured in the teleport.yaml directly, in a new section
under the existing `ca_key_params` used for HSM configuration.
The GCP credentials are expected to be provided to the Teleport auth
server via the
[Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc).
This means that it "just works" if the auth server is running on a GCP
compute instance with the correct attached role, and you can run tests
locally by authenticating with `gcloud auth login`.
This does not support Teleport Cloud, as our current HSM support does
not, because the Auth server needs the configuration and the access to
the GCP account. That would be a larger effort probably requiring a new
Teleport service.
* shrink/remove keystore interface
This commit introduces the keystore.Manager type to handle all
interaction between CA and the keystore backends.
Why:
* reduces the code that needs to be implemented per keystore backend to
only the necessary operations
* separate concerns of managing key material and handling CA data
structures
* define interfaces where they're used, not implemented
* delete net 245 lines of code
* reduce keystore.KeyStore stutter
This PR introduces Auth Server CRUD API for managing the lifecycle of dynamic `kube_cluster` resources.
Currently, `kube_cluster` resources cannot be manipulated without using the API, nor are they being listened to. It will be part of future developments.
# Future work
- The `kubernetes_service` will have to watch these `kube_cluster` resources.
- The discovery service will have to create these resources.
Resource manipulation via `tctl` will be introduced later and is not part of the scope.
Part of #16135, #13376
Related to #12048
* Add Default Allow Rules for new resources
When adding a new resource, we usually also set some default access
rules on the preset Roles (editor, access, auditor).
However, for existing clusters those new rules are not applied when we
start the cluster with the new teleport version.
This then requires manual intervention for setting up access to those
resources.
This PR adds a new mechanism to remove the need for manual intervention.
For each of the preset rules, we'll have default rules.
When the server starts, it tries to create the preset roles but if they
exist, instead of failing we will:
- check if the resource exists either in allow or deny rules
- if so, then we don't do anything else
- add the default rule
This assumes it's ok to re-create the default rules when the admin
removes those resources from the allow/deny rule set.
* add log when adding default allow rules
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
Primary Changes:
- Remove reliance on Private Key PEM:
- Update native and keygen packages to return PrivateKey instead of PEM key
- Add new PrivateKey interface which implements crypto.Signer
- Replace PEM encoded private key usage where possible
- Replace calls to tls.(Load)X509KeyPair with keys.(Load)X509KeyPair in
client packages
Minor Changes:
- Remove unused agent.AddedKey return from LoadKey
- Simplify sshutils and removed unused code paths
- Add ecdsa and ed25519 key support
* Tag forwarded spans with custom attributes
Adds a `teleport.forwarded.for` attribute to a resource or
all spans that are forwarded to the auth server. This allows
consumers of the spans to identify where the spans are coming
from and take possible action. In some scenarios it may
be desirable to drop forwarded spans along the collection
process, by tagging them we can provide a way for those
consumers to identify them. It also allows for potentially
identifying a malicious user that may be trying to spam the
telemetry backend with spans.
Part of #12241
In Teleport Discover, we want to improve the UX of adding a resource.
We'll suggest users to test accessing a resource, and give them immediate feedback using the UI.
Note: we'll add this feature to Server Access, later one we'll add other resources.
After adding a server resource, the user will be presented with a command to test the connection:
`tsh ssh --test-id=123 user@host`
Running that command will provide the user feedback right in the UI of whether or not it worked.
If not, a short message will be displayed describing why it didn't work and how the user can solve the problem.
---
To achieve this we'll need a couple of changes:
1. new resource that stores the result of trying to connect to a given resource, so that the UI can ask for it
2. change `tsh ssh` to receive the `--test-id` flag and write the result to a record of this new resource
This PR addresses 1 - new resource.
A new resource is created:
```
Connection Diagnostics
- success (bool)
- message (string)
```
Two new rpc methods:
- create connection diagnostics
- read connection diagnostics
One new HTTP Endpoint:
- read connection diagnostics
Related to
https://github.com/gravitational/teleport/issues/13957
* Embed auth.Cache in auth.Server
* Hit the backend during Auth initialization
* Bypass the cache when rotating CAs
* Services.UpsertTrustedCluster is different
* Bypass the cache in waitForTunnelConnections
* Fix infinite recursion
* More cache bypassing during init and rotations
* Rename Services to Uncached in auth.Server
* Further cleanups
* Don't start the auth cache immediately
* Go back to Services rather than Uncached
* Comments and a missing method
After the merge of https://github.com/gravitational/teleport/pull/12674 we no longer use the following configuration:
```yaml
teleport:
ca_signature_algo: "rsa-sha2-512"
```
As we now rely upon the `x/crypto` package to choose the signing algorithm (it defaults to `rsa-sha2-512`)
**Demo**
If we set `ca_signature_algo` (the value is irrelevant) and start `teleport` we get:
```shell
root@marco:/workspace# teleport start --debug
2022-06-02T09:33:58Z WARN ca_signing_algo config option is deprecated and will be ignored, we'll always default to rsa-sha2-512. config/configuration.go:348
2022-06-02T09:33:58Z INFO Generating new host UUID: b001159a-10e0-49a7-b4dc-61c73fbe9e42. service/service.go:726
...
```
Fixes#12905
Database CA has been added in Teleport 10.0. When migrating from 9.0 local Host CA is copied to keep backward compatibility. Because only local CA is copied trusted cluster fails to connect as described in attached issue, as leaf doesn't have a copy of remote Database CA.
Now remote Host CA will be copied in the same way as we did for local Database CA.
Introduce Database Certificate Authority. New CA is used by Database Access to sign database certificates making them independent from Host CA.
Co-authored-by: Marek Smoliński <marek@goteleport.com>
* Fix goroutine and memory leak in watchCertAuthorities
The CA Watcher was blocking both on writing to a channel when the watcher
was closed and on HTTP calls that had no request timeout or context passed
to cause cancellation.
All resourceWatcher implementations that had a bug which may cause them to block
on writing to a channel forever were fixed by selecting on the write and ctx.Done.
Adding context.Context to all Get/Put/Post/Delete methods on the auth HTTPClient to
force callers to propagate context. Prior all calls used context.TODO which
prevents requests from being properly cancelled.
Add context propagation to RotateCertAuthority, RotateExternalCertAuthority,
GetCertAuthority, GetCertAuthorities. This is needed to get the correct ctx
from the CertAtuhorityWatcher all the way down to the HTTPClient that makes
the call.
Closes#10648
Removing the old roles migration allows Teleport to start even in the face of
invalid roles. The system will still be largely unusable, but `tctl rm` is now
possible as a fallback.
Added logging makes it easier to determine the bad role.
Turns this scenario:
```shell
$ teleport start
> (...)
> ERROR: initialization failed
> could not parse 'where' rule: "!contains(ssh_session.participants, user.metadata.name)", error: ssh_session.participants is not defined
> (teleport exits)
```
into this:
```shell
$ teleport start
> (...)
> 2021-11-18T16:50:29-03:00 WARN [AUTH:1:CA] "Re-init the cache on error: role \"join_own_sessions_only\"\n\tcould not parse 'where' rule: \"!contains(ssh_session.participants, user.metadata.name)\", error: ssh_session.participants is not defined." cache/cache.go:725
> 2021-11-18T16:50:29-03:00 WARN [AUTH:1:CA] Cache "auth" first init failed, continuing re-init attempts in background. error:[
> ERROR REPORT:
> Original Error: *trace.BadParameterError could not parse 'where' rule: "!contains(ssh_session.participants, user.metadata.name)", error: ssh_session.participants is not defined
> Stack Trace:
> (...)
> User Message: role "join_own_sessions_only"
> could not parse 'where' rule: "!contains(ssh_session.participants, user.metadata.name)", error: ssh_session.participants is not defined] cache/cache.go:678
> 2021-11-18T16:50:35-03:00 WARN [AUTH:1:CA] "Re-init the cache on error: role \"join_own_sessions_only\"\n\tcould not parse 'where' rule: \"!contains(ssh_session.participants, user.metadata.name)\", error: ssh_session.participants is not defined." cache/cache.go:725
> (teleport running, tctl works)
```
See https://github.com/gravitational/teleport/issues/9059 for the larger
context.
* Remove Teleport 4.3 role migration
* Remove unused parameters
* Add role name to GetRoles validation failures
* Add RBAC for Windows desktop access
This commit adds RBAC checks for Windows Desktops as described in
RFD 33 and RFD 34:
- add Windows desktop logins & labels to role definition
- introduce new file config for host labels based on a regexp match
- auth server API performs access checking for Windows desktop resources
- add RDP client callback to authorize the user
- support user/role locks
- respect the client idle timeout setting
Note: in cases where an connection is terminated to to RBAC, the web UI
currently displays "websocket connection failed" because the connection
is closed from the server. We'll need to follow up with a nice error
message for the client side to improve the UX here.
Other changes:
* Remove OSS RBAC migration marked for deletion
* Stop creating a default admin role
* add wildcard desktop access to the preset access role
Updates #7761
rosstimothy