This PR removes the `types.KindKubeService` objects that were
deprecated in Teleport 11.
Teleport 12 removed the double heartbeat and for Teleport 13 we can
safely drop the support of the old hearbeat based on `types.Server`
objects.
Clients older than Teleport 12 connecting to a Teleport 13 server will
fail since they cannot heartbeat using the old mechanism.
* Add UiConfig option to proxy service config
* Add uiConfig to GRV_CONFIG window object and use in terminal constructor
* Update configuration reference docs
* Remove extra structs and change property to scrollbackLines
* Update web/packages/teleport/src/config.ts
Co-authored-by: Jeff Pihach <hatched@users.noreply.github.com>
* Use a common struct and apply in applyProxyConfig
* Remove webclient dependency for fileconf
* Add missing doc comments
* Set individual properties of cfg.Proxy.UI
* Remove unneeded comment
* Add nolint rule
* Fix nolint
* Add test for UIConfig
* Cleanup tests
* Fix TestUIConfig context
* Allow dynamic config for web ui (#22097)
* Update protofiles
---------
Co-authored-by: Jeff Pihach <hatched@users.noreply.github.com>
* handle tctl create device.yaml
* delete + fix create
* delete by asset tag
* factor out finder
* tctl get device
* gci
* use unmarshaler
* device
* test + unmarshal version check
* improve error
* fix device asset search
* fix parse shortcut bug
* various fixes
* rename CheckAndSetDefaults to checkAndSetDefaults
* Update tool/tctl/common/device/resource.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update tool/tctl/common/collection.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update tool/tctl/common/resource_command.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* fix import
* improve findDeviceByIDOrTag and allow multiple returns for fuzzy asset tag searching in tctl get devices/
* update create/delete messages
* print warn instead of error
* mark sanity check
* resource rework
* add comments
* fix typo
* Update tool/tctl/common/resource_command.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update tool/tctl/common/resource_command.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update tool/tctl/common/resource_command.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* drop findmultiple
* Update tool/tctl/common/resource_command.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update tool/tctl/common/device/resource.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* various feedback updates
* user friendly enum for ostypes
* add comments
* Update tool/tctl/common/resource_command.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update tool/tctl/common/device/resource.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update tool/tctl/common/device/resource.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update tool/tctl/common/device/resource.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update tool/tctl/common/device/resource.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* gci
* feedback
* typos
* Update tool/tctl/common/device/resource.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
---------
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Initial SAML IdP service provider object.
The SAML IdP service provider object has been added to support the use of
Teleport as a SAML identity provider. Please see
https://github.com/gravitational/teleport.e/pull/668 for more information.
The local service for handling backend management of the SAML IdP
service provider objects has been added as well. This will be expanded upon
in a later PR.
* Fix imports.
* Copy paste error.
* Update api/types/saml_idp_service_provider.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Update api/types/saml_idp_service_provider.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Update lib/services/local/saml_idp_service_provider.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Use errAssertionFunc, paginate service provider resources.
* Marco's and Tim's feedback.
* Update proto comments.
* Update NewSAMLIdPServiceProvider comment.
* Fix linting, test issues.
* Remove unnecessary functions already given by ResourceHeader.
* Apply suggestions from code review
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Fixes, removal of explict getter for service provider.
* Remove unnecessary functions.
* Update api/types/saml_idp_service_provider.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Linting fix.
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* feat: login rule tctl CRUD commands
This commit implements the `create`, `get`, and `rm` `tctl` commands for
login rule resources.
Much of the Resource <-> Protobuf type conversion code is already
present in teleport.e and is being copied here so that it can be used in
OSS tctl (there is no longer an enterprise-specific resource command for
tctl, everything is in OSS).
https://github.com/gravitational/teleport.e/pull/699 removes the now
redundant code in teleport.e
n the context of Teleport Discover, we must be able to known if there's any DatabaseService available to proxy a given Database resource.
If there's none available, we will offer a script for the user to run and install a DatabaseService which proxies the desired Database resource.
By DatabaseService, we mean the process that Teleport binary manages when the `teleport.yaml` config has the following section:
```yaml
db_service:
enabled: "yes"
```
To accomplish this we are creating a new resource: DatabaseService.
The UI will fetch all DatabaseServices and check if there's any ResourceMatcher that matches the DatabaseLabels.
Previous PRs created the DatabaseService resource and its CRUD methods.
This PR adds an heartbeat for DatabaseServices similar to what we have for Databases.
There's also a new command to list DatabaseServices using `tctl`:
```
$ tctl get db_service --format text
Name Resource Matchers
------------------------------------ --------------------------------------
a6065ee9-d5ee-4555-8d47-94a78625277b (Labels: <all databases>)
d4e13f2b-0a55-4e0a-b363-bacfb1a11294 (Labels: env=[prod],aws-tag=[xyz abc])
```
Test using Teleport Cloud
```
dinis@lenix ~/p/cloudagents> tctl get db_services
kind: db_service
metadata:
expires: "2022-12-21T18:05:10Z"
id: 1671645310983808522
name: 2a28d394-900c-42ea-a120-eed918e4526b
spec:
resources:
- labels:
aws-tag:
- xyz
- abc
env: prod
version: v1
dinis@lenix ~/p/cloudagents> tctl status
Cluster marcoacme.cloud.gravitational.io
Version 12.0.0-dev
```
Part of #19032
Related #19363#19469
* added Github auth creator
This will allow for multiple implementations of Github
auth connectors.
* work to allow multiple github auth connector implementations
* auth works, need to fix upserts
* fix upserting, add comments
* fix imports from merge
* removed now unused error
* point e submodule to master branch
* fix unit test
* use correct capitalization of 'GitHub'
* move Github auth connector conversions out of api module
* fix linter warning
This PR introduces Auth Server CRUD API for managing the lifecycle of dynamic `kube_cluster` resources.
Currently, `kube_cluster` resources cannot be manipulated without using the API, nor are they being listened to. It will be part of future developments.
# Future work
- The `kubernetes_service` will have to watch these `kube_cluster` resources.
- The discovery service will have to create these resources.
Resource manipulation via `tctl` will be introduced later and is not part of the scope.
Part of #16135, #13376
Related to #12048
* Add an installer script resource type and HTTP endpoint
* Add default installer scripts
* Resolve comments
- run `shfmt` over the installer script
- add MustNewInstaller
- move default installer logic to GetInstaller
- only include PublicProxyAddr in server.Config
* Resolve comments
- move from auth_with_roles -> auth/grpcserver
- document `MustNewInstallerV1`
## What
First part of the Kubernetes [Discovery RFD](https://github.com/gravitational/teleport/pull/13376/) to introduce a Kubernetes server per cluster.
This PR introduces a separate Kubernetes server that uses the already introduced `KubernetesClusterV3`.
## Compatibility
In previous versions, Kubernetes Clusters were part of regular `ServerV2` resource and this refactoring deprecates the `ServerV2` usage but keeps them for compatibility with previous version.
Everything is backward compatible, so v10 kubernetes agents and trusted clusters can connect fine.
## Next steps
Once this is merged, a new PR will introduce dynamic registration for Kubernetes Clusters discovered through EKS Discovery.
* configure golangci-lint misspell to check for anglicized spellings
* Americanize spellings
* fix aws constant value with british spelling 🇬🇧
* update api types with americanized spellings
* use american spellings .cloudbuild/scripts
Boilerplate for a new service and API objects:
- windows_desktop_service config section
- service registration and heartbeats
- static host registration and heartbeats
- caching, permissions, etc
- "tctl get" support
For new connections the service aborts after authentication, since the
RDP client implementation is not ready yet (pending in
https://github.com/gravitational/teleport/pull/7824).
Tested that the service starts, registers (both over a tunnel and
directly) and creates the API objects.
Adds the ability to block network traffic on SSH sessions.
The deny/allow lists of IPs are specified in teleport.yaml file.
Supports both IPv4 and IPv6 communication.
This feature currently relies on enhanced recording for
cgroup management so that needs to be enabled as well.
-- Design rationale:
This patch uses Linux Security Module (LSM) hooks, specifically
security_socket_connect and security_socket_sendmsg, to control
egress traffic. The LSM provides two advantages over socket filtering
program types.
- It's executed early enough that the task information is available.
This makes it easy to report PID, COMM, etc.
- It becomes a model for extending restrictions beyond networking.
The set of enforced cgroups is stored in a BPF hash map and the
deny/allow lists are stored in BPF trie maps. An IP address is
first checked against the allow list. If found, it's checked for
an override in the deny list. The policy is default deny. However,
the absence of the NetworkRestrictions API object is allow all.
IPv4 addresses are additionally registered in IPv6 trie (as mapped)
to account for dual stacks. However it is unclear if this is sufficient
as 4-to-6 transition methods utilize a multitude of translation and
tunneling methods.
* Remove JSON schema validation
Removing JSON schema validation from all resource unmarshalers.
--- what JSON schema gets us
Looking at the JSON schema spec and our usage, here are the supposed benefits:
- type validation - make sure incoming data uses the right types for the right fields
- required fields - make sure that mandatory fields are set
- defaulting - set defaults for fields
- documentation - schema definition for our API objects
Note that it does _not_ do:
- fail on unknown fields in data
- fail on a required field with an empty value
--- what replaces it
Based on the above, it may seem like JSON schema provides value.
But it's not the case, let's break it down one by one:
- type validation - unmarshaling JSON into a typed Go struct does this
- required fields - only checks that the field was provided, doesn't actually check that a value is set (e.g. `"name": ""` will pass the `required` check)
- so it's pretty useless for any real validation
- and we already have a separate place for proper validation - `CheckAndSetDefaults` methods
- defaulting - done in `CheckAndSetDefaults` methods
- `Version` is the only annoying field, had to add it in a bunch of objects
- documentation - protobuf definitions are the source of truth for our API schema
--- the benefits
- performance - schema validation does a few rounds of `json.Marshal/Unmarshal` in addition to actual validation; now we simply skip all that
- maintenance - no need to keep protobuf and JSON schema definitions in sync anymore
- creating new API objects - one error-prone step removed
- (future) fewer dependencies - we can _almost_ remove the Go libraries for schema validation (one transient dependency keeping them around)
* Remove services.SkipValidation
No more JSON schema validation so this option is a noop.
For some resource kinds, `tctl get` used to always return the list of
all resources of that kind, irrespective of whether a specific resource
name was given in the argument to `tctl get`.
For instance, `tctl get node/node-id` would have always listed all the
nodes (just like mere `tctl get node`).
Here, all `tctl get` handlers are adjusted to filter by name when a name
is provided as part of the input ref.
* Use fake clock consistently in units tests.
* Split web session management into two interfaces and implement them separately for clear separation
* Split session management into New/Validate to make it aparent where the sessions are created and where existing sessions are managed. Remove ttlmap in favor of a simple map and handle expirations
explicitly.
Add web session management to gRPC server for the cache.
* Reintroduce web sessions APIs under a getter interface.
* Add SubKind to WatchKind for gRPC and add conversions from/to protobuf. Fix web sessions unit tests.
* lib/web: create/insert session context in ValidateSession if the session has not yet been added to session cache.
lib/cache: add event filter for web session in auth cache.
lib/auth: propagate web session subkind in gRPC event.
* Add implicit migrations for legacy web session key path for queries.
* Integrate web token in lib/web
* Add a bearer token when upserting a web session
* Fix tests. Use fake clock wherever possible.
* Converge session cache handling in lib/web
* Clean up and add doc comments where necessary
* Use correct form of sessions/tokens controller for ServerWithRoles. Use fake time in web tests
* Converge the web sessions/tokens handling in lib/auth to match the old behavior w.r.t access checking (e.g. implicit handling of the local user identity).
* Use cached reads and waiters only when necessary. Query sessions/tokens using best-effort - first looking in the cache and falling back to a proxy client
* Properly propagate events about deletes for values with subkind.
* Update to retrofit changes after recent teleport API refactorings
* Update comment on removing legacy code to move the deadline to 7.x
* Do not close the resources on the session when it expires - this beats the purpose of this PR.
Also avoid a race between closing the cached clients and an existing reference to the session by letting the session linger for longer before removing it.
* Move web session/token request structs to the api client proto package
* Only set HTTP fs on the web handler if the UI is enabled
* Properly tear down web session test by releasing resources at the end. Fix the web UI assets configuration by removing DisableUI and instead use the presence of assets (HTTP file system) as an indicator that the web UI has been enabled.
* Decrease the expired session cache clean up threshold to 2m. Only log the expiration error message for errors other than not found
* Add test for terminal disconnect when using two proxies in HA mode
Added support for an identity aware, RBAC enforcing, mutually
authenticated, web application proxy to Teleport.
* Updated services.Server to support an application servers.
* Updated services.WebSession to support application sessions.
* Added CRUD RPCs for "AppServers".
* Added CRUD RPCs for "AppSessions".
* Added RBAC support using labels for applications.
* Added JWT signer as a services.CertAuthority type.
* Added support for signing and verifying JWT tokens.
* Refactored dynamic label and heartbeat code into standalone packages.
* Added application support to web proxies and new "app_service" to
proxy mutually authenticated connections from proxy to an internal
application.
* Implement kubernetes_service registration and sratup
The new service now starts, registers (locally or via a join token) and
heartbeats its presence to the auth server.
This service can handle k8s requests (like a proxy) but not to remote
teleport clusters. Proxies will be responsible for routing those.
The client (tsh) will not yet go to this service, until proxy routing is
implemented. I manually tweaked server addres in kubeconfig to test it.
You can also run `tctl get kube_service` to list all registered
instances. The self-reported info is currently limited - only listening
address is set.
* Address review feedback
Adds support for Concurrent Session Control and a new
semaphore API. Roles now support two new configuration
options, `max_ssh_connections` and `max_ssh_sessions`
which correspond to the total number of authenticated
ssh connections per cluster, and the number of ssh sessions
within a connection respectively. Attempting to exceed
these limits generate variants of the `session.rejected`
audit event and cause the connection/session to be
rejected.
* Add monorepo
* Add reset/passwd capability for local users (#3287)
* Add UserTokens to allow password resets
* Pass context down through ChangePasswordWithToken
* Rename UserToken to ResetPasswordToken
* Add auto formatting for proto files
* Add common Marshaller interfaces to reset password token
* Allow enterprise "tctl" reuse OSS user methods (#3344)
* Pass localAuthEnabled flag to UI (#3412)
* Added LocalAuthEnabled prop to WebConfigAuthSetting struct in webconfig.go
* Added LocalAuthEnabled state as part of webCfg in apiserver.go
* update e-refs
* Fix a regression bug after merge
* Update tctl CLI output msgs (#3442)
* Use local user client when resolving user roles
* Update webapps ref
* Add and retrieve fields from Cluster struct (#3476)
* Set Teleport versions for node, auth, proxy init heartbeat
* Add and retrieve fields NodeCount, PublicURL, AuthVersion from Clusters
* Remove debug logging to avoid log pollution when getting public_addr of proxy
* Create helper func GuessProxyHost to get the public_addr of a proxy host
* Refactor newResetPasswordToken to use GuessProxyHost and remove publicUrl func
* Remove webapps submodule
* Add webassets submodule
* Replace webapps sub-module reference with webassets
* Update webassets path in Makefile
* Update webassets
1b11b26 Simplify and clean up Makefile (#62) https://github.com/gravitational/webapps/commit/1b11b26
* Retrieve cluster details for user context (#3515)
* Let GuessProxyHost also return proxy's version
* Unit test GuessProxyHostAndVersion & GetClusterDetails
* Update webassets
4dfef4e Fix build pipeline (#66) https://github.com/gravitational/webapps/commit/4dfef4e
* Update e-ref
* Update webassets
0647568 Fix OSS redirects https://github.com/gravitational/webapps/commit/0647568
* update e-ref
* Update webassets
e0f4189 Address security audit warnings Updates "minimist" package which is used by 7y old "optimist". https://github.com/gravitational/webapps/commit/e0f4189
* Add new attr to Session struct (#3574)
* Add fields ServerHostname and ServerAddr
* Set these fields on newSession
* Ensure webassets submodule during build
* Update e-ref
* Ensure webassets before running unit-tests
* Update E-ref
Co-authored-by: Lisa Kim <lisa@gravitational.com>
Co-authored-by: Pierre Beaucamp <pierre@gravitational.com>
Co-authored-by: Jenkins <jenkins@gravitational.io>
* Support resource-based bootstrapping for backend.
Outside of static configuration, most of the persistent state of an
auth server exists as a collection of resources, stored in its
backend. The resource API also forms the basis of Teleport's more
advanced dynamic configuration options.
This commit extends the usefulness of the resource API by adding
the ability to bootstrap backend state with a set of previously
exported resources. This allows the resource API to serve as a
rudimentary backup/migration tool.
Notes: This features is a work in progress, and very easy to misuse;
while it will prevent you from overwriting the state of an existing
auth server, it won't stop you from bootstrapping into a wildly
misconfigured state. In general, resource-based bootstrapping is
not a complete solution for backup or migration.
* update e-ref
* Improve help text and error messages for tctl rm, fixes#2594
* Change 'kind' to 'type' for consistency
* Changed examples from role/admin to connector/github
* Added link to Teleport Enterprise
* Update e ref
This commit expands the usage of the caching layer
for auth server API:
* Introduces in-memory cache that is used to serve all
Auth server API requests. This is done to achieve scalability
on 10K+ node clusters, where each node fetches certificate authorities,
roles, users and join tokens. It is not possible to scale
DynamoDB backend or other backends on 10K reads per seconds
on a single shard or partition. The solution is to introduce
an in-memory cache of the backend state that is always used
for reads.
* In-memory cache has been expanded to support all resources
required by the auth server.
* Experimental `tctl top` command has been introduced to display
common single node metrics.
Replace SQLite Memory Backend with BTree
SQLite in memory backend was suffering from
high tail latencies under load (up to 8 seconds
in 99.9%-ile on load configurations).
This commit replaces the SQLite memory caching
backend with in-memory BTree backend that
brought down tail latencies to 2 seconds (99.9%-ile)
and brought overall performance improvement.
Tiago Silva