* Extend Teleport RBAC to suport Kubernetes Verbs
This PR extends Teleport per-Resource RBAC to support Kubernetes verbs
restriction. With this change it's possible to restrict certain actions
allowed by the underlying `kubernetes_users` and `kubernetes_resources`.
Supported verbs:
- `get`
- `create`
- `update`
- `patch`
- `delete`
- `list`
- `watch`
- `deletecollection`
Fixes#27095
* address timr's comments
* assign wildcard to verbs for role <7
* address marco's reviews
* tsh: Implement puttyconfig command to add saved PuTTY sessions to Windows registry
* Addressed comments from code review
* Add support for leaf clusters
* Refactoring from code review
Also moved registry/hostname functions into external packages
* Address more feedback from code review
* Rebase following tsh/common changes
* Fix up putty_config_windows
* Reorder command
* Remove surplus comment
* Use a separate list instead of overloading the 'extra' key
* Address Tim's code review comments
* Address some of Zac's comments
* Refactor formatLocalCommandString to use text/template
* Refactor non-Windows logic into puttyhosts
* Fix subcommand name
* Fix test structure
* Add some more hostnames test cases
* Apply suggestions from code review
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Fix up
---------
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Generalize Kubernetes Resource RBAC
This PR generalizes the work introduced by #19613 to other resources
existent in the Kubernetes API.
It allows restricting access to different Kubernetes resources using a
single `kubernetes_group`.
`kubernetes_resource.kind` supports `*` and the following resources:
* Deployments
* Statefulset
* Secrets
* Configmap
* DaemonSet
* Job
* CronJob
* Namespace
* Services
* Nodes
* ReplicaSets
* Roles
* RoleBindings
* ClusterRoles
* ClusterRoleBindings
* PersistentVolumes
* PersistentVolumeClaims
* Ingress
* Certificates
* ServiceAccounts
Part of #20811
* address zac's feedback
The t.Log() function calls have been removed from most tests. This is to keep our test logs cleaner and more focused on error messages. Logging every detail in tests makes it harder to spot actual errors and relevant information.
This PR replaces the usage of `gravitational/oxy.Forwarder` in our codebase with `httputil.ReverseProxy`.
`httputil.ReverseProxy` is part of Go's standard library and thus it is
maintained by the Go team. Converting everything to use the std lib
allows us to drop our `oxy` fork.
There are some operational changes between the two versions:
1. `httputil.ReverseProxy` handles WebSockets upgrades internally and
reuses the `Transport` provided for the initial request. Thus, it does
not require any special dialer or rewriter mechanisms for WebSockets only.
2. `httputil.ReverseProxy` calls the `ErrorHandler` with the request
after it was rewritten and after roundtrip was called. This was a
problem because our `web.transport` deletes Teleport associated
cookies and when the request failed, the request received by the
`ErrorHandler` missed the cookies to generate a new session. This was
fixed by cloning the request at `web.transport` round tripper.
3. `httputil.ReverseProxy` uses std's `log.Logger`. `logrus` exposes a
method `Writer` that allows it to be used with standard lib logger.
The API is similar to what our `oxy` fork has available.
Fixes#27412
* Teleport Proxy Behind ALB support for IP Pinning
* move X-Forwarded-For handling to a middleware
* fix where middleware should be nil
* minor refactor based on review comments
* remove TODO and use contains
* use_x_forwarded_for -> trust_x_forwarded_for
* add NoopHTTPMiddleware
* InstallScripts: pin teleport version using ServerVersion
When Automatic Upgrades are enabled and the current installation is an
enterprise build, it will install teleport using:
- stable/cloud repo channel (yum, apt)
- pin the version to the one present at:
https://updates.releases.teleport.dev/v1/stable/cloud/version
* improve comments
* Add option to allow for host users not to be deleted
This adds a new role option called create_host_users_mode which allows
for it to be configured to not delete users when a session ends. The
old `create_host_user` option will be deprecated with this.
If the deprecated option is set, the new mode option will default to
dropping users as is the current behavior.
* add generated protos
* use combined output for commands, dont set CreateHostUser
* fix tests
* ci fixes
* add godocs to HostUserMode constants, fix tests
* remain -> keep
* Use an enum instead of a string
* fix merge errors
* Add tests & resolve comments
* Resolve issues
* fmt
* ai: add embeddings basic support
- add Embeddings service and its local implementation
- add Embedding type and proto message
- add nodeEmbeddingCollector tracking nodes
- add NodeEmbeddingWatcher watching for events adn sending them to the
collector
- add the Embedder interface and its openai implementation
* ai: adapt embeddings to the vector index
* fixup! ai: adapt embeddings to the vector index
* fixup! fixup! ai: adapt embeddings to the vector index
* Update lib/service/service.go
Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
* address feedback pt.1
* address feedback pt.2: store protobuf message in backend
* address feedback pt.3: have GetEmbeddings return a stream
* Update lib/services/embeddings.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* address feedback pt.4: extract embedding logic out of Embeddings service
* fixup! address feedback pt.4: extract embedding logic out of Embeddings service
* address feedback pt.5: simpler error handling when embedding fails
* fix tests pt.1
* fix tests pt.2
* fix tests pt.3
* [Assist] Replace embedding watcher (#27953)
Change the way how the embeddings are calculated. Instead of creating a watcher in Auth, we will process all nodes every hour and process embeddings if any embeddings are missing or any node has been updated.
---------
Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
This PR introduces a new role version - `v7` - to support other
Kubernetes resources such as `Deployments`, `Statefulsets`... while
keeping compatibility with previous role versions.
This PR adds the following validations:
- `v6`, `v5`, `v4`: `kubernetes_resources` can only include `kind: pod`
resources
- `v7`: `kubernetes_resources` can include wildcards or other
resourceces (to be introduced by #26244.
It also adds a role version downgrade (v13 clients connecting to v14
auth server) where we downgrade the role's version and remove unsuported
kubernetes_resources from the resource list.
Part of #20811
This commit adds the new function `labels_matching` to the label
expression syntax. It accepts a Teleport-style glob or regular
expression, and returns the aggregate of all values for keys matching
that expression.
Since it is now more likely for users to want to compare two slices,
I've also added the `contains_any` and `contains_all` functions to
complement the existing `contains` function.
This commit also adds the `email.local` function that is already
available in role templates and should be useful.
Some examples are included in the new test cases.
* feat: support authenticating with AWS IAM for MongoDB Atlas
* chore(lint): fix errors
* test(tsh): add missing database field
* refactor(mongodb): check for error on each authenticator branch
* refactor(mongodb): update log messages and atlas check
* refactor(auth): use IsRoleARN helper instead of IsARN
* chore(db): remove unused line
* chore(mongodb): split authenticator func
* refactor(db): rename get atlas token function
* tests(db): reuse already existent auth property
* chore(mongodb): add docs reference
* refactor(db): support role chaining
* feat(types): "require" iam role for atlas users
* refactor(db): use external id only on the first session
* refactor(services): add new database matcher for regular users and aws
* chore(db): rename functions to be more assertive
* chore(types): fix lint
* test(db): remove duplicated test
The test being removed here is covered by `TestMongoDBAtlas`
(lib/srv/db/auth_test.go).
Okta Import Rules will now use Teleport style regexes as opposed to raw
regexp. This will maintain parity with the way the rest of Teleport handles
matching.
`lib/utils/typical` is a new library for building predicate parsers
based on the examples set by `lib/utils/parse` and `e/lib/loginrule`.
The commit updates `lib/utils/parse` to use the `typical` library to
implement the two parsers defined here.
This removes a lot of code in `ast.go` that was written for each
specific function to do type checking and deal with subexpressions. This
is all handled by the `typical` library now. My hope is that this will
make it much easier to add new functions in the future and to share them
with other parsers.
* feat: library for building predicate parsers
Took some things I learned while building Login Rules and implemented a
library that can be used to quickly implement new and improved predicate
parsers.
Copied from the package comment:
typical (TYPed predICAte Library) is a library for building better
predicate expression parsers faster. It is built on top of
[predicate](github.com/gravitational/predicate).
typical helps you (and forces you) to separate parse and evaluation into
two distinct stages so that parsing can be cached and evaluation can be
fast. Expressions can also be parsed to check their syntax without
needing to evaluate them with some specific input.
Functions can be defined by providing implementations accepting and returning
values of ordinary types, the library handles the details that enable any
function argument to be a subexpression that evaluates to the correct type.
By default, typical provides strong type checking at parse time without any
reflection during evaluation. If you define a function that evaluates to
type any, you are opting in to evaluation-time type checking everywhere that
function is called, but types will still be checked for all other parts of
the expression. If you define a function that accepts an interface type it
will be called via reflection during evaluation, but interface satisfaction
is still checked at parse time.
* fix typo in error message
Co-authored-by: Tim Buckley <tim@goteleport.com>
---------
Co-authored-by: Tim Buckley <tim@goteleport.com>
* Add a no-op syslog impl for Windows
* Break tctl's dependency on lib/srv
tctl's auth client depends on the lib/reversetunnel package in order
to dial the auth server via reverse tunnel. This package in turn
depends on lib/srv/forward and lib/srv, which is a bunch of
Linux-specific code for Teleport services (re-exec, etc).
By extracting the client side code for dialing via reverse tunnels
into its own package, we can now build tctl for Windows with the
following incantation:
CGO_ENABLED=1 GOOS=windows GOARCH=amd64 CC=x86_64-w64-mingw32-gcc go build -o tctl.exe ./tool/tctl
This commit updates all `err == io.EOF` comparisons to use
`errors.Is(err, io.EOF)`. This is necessary when the error may have been
wrapped and fixes at least one current breakage (in `tsh request ls`).
`golang.org/x/tools/refactor/eg` was very handy for this, I used the
following template:
```go
package teleport
import (
"errors"
"io"
)
func before(err error) bool { return err == io.EOF }
func after(err error) bool { return errors.Is(err, io.EOF) }
```
* check for correct kube listen address in starting message
* fix for ssh proxy and usage of String function
* remove consolef output and simplify output
* remove consolef output
* Remove consolef from services for startup
* fix imports
* remove all consolef usage
* * Use Stderr during headless login to fix rsync compatibility
* Improve error messages when attempting to login without a terminal
* Skip MOTD acknowledement when logging in without a terminal (headless,
sso)
* When attempting re-login without a terminal, make a debug log and
return the original error
* Return access denied error on unexpected handshake failures in tsh
proxy ssh
* Fix TestProxySSH test and add it to the flaky test detector testsToSkip
list.
* Add full IP pinning enforcement
We're adding IP pinning check to `authorizer.Authorize` which is used for every call,
so now all communications with teleport should enforce IP pinning.
Also making sure we always provide login IP for user certificate creation
and correct client IP propagation everywhere.
* Add integration test for App IP pinning.
* Fix wording
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
* Wrap error
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
* Add godocs
* Clone TLS config
* Improve proxyHeaderSigner usage
* Wider use proxyHeaderDialer and remove adhoc writing of singed header
* Add helper function TLSDial
* Use proxyHeader dialer in authConnect
* Simplify tlsConfig manipulation
Co-authored-by: Przemko Robakowski <przemko.robakowski@goteleport.com>
* Remove redundant channels processing in TLSDial
* Reduce nesting
* Update generated protobufs
* Remove ignoring of bad IP on signed PROXY header generation
* Provide logger to CheckIPPinning function
---------
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
Co-authored-by: Przemko Robakowski <przemko.robakowski@goteleport.com>
A complex regex was previously used to parse 'tsh scp'
destinations, which was hard to understand and even harder
to maintain for those not intimately familiar with regex,
despite it being heavily commented. Instead parse destinations
directly and add more test cases and a fuzz test to ensure
parsing won't panic and the new parse function doesn't
have any regressions.
* ALPN connect test improvements
* fix typos
* remove extra period
* simplify error check
* moving things over
* tsh dials
* reverse tunnel
* fix auth connect
* move ping
* add ssh support
* add HTTP client support
* Move ALPN dialer, ALPN conn upgrade, Ping conn to api
* beatify
* add test
* beautify round 2
* fix timeout
* Implement alpn-ping upgrade for reversetunnel and ssh
* clean up
* fix proxy test
* minor refactor
* remove WebProxyAddr
* require IsALPNConnUpgradeRequiredFunc
* add tlsRoutingWithConnUpgradeConnect
* fix lint
* simplify
* remove debug log and change unknown upgrade type to 404
* Force new proxy client to use web proxy when TLS routing is enabled
* Default headless to --mlock=best_effort to reduce errors to a debug
log.
* Add error for non-linux operating systems using headless.
* Add a better mlock error message and add corresponding troubleshooting
docs.
* Prevent tunneling if the os login doesn't exist
A user.Lookup was added to srv.RunForward to prevent dialing
and forwarding any data if the os login is not found. The check
alone only terminates the direct-tcpip ssh channel and not the
underlying ssh connection.
In order for the parent process to determine if the ssh connection
should be terminated it needs to know why the child exited. That was
not possible by looking at the exit code and any data written to
standard error of the child process was forwarded to standard error
on the parent; which was used to simply log the error and move on.
To pass more detailed errors to the parent, the child process spawned
by srv.RunForward now json marshals the trace.Error to standard
error which is then decoded by the parent process. If the parent
detects the error was due to a missing user it terminates the ssh
connection.
tsh ssh -N was also modified to terminate if the command context
of tsh OR the ssh connection to the node is closes. Prior, it
only terminated if the user cancelled the process by blocking on
ctx.Done(). While this was necessary to end session if the os
login does not exit, it also forces tsh to exit if the node
goes offline.
Note: This does not include any propagation of error messages to the user,
so there won't be any indication from tsh about why the connection was closed.
The session also will not be terminated until the first attempt to forward data and
NOT when the session is created due to the way -N is implemented.
Fixes#217
* Prevent unauthorized access to kube clusters by upserting kube_servers
This PR changes the behavior of the kubernetes_service when validating access
to kubernetes clusters. Previously, the kubernetes_service would use the first
kubernetes cluster it found in the Auth server backend to validate access. This was
problematic because if the first kubernetes cluster was upserted with a
the same name as a kubernetes cluster the user was trying to access but
with different labels, the user would be able to access the cluster even
though they shouldn't be able to.
This PR changes the behavior of the kubernetes_service to use the
in memory kubernetes cluster representation used for heartbeats
instead of relying on the information received from the auth server. This would
block the user from accessing the cluster if the cluster was upserted
with a different set of labels since the kubernetes_service would not
have the updated labels in memory and would deny access.
Fixes#469
---------
Co-authored-by: Tim Ross <tim.ross@goteleport.com>
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
* sync prehog/v1alpha/teleport.proto from cloud
* AnonymizeAndSubmit does not need to be fallible
* DELETE IN: 5.1.0
* Add AnonymizeNonEmpty method to HMACAnonymizer
* Add usagereporter encoding for ResourceHeartbeatEvent
* Emit ResourceHeartbeatEvent on heartbeats
* Use "static" field to ResourceHeartbeatEvent
* KeepAlive's Name field should never be empty
* Use the kube cluster name
* launch submitEvents in a goroutine in AddEventsToQueue
* Handle the OpenSSH node resources
* Skip unknown heartbeats
* Add better docs for KeepAliveType
* Add benchmark for ListNodes
* Move RBAC logging to trace level
BenchmarkListNodes is twice as slow when RBAC logging is enabled.
By switching RBAC logging from debug to trace we can eliminate
the performance hit while still providing a way for users to opt
in to the behavior if they need to debug RBAC.
* Intern compiled regular expressions
Profiles of the benchmark test revealed that the `regexp.Compile`
done within `utils.matchString` was the most cpu and memory intensive
portion of the tests. By leveraging a `lru.Cache` to intern the
compiled regular expressions we get quite a performance improvement.
* Only fetch a single page of resources
Increases the request limit prior to loading the resources from
the cache so that we load enough items in a single page to determine
the start key of the next page.
* Remove version checking from `services.UnmarshalServer`
Unmarshal directly to a `types.ServerV2` instead of first creating
a `types.ResourceHeader` to inspect the version. There is only a
single version for `types.ServerV2` making the check unnecessary.
* Add `GetLabel` to `types.ResourceWithLables`
`GetAllLabels` can be overkill if one simply needs to look up the
value for a particular label. It creates a new `map[string]string`
and copies all of a resources existing labels. RBAC decisions
driven by labels incurred the penalty of the copy each time access
was checked. The impact of the copy is much more noticeable when
a resource has several labels or really long strings in the key
or value.
By leveraging `GetLabel` RBAC can avoid copying the labels altogether
and simply lookup each label key when required.
* log a warning when an invalid AWS IAM role ARN is filtered out
* add tests for nested role arn path and remove superfluous test parallelism
* add extra godoc info for BuildRoleARN
* improve mock IAM error messages
- Failing tests' error messages were not specific enough to know what
happened.
* refactor code to call GetAWS() once
- Avoids constantly fetching a full copy of the AWS metadata.
- Reduces verbosity of code.
* reuse MetadataFromRedshiftCluster in meta fetcher
- Avoids repeating the same logic inside the fetcher.
* improve database-access DynamoDB config checking
- Don't early return from database CheckAndSetDefaults so AWS account ID
is checked properly, and any other future checks added will apply to
DynamoDB as well.
* add aws utils for parsing AWS IAM roles
- ParseRoleARN will parse an AWS ARN and verify that it is a valid
AWS IAM Role resource
- CheckARNPartitionAndAccount is a helper that verifies an ARN partition
and account ID matches the caller's expectations.
* use cmp.Diff instead of require.Equal in db tests
- require.Equal spams a lot of protobuf noise.
- cmp.Diff yields a precise, readable diff on test failure.
This tag includes the most recent updates from the upstream in order to address recently fixed CVE's.
Changes included: https://github.com/gravitational/crypto/compare/903e65687e59...c3f983bc73dcf711908e49d91bda9cffbecb13cd
In addition this adds `utils/bcrypto.go` to restore the prior crypto behavior of truncating input > 72 chars.
This commit in Crypto breaks functionality with long passwords or more commonly our reset tokens which are > 72 characters: https://github.com/gravitational/crypto/commit/bc7d1d1eb54b3530da4f5ec31625c95d7df40231
As described in the commit message, the prior behavior was to ignore the extra data. Since 72 bytes are plenty of data to prevent brute force attacks, there is not a significant need to report this error to the user.
A new `api/client/proxy/Client` has been added to interact with
the SSH and gRPC servers that the Proxy serves on its SSH port.
The client will first try connecting to the gRPC server and if
that fails it will fall back to the SSH server.
Much of the SSH functionality mimics the existing behavior of the
`ProxyClient` in `lib/client`. This is the first part of phasing
out that client in favor of the new client. There will be a follow
up PR that migrates `lib/client` to make use of the new client instead.
Part of #19812
Tiago Silva