* Add warning to previous command
* Hash out basics of new tbot proxying command use
* Avoid passing full botConfig into `tbot ssh-proxy-command`
* wip
* simplify impl
* Set up test suite for ssh config generation
* Add support for checking if ALPNConnUpgrade is required
* Cache the alpn upgrade result
* Americanize Spellings
* Update lib/tbot/config/template_ssh_client.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Add CHANGELOG.md entry for breaking change
* Update tool/tbot/proxy.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Update lib/tbot/tbot.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Switch to `--no` variants rather than `={bool}`
* Use SingleFlight on alpn proxy upgrade cache
* Improve singleflight
* Add rudimentary shell quoting
* Uniformly pad the templating
* Rename shellQuote -> proxyCommandQuote
* Fix imports
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* AWS OIDC App Access: add PublicAddr to App
This PR adds a missing field to the App created to handle the AWS App
Access when using the AWS OIDC credentials
The PublicAddr is required and was not being set.
* fix identation
This starts separating the lib/auth client and server code into
distinct packages to reduce the dependency tree of client tools.
For the time being aliases were left to all things moved into
lib/auth/authclient to reduce the changeset and ensure code still
compiles. Future changes will be built off this to convert code
to use lib/auth/authclient where applicable, this was intentionally
left to a minimum amount of changes to facilitate backports.
The new subcommand is meant to be a replacement for tbot proxy ssh
which does the ssh proxying entirely within tbot to reduce resource
consumption. The old mechanism spawned a tsh process to perform the
proxying which leads to increased CPU and memory consumption - which
can be quite problematic when used as an OpenSSH ProxyCommand for
an ansible playbook.
The new command also deviates in behavior from tsh proxy ssh in that,
for the moment, it requires users to explicitly dictate whether
tls routing, alpn connection upgrades instead of guessing by making a
call to webapi/ping. This again turns out to be quite resource
intensive when tbot is used as an OpenSSH ProxyCommand for an ansible
playbook.
In the future tbot can look this information up out of band and store
it in the proper location so that tbot ssh-proxy-command can consume
it directly without users needing to provide it. The behavior as is
provides a path toward the future while also allowing users to gain
the performance improvements now.
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* dont allow disabling second factor
* update docs for 2fa change
* add environmental variable to work around disabled 2fa
* check if its cloud
* update todos
* add a test
* introduce insecure test mode
* fixes for tests
We were never actually using the passed-in value, and instead
always read the format field of the reciever.
No functional change, as the value we passed in always happened
to be the same as the format field we were using.
* AWS OIDC: Script to configure EC2 Auto Discover with SSM
There's a new teleport command that adds an inline policy which enables
the AWS OIDC Integration to handle EC2 Auto Discover using the script
mode (SSM).
It also adds the SSM Document used in the teleport installation.
There's also a new endpoint which returns the one off script that uses
that command.
* Update lib/integrations/awsoidc/ec2_ssm_iam_config.go
Co-authored-by: Edward Dowling <EdwardDowling@users.noreply.github.com>
---------
Co-authored-by: Edward Dowling <EdwardDowling@users.noreply.github.com>
* add spanner to tsh and teleterm
* handle spanner events in web ui
* update web audit events test snapshot
* customize web ui connect for spanner
* update ConnectDialog snapshot
* Update labels datatype in Teleport crownjewel proto
This commit modifies the labels from a map to repeated TeleportLabel in CrownJewel proto, and accordingly updates the getter function in proto generated code. The newly introduced TeleportLabel type contains a Key and repeated values for improved flexibility.
Clean up
Update and restructure CrownJewel service proto
The code modifications include renaming and restructuring several types and altering various functions in the CrownJewel service proto. Major changes involve renaming DeleteCrownJewel
Rename 'kind' to 'kinds' in crownjewel API
This commit changes the 'kind' field to 'kinds' in the crownjewel API to better reflect its purpose of holding multiple resource types. Changes are made across several files - 'crownjewel.go', 'crownjewel.pb.go' and 'crownjewel.proto'.
After rebase fixes
Remove CrownJewel type from several files
This commit represents the removal of the CrownJewel type from several files. CrownJewel type references are deleted in certain Go files, and imports of CrownJewel in multiple files are being adjusted. The specifics of why this type is removed is not evident from the diffs, but the deletions will affect the related functionalities.
Add CrownJewel update functionality
This update enables the 'UpdateCrownJewel' functionality. If a 'CrownJewel' resource already exists, it can now be updated instead of returning an error. This enhances flexibility when dealing with resource management. It also introduces the ARN field into the 'AWSMatcher' service.
Update CrownJewel service with pagination and improve matchers
This update adds pagination to the CrownJewel service for more efficient data handling. It also improves the matching capabilities with the inclusion of Teleport and AWS matchers and allows for selective querying with these matchers. Additionally, it includes various changes to structure and naming for better clarity and ease of use.
Refactor CrownJewel protos to use ResourceHeader
This refactoring replaces the previous structure of CrownJewel proto's fields Kind, Sub_kind, Version, and Metadata with a single object called Header (ResourceHeader). It simplifies the code structure and harmonizes it with other protos. Now, Header contains the essential attributes that were previously distributed across multiple separate fields.
Add GetCrownJewels method to cache and accessgraph
The function GetCrownJewels has been added to the cache and the accessgraph. It retrieves all CrownJewel instances from the cache. Additionally, CrownJewel retrieval functionality has been removed from auth server and existing usages have been refactored.
Implement Crown Jewel creation feature
Updated the method `createCrownJewel` in the resources command to handle Crown Jewel creation. Introduced a new gRPC service `crownjewelv1` to handle client-server interactions for Crown Jewel objects. Crown Jewel expiry configuration has now also been enabled.
Add missing files
I have no words...
Update authentication and event handling
Refactored the argument from 'auth.ClientI' to '*auth.Client' in 'createCrownJewel' function in 'resource_command.go'. Removed unused import 'durationpb' in 'auth_with_roles.go'. Added a new event code 'AccessGraphAccessPathChangedCode' in 'codes.go' and 'AccessGraphAccessPathChanged' in 'api.go' for better monitoring and logging of access path changes.
Add access path changed event and relevant changes.
update api
add grpc
update api
add cache support
add protos
* Fixes after rebase
* Refactor Crown Jewel resource methods
Updated API methods related to the Crown Jewel resource to improve clarity and consistency. Renamed `GetCrownJewels` to `ListCrownJewels`, added pagination parameters, and ensured uniform handling of the Crown Jewel resource across different parts of the codebase. Updated relevant comments for better code understanding.
* Removed CrownJewel converter, updated CrownJewel references
This commit removes the CrownJewel converter and updates references from the old CrownJewel type to the new protobuf-defined CrownJewel type, streamlining the codebase. The commit also disables CrownJewel deletion since it is not supported yet. Some minor errors were fixed regarding CrownJewel resource names.
* Implement Crown Jewel deletion functionality
The crown jewel deletion feature has been enabled by uncommenting and simplifying the code block. Now, the delete action is directly performed on the specific 'crown jewel' mentioned based on its name. The previously unsupported action returns an informative success message after the deletion.
* Refactored function 'allCombinations' and added authorization checks
Moved 'allCombinations' function into 'utils' package, renamed it to 'Combinations', and updated its references in 'dbobjectimportrulev1' and 'dbobjectv1' tests. Additionally, introduced admin authorization checks to 'CreateCrownJewels', 'UpdateCrownJewels', and 'DeleteCrownJewels'. Created corresponding tests for these changes.
* Add CrownJewel service to cache tests
The CrownJewel service has been added to the cache test file. The service creation is implemented and assigned to new fields in multiple methods within the cache tests. A new test function `TestCrownJewel` has also been added to validate CRUD operations for the service.
* Add access path changed event and update Crown Jewel tests
Added a new event code for when an access path changes in the access graph, and introduced new API calls to handle these situations. Also, updated crown jewel related tests and fixed copyright year in several files.
* Update protobuf import path
The protobuf import path was updated from "github.com/golang/protobuf/proto" to "google.golang.org/protobuf/proto" to ensure correct use and to prevent potential issues, while maintaining functionality in crown_jewels.go.
* Refactor CrownJewel and AccessGraph services
This commit optimizes the ListCrownJewelsResponse in the CrownJewel service by directly assigning the response instead of using an extra loop. Additionally, the EventsStream in the AccessGraph service and related tests have been updated to EventsStreamV2, reflecting the implementation of new versions of these methods.
* Add CrownJewel resource in access controls and tests
In both the cache test and access control files, CrownJewel resource has been included. This provides the system's ability to manage and test CrownJewel resources. Additionally, it allows the handling of CrownJewel resources in client events.
* Simplify implementation
* Update copyright dates and add license headers
Updated the copyright dates in several files to reflect the new year. Additionally, license headers have been added to some files where they were previously missing, ensuring the proper copyrights and GPL notice are visible.
* Refactor CrownJewel codebase for better readability
Fixed various typographical errors in CrownJewel's related functions and removed a TODO comment. Improved function and function calls' names that were incorrectly referring to different modules, making the code easier to read and understand across different CrownJewel services.
* Add validation for Teleport and AWS matchers in service
The validation has been added for Teleport and AWS matchers in the service layer to ensure that essential details are not missed when setting matchers. This includes checks to ensure that kinds and labels for Teleport, and types and tags for AWS matchers are properly set.
* gci
* Add CrownJewel update functionality and refactor related services
This commit adds the ability to update a CrownJewel resource within the ResourceCommand. It also includes refactoring of related services, condensed code logic, and improved utilization of existing methods for marshaling/unmarshaling CrownJewel. Specific changes in service.go and crown_jewels.go files have reduced code redundancy and enhanced the code structure.
* Relocate and update crown_jewels_test file
This commit moves the crown_jewels_test from the local to the services library and adds test cases for unmarshalling Crown Jewels data. Also, import statements for labelv1 and utils were introduced to support these changes.
* Update crown jewels test with AWS matchers
Revised the crown jewels test file by including AWS matchers section in the CrownJewel object. The change ensures to specifically match AWS infrastructure following certain parameters, thus enhancing the specificity and flexibility of our matchers. Apart from this, minor reordering of objects has also been done for better code organization.
* Added upsertCrownJewel function and related proto message
The commit adds the UpsertCrownJewel function in the service.go file, which enables upsert operations on crown jewel resources. Alongside, the UpsertCrownJewelRequest proto message is added to the crownjewel_service.pb.go file to handle such requests.
* Refactor CrownJewelsClient to use upsert
This change refactors the CrownJewelsClient to use the upsert method when the force flag is on. The code has been streamlined by eliminating the need for 'already exist' error checking and separate create and update methods.
* Add GetCrownJewel functionality and refactor validation
A GetCrownJewel function has been added to the Service, which retrieves a specific CrownJewel resource. Also, the CrownJewel validation process has been refactored, moving the validateCrownJewel function from service.go into the crownjewel library and renaming it to ValidateCrownJewel. These changes improve modularity and code organization.
* Add VerbCreate check and test for CrownJewel validation
The checkAccessToKind function now also checks for the VerbCreate permission. Additionally, a new test file, object_test.go, was added to validate the functionality of the CrownJewel object. This includes various validation checks such as "NilCrownJewel", "ValidCrownJewel", "MissingMatchers", among others.
* Refactor CrownJewel resource handling and validation
The codebase was updated to improve CrownJewel resources handling. This includes modifications to list, get, create, update, upsert, and delete functions in several files. Additionally, a validation function has been implemented for CrownJewel objects, ensuring required fields are present. Depreciated logger service has also been removed.
* Add cache implementation to CrownJewel service
The CrownJewel service has been enhanced by integrating it with cache. The main changes include adding "Cache" as part of ServiceConfig and using cache to fetch crown jewels instead of directly using the backend services. The cache interface with the methods "ListCrownJewels" and "GetCrownJewel" has been added to handle these operations.
* Add Cache to service configuration in crownjewelv1 tests
In this commit, we added the Cache field to the service configuration used in the tests for the crownjewelv1 service. This ensures that the service configuration used in the tests matches the expected real-world configuration.
* Update CrownJewelsService to use ConditionalUpdateResource
This commit changes the CrownJewelsService's `UpdateCrownJewel` method to use the `ConditionalUpdateResource` method instead of the `UpdateResource` method. This change will provide a more appropriate update strategy for the specific needs of the CrownJewelsService.
* Replace 'Cache' with 'Reader' in crownjewel service & remove DeleteAllCrownJewelsRequest
The 'Cache' in crownjewel service has been replaced with 'Reader' to improve code readability and to match with its functionality. Also, the DeleteAllCrownJewelsRequest functionality was removed as it posed a high risk potential for data loss and it wasn
* Update tests for Crown Jewel service
This commit updates the test suite of the Crown Jewel service. An object is fetched from the backend before updating its expiry time, ensuring the revision is populated. In addition, a new test is introduced to handle the scenario where the revision is missing when updating a Crown Jewel.
* Refactor protobuf marshaling with utils
This commit replaces the usage of the `protojson` method for marshaling and unmarshaling protobuf objects in `crown_jewel.go` with the `FastMarshal` and `FastUnmarshal` methods from `utils`.
protojson is failing to unmarshal resources.
* Update CrownJewel test and serialization methods
Updated the test condition in crown_jewels_test.go to use Proto.Equal for object comparisons. Also, swapped out the FastMarshal and FastUnmarshal methods with protojson's Marshal and Unmarshal in the CrownJewels service.
* Add missing license
---------
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
* API for creating AppServer that uses AWS OIDC Integration
This PR adds two new APIs:
- web api endpoint
- `tctl create` for AppServers
This allows the client to create AppServer which use the integration
credentials.
This also updates the `editor` role so that they are able to upsert the
AppServer.
* return ui.App after creation
* improve comment
* Save the WebAuthn UserHandle in the mock Key
* Simplify passwordless tests
* fix: Forbid SSO users from logging in using passwordless
* Rename tests to TestPasswordlessProhibitedForSSO
Historically proxy templates have only supported an exact hostname
match via the `Host` field defined. This limits some of the benefits
of using proxy templates and forces users to do a bunch of extra
work in their ssh config. For example, if a user today wanted to
leverage proxy templates to connect to a host via a label, predicate
expression, or fuzzy match, they would be required to use tsh ls,
find the matching UUID or hostname and provide that to tsh for
evaluation with the template expression.
SSH Config:
```
Match Host *.example.com
ProxyCommand tsh proxy ssh %r@$(tsh ls --search=$(echo %h | cut -d. -f1) -f json | jq -r '.[0].metadata.name'):%p
```
The above requires multiple subshells, a connection to the Auth
server, and jq to be installed on the client machine in order to
work.
To reduce the amount of effort required to connect to a resource
proxy templates have been updated with a `Search` and a `Query`
field to support fuzzy search and predicate expressions much like
`tsh ls --search` and `tsh ls --query`. The order of operations
for matching the target host is: 1) Host, 2) Predicate Expression,
3) Query.
In addition to the proxy template changes, tsh proxy ssh
has also been updated to allow resolving the target via
searching or querying -- though ONLY via proxy templates.
There are no additional `--query` or `--search` flags added
to tsh proxy ssh for the time being. This is mostly for CLI UX
reasons since neither tsh ssh or tsh proxy ssh support those
flags today, and only allow filtering by label via supplying
key value pairs in the hostname (tsh ssh alice@foo=bar,env=dev).
tsh proxy ssh is however still limited to connecting to a single
host at a time. If there are no matches, or multiple matches found
as a result of target resolution, then an error is returned.
With the new functionality, the above example can be simplified to
the following.
SSH Config:
```
Match Host *.example.com
ProxyCommand tsh proxy ssh %r@%h
```
Proxy Template:
```
- template: '^(.*).example.com:\d+$'
search: "$1"
```
This shifts the resposibility of host resolution from the caller
to tsh. It allows resolution to happen in the same process that
will initiate the connection, reducing a dial to the cluster, and
eliminating the need for any subshells, or extra tools outside
of tsh to be present.
* Add CreateAppSessionFromReq.
* Deprecate passing route_to_app.session_id in cert requests.
* Automatically create app sessions for app cert requests.
* Deprecate client use of CreateAppSession for app cert requests.
* Fix affected tests.
* Use traits and roles from existing user certificate for app sessions.
* Fix app access integration tests.
* go mod tidy.
* Fix TestTCP.
* Fix TestALPNSNIProxyAppAccess.
* Revert "go mod tidy."
This reverts commit cc6caa2485.
* Switch `tbot` to use `slog.Logger` from logrus
* Pick up a few spots I forgot to convert
* Fix tctl logging
* Avoid using Context-less variants of logger
* Convert logpoints in tool/tbot to use context based loggers
* Fix use of multiline string
* Move tbot log initialization before first logging message
* Switch "err" to "error" in log attributes
* Use `NewPackageLogger`
* AWS OIDC: IAM set up for AWS App Access
AWS App Access using AWS OIDC Integration requires a new policy in the
AWS OIDC Role.
This PR adds a one-off script that executes the configuration command
that adds said policy.
* improved comments, remove dead code, improve logging
* improve comment and http client var name
* move back to interface
* remove default client
* rename role to rolename
* enable fips endpoint support
* restrict target roles by their tags
* use true for tag value
* Convert recursive listing to use Proxy gRPC API
Removes deprecated usage of ProxyClient in getClusterClients used
for tsh ls -R and tsh db ls -R.
* Remove unsued helper functions
* Replace removed test
* Remove unused db listing function
* convert tsh apps ls -R and tsh kube ls -R
* remove unused function and tweak IsExpired
* create common testcase for listing resources
* Add ability to re-exec into shell when using `tsh proxy kube`.
This PR introduces the `tsh proxy kube --exec` mode which spawns the
kube proxy in the background and re-execs the user shell with the
configured kubeconfig file.
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
* rename
* fix printed message
---------
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
* Keep track of whether the user has a real password set
* lint
* Some comments
* Fix TestCreateResources
* Fix e/ link
* Review
* Review
* Review
* Use early return on error in DeletePassword
* Fix the broken teleterm test
* Remove clockwork dependency and handle time internally
The `clockwork` package was removed and handling of time elements has been refactored in codebase. The current time was set once and then passed as an argument to the functions that need it, reducing calls to retrieve the current time. This change simplifies the code and makes it more robust against time drifts.
* Build fix
* Remove usage of fake clock in tests
Removed the use of fake clock in tsh_test.go and kube_test.go to imitate real-world behavior more accurately. This change is aimed at improving the credibility and reliability of those test cases.
* Refine grammar and punctuation in comments.
This commit updates the comments in `access_request.go` and `access_request_test.go` files to improve grammar and punctuation for better readability and understanding of the code. This includes correction of typographical errors, addition of punctuation where necessary, and clarification of some comments for more precise descriptions.
* Update TTL validation error messages
The code changes refine messages for validation errors related to TTL. The error messages returned when requested TTLs exceed the maximum allowed amounts are improved. Time is no longer rounded to minutes, providing more precise error messages.
* Add automatic role requests
This change adds the optioin to automatically make a role access request
during tsh ssh instead of a resource access request.
* Speed up TestSSHAccessRequest
* Use testServer.MakeTestServer to fix issue with parallel runs using the same ports.
* Add deprecation comments for test server helpers outside of tools/teleport/testenv.
* Update tool/teleport/testenv/test_server.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Disable SSH resumption
Enable LoadAllCAs
Update comments
* Prevent context.Canceled from being wrapped across transport stream
trail doesn't properly handle conversions to and from errors that
are context.Canceled which results in code paths trying to evaluate
errors.Is(err, context.Canceled) failing. The error recevied is
an interceptors.RemoteError, which wraps a trace.TraceErr, which
contains a status.Status with codes.Canceled. To work around this
until trail is updated the server half of the stream was updated
to return an io.EOF if if encouters a canceled error.
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
Co-authored-by: Tim Ross <tim.ross@goteleport.com>
* Fix incorrect expiry when using resource153ToLegacyAdapter
* Change the `tctl` format for database_object_import_rule resource.
Prior to this change the resource wouldn't round-trip through `tctl get ... | tctl create` because of header incompatibilities.
In general, all resources must be compatible with services.UnknownResource for their `tctl` storage format.
* Lint fix
Updates the ClusterClient to support issuing user certificates,
possibly with MFA, for all protocols. This should be the last
bit of functionality required to allow Connect to convert to
using the ClusterClient. Once that is done the deprecated
ProxyClient should be one step closer to deletion.
Updates #38239
* Implement persistence for database_object resource
* Split monolithic service test
* Add `t.Parallel()` to the tests
* lint fix
* Update copyright year
Co-authored-by: Anton Miniailo <anton@goteleport.com>
* `make grpc`
---------
Co-authored-by: Anton Miniailo <anton@goteleport.com>
* Introduce plugins cleanup command line function.
tctl can now explicitly issue a plugins cleanup command, which will allow
admins to remove any extra artifacts that a plugin may have left behind
after being deleted.
* Add in a dry run command, list resources.
* GCI.
* Code cleanup, use ResourceID instead of string.
* Only allow okta cleanup, default to dry-run on.
* Add tctl sso configure and tctl sso test support for OIDC and SAML
Migrates the tctl sso configure and tctl sso test commands for
SAML and OIDC connectors from tctl enterprise to OSS. This will
allow the enterprise tctl binary to be removed.
* Guard CreateOIDCAuthRequest and CreateSAMLAuthRequest
Prevent the RPCs from being called in the open source version of
teleport. Now that the tctl sso test commands are in OSS we want
to ensure that the commands will only function if communicating
with an enterprise Auth server.
* fix: license
* update e ref
* configure GCP workforce api client
* add 'integration configure samlidp gcp-workforce' command to teleport ctl
* update api to fetch metadata file from remote Teleport prox.
- minor comment copy and log copy edits
* gcp resource name validation, checkandsetdefault funcs
* use gcp-workforce subcommand path, rename gcpworkforce config as APIParams
* return early on pending provider status
* update comments and godoc
* use slog
* apply review suggestions:
- use 2*minute instead of 120*seconds
- use default http client and add redirect to the client
- NewGCPWorkforceService test
* update logger to use InfoContext. improve wait message, run fix-imports
* update ticker to 5 second from 1 second
* use underscore in log field instead of space. add missing trace.Wrap in waitForPoolStatus
* update NewGCPWorkforceService comment
* review suggestions:
- increase wait timer to 5 mins
- iamv1 migration comment
- move http redirect checker to checkAndSet default
* check for 409 response so that pool provider create op can continue if we timedout after creating pool
* fix package import comment style to make linter happy
* move default http client to gcpworkforce service checkandset defaults. update tests to relax on httpclient input param
The Proxy web api now requests that Auth include allowed logins
per resource instead of guessing logins per resource based on the
information it has cached. However, due to the way SSH sessions
are authorized, the logins are not provide to users verbatim.
Any sessions created via the root web ui to a leaf resource will use
the SSH certificate created for that user in the root cluster. New
certificates are not minted per leaf cluster. This is important
because the nodes only allow os logins for a session if they are
present in the valid prinicpals of the SSH certificate. So even
though we are now capabale of displaying all allowed logins for
leaf SSH servers in the root web ui, the user is only able to use
a subset of them. To avoid any odd UX, the Proxy will filter out
any allowed logins which do not exist in the principals of the
root SSH certificate.
The above only holds for SSH, windows desktops are not as strict
and any allowed login from a leaf cluster is now visible in the
root web ui.
Fixes https://github.com/gravitational/teleport/issues/5041
