Adds an additional type constraint go FnCacheGet and FnCacheGetWithTTL
to enforce that all keys are comparable. This should allow the
removal of various compile time hashability checks that are in
place for existing keys of custom types.
* Support automatic_approval in AMR spec
* Support filter by automatic_approval_name
* Update AMR validation
* Update terraform schema
* Fix import
* Add additional test case
* Improve amr spec godocs
* make -C integrations/terraform docs
Eliminates the namespace CRUD API as part of
https://github.com/gravitational/teleport/issues/49509.
The most notable change introduce here is with flushCache in the
auth helpers. It was ironically using namespaces because they
were deprecated, however, it has since been updated to use a role
instead. The helpers.go file was also updated not to rely on testify
in attempt to prevent it from being included in production builds.
Updates https://github.com/gravitational/teleport/issues/6394.
* GitHub proxy part 6: proxing Git using SSH transport
* better command parsing and update suite
* refactor
* revert unnecearrty files
* address review comments
* ut fix
* revert localsite_test.go
* change special suffix to teleport-github-org for routing
* fix routing ut
* minor typo edit
* fix ut after sshca change
* add UT to sshutils
* minor review comments
* fix api ut because of special suffix change
* GitServerReadOnlyClient
* downgrade error to warning
* run go mod tidy. not sure why it's needed
* rename mock.go to mock_test.go
* Use math/rand/v2's thread-local RNG for jitters
* Replace NewJitter() with DefaultJitter
* Replace NewFullJitter() with FullJitter
* Replace NewHalfJitter() with HalfJitter
* Replace NewSeventhJitter() with SeventhJitter
* Inline jitters in lib/utils/diagnostics/latency
* Replace var jitters in lib/utils with proxy functions
* Replace utils.HalfJitter with retryutils.HalfJitter
* Replace utils.FullJitter with retryutils.FullJitter
* Replace utils.SeventhJitter with retryutils.SeventhJitter
* Replace sharded jitters with thread-local ones
* Remove NewSeventhJitter reference in integrations
- Recursively check for accesslist membership
- Allow adding/removing/listing included access lists in acl commands
- Add a recursive test
- Use dynamic access lists structure from RFD
- Resolve proto changes
- Exclude 'list' members from Access List memberCount
- Calc Access List member count with members of type 'list' excluded,
return seperately to front end
- Update examples/integrations
- Update crd docs
- Update tf docs
- Perform calculation of inherited roles/traits to AccessList service in
order to utilize cache and minimize number of requests.
- Grant Okta integration RO for Access Lists
- Update AccessListMember-* events
- Include count for inherited grants
- Include MembershipKind of affected member(s)
- Emit inherited grants / members' MembershipKind for AccessListMember-*
events
- Update notified owners for Access Requests
- Ensure dynamic owners are notified for Access Requests
- Ensure dynamic owners are notified via Slack integration
- Optionally pass an AbortSignal to `fetchAccessLists` in Web UI
- Replace usages of `services.IsAccessListOwner/IsAccessListMember` with
equivelant funcs from `Hierarchy`
- Remove final references to AccessListMembershipChecker
- Don't allow ACL deletion when member/owner in other lists
- Guard Access List deletion behind membership/ownership checks for List
- Expose Hierarchy func to recursively get all members
- Tidy UserLoginStateGenerator logic involving ACL Membership/Ownership
Co-authored-by: Maxim Dietz <maxim.dietz@goteleport.com>
* Adds caching support for Identity Center resources
Adds support for caching (and watching) some IdentityCenter resource types.
Specifically:
- identitycenterv1.Account
- identitycenterv1.AccountAsssignment
- identitycenterv1.PrincipalAssignment
Also breaks the cache's `genericcollection` out into its own file as an attempt
to reduce clutter in the main collections source file.
* Update identitycenter_events.go
* Update identitycenter_events.go
* Speed up tests
* SCIM state record storage service
* Address review feedback
* Add page size option to listings
* reduce boilerplate
* It's not a misspelling, it's a perfectly valid regional variant.
* Address review feedback
This PR adds a new method that allows users to list UserTasks by
Integration.
Given this is our main use case when showing the UserTasks in the UI
(we'll be always showing UserTasks for a given integration), we created
a dedicated method in the gRPC.
We'll probably add more filters but they will always include the
integration (at least for the uses cases we have currently).
This PR adds the implementation for the User Tasks:
- services (backend+cache)
- clients (API + tctl)
- light validation to set up the path for later PRs
* Add AutoUpdate Client/Cache implementation
* CR changes
* Add permission for proxy to access resources
* Rename all occurrences auto update to camelcase
* Remove auto update client wrapper
* Drop AutoUpdateServiceClient helper
Rename comments for consistency
* Start adding `lib/services` content for SPIFFEFederation type
* Add SPIFFEFederation resource to cache
* Wire SPIFFEFederation into cache
* Fix NewTestAuthServer
* Start writing tests
* Add more test cases to validation
* Add tests to cache for SPIFFEFederation
* More test coverage
* Add test for DeleteSPIFFEFederations
* Finish off tests for SPIFFEFederation resource
* go mod tidy
* Go mod tidy
* Appease linter
* Avoid forcing kind/version in MarshalSPIFFEFederation
* more linter appeasal
This groups the trusted cluster and cert authority storage into
the same service. While on its own this doesn't change much, it
unlocks the ability to fix racy behavior described in
https://github.com/gravitational/teleport/issues/36400.
* Add initial condition parser for access monitoring rules
* Update integrations/access/accessrequest/app.go
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Check previously unchecked error and minor refactor of AMR
* Simplify check for applicable access monitoring rules
* Refactor access monitoring rules plugin integration
* Fix formating and move lock aquisition
* Add methods for listing access monitoring rules with a filter
* Add contains_any predicate expression func
* Add in is_empty func to predicate expression
* Lock AMR cache in plugins while getting initial rules
* Add in check for access monitoring rule version
* Update integrations/access/accessrequest/app.go
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
* Update integrations/access/accessrequest/app.go
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
* Move lock so it doesnt persist over api calls
* Remove unused constant and add more context to logs
* Appease linter
* Update access monitoring rules tests to pass rule validation
* Add in missing access monitoring rules list with filter code
* Appease linter
* Add back validation code for AMRs
* Fix test plugin role and rename listaccessmonitoringrulewithfilter
* Fix local test for AMR crud operations
* Fix end range for listing rules
* Fix unwrapping of resource153 event for monitoring rules
* Refactor AMR cache init into helper function in plugin app
* Add seperate response type for listAccessMonitoringRulesWithfilter
* Add context to log for plugins failing to fetch recipients
* Grab access monitoring rules cache under lock all at once
* Add clarification for which fields are optional in listAMRfilter req
* Update integrations/access/accessrequest/app.go
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Update integrations/access/accessrequest/app.go
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Add forEach to common recipient set
* Move type check to after AMR event op switch
* Move turn some default parser spec methods to funcs
* Make some predicate func usable as methods as well
* Add len func to common recipient sets
* Add integration test for access monitoring rule and slack plugin
* Fix error types and messages when handling AMRs
* Use generic list resource with filter for AMR
* Add test for generic listResourceWithFilter
* Update listResourceWithFilter to use revision instead of id
* Update generic tests to use revision instead of id
* Fix linting
---------
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
* Update labels datatype in Teleport crownjewel proto
This commit modifies the labels from a map to repeated TeleportLabel in CrownJewel proto, and accordingly updates the getter function in proto generated code. The newly introduced TeleportLabel type contains a Key and repeated values for improved flexibility.
Clean up
Update and restructure CrownJewel service proto
The code modifications include renaming and restructuring several types and altering various functions in the CrownJewel service proto. Major changes involve renaming DeleteCrownJewel
Rename 'kind' to 'kinds' in crownjewel API
This commit changes the 'kind' field to 'kinds' in the crownjewel API to better reflect its purpose of holding multiple resource types. Changes are made across several files - 'crownjewel.go', 'crownjewel.pb.go' and 'crownjewel.proto'.
After rebase fixes
Remove CrownJewel type from several files
This commit represents the removal of the CrownJewel type from several files. CrownJewel type references are deleted in certain Go files, and imports of CrownJewel in multiple files are being adjusted. The specifics of why this type is removed is not evident from the diffs, but the deletions will affect the related functionalities.
Add CrownJewel update functionality
This update enables the 'UpdateCrownJewel' functionality. If a 'CrownJewel' resource already exists, it can now be updated instead of returning an error. This enhances flexibility when dealing with resource management. It also introduces the ARN field into the 'AWSMatcher' service.
Update CrownJewel service with pagination and improve matchers
This update adds pagination to the CrownJewel service for more efficient data handling. It also improves the matching capabilities with the inclusion of Teleport and AWS matchers and allows for selective querying with these matchers. Additionally, it includes various changes to structure and naming for better clarity and ease of use.
Refactor CrownJewel protos to use ResourceHeader
This refactoring replaces the previous structure of CrownJewel proto's fields Kind, Sub_kind, Version, and Metadata with a single object called Header (ResourceHeader). It simplifies the code structure and harmonizes it with other protos. Now, Header contains the essential attributes that were previously distributed across multiple separate fields.
Add GetCrownJewels method to cache and accessgraph
The function GetCrownJewels has been added to the cache and the accessgraph. It retrieves all CrownJewel instances from the cache. Additionally, CrownJewel retrieval functionality has been removed from auth server and existing usages have been refactored.
Implement Crown Jewel creation feature
Updated the method `createCrownJewel` in the resources command to handle Crown Jewel creation. Introduced a new gRPC service `crownjewelv1` to handle client-server interactions for Crown Jewel objects. Crown Jewel expiry configuration has now also been enabled.
Add missing files
I have no words...
Update authentication and event handling
Refactored the argument from 'auth.ClientI' to '*auth.Client' in 'createCrownJewel' function in 'resource_command.go'. Removed unused import 'durationpb' in 'auth_with_roles.go'. Added a new event code 'AccessGraphAccessPathChangedCode' in 'codes.go' and 'AccessGraphAccessPathChanged' in 'api.go' for better monitoring and logging of access path changes.
Add access path changed event and relevant changes.
update api
add grpc
update api
add cache support
add protos
* Fixes after rebase
* Refactor Crown Jewel resource methods
Updated API methods related to the Crown Jewel resource to improve clarity and consistency. Renamed `GetCrownJewels` to `ListCrownJewels`, added pagination parameters, and ensured uniform handling of the Crown Jewel resource across different parts of the codebase. Updated relevant comments for better code understanding.
* Removed CrownJewel converter, updated CrownJewel references
This commit removes the CrownJewel converter and updates references from the old CrownJewel type to the new protobuf-defined CrownJewel type, streamlining the codebase. The commit also disables CrownJewel deletion since it is not supported yet. Some minor errors were fixed regarding CrownJewel resource names.
* Implement Crown Jewel deletion functionality
The crown jewel deletion feature has been enabled by uncommenting and simplifying the code block. Now, the delete action is directly performed on the specific 'crown jewel' mentioned based on its name. The previously unsupported action returns an informative success message after the deletion.
* Refactored function 'allCombinations' and added authorization checks
Moved 'allCombinations' function into 'utils' package, renamed it to 'Combinations', and updated its references in 'dbobjectimportrulev1' and 'dbobjectv1' tests. Additionally, introduced admin authorization checks to 'CreateCrownJewels', 'UpdateCrownJewels', and 'DeleteCrownJewels'. Created corresponding tests for these changes.
* Add CrownJewel service to cache tests
The CrownJewel service has been added to the cache test file. The service creation is implemented and assigned to new fields in multiple methods within the cache tests. A new test function `TestCrownJewel` has also been added to validate CRUD operations for the service.
* Add access path changed event and update Crown Jewel tests
Added a new event code for when an access path changes in the access graph, and introduced new API calls to handle these situations. Also, updated crown jewel related tests and fixed copyright year in several files.
* Update protobuf import path
The protobuf import path was updated from "github.com/golang/protobuf/proto" to "google.golang.org/protobuf/proto" to ensure correct use and to prevent potential issues, while maintaining functionality in crown_jewels.go.
* Refactor CrownJewel and AccessGraph services
This commit optimizes the ListCrownJewelsResponse in the CrownJewel service by directly assigning the response instead of using an extra loop. Additionally, the EventsStream in the AccessGraph service and related tests have been updated to EventsStreamV2, reflecting the implementation of new versions of these methods.
* Add CrownJewel resource in access controls and tests
In both the cache test and access control files, CrownJewel resource has been included. This provides the system's ability to manage and test CrownJewel resources. Additionally, it allows the handling of CrownJewel resources in client events.
* Simplify implementation
* Update copyright dates and add license headers
Updated the copyright dates in several files to reflect the new year. Additionally, license headers have been added to some files where they were previously missing, ensuring the proper copyrights and GPL notice are visible.
* Refactor CrownJewel codebase for better readability
Fixed various typographical errors in CrownJewel's related functions and removed a TODO comment. Improved function and function calls' names that were incorrectly referring to different modules, making the code easier to read and understand across different CrownJewel services.
* Add validation for Teleport and AWS matchers in service
The validation has been added for Teleport and AWS matchers in the service layer to ensure that essential details are not missed when setting matchers. This includes checks to ensure that kinds and labels for Teleport, and types and tags for AWS matchers are properly set.
* gci
* Add CrownJewel update functionality and refactor related services
This commit adds the ability to update a CrownJewel resource within the ResourceCommand. It also includes refactoring of related services, condensed code logic, and improved utilization of existing methods for marshaling/unmarshaling CrownJewel. Specific changes in service.go and crown_jewels.go files have reduced code redundancy and enhanced the code structure.
* Relocate and update crown_jewels_test file
This commit moves the crown_jewels_test from the local to the services library and adds test cases for unmarshalling Crown Jewels data. Also, import statements for labelv1 and utils were introduced to support these changes.
* Update crown jewels test with AWS matchers
Revised the crown jewels test file by including AWS matchers section in the CrownJewel object. The change ensures to specifically match AWS infrastructure following certain parameters, thus enhancing the specificity and flexibility of our matchers. Apart from this, minor reordering of objects has also been done for better code organization.
* Added upsertCrownJewel function and related proto message
The commit adds the UpsertCrownJewel function in the service.go file, which enables upsert operations on crown jewel resources. Alongside, the UpsertCrownJewelRequest proto message is added to the crownjewel_service.pb.go file to handle such requests.
* Refactor CrownJewelsClient to use upsert
This change refactors the CrownJewelsClient to use the upsert method when the force flag is on. The code has been streamlined by eliminating the need for 'already exist' error checking and separate create and update methods.
* Add GetCrownJewel functionality and refactor validation
A GetCrownJewel function has been added to the Service, which retrieves a specific CrownJewel resource. Also, the CrownJewel validation process has been refactored, moving the validateCrownJewel function from service.go into the crownjewel library and renaming it to ValidateCrownJewel. These changes improve modularity and code organization.
* Add VerbCreate check and test for CrownJewel validation
The checkAccessToKind function now also checks for the VerbCreate permission. Additionally, a new test file, object_test.go, was added to validate the functionality of the CrownJewel object. This includes various validation checks such as "NilCrownJewel", "ValidCrownJewel", "MissingMatchers", among others.
* Refactor CrownJewel resource handling and validation
The codebase was updated to improve CrownJewel resources handling. This includes modifications to list, get, create, update, upsert, and delete functions in several files. Additionally, a validation function has been implemented for CrownJewel objects, ensuring required fields are present. Depreciated logger service has also been removed.
* Add cache implementation to CrownJewel service
The CrownJewel service has been enhanced by integrating it with cache. The main changes include adding "Cache" as part of ServiceConfig and using cache to fetch crown jewels instead of directly using the backend services. The cache interface with the methods "ListCrownJewels" and "GetCrownJewel" has been added to handle these operations.
* Add Cache to service configuration in crownjewelv1 tests
In this commit, we added the Cache field to the service configuration used in the tests for the crownjewelv1 service. This ensures that the service configuration used in the tests matches the expected real-world configuration.
* Update CrownJewelsService to use ConditionalUpdateResource
This commit changes the CrownJewelsService's `UpdateCrownJewel` method to use the `ConditionalUpdateResource` method instead of the `UpdateResource` method. This change will provide a more appropriate update strategy for the specific needs of the CrownJewelsService.
* Replace 'Cache' with 'Reader' in crownjewel service & remove DeleteAllCrownJewelsRequest
The 'Cache' in crownjewel service has been replaced with 'Reader' to improve code readability and to match with its functionality. Also, the DeleteAllCrownJewelsRequest functionality was removed as it posed a high risk potential for data loss and it wasn
* Update tests for Crown Jewel service
This commit updates the test suite of the Crown Jewel service. An object is fetched from the backend before updating its expiry time, ensuring the revision is populated. In addition, a new test is introduced to handle the scenario where the revision is missing when updating a Crown Jewel.
* Refactor protobuf marshaling with utils
This commit replaces the usage of the `protojson` method for marshaling and unmarshaling protobuf objects in `crown_jewel.go` with the `FastMarshal` and `FastUnmarshal` methods from `utils`.
protojson is failing to unmarshal resources.
* Update CrownJewel test and serialization methods
Updated the test condition in crown_jewels_test.go to use Proto.Equal for object comparisons. Also, swapped out the FastMarshal and FastUnmarshal methods with protojson's Marshal and Unmarshal in the CrownJewels service.
* Add missing license
---------
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
* fix: prevent deleting AWS OIDC integration used by EAS
The External Audit Storage feature for Teleport Cloud depends on an AWS
OIDC integration to exist in order to get credentials for the customer
AWS account.
It's currently possible for a customer to delete the AWS OIDC
integration referenced by the EAS configuration, leading to the EAS
integration breaking and the Auth service crashing the next time it's
started up.
This commit prevents deleting an AWS OIDC integration currently
referenced by and EAS integration, and prevents creating an EAS
integration referencing an unknown AWS OIDC integration.
* use PreDeleteCheck instead of calling EAS service
* reorder tests and add missing cleanup
* create integrations svc on the fly
* switch to AtomicWrite
* set revision of returned resources
* deprecate gRPC DeleteAllIntegrations and use more general cacheMode
* Don't watch web sessions with secrets by default; Deny users from watching web session secrets.
* Add backwards compatibility logic for old proxies
Fallback to web session permissions for users watching with secrets
Fix uncaught merge conflict
* CopyWithoutSecrets without proto.Clone to avoid gogoproto issue.
* Revert WebSession.CopyWithoutSecrets rename and instead add special case in fanoutv2.go since WebSessions can not implement ResourceWithSecrets without affecting upstream consumers negatively.
* Address slice iteration comment.
* Add conditional update methods for `AccessList` and `AccessListMembers`
This PR adds two new methods for conditinally update an AccessList and
AccessListMember resources to avoid overriding changed resources.
This is a preparation for a ineligible status reconciler.
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
* Update lib/services/simple/access_list.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* handle review comments and drop copy methods
* simplify code
---------
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
The existing resource parser was converted to use typical under
the hood. Aside from the type safety improvements the making the
switch allows a single predicate expression to be parsed once and
evaluated per resource. The previous parser worked backwards, it
created a parser per resource with the same expression which lead
to elevated memory consumption and poor performance.
An additional bug was fixed that prevented predicate filtering from
working properly with the unified resource API due to applying
predicates prior to filtering by kind. This resulted in queriers
for server specific things (resource.spec.hostname) to fail even
when only the server resource was specified in the resource kinds.
Benchamrk results for the changes in this commit compared to master
show a 75% reduction in the test time and a 96% reduction in allocations
during the tests.
```bash
$ benchstat e77e5e080b1848da830a9030d43c38be55a5b8d5.txt 8eff752a93dfe96225a71431af829015e83400fa.txt
goos: darwin
goarch: arm64
pkg: github.com/gravitational/teleport/lib/auth
│ e77e5e080b1848da830a9030d43c38be55a5b8d5.txt │ 8eff752a93dfe96225a71431af829015e83400fa.txt │
│ sec/op │ sec/op vs base │
ListUnifiedResourcesFilter/labels-12 73.88m ± 1% 64.74m ± 6% -12.37% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_path-12 294.06m ± 20% 74.83m ± 5% -74.55% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_index-12 307.63m ± 4% 76.32m ± 4% -75.19% (p=0.000 n=10)
ListUnifiedResourcesFilter/search_lower-12 130.7m ± 5% 130.5m ± 4% ~ (p=0.971 n=10)
ListUnifiedResourcesFilter/search_upper-12 147.4m ± 5% 138.7m ± 2% -5.87% (p=0.003 n=10)
ListUnifiedResources/simple_labels-12 1.748 ± 2% 1.744 ± 2% ~ (p=0.631 n=10)
geomean 246.6m 150.6m -38.93%
│ e77e5e080b1848da830a9030d43c38be55a5b8d5.txt │ 8eff752a93dfe96225a71431af829015e83400fa.txt │
│ B/op │ B/op vs base │
ListUnifiedResourcesFilter/labels-12 735.2Ki ± 0% 724.6Ki ± 1% -1.44% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_path-12 238.873Mi ± 0% 7.585Mi ± 0% -96.82% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_index-12 238.869Mi ± 0% 7.587Mi ± 0% -96.82% (p=0.000 n=10)
ListUnifiedResourcesFilter/search_lower-12 16.77Mi ± 0% 16.77Mi ± 0% ~ (p=0.436 n=10)
ListUnifiedResourcesFilter/search_upper-12 16.78Mi ± 0% 16.76Mi ± 0% ~ (p=0.052 n=10)
ListUnifiedResources/simple_labels-12 1.272Gi ± 0% 1.273Gi ± 0% ~ (p=0.739 n=10)
geomean 49.67Mi 15.69Mi -68.41%
│ e77e5e080b1848da830a9030d43c38be55a5b8d5.txt │ 8eff752a93dfe96225a71431af829015e83400fa.txt │
│ allocs/op │ allocs/op vs base │
ListUnifiedResourcesFilter/labels-12 11.45k ± 0% 11.41k ± 0% -0.37% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_path-12 4962.0k ± 0% 161.5k ± 0% -96.75% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_index-12 4812.0k ± 0% 161.5k ± 0% -96.64% (p=0.000 n=10)
ListUnifiedResourcesFilter/search_lower-12 161.6k ± 0% 161.6k ± 0% ~ (p=0.419 n=10)
ListUnifiedResourcesFilter/search_upper-12 161.7k ± 0% 161.6k ± 0% ~ (p=0.116 n=10)
ListUnifiedResources/simple_labels-12 23.70M ± 0% 23.70M ± 0% +0.01% (p=0.019 n=10)
geomean 743.7k 238.5k -67.93%
```
* add kubernetes waiting container backend service and API
* remove other resource type
* add unit test
* fix after rebasing
* address feedback
* remove KubeWaitingContainerRequest
* check for nil in validation func
* remove UnwrapResource153
* fix test
* address feedback
* address linter issues
* fix cache watch kind test
* Remove logging from circuit breaker
The logging is never output, and if it was would be very verbose.
Instead of dropping the level down to trace the logger is removed
entirely.
* Add constants from trace library
The logging functionality in trace is being removed in an upcoming
release. To prepare the logging component keys are copied from trace
into teleport.
* Convert trace.ComponentFields to teleport.ComponentFields
* Replace trace.Component with teleport.ComponentKey
* Replace trace logging constants
Gavin Frazar