* Convert existing non-gogo codegen to the Hybrid API
Contributes to https://github.com/gravitational/teleport/issues/66776.
All existing protos explicitly set to API_OPEN have been change to
API_HBYRID. The new codegen was performed via make grpc. There are no
other functional changes to the code to start consuming the Hybrid API
those will come later. The intent is to get all Hybrid codegen in and
backported to ease the transition.
* Initial migration to the Opaque API
Contributes to https://github.com/gravitational/teleport/issues/66776.
All of the changes here are mechanical conversions generated from
`open2opaque rewrite -levels=green ./...`. There will be a follow up
to this in teleport.e which does the same. Once all changes have been
merged the process will be repeated with -levels=yellow followed by
-levels=red.
See https://protobuf.dev/reference/go/opaque-migration/ for more
details.
This is a pass on replacing modulestest.SetTestModules with supplying modules
where already possible. Some additional uses of modules.GetModules were also
replaced with localized modules injection to permit the removal of
modulestest.SetTestModules in tests. Most of the changes here are mechanical,
the one notable change is in TestSSHCommands. The test was not passing locally
because the shell (fish) builtin echo behaves differently. The easiest way
to produce expected output was to change the command invocation to use
/bin/echo to bypass any shell builtins for consistent behavior.
Inject modules into tool tests
Inject modules into integration tests
Inject modules into join server
Remove superflous enterprise modules in TestAuthenticationConfig_Parse_deviceTrustPB
Inject modules into signature algorithm config tests
Inject modules into TestGithubConnectorCRUDEventsEmitted
Inject modules into TestGetHierarchyForUser
Inject modules into kube proxy tests
Inject modules into controller tests
make fix-imports
add integration/app/fixture as test package
Part 1 or removing InsecureDevMode - enterprise still relies on it
so a few TODOs were left around for clean up later. This global is
mostly leveraged in tests to permit trust when self-signed certificates
are used. These tests are often spinning up multiple teleport instances
and most of the time exercise trusted clusters which makes them slow.
Due to the global nature of InsecureDevMode these tests cannot be
run in parallel.
To reduce the number of connections established to MongoDB servers,
Teleport Database Service instances will now cache connection pools to
MongoDB servers and reuse the connections when handling connections from
the same Teleport client.
The ability to alter the auth server clock existed
to allow tests to inject a fake clock. While this sounds
great it introduces opportunity for data races if other
components are already consuming the existing clock. For
example:
```
==================
WARNING: DATA RACE
Write at 0x00c003014720 by goroutine 168:
github.com/gravitational/teleport/lib/auth.(*Server).SetClock()
/__w/teleport.e/teleport.e/lib/auth/auth.go:2308 +0x88
github.com/gravitational/teleport/lib/client_test.newStandaloneTeleport()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:629 +0x11ad
github.com/gravitational/teleport/lib/client_test.newStandaloneTeleport()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:621 +0x1092
github.com/gravitational/teleport/lib/client_test.TestTeleportClient_Login_local.func21()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:290 +0x14e
testing.tRunner()
/opt/go/src/testing/testing.go:1934 +0x21c
testing.(*T).Run.gowrap1()
/opt/go/src/testing/testing.go:1997 +0x44
Previous read at 0x00c003014720 by goroutine 9018:
github.com/gravitational/teleport/lib/auth.(*Server).GenerateHostCerts()
/__w/teleport.e/teleport.e/lib/auth/auth.go:5127 +0x59d
github.com/gravitational/teleport/lib/auth.GenerateIdentity()
/__w/teleport.e/teleport.e/lib/auth/init.go:1606 +0x450
github.com/gravitational/teleport/lib/service.(*TeleportProcess).reRegister()
/__w/teleport.e/teleport.e/lib/service/connect.go:450 +0x25a
github.com/gravitational/teleport/lib/service.(*TeleportProcess).rotate()
/__w/teleport.e/teleport.e/lib/service/connect.go:1160 +0xaa7
github.com/gravitational/teleport/lib/service.(*TeleportProcess).syncServiceRotationState()
/__w/teleport.e/teleport.e/lib/service/connect.go:1017 +0x20b
github.com/gravitational/teleport/lib/service.(*TeleportProcess).syncRotationState()
/__w/teleport.e/teleport.e/lib/service/connect.go:996 +0x1c9
github.com/gravitational/teleport/lib/service.(*TeleportProcess).syncRotationStateAndBroadcast()
/__w/teleport.e/teleport.e/lib/service/connect.go:962 +0x4e
github.com/gravitational/teleport/lib/service.(*TeleportProcess).syncRotationStateCycle()
/__w/teleport.e/teleport.e/lib/service/connect.go:906 +0xcb
github.com/gravitational/teleport/lib/service.(*TeleportProcess).periodicSyncRotationState()
/__w/teleport.e/teleport.e/lib/service/connect.go:870 +0x5f3
github.com/gravitational/teleport/lib/service.(*TeleportProcess).periodicSyncRotationState-fm()
<autogenerated>:1 +0x33
github.com/gravitational/teleport/lib/service.(*LocalService).Serve()
/__w/teleport.e/teleport.e/lib/service/supervisor.go:605 +0x35
github.com/gravitational/teleport/lib/service.(*LocalSupervisor).serve.func1()
/__w/teleport.e/teleport.e/lib/service/supervisor.go:328 +0x4a2
Goroutine 168 (running) created at:
testing.(*T).Run()
/opt/go/src/testing/testing.go:1997 +0x9d2
github.com/gravitational/teleport/lib/client_test.TestTeleportClient_Login_local()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:285 +0x691
testing.tRunner()
/opt/go/src/testing/testing.go:1934 +0x21c
testing.(*T).Run.gowrap1()
/opt/go/src/testing/testing.go:1997 +0x44
Goroutine 9018 (running) created at:
github.com/gravitational/teleport/lib/service.(*LocalSupervisor).serve()
/__w/teleport.e/teleport.e/lib/service/supervisor.go:317 +0x10a
github.com/gravitational/teleport/lib/service.(*LocalSupervisor).Start()
/__w/teleport.e/teleport.e/lib/service/supervisor.go:360 +0x2f6
github.com/gravitational/teleport/lib/client_test.startAndWait()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:804 +0x91
github.com/gravitational/teleport/lib/client_test.startAndWait()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:802 +0x45
github.com/gravitational/teleport/lib/client_test.newStandaloneTeleport()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:621 +0x1092
github.com/gravitational/teleport/lib/client_test.TestTeleportClient_Login_local.func21()
/__w/teleport.e/teleport.e/lib/client/api_login_test.go:290 +0x14e
testing.tRunner()
/opt/go/src/testing/testing.go:1934 +0x21c
testing.(*T).Run.gowrap1()
/opt/go/src/testing/testing.go:1997 +0x44
==================
```
The setter is also redundant - Auth already exposes a way
to be constructed with a custom clock. In order to get rid
of the races once and for all, the clock setter was removed and
all tests have been updated to plumb the clock through on
construction.
This fixes a topology monitoring connection leak.
The leak was triggered by an error during server selection or the
initial auth handshake with the MongoDB server.
Moves all test related logger initialization and creation to the
logtest package to reduce testing symbols in production code. The
existing helpers in lib/utils have been left in place until the
enterprise references can be converted.
Updates #51023.
The new package is meant to house a test friendly implementation
of modules.Modules. Due to the pervasive and global nature of
modules, this package also contains a copy of of
modules.SetTestModules. In a perfect world this function wouldn't
be needed and modules would be injected where required, however,
that would take considerable effort. The existing test modules
are temporarily left in place until the enterprise tests can be
updated to use modulestest.
Converts most items to use servicecfg.Config.Logger instead of
servicecfg.Config.Log. The deprecated logrus logger unfortunately
cannot be removed yet as it's still used in e. Additionally all
logging within the config package was converted to use slog.
* disable tls client resumption
* avoid redundant verification in TLSDial
* fix wrong use of InsecureSkipVerify in HTTP auth client
* avoid overwriting GetClientCertificate in api client
* Use a getter function for CAs in TunnelAuthDialer
* Use dynamic connector data for the agent's auth client
* Make getPROXYSigner dynamic
* WIP: the thing (reloadless)
* Fix TestInstanceCertReissue
* Fix TestIntegrations/RotateFoo
* get rid of the whole reload mechanism
* Update most tests to not check for reloads
* Get rid of TestDatabaseAccess/RotateTrustedCluster
The test has been unconditionally skipped for two years now.
* avoid clobbering the auth connector
* overwrite clientState in update_servers anyway
* fix integration/hsm tests
* Document ListenForNewEvent
* fix RunWithSignalChannel godoc
* simplify ListenForNewEvents
* Remove forgotten commented code
* Explain the t.Setenv trick for non-parallelism
* explain waitingForNewEvent in integration/hsm
* Add RotateCertAuthority to gRPC TrustService.
* Add gRPC server implementation.
* Move RotateRequest type to api/types.
* Mark deprecated HTTP rotate endpoint for deletion.
* Add client implemenation and HTTP fallback.
In addition to adding server and backend handling for create and
update roles, the services.Access interface was updated to return
a role from the existing Create/UpsertRole methods. Bumps the e
ref to incorporate the associated changes needed there to prevent
breaking the build.
services.UsersService now takes a context and returns the user
from write operations as shown in the diff below. The bulk of the
changes are from modifying code to account for the additional
parameter and/or return value. Functional changes to better make
use of the new API will come in follow up PRs.
```diff
// UserGetter is responsible for getting users
type UserGetter interface {
// GetUser returns a user by name
- GetUser(user string, withSecrets bool) (types.User, error)
+ GetUser(ctx context.Context, user string, withSecrets bool) (types.User, error)
}
// UsersService is responsible for basic user management
type UsersService interface {
UserGetter
// CreateUser creates user, only if the user entry does not exist
- CreateUser(user types.User) error
+ CreateUser(ctx context.Context, user types.User) (types.User, error)
// UpdateUser updates an existing user.
- UpdateUser(ctx context.Context, user types.User) error
+ UpdateUser(ctx context.Context, user types.User) (types.User, error)
// UpdateAndSwapUser reads an existing user, runs `fn` against it and writes
// the result to storage. Return `false` from `fn` to avoid storage changes.
// Roughly equivalent to [GetUser] followed by [CompareAndSwapUser].
// Returns the storage user.
UpdateAndSwapUser(ctx context.Context, user string, withSecrets bool, fn func(types.User) (changed bool, err error)) (types.User, error)
// UpsertUser updates parameters about user
- UpsertUser(user types.User) error
+ UpsertUser(ctx context.Context, user types.User) (types.User, error)
// CompareAndSwapUser updates an existing user, but fails if the user does
// not match an expected backend value.
CompareAndSwapUser(ctx context.Context, new, existing types.User) error
// DeleteUser deletes a user with all the keys from the backend
DeleteUser(ctx context.Context, user string) error
// GetUsers returns a list of users registered with the local auth server
- GetUsers(withSecrets bool) ([]types.User, error)
+ GetUsers(ctx context.Context, withSecrets bool) ([]types.User, error)
// DeleteAllUsers deletes all users
- DeleteAllUsers() error
+ DeleteAllUsers(ctx context.Context) error
}
```
Depends on gravitational/teleport.e#2346
Implements step 3 of #32949
This PR removes `GetAppSessions` functions in favor of
`ListAppSessions`.
It also removed all the deprecated fields in the `DatabaseServerSpecV3`
message.
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
Adds `UpsertCertAuthority` to the `teleport.v1.trust.TrustService`
and deprecates the HTTP api. The `services.Trust` interface was
also updated to include a `context.Context` in both
`CreateCertAuthority` and `UpsertCertAuthority.
To ensure backwards compatibility the `auth.Client` will first
attempt to use the gRPC client and if that fails with a not
implemented error it will fallback to using HTTP.
Part of #6394
* Add CreatedBy and CreatedOn fields to locks.
* Move the setting of the CreatedBy and CreatedOn for locks to the server instead of the client.
* Add tests for the tctl resource lock import and for the tctl lock create command.
* Use clockwork for time in lock creation tests.
* Make fake clock optional and use the golang 0 date.
* Change CreatedOn to CreatedAt.
* Allow a custom time to be passed through as a fake clock.
* Actually accept a fake clock with useFakeClock.
* Pass Clock through as a configuration value when initializing Teleport for the tests.
* Test cleanup
* Set CreatedBy and CreatedAt values in test mocks.
* Fix failing test from enabling clock passthrough to auth.
* Move configuration from lib/service to lib/service/servicecfg
The new servicecfg package will hold only configuration for services.
This will allow other packages (like tctl and tsh) to depend on
servicecfg without pulling in all of lib/service (which has a number
of platform-specific details).
This is the first step towards being able to build tctl for Windows.
* Move PAM and BPF config into servicecfg
This breaks a compile-time dependency on BPF/PAM for tctl.
* Start removing generateToken
* Refactor `tokens add` to use `CreateToken` RPC
* Correct deprecation version
* Remove calls to GenerateToken
* Start tests for token crud
* Allow user role rules to be provided wth CreateUserAndRole helper
* Add TestCreateToken to tls_test
* Add TestUpsertToken
* Add GetTokens test
* Add TestGetToken
* Add test for DeleteToken RPC
* Dedupe error assertions
* Add support for emitting trusted cluster token creation events to Create/UpsertToken
* Fix imports
* Change parameter order to appease linter
* Fix broken token expiry test
* Fix token expiry time for TestAuth_RegisterUsingToken
* Remove unnecessary cleanup
* Apply suggestions from code review
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Change test comment from deprecation notice
* `make gprc` after code review changes
* Update lib/auth/tls_test.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update lib/auth/tls_test.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update lib/auth/tls_test.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Update lib/auth/tls_test.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Rename tests
* Update lib/auth/tls_test.go
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* More test cases for GetTokens
* Improve message on duplicate token
---------
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Add RBAC labels for Databas Services access
This PR adds a new set of Labels that controls access to the
DatabaseService resource.
We changed the `RoleSet.checkAccess` to allow for checks in Resources
without Labels.
However adding complexity to this function is something that can bit us
in the future.
So, this PR reverts that change and adds a new set of labels.
Those labels will control whether the RoleSet has access to
DatabaseServices.
Given that this is a new condition that we add to the default `editor`
role, we had to create a migration/backfill for the roles that don't
have them.
This allow us to migrate the roles created previously and add this new
allow condition.
* remove preset for remote proxy and database labels
* Remove db_service_labels wildcard from editor preset role
* Fix early exit logic
This PR includes a new Role resource version that is compatible with V5 spec.
The new resource introduces the `kubernetes_resources` definition that allows operators to limit the Kubernetes resources that each member can access. The `kubernetes_resources` entries must follow the following format: `{"kind":"<kind>", "namespace":"<namespace>","name":"<pod>"}`. Currently, it only supports objects of `kind` `pod`. Valid examples `<namespace>/<name>:
- `*/*`: matches all pods in all namespaces.
- `default/*`: matches all pods in the `default` namespace.
- `*/nginx-*`: matches every pod prefixed with `nginx-` in every namespace.
For older resource versions - V5, V4, V3 - `kubernetes_resources` is automatically populated with `{"kind":"pod","namespace":"*","name":"*"}` to keep compatibility. For the newest version, it's mandatory to define its value otherwise access to pods will be denied.
Part of #18434
n the context of Teleport Discover, we must be able to known if there's any DatabaseService available to proxy a given Database resource.
If there's none available, we will offer a script for the user to run and install a DatabaseService which proxies the desired Database resource.
By DatabaseService, we mean the process that Teleport binary manages when the `teleport.yaml` config has the following section:
```yaml
db_service:
enabled: "yes"
```
To accomplish this we are creating a new resource: DatabaseService.
The UI will fetch all DatabaseServices and check if there's any ResourceMatcher that matches the DatabaseLabels.
Previous PRs created the DatabaseService resource and its CRUD methods.
This PR adds an heartbeat for DatabaseServices similar to what we have for Databases.
There's also a new command to list DatabaseServices using `tctl`:
```
$ tctl get db_service --format text
Name Resource Matchers
------------------------------------ --------------------------------------
a6065ee9-d5ee-4555-8d47-94a78625277b (Labels: <all databases>)
d4e13f2b-0a55-4e0a-b363-bacfb1a11294 (Labels: env=[prod],aws-tag=[xyz abc])
```
Test using Teleport Cloud
```
dinis@lenix ~/p/cloudagents> tctl get db_services
kind: db_service
metadata:
expires: "2022-12-21T18:05:10Z"
id: 1671645310983808522
name: 2a28d394-900c-42ea-a120-eed918e4526b
spec:
resources:
- labels:
aws-tag:
- xyz
- abc
env: prod
version: v1
dinis@lenix ~/p/cloudagents> tctl status
Cluster marcoacme.cloud.gravitational.io
Version 12.0.0-dev
```
Part of #19032
Related #19363#19469
While working on integration tests, I noticed that testOptions.leafConfig
is simply ignored because after we set up the leaf cluster config, we call
testOptions.rootConfig instead.
Making all of our integration tests run in entirely parallel requires
a large engineering effort to enforce test isolation and remove all race
conditions between tests.
A lower-effort alternative may be to split apart the various test suites
into their own Go packages, and test those packages in parallel, even if
the tests inside are still executed serially. Auditing the test suites
for races on system-level resources (e.g. files, ports) is much easier
than chasing down every p[ossible race in the testing system.
This patch acts as a trial run, breaking a fairly well-defined and
self-contained test suite out into its own package. Note that the goal of
this change is not necessarily to shave minutes off the build (although
that would be nice), but to act as an illustration of how other, less
well-formed test suites might be broken apart.
See-Also: #12421
See-Also: #14408
rosstimothy