A few of our tests that run certificate rotation fail with an error similar to this:
```
can only switch to phase update_clients from init, the current phase is
```
This happens when a duplicated private key is returned and set as a new key during CA rotation. During the CA rotation, we look for duplicated keys https://github.com/gravitational/teleport/blob/962e5a25bc56b5aad05528867182a43cd53cb551/lib/auth/rotate.go#L218 and try to rotate all of them at one. Because some of our tests use still use `testauthority` that only has 4 keys there is a huge chance that the same key will be returned twice, which causes the issue.
I swapped the logic to use a bigger pool with keys introduced in https://github.com/gravitational/teleport/pull/18750. I also removed the randomization part that, after some testing, I discovered was also causing some problems.
We could probably refactor the code in many tests now to make it simpler, but I want to keep this change as simple as possible. Otherwise, backporting bigger changes to older branches takes a long time.
Many of our tests (db package, I'm looking at you) generate many RSA keys. This has two main side effects; makes our tests slow and flaky as CPU usage spikes in random moments when the tests are run in parallel.
This change pre-generates RSA keys at the beginning of each test module and reuse them in randomized order to reduce the situation that one key has been used multiple times in one test.
I had to move a few files to avoid circular dependencies.
* Add Yubikey PrivateKey implementation for use by Teleport clients.
- Add yubikey login logic, reusing previously stored private keys.
- Fix identity file decoding with PIV keys, which sign ecdsa certificates.
- Add libpcsclite-dev pre-req for building on linux.
- Remove unnecessary keys.Signer interface and move its functionality to keys.PrivateKey.
- Move retry and jitter utils to new api/utils/retryutils package.
Update metalinter, fix a few lint warnings and replace deprecated linters.
`deadcode`, `structcheck` and `varcheck` are abandoned and now replaced by [`unused`][1].
Since 1.19, `go fmt` reformats godocs according to https://go.dev/doc/comment. I've done a bulk-reformatting of the codebase to keep the linter happy. Backporting is mostly harmless (the exception being `lib/services/role_test.go`, that for some reason breaks the _old_ linter using the new format).
[1]: https://golangci-lint.run/usage/linters/
* Bump golangci-lint version
* Replace abandoned linters
* Fix bodyclose on lib/auth/github.com
* Fix bodyclose on lib/kube/proxy/streamproto/proto_test.go
* Fix bodyclose on lib/srv/alpnproxy/proxy_test.go
* Fix bodyclose on lib/web/conn_upgrade_test.go
* Silence staticcheck on lib/kube/proxy/forwarder_test.go
* Silence staticcheck on lib/utils/certs_test.go
* Address BuildNameToCertificate deprecation warnings
* Run `go fmt ./...`
* Run `go fmt ./...` on api/
* Ignore formatting in role_test.go
* Remove redundant initializers in lib/srv/uacc/
* Update e/
This PR implements the SSH Tester for ConnectionDiagnostic feature.
This feature is also known as Test Connection, part of Teleport
Discover.
The goal here is to provide immediante feedback about a newly added
resource. Can the user connect to it?
We are targetting SSH Nodes as a first ResourceKind.
To test the access to an SSH Node we require the ResourceName and a
login username (ssh principal).
Then a series of checks will occur in two places:
- SSH client in the Web server
- SSH server in the SSH agent
The ssh client creates a new Connection Diagnostic with some initial
state.
Then it tries to build up the necessary SSH config
This already gives us a couple of things to check for:
- does the node exist and does the current user (inherited from
websession) can access it?
- is the node accepting TCP connections (in the specific port)?
- is the node accepting SSH protocol on top of the TCP connection?
Then, the ConnectionDiagnosticID is inject into the certificate and the
SSH Server receives it and will also Append traces into it:
- is the requested principal allowed for the current user?
- does the requested principal exist in the target node?
This is not an exhaustive list of checks.
For a complete list of which checks are verified please see the
TestDiagnoseSSHConnection test.
After all those checks, it returns if the Connection was successful and
what all of the traces generated along the way.
Demo:
Primary Changes:
- Remove reliance on Private Key PEM:
- Update native and keygen packages to return PrivateKey instead of PEM key
- Add new PrivateKey interface which implements crypto.Signer
- Replace PEM encoded private key usage where possible
- Replace calls to tls.(Load)X509KeyPair with keys.(Load)X509KeyPair in
client packages
Minor Changes:
- Remove unused agent.AddedKey return from LoadKey
- Simplify sshutils and removed unused code paths
- Add ecdsa and ed25519 key support
This change adds IP-based validation for SSH certificates.
There's new option in role definition:
kind: role
metadata:
name: dev
spec:
options:
pin_source_ip: true
When that is set to true client IP must be the same when generating certificates and using them. It uses source_address critical option that should be supported by both teleport and sshd and only applies to certificates we send to user (like in tsh login), we don't pin IP in certificates issued for web UI as they can't leak.
This change also omits machine ID (it uses different code path) - it will be added in separate PR.
Most of the lines changed are from regenerating types.proto, change itself is not that big
Relates #11719
This commit upgrades the version of x/crypto we use, to the current latest
`go get -u golang.org/x/crypto`
We also replaced the deprecated variables and updated the tests to match the
current default KEX Algos
The x/crypto didn't support RSA-SHA2 algos, so we developed our own algorithm
signer. This is no longer the case, and after upgrading x/crypto to 20220518 we
can safely remove the custom code we have.
From OpenSSH 8.8+, it works if we explicitly add the older algorithm
Somthing like this: `./ssh -vvv -oPubkeyAcceptedAlgorithms=+ssh-rsa-cert-v01@openssh.comteleportadmin@moon.marco.mydemo`
* Add certificate renewal bot
This adds a new `tbot` tool to continuously renew a set of
certificates after registering with a Teleport cluster using a
similar process to standard node joining.
This makes some modifications to user certificate generation to allow
for certificates that can be renewed beyond their original TTL, and
exposes new gRPC endpoints:
* `CreateBotJoinToken` creates a join token for a bot user
* `GenerateInitialRenewableUserCerts` exchanges a token for a set of
certificates with a new `renewable` flag set
A new `tctl` command, `tctl bots add`, creates a bot user and calls
`CreateBotJoinToken` to issue a token. A bot instance can then be
started using a provided command.
* Cert bot refactoring pass
* Use role requests to split renewable certs from end-user certs
* Add bot configuration file
* Use `teleport.dev/bot` label
* Remove `impersonator` flag on initial bot certs
* Remove unnecessary `renew` package
* Misc other cleanup
* Do not pass through `renewable` flag when role requests are set
This adds additional restrictions on when a certificate's `renewable`
flag is carried over to a new certificate. In particular, it now also
denies the flag when either role requests are present, or the
`disallowReissue` flag has been previously set.
In practice `disallow-reissue` would have prevented any undesired
behavior but this improves consistency and resolves a TODO.
* Various tbot UX improvements; render SSH config
* Fully flesh out config template rendering
* Fix rendering for SSH configuration templates
* Added `String()` impls for destination types
* Improve certificate renewal logging; show more detail
* Properly fall back to default (all) roles
* Add mode hints for files
* Add/update copyright headers
* Add stubs for tbot init and watch commands
* Add gRPC endpoints for managing bots
* Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints
* Replace `tctl bot (add|rm|ls)` implementations with gRPC calls
* Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`,
`BotGenerationLabel`
* Fix outdated destination flag in example tbot command
* Bugfix pass for demo
* Fixed a few nil pointer derefs when using config from CLI args
* Properly create destination if `--destination-dir` flag is used
* Remove improper default on CLI flag
* `DestinationConfig` is now a list of pointers
* Address first wave of review feedback
Fixes the majority of smaller issues caught by reviewers, thanks all!
* Add doc comments for bot.go functions
* Return the token TTL from CreateBot
* Split initial user cert issuance from `generateUserCerts()`
Issuing initial renewable certificate ended up requiring a lot of
hacks to skip checks that prevented anonymous bots from getting
certs even though we'd verified their identity elsewhere (via token).
This reverts all those hacks and splits initial bot cert logic into a
dedicated `generateInitialRenewableUserCerts()` function which should
make the whole process much easier to follow.
* Set bot traits to silence log messages
* tbot log message consistency pass
* Resolve lints
* Add config tests
* Remove CreateBotJoinToken endpoint
Users should instead use the CreateBot/DeleteBot endpoints.
* Create a fresh private key for every impersonated identity renewal
* Hide `config` subcommand
* Rename bot label prefix to `teleport.internal/`
* Use types.NewRole() to create bot roles
* Clean up error handling in custom YAML unmarshallers
Also, add notes about the supported YAML shapes.
* Fetch proxy host via gRPC Ping() instead of GetProxies()
* Update lib/auth/bot.go
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* Fix some review comments
* Add renewable certificate generation checks (#10098)
* Add renewable certificate generation checks
This adds a new validation check for renewable certificates that
maintains a renewal counter as both a certificate extension and a
user label. This counter is used to ensure only a single certificate
lineage can exist: for example, if a renewable certificate is stolen,
only one copy of the certificate can be renewed as the generation
counter will not match
When renewing a certificate, first the generation counter presented
by the user (via their TLS identity) is compared to a value stored
with the associated user (in a new `teleport.dev/bot-generation`
label field). If they aren't equal, the renewal attempt fails.
Otherwise, the generation counter is incremented by 1, stored to the
database using a `CompareAndSwap()` to ensure atomicity, and set on
the generated certificate for use in future renewals.
* Add unit tests for the generation counter
This adds new unit tests to exercise the generation counter checks.
Additionally, it fixes two other renewable cert tests that were
failing.
* Remove certRequestGeneration() function
* Emit audit event when cert generations don't match
* Fully implement `tctl bots lock`
* Show bot name in `tctl bots ls`
* Lock bots when a cert generation mismatch is found
* Make CompareFailed respones from validateGenerationLabel() more actionable
* Update lib/services/local/users.go
Co-authored-by: Nic Klaassen <nic@goteleport.com>
* Backend changes for tbot IoT and AWS joining (#10360)
* backend changes
* add token permission check
* pass ctx from caller
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
* fix comment typo
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
* use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event
* Client changes for tbot IoT joining (#10397)
* client changes
* delete replaced APIs
* delete unused tbot/auth.go
* add license header
* don't unecessarily fetch host CA
* log fixes
* s/tunnelling/tunneling/
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* auth server addresses may be proxies
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* comment typo fix
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* move *Server methods out of auth_with_roles.go (#10416)
Co-authored-by: Tim Buckley <tim@goteleport.com>
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
Co-authored-by: Tim Buckley <tim@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
Co-authored-by: Tim Buckley <tim@goteleport.com>
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
Co-authored-by: Nic Klaassen <nic@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* Address another batch of review feedback
* Addres another batch of review feedback
Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()`
calls, clear some TODOs and lints, and address other misc feedback
items.
* Fix lint
* Add missing doc comments to SaveIdentity / LoadIdentity
* Remove pam tag from tbot build
* Update note about bot lock deletion
* Another pass of review feedback
Ensure all requestable roles exist when creating a bot, adjust the
default renewable cert TTL down to 1 hour, and check types during
`CompareAndSwapUser()`
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
Co-authored-by: Nic Klaassen <nic@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
* Allow impersonation of roles without users
This adds the ability to impersonate one or more roles without
impersonating a particular user.
In Teleport today, when creating an impersonator role, both users and
roles must be specified as impersonation is fundamentally tied to an
existing Teleport user:
```yaml
allow:
impersonate:
users: ['jenkins']
roles: ['jenkins']
```
This is inconvenient for two reasons:
1. A user must exist for each set of roles you'd like to
impersonate, creating a UX burden.
2. It makes it difficult to use impersonation to reduce one's
permissions as you always inherit all of the roles granted to the
target user.
For the [certificate bot][bot] we'd instead like to use impersonation
to generate end-user (impersonated) certificates with a reduced set
of permissions. For example, given the following role:
```yaml
allow:
impersonate:
roles: ['jenkins', 'deploy']
```
We can then use `GenerateUserCerts` to issue certifices for a subset
of the allowed roles, e.g. one set of certificates with only the
`jenkins` role attached, and another with only `deploy`.
To that end, this patch:
1. Removes the requirement that roles define both `users` and
`roles` in impersonate conditions
2. Introduces a new `RoleRequests` field in `UserCertsRequest`
3. Modifies `generateUserCerts` to gather `roles` from
`RoleRequests` if allowed by an `allow` (with no `users`)
[bot]: https://github.com/gravitational/teleport/pull/7986
* Add `determineDesiredRolesAndTraits`; audit log on role impersonation
This splits initial role and trait determination into a new function,
`determineDesiredRolesAndTraits`, to improve control flow and clarity
given the new branches introduced for role impersonation.
Additionally, this moves the call to `CheckRoleImpersonation` down
to match regular user impersonation's flow, and emits an audit log
event on failure.
* Formatting fix
* Unit testing for role requests
This adds a new set of unit tests for role requests.
Also discovered the `impersonator` field wasn't being set for
role impersonation, so it's now set to the user's own username.
In other words, role impersonation will appear (in the audit log and
elsewhere) as self-impersonation.
* Clean up testing users between runs
* Deny most reimpersonation cases and add tests
This attempts to deny most cases of reimpersonation, where an
impersonated certificate might be used to generate certificates for
other roles the user is allowed to impersonate.
One test case is currently failing pending a solution.
* Add new DisallowReissue certificate extension
This adds a new DisallowReissue certificate extension that, if set,
prevents that identity from interacting with `GenerateUserCerts`.
This flag is always set when RoleRequests are used to prevent
unintended privilege escalation (while avoiding breaking changes to
Teleport's existing certificate generation behavior).
* Fix test lints
* Fix typo
* Fix test doc typo, add testcase for user impersonation misuse
* Apply suggestions from code review
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Accept context in CreateRole per review feedback
* Fix misleading comment
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Fix node registration backwards compatibility
With the consolidation of PackedKeys and proto.Certs in #8193,
I mistakenly changed the JSON that is returned from the /tokens/register
endpoint and broke the ability for older clients (7.x and earlier)
to register with newer (8.x and up) auth servers.
This wasn't caught by the type system as part of the refactor because
our API server handlers return interface{} and simply JSON encode whatever
is returned.
This commit preserves the original wire format by introducing an
unexported type with the original JSON keys and converting back
to proto.Certs before/after putting data on the wire.
* Fix debug log output
This commit contains 2 changes:
1. Rename GenerateServerKeys to GenerateHostCerts.
This is a more accurate name and consistent with the existing
GenerateUserCerts endpoint.
2. Change the request type to include a single role, rather than a
list of roles. We only ever allowed a single role in the list
anyway, so this change will prevent future mis-use of the API.
Note: a side effect of this change is we now have two similar endpoints:
- GenerateHostCert: old API that generates SSH cert only
- GenerateHostCerts: a newer API that generates SSH and TLS certs
To avoid making this change too big, we'll aim to deprecate
GenerateHostCert in the future.
Fixes#5352
```yaml
allow:
impersonate:
users: ['alice', 'bob']
roles: ['*']
where: 'contains(user.spec.traits["groups"], impersonate_role.traits)'
```
Adds "impersonator" to all X.509 and SSH client certs
issued using impersonation and does best effort to track
requests by impersonators in audit events.
Limits certs TTL to the impersonator's max session TTL.
Prevents impersonating users to recursively impersonate
other users.
Allows impersonating users to renew their own certificate,
for example to set route to cluster.
Adds missing token permission for editor role.
* mfa: per-session MFA certs for SSH and Kubernetes
This is client-side support for requesting single-use certs with an MFA
check.
The client doesn't know whether they need MFA check when accessing a
resource, this is decided during an RBAC check on the server. So a
client will always try to get a single-use cert, and the server will
respond with NotNeeded if MFA is not required. This is an extra
round-trip for every session which causes ~20% slowdown in SSH logins:
```
$ hyperfine '/tmp/tsh-old ssh talos date' '/tmp/tsh-new ssh talos date'
Benchmark #1: /tmp/tsh-old ssh talos date
Time (mean ± σ): 49.9 ms ± 1.0 ms [User: 15.1 ms, System: 7.4 ms]
Range (min … max): 48.4 ms … 54.1 ms 59 runs
Benchmark #2: /tmp/tsh-new ssh talos date
Time (mean ± σ): 60.2 ms ± 1.6 ms [User: 19.1 ms, System: 8.3 ms]
Range (min … max): 59.0 ms … 69.7 ms 50 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
'/tmp/tsh-old ssh talos date' ran
1.21 ± 0.04 times faster than '/tmp/tsh-new ssh talos date'
```
Another few other internal changes:
- client.LocalKeyAgent will now always have a non-nil LocalKeyStore.
Previously, it would be nil (e.g. in a web UI handler or when using an
identity file) which easily causes panics. I added a noLocalKeyStore
type instead that returns errors from all methods.
- requesting a user cert with a TTL < 1min will now succeed and return a
1min cert instead of failing
* Capture access approvals on MFA-issued certs
* Address review feedback
* Address review feedback
* mfa: accept unknown nodes during short-term MFA cert creation
An unknown node could be an OpenSSH node set up via
https://goteleport.com/teleport/docs/openssh-teleport/
In this case, we shouldn't prevent the user from connecting.
There's a small risk of authz bypass - an attacker might know a
different name/IP for a registered node which Teleport doesn't know
about. But a Teleport node will still check RBAC and reject the
connection.
* Validate username against unmapped user identity
IssueUserCertsWithMFA is called on the leaf auth server in case of
trusted clusters. Username in the request object will be that of the
original unmapped caller.
* mfa: add IsMFARequired RPC
This RPC is ran before every connection to check whether MFA is
required. If a connection is against the leaf cluster, this request is
forwarded from root to leaf for evaluation.
* Fix integration tests
* Correctly treat "Username" as login name in IsMFARequired
Also, move the logic into auth.Server out of ServerWithRoles.
* Fix TestHA
* Address review feedback
* mfa: put device UUID in MFAVerified cert extensions
Instead of just a bool, add the device UUID. This will be used in audit
events when a session is started using MFA-issued certs.
* Add WithMFA to session-related audit events
Also extract a common function for parsing SSH certs.
* auth: API for requesting per-connection certificates
See https://github.com/gravitational/teleport/blob/master/rfd/0014-session-2FA.md#api
This API is a wrapper around GenerateUserCerts with a few differences:
- performs an MFA check before generating a cert
- enforces a single usage (ssh/k8s/db for now)
- embeds client IP in the cert
- marks a cert to distinguish from regular user certs
- enforces a 1min TTL
* Apply suggestions from code review
Co-authored-by: a-palchikov <deemok@gmail.com>
Co-authored-by: a-palchikov <deemok@gmail.com>
Store the signing algorithm along the CA private key. When reading old
CAs that don't have it set, default to UNKNOWN proto enum which
corresponds to the old SHA1-based signing alg.
The only time you get a SHA2 signature is when creating a fresh cluster
and generating a new CA. This can be disabled in the config.
This allows users to override the SHA2 signing algorithms we default to
now for compatibility with the (very) old OpenSSH versions.
For host and user certs, use the CA signing algo for their own
handshakes. This allows us to propagate the signing algo from auth
server everywhere else.
Motivation:
x/crypto/ssh defaults to using SHA-1 for signatures:
https://github.com/golang/crypto/blob/master/ssh/keys.go#L963-L982
Because Teleport uses RSA for user, host and CA keys, we end up with
SHA-1 by default.
SHA-1 is now considered weak and OpenSSH plans to deprecate it:
https://www.openssh.com/txt/release-8.3
Fix:
Wrap all RSA `ssh.Signer`s and override `SignWithAlgorithm` to
provide `SigAlgoRSASHA2512` if not otherwise specified. This will
only affect new certs, existing certs will use `SigAlgoRSA` until
rotated. For CA certs (e.g. exported with `tctl auth export`) users
might need to manually rotate.
Limited local testing with openssh 8.2 client and
`-oHostKeyAlgorithms=-ssh-rsa` confirms that this works with a new
cluster and fails with an old one.
- Role options now include a `permit_x11_forwarding` bool
which is set to `false` by default.
- Recording proxies now forward X11 requests and channels
when when permitted by RBAC.
- User certs will now include the `permit-X11-forwarding`
extension when permitted by RBAC.
- If X11 forwarding is requested for a session a new `x11`
audit event is emitted by recording proxies.
Adding following principals:
- `localhost`
- `127.0.0.1`
- `::1`
With these, `tsh` (both `ssh` and `join`) works with a local proxy
without any SSH handshake errors.
Removed the warning from quickstart docs, but keeping `--proxy=grav-00`
since that implies to the reader that proxy is usually remote.
Fixes#2910
If an attacker can force a username change at an IdP, upon second login,
the services.User object of the original user can be updated with new
roles and traits. If these new roles and traits differ, the original
user can have their privileges raised (or lowered).
To mitigate this, encode roles and traits within the certificate and use
these when fetching roles to make RBAC decisions. If roles and traits are
not encoded within an certificate (for example for old style SSH
certificates then fallback to using the services.User object and log a
warning.
This commit implements #2543
In SSH terms ProxyJump is a shortcut for SSH client
connecting the proxy/jumphost and requesting .port forwarding to the
target node.
This commit adds support for direct-tcpip request support
in teleport proxy service that is an alias to the existing proxy
subsystem and reuses most of the code.
This commit also adds support to "route to cluster" metadata
encoded in SSH certificate making it possible to have client
SSH certificates to include the metadata that will cause the proxy
to route the client requests to a specific cluster.
`tsh ssh -J proxy:port ` is supported in a limited way:
Only one jump host is supported (-J supports chaining
that teleport does not utilise) and tsh will return with error
in case of two jumphosts: -J a,b will not work.
In case if `tsh ssh -J user@proxy` is used, it overrides
the SSH proxy coming from the tsh profile and port-forwarding
is used instead of the existing teleport proxy subsystem
Added utils.CertChecker that wraps a ssh.CertChecker. The new
certificate checker first checks if the certificate is a valid
certificate for Teleport. At the moment that is 2048-bit RSA then calls
the underlying certificate checker to perform the requested validation.
This commit introduces several key changes to
Teleport backend and API infrastructure
in order to achieve scalability improvements
on 10K+ node deployments.
Events and plain keyspace
--------------------------
New backend interface supports events,
pagination and range queries
and moves away from buckets to
plain keyspace, what better aligns
with DynamoDB and Etcd featuring similar
interfaces.
All backend implementations are
exposing Events API, allowing
multiple subscribers to consume the same
event stream and avoid polling database.
Replacing BoltDB, Dir with SQLite
-------------------------------
BoltDB backend does not support
having two processes access the database at the
same time. This prevented Teleport
using BoltDB backend to be live reloaded.
SQLite supports reads/writes by multiple
processes and makes Dir backend obsolete
as SQLite is more efficient on larger collections,
supports transactions and can detect data
corruption.
Teleport automatically migrates data from
Bolt and Dir backends into SQLite.
GRPC API and protobuf resources
-------------------------------
GRPC API has been introduced for
the auth server. The auth server now serves both GRPC
and JSON-HTTP API on the same TLS socket and uses
the same client certificate authentication.
All future API methods should use GRPC and HTTP-JSON
API is considered obsolete.
In addition to that some resources like
Server and CertificateAuthority are now
generated from protobuf service specifications in
a way that is fully backward compatible with
original JSON spec and schema, so the same resource
can be encoded and decoded from JSON, YAML
and protobuf.
All models should be refactored
into new proto specification over time.
Streaming presence service
--------------------------
In order to cut bandwidth, nodes
are sending full updates only when changes
to labels or spec have occured, otherwise
new light-weight GRPC keep alive updates are sent
over to the presence service, reducing
bandwidth usage on multi-node deployments.
In addition to that nodes are no longer polling
auth server for certificate authority rotation
updates, instead they subscribe to event updates
to detect updates as soon as they happen.
This is a new API, so the errors are inevitable,
that's why polling is still done, but
on a way slower rate.
Jakub Nyckowski
