This PR implements the core of the kube-agent-updater, which is part of
https://github.com/gravitational/teleport/issues/21516#issue-1576935859
In order to have a fully working updater we still need to:
- implement the interfaces for version retrieval, image validation and maintenance trigger
- add statefulset support (and deal with the potential deadlocks)
- implement the CI and release pipeline (Dockerfile, README, Makefile, github action, drone)
- integrate in the `teleport-kube-agent` helm chart
Those changes will happen in subsequent PRs.
* Skip building webassets in CI
Building webassets is not always needed and many of our CI build just builds them and ends up not using them after.
This PR skips building the webassets for all pipelines where they are not needed. This should save us some time and $$.
* Make some changes to trigger CI
* Create the missing directory
Currently, a lot of makefile logic is copied between OSS end Ent repo. This PR moves BPF-related code to common.mk, so it doesn't need to be copied between reports. I'll move more code in the following PRs. I just want to keep diffs small.
Test ARM64 assets build
Use all available cores when building Clang
Add test trigger
Update assets buildbox name
Build all dependencies on ARM64 including BPF
* Add WEBASSETS_SKIP_BUILD to Makefile
WEBASSETS_SKIP_BUILD allows skipping webassets build for development purposes. Teleport build in that mode displays blank page instead of WebUI, but the build significantly faster.
`make full` - always builds webassets
`make WEBASSETS_SKIP_BUILD=0` - works the same as `full`
`make WEBASSETS_SKIP_BUILD=1` - skips webassets build.
* Copy the root UI index.html file and serve it as it's used to fetch the config in the UI dev server.
---------
Co-authored-by: Jeff Pihach <jeff.pihach@goteleport.com>
Update to libbpf 1.0.1 and github.com/aquasecurity/libbpfgo v0.4.5-libbpf-1.0.1. As we're building our releases on CentOS 7 anyway we can also switch to mainstream libbpf instead of using our fork.
The `tsh appps` family of commands is aliased to `tsh apps`, so both
invocations work correctly. The command itself is defined as `tsh apps`,
so this is what appears in the help message.
Update references to `tsh app` to recommend `tsh apps` instead so that
there isn't confusion when browsing `tsh help` and looking for a missing
`app` subcommand.
Fixes#21367
* Adjust go_package of lib/prehog Go protobufs
This makes them follow the pattern set out by api/proto and proto.
* Adjust go_package of lib/teleterm Go protobufs
* Use single buf.gen.yaml to generate JS protos
This also entailed changing the location of lib/teleterm protos and changing
the value of their package specifier to match the conventions in other parts
of the codebase. This is a breaking change but that is fine for Connect
as the protos are used locally only and each build ships with matching
protobufs.
* Make web/packages/teleterm use protobufs from gen-proto-js
We used to copy protobufs over to web/packages/teleterm/src/services/tshd
since webapps used to be in a separate repo.
This is no longer the case, so we can just make teleterm use protobufs
from gen-proto-js.
* Move prehog & teleterm protos into proto/teleport/lib
* Generate JS protos to gen/proto/js
* Move lib/teleterm Go protobufs to gen/proto/go
* Move lib/prehog Go protobufs to gen/proto/go
* Rename lib/teleterm proto package
* Re-enable linter rules for teleterm & prehog
* Update prehogv1 path in usagereporter_test.go
* Use except instead of ignore_only to allow Google API-style responses
* Add UNARY_RPC to api/proto & proto
* Ignore gen/ when running addlicense
* buf-js.gen.yaml: Remove comment about lack of go_package for JS
* Move prehog protos to proto/prehog/v1alpha
* Adjust prehog's go_package to match proto package
* Add check if protos are up to date.
A new check has been added that will detect if protobufs are up to date. The
script will exit abnormally if protobufs need to be regenerated.
* Alan's feedback.
* Restoring the script.
* Update script comment.
* Add in the set -eu.
* Add a comment for the pull_request/merge_group bit in the new github action.
* Remove helper script.
* Reduce the runner size.
* Conditionally build the UI if there are changes.
`ensure-webassets` and `ensure-webassets-e` will now build the UI if the
hash-of-hashes has changed since the last time the webassets were built.
* Update Makefile
Co-authored-by: Jeff Pihach <hatched@users.noreply.github.com>
* Check for the existence of the make command.
* Choose a shasum exec from a preconfigured list or use an override.
* Shellcheck fix.
* Remove yarn.lock from sha calculation.
* Recalc SHA on successful build.
* Fix linting issues.
* Use Python instead of shasum.
* Remove unnecessary comments from sh file.
* Unconditionally rebuild if Python executable is not found.
* Revert "Unconditionally rebuild if Python executable is not found."
This reverts commit 9d2b0e3409.
* Revert "Remove unnecessary comments from sh file."
This reverts commit 600e02811b.
* Revert use of Python, use simplified shell scripting.
* Linting fixes.
* Apply suggestions from code review
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Remove reference to realpath as apparently the MacOS builder doesn't have it.
* calculate_sha must return 0.
* Use echo in calculate_sha or else set -o pipefail causes failures.
* Suppress SC2005.
---------
Co-authored-by: Jeff Pihach <hatched@users.noreply.github.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Remove CLANG_FORMAT from Makefiles
It was used to format protos but we use Buf for that since v10.
* Move installing grpc_node_plugin into Dockerfile
This commit basically takes grpc_node_plugin compilation from
Dockerfile-teleterm and moves it to Dockerfile.
* Replace Dockerfile-teleterm with Dockerfile
After moving grpc_node_plugin compilation to Dockerfile, the only remaining
thing that Dockerfile-teleterm does is installing rpm so that we can make
an RPM package for Connect during tag builds.
Installing this package can be simply moved to Dockerfile.
* Remove grpc-teleterm Make target in favor of grpc
* Add updated protobufs
It looks like they're a result of someone changing protos in lib/prehog
without running `make grpc-teleterm` separately. Which is why we're getting
rid of grpc-teleterm as a separate Make target in the first place. ;)
Since the release of `tsh proxy app` we no longer need a
Teleport-aware Drone CLI and can leverage the standard
drone tool from https://docs.drone.io/cli/install/
This patch performs the (hopefully final) switchover that will make drone
defer to GHA in order to build Teleport ion arm64.
This patch:
- Replaces all of the Dronegen code to generate arm64 builds locally with
steps that invoke the GHA builder workflow
- Changes the release tagging behavior in the Makefile to tag `teleport.e`
with the same tag as teleport. This is required to for Drone to identify
the revision of the arm64 build workflow to invoke
- Updates the e reference to include a revision of `teleport.e` that
contains the builder workflows
Thanks to everyone involved in getting this working.
The existing `build.assets` makefile targets had the actual build steps
coupled together with building the build box image. Because of how GHA
image builds work, we need to uncouple those tasks.
GHA also builds OSS and Enterprise teleports in parallel, so we needed
a new target to build the Enterprise release without also automatically
building the OSS bundle in series.
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
* [draft] Add a new usage reporter
This adds a new usage reporter service to the auth server. It's
disabled by default in OSS and can only be turned on via startup hook
in Cloud / Enterprise. In OSS, the audit log wrapper is never
configured and any usage events are sent to a no-op discard reporter.
Usage events are defined in prehog and can be sent to the new
UsageReporter Service on the auth server. An audit event wrapper is
used to capture certain events that are otherwise difficult to hook.
Events are anonymized before submission, then held in a non-blocking
queue for batching and submission purposes.
* Remove dead code
* Add SubmitUsageEvent RPC to Auth.
This adds a new SubmitUsageEvent RPC to the Auth API that external
clients (e.g. the UI) can use to submit usage events externally.
* Slight refactor for unit testing
* Add Prometheus metrics and add initial working prehog submitter
* Add more metrics, tweak prehog client, and add unit tests
* Further tweak http transport settings based on Teleport defaults
* Add missing metrics
* Fix goimports
* Add new UI usage events
* Update e ref
* Add prehog directly for now. Improve logging.
* update prehog
* Add new prehog events; use username from request identity
* add HTTP server for user events
* Add username back to pre-onboard events
* unauthenticated user events
* Fix userevent build error
* Use event-provided username where appropriate
* Move barebones prehog reqs to lib/prehog and generate here.
Also, use prod tunable values.
* Fix license lints
* De-flake tests by adding unfortunate amounts of synchronization.
* Add missing license header
* Misc PR cleanup for review
* Update lib/events/usageevents/usageevents.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Address a batch of review comments
Adds `anonymizer.AnonymizeString` and parent loggers
* Update e ref
* Clean up comments
* Remove onboard prefix from recovery code event
* Address another batch of feedback
* Use defaults.HTTPClient()
* Remove a noisy log message
* Demote noisy log message to debug
* Temporarily revert e ref for merge
Co-authored-by: Michelle Bergquist <michelle.bergquist@goteleport.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
If all etcd nodes are unreachable, Teleport exits with the following
message:
```
ERROR: initialization failed
context deadline exceeded
```
With this commit we get instead:
```
ERROR: timed out dialing etcd endpoints: [https://127.0.0.1:2381https://127.0.0.1:12381]
initialization failed
context deadline exceeded
```
Note that if only some of the etcd nodes are unreachable, the etcd
client already provides a useful error message:
```
ERROR: initialization failed
failed to dial endpoint https://127.0.0.1:12381 with maintenance client: context deadline exceeded
```
Devices running these architectures are likely not powerful enough
to handle desktop sessions. This will also reduce the binary size
for these builds, making them slightly more convenient for smaller
resource-constrained devices.
* Added multiarch build support for teleport oss, ent, and fips
* Exported image/imageTag types
* Resigned dronegen
* Removed remainder of testing changes
* Removed changes to submodules
* Reverted dockerfile-fips change
* FIxed docs wording
* Un-exported most constants
* Removed teleport.e makefile deb call
* Moved "sed | cut magic" to files
* Re-added `mkdir -pv /go/cache` to push.go
* Command deterministic order fix
* Added staging-only tag pipeline
* Moved PR to teleport operator to minimize potential issue impact
* Updated promote to pull and push without build
* Made cron triggers not affect canonical tags
* Added check for pre-existing tags on immutable CRs
* Added immutability check to manifests
* Updated staging ecr to only apply $TIMESTAMP tag on cron triggers
* Updated triggerinfo struct to use a triggerflag struct
* Fixed makefile after git mistake
* Makefile fix
* PR fixes
* Moved internal tools Go version to constant
* Separated container images gofile into multiple files
* Moved testing comment
* Added licenses
* Reorganized and added docs for container images
* Moved const to correct file
* Tag trigger logic test
* Testing specific fix
* Moved testing to v10.3.2
* Make semver dirs
* Refactored local registry name/socket
* Merged previous dockerfile changes
* Added TARGETOS TARGETARCH args
* Updatd tag to testing tag
* Promotion logic test
* Promotion fixes
* Testing specific fix
* Removed prerelease check for testing
* Added staging login commands to promote
* Fixed missing credentials on promotion pull
* Rerun tag test with new "full" semver
* Made staging builds only publish full semver
* Added semver logging command
* Empty commit to trigger Drone
* Promotion test
* Fixed preceeding v on promote pull
* Empty commit to trigger Drone
* Re-enabled verify not prerelease step on promote
* Cron trigger test
* Testing fix
* Testing fix 2
* Added sleep timer on docker buildx build
* Testing cleanup
