Removes unnecessary workflow arguments from the Drone workflow
invocation. Refactors some of the `dronegen` generators to make
this easier to express insife `dronegen`
* Integrate GHA OCIU publishing pipeline with Drone
Adds new drone pipelines for building and promoting distroless
images. The pipelines themselves invoke workflows on GHA and
await their completion.
* Revert dev version
* restore -dev suffix
* Review Comments
* Address review comments
* Added `--target-cloud` flag to OS package repo tool
* Updated OS package repo tool to use "version channel" instead of "artifact version"
* Added help flag examples
Since the release of `tsh proxy app` we no longer need a
Teleport-aware Drone CLI and can leverage the standard
drone tool from https://docs.drone.io/cli/install/
This patch performs the (hopefully final) switchover that will make drone
defer to GHA in order to build Teleport ion arm64.
This patch:
- Replaces all of the Dronegen code to generate arm64 builds locally with
steps that invoke the GHA builder workflow
- Changes the release tagging behavior in the Makefile to tag `teleport.e`
with the same tag as teleport. This is required to for Drone to identify
the revision of the arm64 build workflow to invoke
- Updates the e reference to include a revision of `teleport.e` that
contains the builder workflows
Thanks to everyone involved in getting this working.
* Enable nolintlint linter
* Fix nolint comments in the api package
* Fix RDP client comment
* Address review comment
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Allow unused for nolintlint linter
* Remove redundant casting
* Add comment on why allowed unused is enabled
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
It was added in effort to debug flaky Connect builds (#15836).
However, we discovered that the v11.1.0 macOS version of Connect stopped
working. This was likely due to upgrade of electron-builder which recently
updated its process of building native deps
(electron-userland/electron-builder#7196).
In the Node.js ecosystem, the DEBUG env var is typically used to control
which packages emit debug messages [1]. However, after the update of
electron-builder, the env var also changed the behavior of one of the
packages responsible for building the apps.
This was confirmed by inspecting file tree between different app bundles
and running the build locally with DEBUG set to electron-*.
[1] https://www.npmjs.com/package/debug
* Bump relcli (attach assets atomically)
* Create/attach assets atomically
* Order publish-rlz last in promote target
* Move publish-rlz down
Drone needs dependencies to be defined above the dependent pipeline
* Update relcli (enable publishing "dev" builds)
* Bump relcli to a stable version
Adds a drone node selector so builds will be routed to the correct buildbox
Also:
1. changes the version of Visual Studio expected by npm to vs 2022 to match buildbox installation
2. Removes the concurrency limit for window push builds. As there are separate builders for v10, v11
and v12. Given that the concurrency limits are enforced across all builds with the same name, they
serialise builds from different teleport versions. it seems wasteful to have a builder for v10 sit idle
while waiting a v11 build completes.
The concurrent limits have been left for tag builds, as there are flow-on concurrency concern due to artifact
uploading, etc.
* Serialize apt/yum promote pipelines
These were running in parallel, but we want them to run serially.
Therefore, we add a dependency between each step and its previous step.
* Allow dev build promotes to proceed in deb/rpm pipelines
This helps test a couple more changes from this pipeline when cutting a
dev build. Particularly, we saw the download and role assumption steps
fail in https://github.com/gravitational/teleport/pull/17334, and this
change would have allowed us to catch that error during testing.
* Fix globbing bug
This bug does not appear to affect anything currently. However it
should be fixed in case the rm is important at some point in the future.
The bug is: when a wildcard is inside quotes, it is treated as a literal
filename. So rm -rf "$ARTIFACT_PATH/*" tries to remove the file named
'*' instead of trying to remove everything in artifact path.
* Swap YUM_REPO_NEW_ROLE to YUM_REPO_NEW_AWS_ROLE
All other roles environment variables end in AWS_ROLE, and consistency
is our friend here.
* Added multiarch build support for teleport oss, ent, and fips
* Exported image/imageTag types
* Resigned dronegen
* Removed remainder of testing changes
* Removed changes to submodules
* Reverted dockerfile-fips change
* FIxed docs wording
* Un-exported most constants
* Removed teleport.e makefile deb call
* Moved "sed | cut magic" to files
* Re-added `mkdir -pv /go/cache` to push.go
* Command deterministic order fix
* Added staging-only tag pipeline
* Moved PR to teleport operator to minimize potential issue impact
* Updated promote to pull and push without build
* Made cron triggers not affect canonical tags
* Added check for pre-existing tags on immutable CRs
* Added immutability check to manifests
* Updated staging ecr to only apply $TIMESTAMP tag on cron triggers
* Updated triggerinfo struct to use a triggerflag struct
* Fixed makefile after git mistake
* Makefile fix
* PR fixes
* Moved internal tools Go version to constant
* Separated container images gofile into multiple files
* Moved testing comment
* Added licenses
* Reorganized and added docs for container images
* Moved const to correct file
* Tag trigger logic test
* Testing specific fix
* Moved testing to v10.3.2
* Make semver dirs
* Refactored local registry name/socket
* Merged previous dockerfile changes
* Added TARGETOS TARGETARCH args
* Updatd tag to testing tag
* Promotion logic test
* Promotion fixes
* Testing specific fix
* Removed prerelease check for testing
* Added staging login commands to promote
* Fixed missing credentials on promotion pull
* Rerun tag test with new "full" semver
* Made staging builds only publish full semver
* Added semver logging command
* Empty commit to trigger Drone
* Promotion test
* Fixed preceeding v on promote pull
* Empty commit to trigger Drone
* Re-enabled verify not prerelease step on promote
* Cron trigger test
* Testing fix
* Testing fix 2
* Added sleep timer on docker buildx build
* Testing cleanup
* Refactor build-buildboxes to uses multiple profiles
This greatly reduces the number of steps in the pipeline, allowing drone-runner-kube to successfully schedule the pipeline.
Fixes https://github.com/gravitational/teleport/issues/17310
Furthermore, I also updated un-dronegen'ed pipelines to have same syntax as dronegen'd ones, which is nice for consistency.
Previously, "${ARTIFACT_PATH}" was interpreted as Drone variable
subsitution, resulting in "rm -rf ${ARTIFACT_PATH}/*" becoming
"rm -rf /*", which deleted credentials on the filesystem.
* add windows submodule support for connect builds
* added connect enterprise support to mac builds
* update submodule command
* removed ability to fail webapps.e and recursive
* dronegen
This is follow up to #17201, that fixes the buildbox pipeline error seen here:
An error occurred (AccessDeniedException) when calling the GetAuthorizationToken operation: User: arn:aws:iam::146628656107:user/teleport_build_user_read_only is not authorized to perform: ecr-public:GetAuthorizationToken on resource: * because no identity-based policy allows the ecr-public:GetAuthorizationToken action
Contributes to gravitational/SecOps#213.
This PR updates our various Drone pipelines to use AWS roles for publishing.
Our AWS FTR requires that we do not use any long lived credentials in our AWS accounts and instead use roles. This means we need to move from attaching policies directly to users to attaching policies to roles and having policyless users assume those roles.
https://aws.amazon.com/partners/foundational-technical-review/
Contributes to https://github.com/gravitational/SecOps/issues/213
Without this any tag that isn't part of the history on master will fail
to successfully promote. This breaks most dev builds, which don't end
up as part of master or a release branch.
Without these changes, the promote step will always fail because of a
mismatch between where the repo is cloned and where it is referenced:
/go/src/.../teleport.git
vs
/go/src/.../teleport
(cherry picked from commit b209b98f0d)
* Port publish-rlz pipeline to dronegen
* Add artifact cleanup pipeline on tag, "before-all"
* Bump relcli in all pipelines
* Add missing license header
* Refactoring to relcli steps per code review
Only functionally-equivalent changes, .drone.yml identical to previous
* Factor out relcli pipeline
* Use docker:cli to pull relcli
* Add "wait for docker" to relcli pipelines
* Add piv build dependencies.
- Add LIBPCSCLITE build tag.
- Add libpcsclite static linking using gravitational/pcsc fork.
- Enable use of dynamic pcsc library with LIBPCSCLITE=dynamic.
- Refactor CGOFLAG in Makefile.
- Update Centos7 Dockerfile and drone.
* Refactor RELEASE_MESSAGE for readability. Now produces message like: "RELEASE_MESSAGE=Building with GOOS=linux GOARCH=amd64 REPRODUCIBLE= and with PIV support and without PAM support, FIPS support, BPF support, Windows RDP client, libfido2, Touch ID."
Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
Update metalinter, fix a few lint warnings and replace deprecated linters.
`deadcode`, `structcheck` and `varcheck` are abandoned and now replaced by [`unused`][1].
Since 1.19, `go fmt` reformats godocs according to https://go.dev/doc/comment. I've done a bulk-reformatting of the codebase to keep the linter happy. Backporting is mostly harmless (the exception being `lib/services/role_test.go`, that for some reason breaks the _old_ linter using the new format).
[1]: https://golangci-lint.run/usage/linters/
* Bump golangci-lint version
* Replace abandoned linters
* Fix bodyclose on lib/auth/github.com
* Fix bodyclose on lib/kube/proxy/streamproto/proto_test.go
* Fix bodyclose on lib/srv/alpnproxy/proxy_test.go
* Fix bodyclose on lib/web/conn_upgrade_test.go
* Silence staticcheck on lib/kube/proxy/forwarder_test.go
* Silence staticcheck on lib/utils/certs_test.go
* Address BuildNameToCertificate deprecation warnings
* Run `go fmt ./...`
* Run `go fmt ./...` on api/
* Ignore formatting in role_test.go
* Remove redundant initializers in lib/srv/uacc/
* Update e/
WARNING: Due to issues with the windows drone executor's poor escaping when it echoes commands, I have moved the error message functionality into the PS build functions in build.assets/windows/build.ps1. This means that any failures that occur during the code checkout step will not be reported.
I'm not sure that this is the correct tradeoff, but it may well suffice for now.
Uses Drone to build Teleport Connect for Windows on a Native
Windows builder.
This PR adds 2 pipelines to the Drone YAML:
1. `push-build-native-windows-amd64`: Invoked on a push to master,
branch/v*, etc., and asserts that Teleport Connect can be built, and
2. `build-native-windows-amd64`: Invoked when a branch tag is
committed to the teleport Repo. Builds Teleport Connect and uploads
it to dronestorage
These builds are run on a native windows builder (as opposed to tsh,
which is built in a linux environment and cross-compiled for Windows)
* Add proof of concept of Connect pipeline
The proof of concept includes a lot of copy-pasted lines which will get
cleared up in subsequent commits.
* Extract copying artifacts into separate functions
The tag pipeline no longer needs to worry about Connect artifacts.
* Reuse steps to install & cleanup toolchains
* Share toolchain configuration commands between pipelines
* Share build commands among different pipelines
* Download webapps only if a pipeline builds Connect
As seen by the changes to .drone.yml, this removes unnecessary webapps
clones from these tag pipelines: build-darwin-amd64, build-darwin-amd64-pkg,
build-darwin-amd64-pkg-tsh. None of them needs webapps to function anymore
and the pkg pipelines never needed webapps in the first place.
Trent Clarke