mirror of
https://github.com/gravitational/teleport.git
synced 2026-09-24 16:17:11 +08:00
a13ff879168709acbc2464ec8fb8d00a96b257ba
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Luke Okraszewski
|
e9356dbd90 |
[tools] add benchstat tool (#63694)
* [tools] add benchstat tool Adds `golang.org/x/perf/cmd/benchstat` to tools. Required for differential benchmarking. Example: ``` make test-go-bench BENCH_OUTPUT=a.txt make test-go-bench BENCH_OUTPUT=b.txt make benchstat "BENCH_FILES=a.txt b.txt" ``` While the tool can be invoked directly, the Make target provides a convient command for CI workflows. * missing newline * bump benchstat deps |
||
|
Luke Okraszewski
|
8f4131ff15 |
[tools] move go tools to isolated dir (#63724)
Each go tool will now reside in a isolated dir under `build.assets/tools`, this allows us to isolate the deps of each tool from both main module without merging them for tools. This should make reviewing tools and dep changes easier at a glance and remove noise from main module dep manifest. Each tool is a Makefile var to allow easy overrides over a macro which cannot be temporarily unset/overriden. The tools have been added to common.mk for easy reuse across OSS and e. |
||
|
|
7a7f08cb39 |
Replace aquasecurity/libbpfgo with cilium/ebpf (#60541)
* start the refactor away from aquasecurity/libbpfgo to use cilium/ebpf Remove debug statements and unused code in BPF library This commit eliminates unnecessary debug log statements from bpf_test.go and removes unused code in disk.go and network.go files within BPF library. The aim is to make codebase cleaner, enhancing readability and maintainability without losing track of functional aspects. It's part of an ongoing effort to maintain optimal code quality across the project. Revise Dockerfile and Makefile instructions Adjusted Dockerfile and Makefile commands to optimize build process. The changes ensure better readability and maintainability of code, especially with respect to BPF bytecode and test preparations. A clause was also added to check for the presence of clang before building BPF bytecode. Refactor BPF code and tests Refactored BPF source code and its corresponding tests for better readability and efficiency. This includes updating data structures like rawExecEvent, rawOpenEvent, rawConn4Event and rawConn6Event to their more meritorious versions like commandDataT, diskDataT, networkIpv4DataT and networkIpv6DataT respectively. Also, generated concise and clear bindigs for eBPF programs to replace the previous multi-arch setup. Furthermore, correspondingly adapted all related test cases arising in bpf_test.go, command.go and other files. Remove counter_test from BPF and related test code This commit removes the file counter_test.bpf.c from the enhanced recording in BPF. Following this file deletion, corresponding test functions in bpf_test.go, that were originally testing the functionality provided by counter_test.bpf.c, have been removed as well. Refactor BPF test and improve error handling The commit refactors the BPF test suite to run each test as a sub-test by using t.Run, easing debugging and error tracing. It also simplifies the 'executeCommand' function by removing redundant function calls related to finding executable files, and improves error handling. In 'cgroup.go', the code has been adjusted to retry on EBUSY errors when removing a cgroup, instead of ignoring them. Update cgroup deletion process and error handling The changes made introduce a retry mechanism when deleting a cgroup in case the cgroup is busy. This prevents a failed attempt from immediately returning an error. Additionally, error handling has been improved in the 'writePids' function to better handle 'no such process' errors. Enhance cgroup monitoring in disk.bpf.c The code has been updated to check if current cgroup is being monitored in the "enter_open" method of "disk.bpf.c". If the cgroup is not marked for monitoring, it will be ignored, enhancing overall cgroup management. Improve resource handling This commit introduces a deferred close to the sendEvents function to ensure proper resource handling of the eventBuf object. Add rlimit package to remove resource limits Introduced the "github.com/cilium/ebpf/rlimit" package and added a function to remove resource limits for kernels less than 5.11. This operation is necessary for kernels with a lower version because they may face issues when running applications with allocated memory limitations. Fix a test after rebase Refactor BPF testing setup and implement session closing This commit updates the BPF testing structure to remove the context dependency in event generation functions, simplifying the test setup. Additionally, it adds a 'closed' field to the 'open', 'conn', and 'exec' structs alongside mutex protection. This implementation allows checking if a session is closed before any operations, correctly handling already closed sessions. Furthermore, it corrects tracepoints for handling 'openat' and 'openat2' system calls for different platforms. This refactoring effort aims to improve the robustness and readability of the BPF testing suite. Update conditions for BPF bytecode building This update modifies the Makefile to add a conditional check for the presence of /usr/include/bpf/bpf_helpers.h. This new requirement must be met along with the existence of /usr/include/linux/bpf.h and the installation of Clang, in order to facilitate the correct build of BPF bytecode. Consequently, this change enhances the reliability of the build process. Add conditional for bpf_helpers.h in Makefile This commit adds an additional check in the Makefile for the existence of /usr/include/bpf/bpf_helpers.h. This ensures that BPF bytecode only gets built when both bpf.h and bpf_helpers.h exist, and Clang is installed, improving the build process reliability. Refactor and improve BPF bytecode generation This commit refactors and improves the BPF bytecode generation process. It introduces distinct bytecode structures for different architectures, ensuring better compatibility and execution. The BPF test is adjusted due to its tendency to fail unpredictably, hence it's commented out to avoid unnecessary build failures. This commit also eliminates certain redundancies in `common.mk`, improving the clarity of the build process. Comment out test code in bpf_test.go The test code in the file bpf_test.go has been commented out. These changes are circumstantial and do not imply that the test code is obsolete or unnecessary. The use of such tests may be needed in future commits or for local development testing and debugging. Remove libbpf dependency from Vagrantfile and update README The Vagrantfile and README have been updated to remove the no longer needed libbpf dependency. This change simplifies the project set up by eliminating unused build steps in the Vagrantfile and correcting related instructions in the README. This results in a more streamlined, intuitive build process and reduced potential for code complexity and dependency conflicts. Remove obsolete dependencies and related code blocks The 'aquasecurity/libbpfgo' library referencing and the related code blocks have been removed from the test files and core modules. The corresponding setup instructions in the dockerfiles and build scripts have also been eliminated. This update simplifies the build process and overall project structure while reducing unnecessary dependency risks. Remove aquasecurity/libbpfgo dependency Remove restricted.bpf.c Migrate disk events to cilium Migrate network events to cilium Remove restricted session Use cilium/ebpf for BPF * fix build issues caused by stale branch Also updated bpf bytecode, updated github.com/cilium/ebpf to 0.19.0, and updated bpf/README to reflect changes * fix tests and a bit of cleanup Also added a subtest and fixed receiving IPv6 network events and fixed attempting to create tracepoints for syscalls that are not present on arm64. * repurpose buffer size configs to set event channel buffer size * add new GHA job to check that generated eBPF files are up to date * fix lost event promethus counters and address feedback * address feedback * use only unsafe instead of CGO to convert C strings * rename common_data_t comm -> command * convert strings in bpf programs from char arrays to u8 or uint8 arrays so we can convert them to Go strings without the use of the unsafe package --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com> Co-authored-by: Tim Ross <tim.ross@goteleport.com> |
||
|
Cam Hutchison
|
5f7e1f65f4 |
buildbox: Add new cross-compiling buildbox for Teleport (#44130)
* buildbox: Add new cross-compiling buildbox for Teleport
Add Dockerfiles and make targets to build a new buildbox that has a set
of cross compilers for the four architectures we target for Linux -
amd64, arm64, 386, and arm (32-bit). The buildbox also contains the
third-party C libraries that Teleport needs to statically link against
for a full set of features, again compiled for each target architecture.
The cross compilers are built using crosstool-NG. The compilers must
have a "vendor" field in the "target triple" of "unknown" (default).
Setting it to anything else means rust does not find the cross compilers
properly. Using "unknown" causes the triple to match the rust target.
(This may ultimately be unnecessary to make these match, but is needed
if we have the boring crate build boringssl itself - see last
paragraph).
Two container images are produced:
* `buildbox-thirdparty` - a base image that contains the third-party
tools (compilers, etc) and libraries that we build from source. Once
build, this should never need to be rebuilt unless we change the
version of one of the components it builds.
* `buildbox-ng` - the buildbox used to build Teleport. It copies out the
third-party components from the previous image and installs whatever
other tools are needed by the build, whether from the distro archive
or directly from upstream (such as Go and Rust compilers).
Additionally, the three intermediate build stages of
`buildbox-thirdparty` can be built for working on the buildbox; `ctng`,
`compilers` and `tplibs`.
`buildbox-ng` will become just `buildbox` once it can replace the others
entirely, at which point those others will be removed.
Currently the build tools to build a FIPS version of teleport is not in
the buildbox. That has been troublesome as the rust boring crate wants
to build boringssl itself and it is not being done correctly with the
cross compilers. Further work is needed here, likely building boringssl
ourselves and pointing the rust boring crate at the pre-built libraries.
* buildbox: Add libelf.pc for pkg-config/pkconf
Add a libelf.pc file taken from elfutils and minorly adjusted for how we
install it separately. libelf was taken out of elfutils but did not take
the libelf.pc file, so we do that ourselves.
This will allow the build to use pkg-config/pkconf for selecting the
libraries to link teleport to as some later versions of libelf also need
`-lzstd`, such as this buildbox but also ubuntu-24.04.
* buildbox: Set prefix for zstd, add sh-cross-vars target
Set `PREFIX` when building `zstd` to ensure it's pkg-config file has the
correct prefix set in it, otherwise it has `-L/usr/local/lib` which we
do not want for cross-compiling.
Add a `sh-cross-vars` target to echo the cross-compiling vars for an
architecture, in a form that can be sourced by the shell:
eval $(make -s -f cross-compile.mk ARCH=arm64 sh-cross-vars)
This will set up your shell for cross-compiling for the given
architecture, which is useful when working on the libraries in the
buildbox.
* Address Roman's review comments
* Rename ctng to crosstoolng throughout
* Rename tplibs.mk to thirdparty-libs.mk
* Fix spelling mistakes.
* Address Jakub's review comments
* Remove `--hostname $(HOSTNAME)` when running containers.
* Remove `--volume /tmp:/tmp` when running containers.
* Rename bbcommon.mk to buildbox-common.mk
* buildbox: Fix up broken renaming
Renaming things was done poorly in the last few commits. Fix all that
up.
* buildbox: Use gold linker for arm64 builds
Use the `gold` linker when building arm64 binaries, as Enterprise
Teleport will not build with the binutils (bfd) linker; it gives
numerous errors when linking of the form:
something.rs: (.text.unlikely._XXX): relocation truncated to fit: R_AARCH64_CALL26 against symbol ...
These errors do not occur when using the `gold` linker, which is already
included and built by crosstool-NG.
* buildbox: Change crosstool arm tuple to match rust
Tweak the crosstool-NG configuration for the arm cross-compiler so that
the tuple for it matches the tuple currently used in the build by rust
for the same target. This is mostly to cause less confusion as there's
no real need for them to be different.
* buildbox: Add support for rust cross-compiling
Add environment variables so that cargo can find the appropriate
architecture-specific linker when cross-compiling. This is needed now as
we have a rust binary (fdpass-teleport) in the build, as opposed to just
rust compiled to a library linked to Go code. Rust does not have a cross
platform linker and relies an a linker in the toolchain for the target
architecture.
To make this work without needing per-architecture setup when building,
all of the toolchain binaries are symlinked into
/opt/thirdparty/host/bin so they are all in the path. Because each of
the binaries is prefixed with the target tuple, there are no name
clashes.
* buildbox: Add stages to buildbox Dockerfile for better caching
Download Go and Rust in separate stages and copy their installation into
the final container image. This helps as these stages no longer have
unrelated layers before them so they cache better. The Go and Rust
stages should only need to be rebuilt if the versions of the compilers
change, or some other aspect of the installation of the compilers
change. Previously adding a new package to be installed would cause Go
and Rust to be re-installed. This no longer happens.
* buildbox: Remove include of buildbox-common.mk in thirdparty-libs.mk
Remove the include of `buildbox-common.mk` from `thirdparty-libs.mk` as
it is included by `cross-compile.mk`. This caused some duplicate target
warnings when make ran.
* build: Add support for buildbox-ng to Makefile
Add supoprt for building in buildbox-ng to the build by including the
cross-compiling definitions if we are running in that buildbox and
gating some older cross-compiling definitions from being used.
* build: Add release targets using buildbox-ng
Add a set of targets to build releases using buildbox-ng instead of the
standard buildboxes. Ultimately these targets will be removed when the
old buildboxes are removed. These new targets are temporary until that
happens supporting testing of the new buildbox.
* buildbox: Add teleport user, expand comments
Add the teleport user (1000:1000) to the buildbox for running in CI
where a repository is checked-out inside the container instead of
mounting a volume on to /home/teleport. Make that home directory
world-writable so that it can still be used if 1000:1000 cannot be used
for some reason.
Expand and clean up comments.
Re-order ENV vars a little for better grouping.
Put Rust and Go temp directories under /tmp/build so a single volume can
be mounted there for persistent caches across runs.
* build: Fix invocation of pkg-config in $(shell ...)
Explicitly set `PKG_CONFIG_PATH` when running `$(PKGCONF)` as when
running that in a `$(shell ...)` expression, variables exported in the
Makefile are not exported for that shell, so `$(PKGCONF)` does not see
`PKG_CONFIG_PATH`. GNU make 4.4 changes this so that exported variables
are exported to `$(shell ...)` expressions, but the buildbox has GNU
make 4.3.
This was causing libbpf to not be detected properly so was building
teleport without bpf.
|
||
|
Cam Hutchison
|
10932655f8 |
build: Find feature headers outside system include dir (#43946)
* build: Use pkg-config to find bpf headers and libraries Use `pkg-config` / `pkgconf` to find the specific version of `libbpf` that we want to link to as well as the dependent libraries such as `libelf`, `libz`, and sometimes `libzstd`. This resolves the variable dependency on `libzstd` depending on the base OS. We still take preference of `/lib/libbpf-$(LIBBPF_VER)` if it exists as it does for the standard buildbox, but that is deprecated and will be removed when we move to the new buildbox. If there is no `libelf.pc` config, fall back to hard-coded libraries as it was previously. Again, this is for use with our existing CentOS 7 buildbox. This cleans up the BPF detection in `common.mk` as it had some head-scratching failure modes when clang or llvm-strip were not found. The logic for enabling BPF is clearer now and safer. * build: Also look in C_INCLUDE_PATH for pam_appl.h When looking for the PAM header `pam_appl.h` to determine if PAM should be enabled in teleport, first look in the directories specified in the `C_INCLUDE_PATH` environment variable before checking the system header directories. The buildbox-ng has per-architecture include directories and configures `C_INCLUDE_PATH` to find the correct headers. Fall back to the existing default directories if not found in `C_INCLUDE_PATH`. |
||
|
Jakub Nyckowski
|
988fea60c0 |
Enhance library dependency management with libelf check (#43981)
* Enhance library dependency management with libelf check Added logic to check for libelf presence and use pkgconf to dynamically include necessary static libraries when available. If libelf is not found, default to manually specified -lelf and -lz libraries. This ensures more reliable builds across different environments and addresses issues with Ubuntu 24.04. * Remove duplicated pkgconf library |
||
|
Isaiah Becker-MayerandZac Bergquist
|
d6fe42b2a1 |
Restrict cipher suites for Desktop Access in FIPS mode (#42277)
* Updates to the new boring fork's hash with FIPS enforcement. This hash is on a branch and should be changed once that branch is merged. It includes the `set_fips_compliance_policy` function which is used to enforce FIPS-valid ciphers in the connection. This commit also updates `Cargo.lock` generally by having called `cargo update` before committing the changes. * reverts previous changes such that we are again using clang 12.0.0 rather than 14.0.6 * Updates to latest boring hash * removes rdp client for fips builds on arm64 * updates e to isaiah/enforce-fips head * Updates boring ref * Updates boring ref * Update Makefile Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> * Adds note about desktop access fips to fedramp docs * Updates boring hash to now-merged HEAD of the `teleport` branch * update e to latest head * try adding --nocheck to see if that fixes arm64 builds * udates to latest HEAD * latest e * Update docs/pages/access-controls/compliance-frameworks/fedramp.mdx Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> * Update docs/pages/access-controls/compliance-frameworks/fedramp.mdx Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> * Update docs/pages/access-controls/compliance-frameworks/fedramp.mdx Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> * noteable --> notable --------- Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> |
||
|
|
f00f580cc2 |
Update to BoringSSL fips-20220613 tag (#36166)
* Add ninja-build setup to Dockerfile-centos7 The Dockerfile-centos7 has been updated to include the setup and installation of ninja-build. The version of ninja-build included in CentOS 7 AMR64 was too old for building boringssl, hence it needs to be built from source. Both additional dependencies and the source build steps have been added as part of this update. * Add CMake to the ninja build * Update Dockerfile to fix ARM64 build issue The Dockerfile for Centos7 was updated to better support ARM64 architectures. Instead of fetching a precompiled binary of CMake, we now clone CMake v3.28.1 and compile it specifically for the target. This change includes the addition of the `-mno-outline-atomics` flag, which resolves ARM64-specific issues with compilation. * Update Dockerfile to conditionally set CFLAGS for arm64 architecture The Dockerfile for CentOS7 now includes a condition that sets the CFLAGS environment for arm64 architecture. This change was made during the process of enabling CMake to bootstrap itself and ensures that correct flags are set for specific architectures during the build process. * Update Clang from 12 to 14 to match BoringSSL `fips-20220613` tag Matches what upstream Go is using for BoringCrypto build: https://github.com/golang/go/commit/7383b2a4db5dc93c9b875b42d5add73d27cc4b9f Also set `CMAKE=cmake3`, as `boring-rs` (by way of `cmake-rs`) calls `cmake`. Additionally, make `build-centos7-assets` a dependency. * Removes build-centos7-assets as a dependency for the build-centos7 target. Adds a comment to the Makefile to explain why. * Updated Clang build configuration in Dockerfile Updates clang to be built with the inclusion of the header files necessary for building boring. Also changes the build to use Ninja. * moves the ninja copy closer to the clang copy * Swap to fork of `boring-rs` for build fixes (#36720) BoringSSL does not build properly on CentOS 7 without some build fixes. As these fixes are not yet upstreamed, a separate fork is being used -- https://github.com/gravitational/boring. Co-authored-by: Isaiah Becker-Mayer <isaiah@goteleport.com> * Ninja build snuck back into Dockerfile-centos7, but we don't need it since it's in Dockerfile-centos7-assets now. Removing it. * Prevent docker buildkit from caching Use docker build to build fips image to prevent docker from fetching the base centos image from the registry. * Add 'nolint' comment in boringtest.go A 'nolint' comment was added in the boringtest go file to ignore linting this file * Rename test Go file --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com> Co-authored-by: Isaiah Becker-Mayer <isaiah@goteleport.com> |
||
|
Jakub Nyckowski
|
9accc2f9ff |
Remove restricted sessions (#36814)
* Remove restricted sessions * Remove missing reference to restricted session * Update CHANGELOG.md to remove outdated SSH feature The commit removes the reference to the deprecated and removed SSH restricted sessions feature from the CHANGELOG.md file. The restricted session feature had been phased out since Teleport 14, and this update reflects its removal in Teleport 15. * Rephrase the changelog note that it's recommends the implementation of network restrictions outside of Teleport like iptables and security groups. |
||
|
Reed Loden
|
1b62c1a077 |
Enable PIV and FIDO2 support on Linux FIPS builds (#35957)
Thanks to our new buildboxes, PIV and FIDO2 support are easy to enable. |
||
|
Reed Loden
|
91d3bedb7b |
Remove custom BoringSSL build (#33886)
The `boring` crate will compile BoringSSL on demand. Remove unneeded Clang 7 build and replace Clang 10 with Clang 12. BoringSSL in FIPS mode explicitly requires Clang 12.0.0, while libbpf and related tools only require Clang 10+, so standardized everything on Clang 12.0.0 so that we don't need multiple Clang installations. This also required libbpf to be bumped, as 1.0.1 no longer compiled. Both 1.1.x and 1.2.x seem to build fine, so went ahead and bumped to 1.2.2 (latest libbpf). As a result, `aquasecurity/libbpfgo` was also bumped to match the new version. Additionally, add a few missing git commit hash validations that were noticed as all the `Dockerfile`s were being reviewed/updated. |
||
|
Jakub Nyckowski
|
942c0652d1 |
Move makefile BPF to common.mk (#22485)
Currently, a lot of makefile logic is copied between OSS end Ent repo. This PR moves BPF-related code to common.mk, so it doesn't need to be copied between reports. I'll move more code in the following PRs. I just want to keep diffs small. |