* Bump gravitational/predicate to WIP commit
* Remove some allocations in predicate expressions
* Optimize contains(split()) in resource expression parser
* Do a type-safe assert on the result of astutil.Apply
* Keep the full value argument in the early exit of optimizeSplitContains
* Handle degenerate cases in splitContains
* Expand on the benchmark
* Rename LazyStringSlice to StringSlice
* Document StringSlice
* Fix test
* Fix handling of split on empty string
* Bump gravitational/predicate to 1.4.0
* Optimize multibyte delimiters too
* Update godoc for optimizeSplitContains
* Clarify the logic in splitContainsAffix
* Actually use multi-byte delimiters in BenchmarkSplitContains
* Revert "Actually use multi-byte delimiters in BenchmarkSplitContains"
This reverts commit 595bb85ba9.
* Revert "Optimize multibyte delimiters too"
This reverts commit f285928e99.
* Add some tests with overlap of target and delimiter
* Clarify the return value of the AST optimization visitors
* Add unit test just for the new string splitting
* Add some broken utf8 coverage too
* lib/utils/typical: support `==` and `!=` operators for integers
In #52460, we're adopting the Teleport predicate language for Workload Identity
rules. We currently allow you to match on the equality/inequality of the attested
process' PID, GID, and UID, but this wasn't supported in the predicate language.
We now check the operand types at compile-time and choose the correct
booleanOperator. If both operands are dynamically typed (e.g. `ifelse`) we do
not allow you to use the `==` or `!=` operators.
* Fix compile/parse time nomenclature
* Rename type parameters for clarity
This only converts the places in the utils package that were
logging with logrus, but it does not fully remove the logrus
package as a dependency. The package is home to the logrus text
formatter which needs to exist until slog is the only logging
mechanism in use.
The existing resource parser was converted to use typical under
the hood. Aside from the type safety improvements the making the
switch allows a single predicate expression to be parsed once and
evaluated per resource. The previous parser worked backwards, it
created a parser per resource with the same expression which lead
to elevated memory consumption and poor performance.
An additional bug was fixed that prevented predicate filtering from
working properly with the unified resource API due to applying
predicates prior to filtering by kind. This resulted in queriers
for server specific things (resource.spec.hostname) to fail even
when only the server resource was specified in the resource kinds.
Benchamrk results for the changes in this commit compared to master
show a 75% reduction in the test time and a 96% reduction in allocations
during the tests.
```bash
$ benchstat e77e5e080b1848da830a9030d43c38be55a5b8d5.txt 8eff752a93dfe96225a71431af829015e83400fa.txt
goos: darwin
goarch: arm64
pkg: github.com/gravitational/teleport/lib/auth
│ e77e5e080b1848da830a9030d43c38be55a5b8d5.txt │ 8eff752a93dfe96225a71431af829015e83400fa.txt │
│ sec/op │ sec/op vs base │
ListUnifiedResourcesFilter/labels-12 73.88m ± 1% 64.74m ± 6% -12.37% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_path-12 294.06m ± 20% 74.83m ± 5% -74.55% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_index-12 307.63m ± 4% 76.32m ± 4% -75.19% (p=0.000 n=10)
ListUnifiedResourcesFilter/search_lower-12 130.7m ± 5% 130.5m ± 4% ~ (p=0.971 n=10)
ListUnifiedResourcesFilter/search_upper-12 147.4m ± 5% 138.7m ± 2% -5.87% (p=0.003 n=10)
ListUnifiedResources/simple_labels-12 1.748 ± 2% 1.744 ± 2% ~ (p=0.631 n=10)
geomean 246.6m 150.6m -38.93%
│ e77e5e080b1848da830a9030d43c38be55a5b8d5.txt │ 8eff752a93dfe96225a71431af829015e83400fa.txt │
│ B/op │ B/op vs base │
ListUnifiedResourcesFilter/labels-12 735.2Ki ± 0% 724.6Ki ± 1% -1.44% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_path-12 238.873Mi ± 0% 7.585Mi ± 0% -96.82% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_index-12 238.869Mi ± 0% 7.587Mi ± 0% -96.82% (p=0.000 n=10)
ListUnifiedResourcesFilter/search_lower-12 16.77Mi ± 0% 16.77Mi ± 0% ~ (p=0.436 n=10)
ListUnifiedResourcesFilter/search_upper-12 16.78Mi ± 0% 16.76Mi ± 0% ~ (p=0.052 n=10)
ListUnifiedResources/simple_labels-12 1.272Gi ± 0% 1.273Gi ± 0% ~ (p=0.739 n=10)
geomean 49.67Mi 15.69Mi -68.41%
│ e77e5e080b1848da830a9030d43c38be55a5b8d5.txt │ 8eff752a93dfe96225a71431af829015e83400fa.txt │
│ allocs/op │ allocs/op vs base │
ListUnifiedResourcesFilter/labels-12 11.45k ± 0% 11.41k ± 0% -0.37% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_path-12 4962.0k ± 0% 161.5k ± 0% -96.75% (p=0.000 n=10)
ListUnifiedResourcesFilter/predicate_index-12 4812.0k ± 0% 161.5k ± 0% -96.64% (p=0.000 n=10)
ListUnifiedResourcesFilter/search_lower-12 161.6k ± 0% 161.6k ± 0% ~ (p=0.419 n=10)
ListUnifiedResourcesFilter/search_upper-12 161.7k ± 0% 161.6k ± 0% ~ (p=0.116 n=10)
ListUnifiedResources/simple_labels-12 23.70M ± 0% 23.70M ± 0% +0.01% (p=0.019 n=10)
geomean 743.7k 238.5k -67.93%
```
Provides an opt-in mechanism to allow parsers to support dynamically
resolving variables at runtime. While this defeats some of the
static compile type benefits that typical guarantees, it is required
to allow existing parsers to be converted to use typical.
The existing resource parsers allow searching for variabls by their
fully qualified path. For example, the hostname of a server may
be quiered via `resource.spec.hostname`. In order to support this
typical could use some reflection to automatically add variables
to every nested field but that would be complicated and expensive.
Instead the ParserSpec can now be configured with
`GetUnknownIdentifier`, which allows creators of the parsers to
have full control over how a dynamic variable is resolved. The
parser prevents any dynamic resolution if the value is not configured
to limit the unsafe behvior to only parsers that explicitly opt in.
Prefer the stdlib version of the experimental package.
Includes a depguard update to prevent future regressions,
as well as an e-ref update to appease depguard.
Generated with:
$ gofmt -w -r '"golang.org/x/exp/slices" -> "slices"' . && \
make fix-imports
This commit adds the new function `labels_matching` to the label
expression syntax. It accepts a Teleport-style glob or regular
expression, and returns the aggregate of all values for keys matching
that expression.
Since it is now more likely for users to want to compare two slices,
I've also added the `contains_any` and `contains_all` functions to
complement the existing `contains` function.
This commit also adds the `email.local` function that is already
available in role templates and should be useful.
Some examples are included in the new test cases.
`lib/utils/typical` is a new library for building predicate parsers
based on the examples set by `lib/utils/parse` and `e/lib/loginrule`.
The commit updates `lib/utils/parse` to use the `typical` library to
implement the two parsers defined here.
This removes a lot of code in `ast.go` that was written for each
specific function to do type checking and deal with subexpressions. This
is all handled by the `typical` library now. My hope is that this will
make it much easier to add new functions in the future and to share them
with other parsers.
* feat: library for building predicate parsers
Took some things I learned while building Login Rules and implemented a
library that can be used to quickly implement new and improved predicate
parsers.
Copied from the package comment:
typical (TYPed predICAte Library) is a library for building better
predicate expression parsers faster. It is built on top of
[predicate](github.com/gravitational/predicate).
typical helps you (and forces you) to separate parse and evaluation into
two distinct stages so that parsing can be cached and evaluation can be
fast. Expressions can also be parsed to check their syntax without
needing to evaluate them with some specific input.
Functions can be defined by providing implementations accepting and returning
values of ordinary types, the library handles the details that enable any
function argument to be a subexpression that evaluates to the correct type.
By default, typical provides strong type checking at parse time without any
reflection during evaluation. If you define a function that evaluates to
type any, you are opting in to evaluation-time type checking everywhere that
function is called, but types will still be checked for all other parts of
the expression. If you define a function that accepts an interface type it
will be called via reflection during evaluation, but interface satisfaction
is still checked at parse time.
* fix typo in error message
Co-authored-by: Tim Buckley <tim@goteleport.com>
---------
Co-authored-by: Tim Buckley <tim@goteleport.com>
Edoardo Spadolini