Files
teleport/lib/utils/cli.go
T
32e48017d3 Implement tbot init subcommand and ACL management (#10289)
* Add certificate renewal bot

This adds a new `tbot` tool to continuously renew a set of
certificates after registering with a Teleport cluster using a
similar process to standard node joining.

This makes some modifications to user certificate generation to allow
for certificates that can be renewed beyond their original TTL, and
exposes new gRPC endpoints:
 * `CreateBotJoinToken` creates a join token for a bot user
 * `GenerateInitialRenewableUserCerts` exchanges a token for a set of
   certificates with a new `renewable` flag set

A new `tctl` command, `tctl bots add`, creates a bot user and calls
`CreateBotJoinToken` to issue a token. A bot instance can then be
started using a provided command.

* Cert bot refactoring pass

* Use role requests to split renewable certs from end-user certs
* Add bot configuration file
* Use `teleport.dev/bot` label
* Remove `impersonator` flag on initial bot certs
* Remove unnecessary `renew` package
* Misc other cleanup

* Do not pass through `renewable` flag when role requests are set

This adds additional restrictions on when a certificate's `renewable`
flag is carried over to a new certificate. In particular, it now also
denies the flag when either role requests are present, or the
`disallowReissue` flag has been previously set.

In practice `disallow-reissue` would have prevented any undesired
behavior but this improves consistency and resolves a TODO.

* Various tbot UX improvements; render SSH config

* Fully flesh out config template rendering
* Fix rendering for SSH configuration templates
* Added `String()` impls for destination types
* Improve certificate renewal logging; show more detail
* Properly fall back to default (all) roles
* Add mode hints for files
* Add/update copyright headers

* Add stubs for tbot init and watch commands

* Add gRPC endpoints for managing bots

* Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints
* Replace `tctl bot (add|rm|ls)` implementations with gRPC calls
* Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`,
  `BotGenerationLabel`

* Fix outdated destination flag in example tbot command

* Bugfix pass for demo

* Fixed a few nil pointer derefs when using config from CLI args
* Properly create destination if `--destination-dir` flag is used
* Remove improper default on CLI flag
* `DestinationConfig` is now a list of pointers

* Address first wave of review feedback

Fixes the majority of smaller issues caught by reviewers, thanks all!

* Add doc comments for bot.go functions

* Return the token TTL from CreateBot

* Split initial user cert issuance from `generateUserCerts()`

Issuing initial renewable certificate ended up requiring a lot of
hacks to skip checks that prevented anonymous bots from getting
certs even though we'd verified their identity elsewhere (via token).

This reverts all those hacks and splits initial bot cert logic into a
dedicated `generateInitialRenewableUserCerts()` function which should
make the whole process much easier to follow.

* Set bot traits to silence log messages

* tbot log message consistency pass

* Implement `tbot init` subcommand

This adds a new CLI subcommand to initialize a tbot destination
directory by creating required files ahead of time and assigning
proper permissions (and ACLs, where possible).

* Resolve lints

* Add config tests

* Remove CreateBotJoinToken endpoint

Users should instead use the CreateBot/DeleteBot endpoints.

* Create a fresh private key for every impersonated identity renewal

* Hide `config` subcommand

* Rename bot label prefix to `teleport.internal/`

* Use types.NewRole() to create bot roles

* Clean up error handling in custom YAML unmarshallers

Also, add notes about the supported YAML shapes.

* Fetch proxy host via gRPC Ping() instead of GetProxies()

* Update lib/auth/bot.go

Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>

* Fix some review comments

* Add renewable certificate generation checks (#10098)

* Add renewable certificate generation checks

This adds a new validation check for renewable certificates that
maintains a renewal counter as both a certificate extension and a
user label. This counter is used to ensure only a single certificate
lineage can exist: for example, if a renewable certificate is stolen,
only one copy of the certificate can be renewed as the generation
counter will not match

When renewing a certificate, first the generation counter presented
by the user (via their TLS identity) is compared to a value stored
with the associated user (in a new `teleport.dev/bot-generation`
label field). If they aren't equal, the renewal attempt fails.
Otherwise, the generation counter is incremented by 1, stored to the
database using a `CompareAndSwap()` to ensure atomicity, and set on
the generated certificate for use in future renewals.

* Add unit tests for the generation counter

This adds new unit tests to exercise the generation counter checks.

Additionally, it fixes two other renewable cert tests that were
failing.

* Remove certRequestGeneration() function

* Emit audit event when cert generations don't match

* Fully implement `tctl bots lock`

* Show bot name in `tctl bots ls`

* Lock bots when a cert generation mismatch is found

* Make CompareFailed respones from validateGenerationLabel() more actionable

* Update lib/services/local/users.go

Co-authored-by: Nic Klaassen <nic@goteleport.com>

* Backend changes for tbot IoT and AWS joining (#10360)

* backend changes

* add token permission check

* pass ctx from caller

Co-authored-by: Roman Tkachenko <roman@goteleport.com>

* fix comment typo

Co-authored-by: Roman Tkachenko <roman@goteleport.com>

* use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event

* Client changes for tbot IoT joining (#10397)

* client changes

* delete replaced APIs

* delete unused tbot/auth.go

* add license header

* don't unecessarily fetch host CA

* log fixes

* s/tunnelling/tunneling/

Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>

* auth server addresses may be proxies

Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>

* comment typo fix

Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>

* move *Server methods out of auth_with_roles.go (#10416)

Co-authored-by: Tim Buckley <tim@goteleport.com>

Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
Co-authored-by: Tim Buckley <tim@goteleport.com>

Co-authored-by: Roman Tkachenko <roman@goteleport.com>
Co-authored-by: Tim Buckley <tim@goteleport.com>
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>

Co-authored-by: Nic Klaassen <nic@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>

* Address another batch of review feedback

* Addres another batch of review feedback

Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()`
calls, clear some TODOs and lints, and address other misc feedback
items.

* Fix lint

* Add missing doc comments to SaveIdentity / LoadIdentity

* Remove pam tag from tbot build

* Update note about bot lock deletion

* Another pass of review feedback

Ensure all requestable roles exist when creating a bot, adjust the
default renewable cert TTL down to 1 hour, and check types during
`CompareAndSwapUser()`

* Remove ModeHint

* Rename Identity.Cert and Identity.XCert

* Add `symlinks` flag to tbot config

The optional symlinks flag for directory destinations allows users to
opt in / out of whichever symlink attack hardening mode is selected
by default.

* Add mostly-working secure implementation of botfs.Create/Write

This adds symlink mode selection (secure, try-secure, insecure) and
Linux `Create()`/`Write()` implementations to open files safely.

* Add configurable ACL modes and verify ACL support in tbot init

* Initialize destinations at startup and test before renewal

This initializes destinations at startup (to create directories if
not using `tbot init`) and tests them to ensure the bot can write
_before_ attempting to renew certificates; this should prevent most
accidental generation counter locks.

* Hide watch for now

* Issue a new identity if a token change is detected

* Warn if identity appears to be expired on startup

* Fully implement ACL Verify and Configure

 - Fully implements ACL support for Linux
 - Adds bot-side verification support to ensure ACLs are configured
   properly at runtime.
 - Gracefully falls back to no ACLs if the platform / filesystem
   doesn't support them
 - Clear up outstanding lints

* Make `tbot init` work without a config file

* Show init instructions in tctl bots add

Also:
 - Make --bot-user a flag in init (the tctl instructions were
   confusing otherwise)
 - Handle IsOwnedBy sanely on unsupported platforms
 - Add Bold colorizing support

* Clear some TODOs and rephrase tctl help

* Fix typo

* Fix token hash detection bug

* Actually read and write certs with symlink enforcement

Also, fix a config loading bug where CheckAndSetDefaults() wasn't
being called in all cases with CLI destinations.

* Add workaround for OpenSSH permissions check with ACLs

OpenSSH has an overly-paranoid permissions check that forces key
files to be exclusively owner-readable. Unfortunately, for POSIX
compatibility purposes, when ACLs are set, the ACL mask is set as
the group permissions. This effectively makes any ACL incompatible
with OpenSSH.

However, OpenSSH's check does have an escape hatch: it only applies
if the current user is the owner of the file. Therefore, this change
tweaks the `tbot init` flow to create files as root, owned by a
separate user (either `nobody` or even the bot user), with ACL
permissions granting both the bot and reader user access to the
certificates. This effectively bypasses OpenSSH's permissions check
and should preserve our security boundaries.

* Fix lints

* Fix an improper directory chmod to 0600 if ACL test fails

* First pass of tbot init unit tests

* Add symlink tests and fix bug with resolving the default owner

* Fix err misuse

* Fix an ACL error if the bot or reader user is the owner.

* Fix typo

* Fix missing error case in VerifyACL causing unreadable directories

* Address review feedback

- Rename ACLOn -> ACLRequired
- Simplify fs_linux.Read()
- Add missing fs_other.Read()
- Hoist renewal loop logic into its own function
- A few misc bugfixes

* Apply suggestions from code review

Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>

* Address review feedback

- Only log syscall warning once
- Formatting and wording changes
- Improve error handling for `--clean`

* Fix lint error

* Fix imports in fs_other

* Fix possible nil pointer deref if storage is unset

* Use the bot user as default owner

This is more likely to be a safe owner choice than `nobody:nobody`.

* Apply suggestions from code review

Co-authored-by: Roman Tkachenko <roman@goteleport.com>

* Code review fixes

Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
Co-authored-by: Nic Klaassen <nic@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00

441 lines
13 KiB
Go

/*
Copyright 2016-2021 Gravitational, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package utils
import (
"bytes"
"crypto/x509"
"flag"
"fmt"
"io"
"io/ioutil"
stdlog "log"
"math"
"os"
"runtime"
"strconv"
"strings"
"testing"
"unicode"
"github.com/gravitational/teleport"
"github.com/gravitational/teleport/api/constants"
"github.com/sirupsen/logrus"
log "github.com/sirupsen/logrus"
"github.com/gravitational/kingpin"
"github.com/gravitational/trace"
)
type LoggingPurpose int
const (
LoggingForDaemon LoggingPurpose = iota
LoggingForCLI
)
// InitLogger configures the global logger for a given purpose / verbosity level
func InitLogger(purpose LoggingPurpose, level log.Level, verbose ...bool) {
log.StandardLogger().ReplaceHooks(make(log.LevelHooks))
log.SetLevel(level)
switch purpose {
case LoggingForCLI:
// If debug logging was asked for on the CLI, then write logs to stderr.
// Otherwise, discard all logs.
if level == log.DebugLevel {
log.SetFormatter(NewDefaultTextFormatter(trace.IsTerminal(os.Stderr)))
log.SetOutput(os.Stderr)
} else {
log.SetOutput(ioutil.Discard)
}
case LoggingForDaemon:
log.SetFormatter(NewDefaultTextFormatter(trace.IsTerminal(os.Stderr)))
log.SetOutput(os.Stderr)
}
}
// InitLoggerForTests initializes the standard logger for tests.
func InitLoggerForTests() {
// Parse flags to check testing.Verbose().
flag.Parse()
logger := log.StandardLogger()
logger.ReplaceHooks(make(log.LevelHooks))
log.SetFormatter(NewTestTextFormatter())
logger.SetLevel(log.DebugLevel)
logger.SetOutput(os.Stderr)
if testing.Verbose() {
return
}
logger.SetLevel(log.WarnLevel)
logger.SetOutput(ioutil.Discard)
}
// NewLoggerForTests creates a new logger for test environment
func NewLoggerForTests() *log.Logger {
logger := log.New()
logger.ReplaceHooks(make(log.LevelHooks))
logger.SetFormatter(NewTestTextFormatter())
logger.SetLevel(log.DebugLevel)
logger.SetOutput(os.Stderr)
return logger
}
// WrapLogger wraps an existing logger entry and returns
// an value satisfying the Logger interface
func WrapLogger(logger *log.Entry) Logger {
return &logWrapper{Entry: logger}
}
// NewLogger creates a new empty logger
func NewLogger() *log.Logger {
logger := log.New()
logger.SetFormatter(NewDefaultTextFormatter(trace.IsTerminal(os.Stderr)))
return logger
}
// Logger describes a logger value
type Logger interface {
log.FieldLogger
// GetLevel specifies the level at which this logger
// value is logging
GetLevel() log.Level
// SetLevel sets the logger's level to the specified value
SetLevel(level log.Level)
}
// FatalError is for CLI front-ends: it detects gravitational/trace debugging
// information, sends it to the logger, strips it off and prints a clean message to stderr
func FatalError(err error) {
fmt.Fprintln(os.Stderr, UserMessageFromError(err))
os.Exit(1)
}
// GetIterations provides a simple way to add iterations to the test
// by setting environment variable "ITERATIONS", by default it returns 1
func GetIterations() int {
out := os.Getenv(teleport.IterationsEnvVar)
if out == "" {
return 1
}
iter, err := strconv.Atoi(out)
if err != nil {
panic(err)
}
log.Debugf("Starting tests with %v iterations.", iter)
return iter
}
// UserMessageFromError returns user-friendly error message from error.
// The error message will be formatted for output depending on the debug
// flag
func UserMessageFromError(err error) string {
if err == nil {
return ""
}
if log.GetLevel() == log.DebugLevel {
return trace.DebugReport(err)
}
var buf bytes.Buffer
if runtime.GOOS == constants.WindowsOS {
// TODO(timothyb89): Due to complications with globally enabling +
// properly resetting Windows terminal ANSI processing, for now we just
// disable color output. Otherwise, raw ANSI escapes will be visible to
// users.
fmt.Fprint(&buf, "ERROR: ")
} else {
fmt.Fprint(&buf, Color(Red, "ERROR: "))
}
formatErrorWriter(err, &buf)
return buf.String()
}
// FormatErrorWithNewline returns user friendly error message from error.
// The error message is escaped if necessary. A newline is added if the error text
// does not end with a newline.
func FormatErrorWithNewline(err error) string {
message := formatError(err)
if !strings.HasSuffix(message, "\n") {
message = message + "\n"
}
return message
}
// formatError returns user friendly error message from error.
// The error message is escaped if necessary
func formatError(err error) string {
var buf bytes.Buffer
formatErrorWriter(err, &buf)
return buf.String()
}
// formatErrorWriter formats the specified error into the provided writer.
// The error message is escaped if necessary
func formatErrorWriter(err error, w io.Writer) {
if err == nil {
return
}
if certErr := formatCertError(err); certErr != "" {
fmt.Fprintln(w, certErr)
return
}
// If the error is a trace error, check if it has a user message embedded in
// it, if it does, print it, otherwise escape and print the original error.
if traceErr, ok := err.(*trace.TraceErr); ok {
for _, message := range traceErr.Messages {
fmt.Fprintln(w, AllowNewlines(message))
}
fmt.Fprintln(w, AllowNewlines(trace.Unwrap(traceErr).Error()))
return
}
strErr := err.Error()
// Error can be of type trace.proxyError where error message didn't get captured.
if strErr == "" {
fmt.Fprintln(w, "please check Teleport's log for more details")
} else {
fmt.Fprintln(w, AllowNewlines(err.Error()))
}
}
func formatCertError(err error) string {
switch innerError := trace.Unwrap(err).(type) {
case x509.HostnameError:
return fmt.Sprintf("Cannot establish https connection to %s:\n%s\n%s\n",
innerError.Host,
innerError.Error(),
"try a different hostname for --proxy or specify --insecure flag if you know what you're doing.")
case x509.UnknownAuthorityError:
return `WARNING:
The proxy you are connecting to has presented a certificate signed by a
unknown authority. This is most likely due to either being presented
with a self-signed certificate or the certificate was truly signed by an
authority not known to the client.
If you know the certificate is self-signed and would like to ignore this
error use the --insecure flag.
If you have your own certificate authority that you would like to use to
validate the certificate chain presented by the proxy, set the
SSL_CERT_FILE and SSL_CERT_DIR environment variables respectively and try
again.
If you think something malicious may be occurring, contact your Teleport
system administrator to resolve this issue.
`
case x509.CertificateInvalidError:
return fmt.Sprintf(`WARNING:
The certificate presented by the proxy is invalid: %v.
Contact your Teleport system administrator to resolve this issue.`, innerError)
default:
return ""
}
}
const (
// Bold is an escape code to format as bold or increased intensity
Bold = 1
// Red is an escape code for red terminal color
Red = 31
// Yellow is an escape code for yellow terminal color
Yellow = 33
// Blue is an escape code for blue terminal color
Blue = 36
// Gray is an escape code for gray terminal color
Gray = 37
)
// Color formats the string in a terminal escape color
func Color(color int, v interface{}) string {
return fmt.Sprintf("\x1b[%dm%v\x1b[0m", color, v)
}
// Consolef prints the same message to a 'ui console' (if defined) and also to
// the logger with INFO priority
func Consolef(w io.Writer, log log.FieldLogger, component, msg string, params ...interface{}) {
msg = fmt.Sprintf(msg, params...)
log.Info(msg)
if w != nil {
component := strings.ToUpper(component)
// 13 is the length of "[KUBERNETES]", which is the longest component
// name prefix we have *today*. Use a Max function here to avoid
// negative spacing, in case we add longer component names.
spacing := int(math.Max(float64(12-len(component)), 0))
fmt.Fprintf(w, "[%v]%v %v\n", strings.ToUpper(component), strings.Repeat(" ", spacing), msg)
}
}
// InitCLIParser configures kingpin command line args parser with
// some defaults common for all Teleport CLI tools
func InitCLIParser(appName, appHelp string) (app *kingpin.Application) {
app = kingpin.New(appName, appHelp)
// hide "--help" flag
app.HelpFlag.Hidden()
app.HelpFlag.NoEnvar()
// set our own help template
return app.UsageTemplate(defaultUsageTemplate)
}
// SplitIdentifiers splits list of identifiers by commas/spaces/newlines. Helpful when
// accepting lists of identifiers in CLI (role names, request IDs, etc).
func SplitIdentifiers(s string) []string {
return strings.FieldsFunc(s, func(r rune) bool {
return r == ',' || unicode.IsSpace(r)
})
}
// EscapeControl escapes all ANSI escape sequences from string and returns a
// string that is safe to print on the CLI. This is to ensure that malicious
// servers can not hide output. For more details, see:
// * https://sintonen.fi/advisories/scp-client-multiple-vulnerabilities.txt
func EscapeControl(s string) string {
if needsQuoting(s) {
return fmt.Sprintf("%q", s)
}
return s
}
// AllowNewlines escapes all ANSI escape sequences except newlines from string and returns a
// string that is safe to print on the CLI. This is to ensure that malicious
// servers can not hide output. For more details, see:
// * https://sintonen.fi/advisories/scp-client-multiple-vulnerabilities.txt
func AllowNewlines(s string) string {
if !strings.Contains(s, "\n") {
return EscapeControl(s)
}
parts := strings.Split(s, "\n")
for i, part := range parts {
parts[i] = EscapeControl(part)
}
return strings.Join(parts, "\n")
}
// NewStdlogger creates a new stdlib logger that uses the specified leveled logger
// for output and the given component as a logging prefix.
func NewStdlogger(logger LeveledOutputFunc, component string) *stdlog.Logger {
return stdlog.New(&stdlogAdapter{
log: logger,
}, component, stdlog.LstdFlags)
}
// Write writes the specified buffer p to the underlying leveled logger.
// Implements io.Writer
func (r *stdlogAdapter) Write(p []byte) (n int, err error) {
r.log(string(p))
return len(p), nil
}
// stdlogAdapter is an io.Writer that writes into an instance
// of logrus.Logger
type stdlogAdapter struct {
log LeveledOutputFunc
}
// LeveledOutputFunc describes a function that emits given
// arguments at a specific level to an underlying logger
type LeveledOutputFunc func(args ...interface{})
// GetLevel returns the level of the underlying logger
func (r *logWrapper) GetLevel() logrus.Level {
return r.Entry.Logger.GetLevel()
}
// SetLevel sets the logging level to the given value
func (r *logWrapper) SetLevel(level logrus.Level) {
r.Entry.Logger.SetLevel(level)
}
// logWrapper wraps a log entry.
// Implements Logger
type logWrapper struct {
*logrus.Entry
}
// needsQuoting returns true if any non-printable characters are found.
func needsQuoting(text string) bool {
for _, r := range text {
if !strconv.IsPrint(r) {
return true
}
}
return false
}
// Usage template with compactly formatted commands.
var defaultUsageTemplate = `{{define "FormatCommand"}}\
{{if .FlagSummary}} {{.FlagSummary}}{{end}}\
{{range .Args}} {{if not .Required}}[{{end}}<{{.Name}}>{{if .Value|IsCumulative}}...{{end}}{{if not .Required}}]{{end}}{{end}}\
{{end}}\
{{define "FormatCommands"}}\
{{range .FlattenedCommands}}\
{{if not .Hidden}}\
{{.FullCommand | printf "%-12s" }}{{if .Default}} (Default){{end}} {{ .Help }}
{{end}}\
{{end}}\
{{end}}\
{{define "FormatUsage"}}\
{{template "FormatCommand" .}}{{if .Commands}} <command> [<args> ...]{{end}}
{{if .Help}}
{{.Help|Wrap 0}}\
{{end}}\
{{end}}\
{{if .Context.SelectedCommand}}\
usage: {{.App.Name}} {{.Context.SelectedCommand}}{{template "FormatUsage" .Context.SelectedCommand}}
{{else}}\
Usage: {{.App.Name}}{{template "FormatUsage" .App}}
{{end}}\
{{if .Context.Flags}}\
Flags:
{{.Context.Flags|FlagsToTwoColumnsCompact|FormatTwoColumns}}
{{end}}\
{{if .Context.Args}}\
Args:
{{.Context.Args|ArgsToTwoColumns|FormatTwoColumns}}
{{end}}\
{{if .Context.SelectedCommand}}\
{{ if .Context.SelectedCommand.Commands}}\
Commands:
{{if .Context.SelectedCommand.Commands}}\
{{template "FormatCommands" .Context.SelectedCommand}}
{{end}}\
{{end}}\
{{else if .App.Commands}}\
Commands:
{{template "FormatCommands" .App}}
Try '{{.App.Name}} help [command]' to get help for a given command.
{{end}}\
{{ if .Context.SelectedCommand }}\
Aliases:
{{ range .Context.SelectedCommand.Aliases}}\
{{ . }}
{{end}}\
{{end}}
`