* Remove logging from circuit breaker
The logging is never output, and if it was would be very verbose.
Instead of dropping the level down to trace the logger is removed
entirely.
* Add constants from trace library
The logging functionality in trace is being removed in an upcoming
release. To prepare the logging component keys are copied from trace
into teleport.
* Convert trace.ComponentFields to teleport.ComponentFields
* Replace trace.Component with teleport.ComponentKey
* Replace trace logging constants
* ensure moderated file transfers only preform allowed operations
* fix sess test
* wip integration test
* integration test working
* tweak some error messages and test cases
* allow setstat for uploads
* address nits from code review
* expand paths when necessary when creating file transfer request
* improve UX of errors in web UI
* deny request when an invalid user tries to transfer files
* don't allow open requests, they aren't done by the webui
* address feedback
* use a buffered reader when reading the file request, address a few nits
Adds the new linter and configures it to prevent direct error
comparisions and type assertions in favor of errors.Is and errors.As.
All current violations caught by the linter have been updated
accordingly.
* Special case the subsystems handled by teleport exec
This only applies to sftp, and lets us do a real reexec where that's
possible (i.e. launching /proc/self/exe on Linux), making sftp work even
if Teleport has been replaced by some wrapper or has been deleted.
* Delete leftover comment
* Update godoc for ExecCommand.Command
* Added Fuzz Seeds for services/fuzz_test.go
* Fuzzing improvements for `lib/srv/db` fuzzing
This ensures that all fuzzing under `lib/srv/db` has the following:
* Good seeds that provide basic levels of coverage
* Enabled in OSS-Fuzz
* Fuzzing improvements for `lib/sshutils`
This improves the fuzzing in `lib/sshutils/sftp/parse_test.go` and `lib/sshutils/x11/display_test.go`.
Seeds were added and additional local fuzzing was done. In addition these functions are now enabled in `oss-fuzz`.
A complex regex was previously used to parse 'tsh scp'
destinations, which was hard to understand and even harder
to maintain for those not intimately familiar with regex,
despite it being heavily commented. Instead parse destinations
directly and add more test cases and a fuzz test to ensure
parsing won't panic and the new parse function doesn't
have any regressions.
Previously a Teleport client using SFTP would resolve remote host user
home directories by making a subsystem request to a Teleport server
which would return the home directory. The problem was the subsystem
request counted as an open session, which could make the SFTP file
transfer fail. This was frustrating and didn't make much sense, but
after reading the SFTP specification again I realized that SFTP servers
are to handle relative paths by assuming they start at the user's home
directory. So let the server figure out the correct path and remove any
tilde prefixes from remote paths.
* refactor SFTP backend to use upstream dep, not our fork
This change also greatly reduces the number of SFTP audit logs.
Now SFTP events are only sent when files are opened or modified
in any way, instead of for *every* SFTP request.
* added to SFTP integration test
* fix error when handling setstat on dirs
* fix linter warning
* move file/dir permission constants to lib/defaults package
* use 'path.Join' instead of 'filepath.Join' to build remote file paths
tsh on Windows when using 'filepath.Join' would create paths with forward
slashes in them, which non-Windows Nodes couldn't handle. Consistently use
'path.Join' to avoid issues.
Fixes#18919
* add comment explaining why 'path.Join' is used
* move comment to make linter happy
* Our current SFTP implementation uses a single channel to transfer data. Connections with a high latency cause very slow transfer. github.com/pkg/sftp allows transferring data using concurrent streams, which seems to solve the problem. In order to utilize the feature, the remote file must implement the WriteTo() or ReadFrom() and Stat() methods. I've moved the progress bar to relay the data from a local file, as we can still call Read() or Write() on it to update the progress.
This PR doesn't address the high CPU usage by github.com/pkg/sftp on low latency links.
Closes#20579
* Cleanup
* Fix fileTransferProgress
* Simplify interfaces
* Move helper to utils package
Add comments
* Added comments
* Change canceler implementation to return correct values from Read/Write
* Fix infinite recursive call in cancelWriter.
* switch underlying protocol used for 'tsh scp' to SFTP
* address TODO
* appease linter
* add method to make it easier for other callers to transfer files
* add tests
* print transfer progress with progress bar by default
Also allow a SIGINT to gracefully stop the SFTP connection. This is
necessary because the progress bar will ignore signals and prevent the
process from exiting.
* address SFTP fork issues
* make tests less flakey
* fix specifying dir for dst not copying files to correct paths
* make tests less flakey (again)
* don't check file access times, often differs when run in CI
* few small fixes from review, simplify Create method now that HTTP FS isn't needed
* create dst files and dirs with src mode
* improved error messages when doing file operations
* expand home dirs in remote paths
* addressed more feedback
* add license to get_home_dir.go
* address minor feedback of tests, add home dir expansion test
* update sftp fork to point to latest commit on master branch
* addressed feedback
* don't cache home dir lookups, only one remote path can ever be used
rosstimothy