* SCP -Change file attrs only when requested
OpenSSH sets 0644 by default and allows copy the source permissions by passing `-p` flag.
Our current implementation sets the file permissions even when client didn't use `-p` flag which can fail if the user doesn't have permissions to set it. This patch matches the OpenSSH behavior.
* Fix formatting
* Our current SFTP implementation uses a single channel to transfer data. Connections with a high latency cause very slow transfer. github.com/pkg/sftp allows transferring data using concurrent streams, which seems to solve the problem. In order to utilize the feature, the remote file must implement the WriteTo() or ReadFrom() and Stat() methods. I've moved the progress bar to relay the data from a local file, as we can still call Read() or Write() on it to update the progress.
This PR doesn't address the high CPU usage by github.com/pkg/sftp on low latency links.
Closes#20579
* Cleanup
* Fix fileTransferProgress
* Simplify interfaces
* Move helper to utils package
Add comments
* Added comments
* Change canceler implementation to return correct values from Read/Write
* Fix infinite recursive call in cancelWriter.
* Enable nolintlint linter
* Fix nolint comments in the api package
* Fix RDP client comment
* Address review comment
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Allow unused for nolintlint linter
* Remove redundant casting
* Add comment on why allowed unused is enabled
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
Our need for customizing our security response headers primarily exists with the Content-Security-Policy. Other headers can be applied more broadly to provide safe defaults across our service.
This commit changes httpheaders.go to allow for the CSP to be applied separate from the other headers. Then the changes to httplib.go's handlers help make sure these headers are set generally (which we were already partially doing for caching directives).
Other changes were primarily making sure these tools are configured or used. Or a removal of previous settings as this function is now being handled more centrally.
* switch underlying protocol used for 'tsh scp' to SFTP
* address TODO
* appease linter
* add method to make it easier for other callers to transfer files
* add tests
* print transfer progress with progress bar by default
Also allow a SIGINT to gracefully stop the SFTP connection. This is
necessary because the progress bar will ignore signals and prevent the
process from exiting.
* address SFTP fork issues
* make tests less flakey
* fix specifying dir for dst not copying files to correct paths
* make tests less flakey (again)
* don't check file access times, often differs when run in CI
* few small fixes from review, simplify Create method now that HTTP FS isn't needed
* create dst files and dirs with src mode
* improved error messages when doing file operations
* expand home dirs in remote paths
* addressed more feedback
* add license to get_home_dir.go
* address minor feedback of tests, add home dir expansion test
* update sftp fork to point to latest commit on master branch
* addressed feedback
* don't cache home dir lookups, only one remote path can ever be used
Update metalinter, fix a few lint warnings and replace deprecated linters.
`deadcode`, `structcheck` and `varcheck` are abandoned and now replaced by [`unused`][1].
Since 1.19, `go fmt` reformats godocs according to https://go.dev/doc/comment. I've done a bulk-reformatting of the codebase to keep the linter happy. Backporting is mostly harmless (the exception being `lib/services/role_test.go`, that for some reason breaks the _old_ linter using the new format).
[1]: https://golangci-lint.run/usage/linters/
* Bump golangci-lint version
* Replace abandoned linters
* Fix bodyclose on lib/auth/github.com
* Fix bodyclose on lib/kube/proxy/streamproto/proto_test.go
* Fix bodyclose on lib/srv/alpnproxy/proxy_test.go
* Fix bodyclose on lib/web/conn_upgrade_test.go
* Silence staticcheck on lib/kube/proxy/forwarder_test.go
* Silence staticcheck on lib/utils/certs_test.go
* Address BuildNameToCertificate deprecation warnings
* Run `go fmt ./...`
* Run `go fmt ./...` on api/
* Ignore formatting in role_test.go
* Remove redundant initializers in lib/srv/uacc/
* Update e/
* Fixes the scp logic to take target directory into account in sink mode.
Also expose channel error in scp client so the error is more visible to
the user. Old behavior will only output the 'exit code n' if anything
breaks.
Fixes https://github.com/gravitational/teleport/issues/5497.
* Silence 'wait: remote command exited without exit status or exit signal' error when interrupting the scp session. Leave a TODO to fix properly in a future PR
* Address review comments
* Add -p flag to scp
* Add support for preserving access/modification times on files/directories when copying files between hosts.
* lib/sshutils/scp: add time statting for directories
* Add directory handling for scp
* Rewrite scp tests with testify
* Address review comments
Caught by `ineffassign` linter.
Note the change in lib/srv/regular/sshserver.go - looks like we
advertised the custom listening port (`aport`), if it differed from
regular listening port (`port`).
This serves two purposes:
1. on upload, the receiver (node) will exit with error and generate an
SCP error audit event.
2. on download, the receiver (tsh) will print a meaningful message from
the sender (node)
Fixes#2861
Spring cleaning!
A very mechanical cleanup using several linters (unused, deadcode,
structcheck). Build and tests still pass so no behavior should be
affected.
On the client side, make sure directory mode is requested when sending
multiple files. On the server side, when in directory mode, validate
that the target is a directory.
This was fixed running the `misspell` linter in fix mode using
`gometalinter`. The exact command I ran was :
```
gometalinter --vendor --disable-all -E misspell --linter='misspell:misspell -w {path}:^(?P<path>.*?\.go):(?P<line>\d+):(?P<col>\d+):\s*(?P<message>.*)$' ./...
```
Some typo were fixed by hand on top of it.
In this commit:
1. Minor addition to Makefile to pull new .go files from
tool/teleport/common
2. os.Glob() returns an empty list (instead of an error) if the
file/pattern is not found, so added check for that.
3. sendFile was prematurely sending 'C' command before trying to open a
file. This used to lead to creation of empty files for invlaid
sources.
Also, removed some confusing comments.
1) -r flag handling
Proper handling of `-r` flag on the tsh side (recursive).
Prior to this commit it was all uploads were _always_ recursive.
2) less verbose logging.
Replaced a bunch of log.Info() with log.Debug()
Tobiasz Heller