Commit Graph
6 Commits
Author SHA1 Message Date
Andrew LeFevre d632c6ab71 improve 'tsh scp' error message when no remote path is specified (#21106)
While here, move package comment in SFTP client code to the specific
import it mentions.

Fixes #21052
2023-02-03 18:07:00 +00:00
Andrew LeFevre 629d072797 Use 'path.Join' instead of 'filepath.Join' to build remote file paths (#20961)
* use 'path.Join' instead of 'filepath.Join' to build remote file paths

tsh on Windows when using 'filepath.Join' would create paths with forward
slashes in them, which non-Windows Nodes couldn't handle. Consistently use
'path.Join' to avoid issues.

Fixes #18919

* add comment explaining why 'path.Join' is used

* move comment to make linter happy
2023-01-31 15:56:14 +00:00
Jakub Nyckowski a98c43f6e8 Use concurrent streams for SFTP connections (#20751)
* Our current SFTP implementation uses a single channel to transfer data. Connections with a high latency cause very slow transfer. github.com/pkg/sftp allows transferring data using concurrent streams, which seems to solve the problem. In order to utilize the feature, the remote file must implement the WriteTo() or ReadFrom() and Stat() methods. I've moved the progress bar to relay the data from a local file, as we can still call Read() or Write() on it to update the progress.

This PR doesn't address the high CPU usage by github.com/pkg/sftp on low latency links.

Closes #20579

* Cleanup

* Fix fileTransferProgress

* Simplify interfaces

* Move helper to utils package
Add comments

* Added comments

* Change canceler implementation to return correct values from Read/Write

* Fix infinite recursive call in cancelWriter.
2023-01-30 20:41:21 +00:00
Jakub Nyckowski 0ee91f6c37 Enable GCI linter (#17894) 2022-10-28 20:20:28 +00:00
Grzegorz Zdunek 2c6a898983 Add file transfer support to Connect (#16880) 2022-10-10 11:09:08 +00:00
Andrew LeFevre 77f8a4ef10 Switch underlying protocol used for 'tsh scp' to SFTP (#16601)
* switch underlying protocol used for 'tsh scp' to SFTP

* address TODO

* appease linter

* add method to make it easier for other callers to transfer files

* add tests

* print transfer progress with progress bar by default

Also allow a SIGINT to gracefully stop the SFTP connection. This is
necessary  because the progress bar will ignore signals and prevent the
process from exiting.

* address SFTP fork issues

* make tests less flakey

* fix specifying dir for dst not copying files to correct paths

* make tests less flakey (again)

* don't check file access times, often differs when run in CI

* few small fixes from review, simplify Create method now that HTTP FS isn't needed

* create dst files and dirs with src mode

* improved error messages when doing file operations

* expand home dirs in remote paths

* addressed more feedback

* add license to get_home_dir.go

* address minor feedback of tests, add home dir expansion test

* update sftp fork to point to latest commit on master branch

* addressed feedback

* don't cache home dir lookups, only one remote path can ever be used
2022-09-28 16:37:18 +00:00