Commit Graph
81 Commits
Author SHA1 Message Date
davelopez e73723b3e2 Add test to ensure local file access is restricted 2025-05-08 23:57:55 +02:00
davelopez f080d5d73e Add basic test for remote zip extractor file source 2025-05-08 23:51:48 +02:00
David López 28e5f7d84b Merge pull request #19154 from kysrpex/file_sources_assign_paths
Let file sources choose a path for uploaded files
2025-01-20 16:11:57 +01:00
Nicola Soranzo 94bff7786c Type annotation fixes for mypy 1.14.0 2024-12-30 15:23:21 +00:00
José Manuel Domínguez 6fe92643af Merge branch 'dev' into file_sources_assign_paths 2024-11-28 13:11:29 +01:00
José Manuel Domínguez 53012bf059 Let file sources choose a path for uploaded files
The method `write_from()` of `SingleFileSource` and `BaseFilesSource` reads a local file from `native_path` and saves it to `target_path` on a file source.

This commit allows the service backing the file source to choose which will be the path of the saved file, meaning that `target_path` and the actual path where the file can be recovered later do not have to match. The latter is the return value of `write_from()`.

Therefore, all usages of `write_from()` have also been refactored to consider the paths chosen by the file source's backing service. In addition, when exporting a history, the URI that the service backing the file source assigns to it will be saved to the history export result metadata object.
2024-11-18 17:02:29 +01:00
John Chilton edc5bc2a8f Allow oauth2 file sources with Dropbox initial implementation. 2024-10-14 15:16:41 +02:00
John Chilton eea3909cb0 Fixes and expansion of webdav unit tests. 2024-10-05 17:23:26 -04:00
John Chilton 77e6df4303 Fill out missing file source management tests. 2024-05-28 11:31:35 -04:00
davelopez 9cfc7e2c3a Add unit test to check empty query string 2024-05-21 15:16:23 +02:00
davelopez 40fbc946bb Add unit tests for invalid parameters 2024-05-21 11:28:02 +02:00
David LópezandJohn Davis 03d9d12f22 Apply suggestions from code review
Replace unused tuple value with `_`

Co-authored-by: John Davis <jdavcs@gmail.com>
2024-05-21 10:45:17 +02:00
davelopez dd60362aa6 Add total_matches header to file sources listing
This is necessary to handle pagination properly in the UI.
2024-05-21 10:45:09 +02:00
davelopez ff83166bc5 Add unit tests for search query 2024-05-21 10:42:48 +02:00
davelopez d0632a5d14 Refactor unit tests for TempFilesSource 2024-05-21 10:42:45 +02:00
davelopez 080a724deb Add unit tests for TempFilesSource 2024-05-21 10:42:43 +02:00
John Chilton f95da89b3d Implement file source templates 2024-05-20 12:18:47 -04:00
John Chilton 076fff8302 Update s3fs dependency. 2024-05-14 08:50:44 -04:00
John Chilton e03d162429 Update Azure infrastructure. 2024-05-03 13:57:58 -04:00
Björn Grüning 4e2aaf8e2b Merge pull request #18050 from davelopez/file_sources_typings
Add stronger type annotations in file sources + refactoring
2024-04-29 15:07:08 +02:00
Nicola Soranzo 0629908541 Fix new UP031 errors from ruff 0.4.2 2024-04-29 00:09:38 +01:00
davelopez 4a6958e059 Refactoring + stronger type annotations in file sources 2024-04-26 10:21:26 +02:00
John Chilton ac197e7f25 Refactor galaxy.files plugin + config handling.
- Downstream I want a way to load plugins from outside the ConfiguredFileSources class.
- Adds some more typing as I went in there and refactored things.
2024-04-25 13:05:02 -04:00
Nicola Soranzo dcb6f061f6 Don't install Galaxy packages in editable mode when testing them
When enabling the `warn_unused_ignores = True` option of mypy, this
was making the `test_galaxy_packages` tests fail with:

```
mypy .
Traceback (most recent call last):
  File "/tmp/gxpkgtestenvu3fgDr/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/tmp/gxpkgtestenvu3fgDr/lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "mypy/main.py", line 100, in main
  File "mypy/main.py", line 182, in run_build
  File "mypy/build.py", line 192, in build
  File "mypy/build.py", line 266, in _build
  File "mypy/build.py", line 2942, in dispatch
  File "mypy/build.py", line 3333, in process_graph
  File "mypy/build.py", line 3414, in process_fresh_modules
  File "mypy/build.py", line 2110, in fix_cross_refs
  File "mypy/fixup.py", line 52, in fixup_module
  File "mypy/fixup.py", line 127, in visit_symbol_table
  File "mypy/lookup.py", line 49, in lookup_fully_qualified
AssertionError: Cannot find component 'util' for 'galaxy.util'
make: *** [Makefile:96: mypy] Error 1
```

Moreover, installing packages in editable mode was hiding some issues
with how files from other packages were accessed.
2024-04-16 01:12:57 +01:00
Nicola Soranzo 330599c70f Fixes for new errors from ruff 0.3.2 2024-03-12 11:06:57 +00:00
Nicola Soranzo e975ca3754 Update to black 2024 stable style 2024-01-30 18:09:51 +00:00
mvdbeek 68211090e6 Skip test when ftp.gnu.org not available 2024-01-01 11:21:49 +02:00
Dannon Baker d01b039415 Fix unittest mocks to support us checking geturl() after opening connections. 2023-09-22 11:05:24 -04:00
nuwang b997b00456 Run isort against test_http.py 2023-06-14 19:27:31 +05:30
nuwang 6a54e6ea1b Add stock filesoures to example file and associated test case 2023-06-14 19:27:31 +05:30
Nuwan GoonasekeraandNicola Soranzo 58fd1e039c Apply suggestions from code review
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2023-03-18 00:12:16 +05:30
nuwang cafcec67bb Fix mypy errors in test_posix 2023-03-17 22:12:50 +05:30
nuwang c1fb122828 Fix linting error in regex 2023-03-17 20:16:13 +05:30
nuwang 31927d96f6 Add tests for file:// urls referring to allowed roots 2023-03-17 02:20:40 +05:30
nuwang fe4a3aa7ec Add test for malformed file:// url 2023-03-16 20:10:41 +05:30
Nuwan GoonasekeraandNicola Soranzo 3e53fcf916 Apply suggestions from code review
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2023-03-16 19:35:50 +05:30
nuwang d7abd7dd25 Preliminary addition of file:// support to posix 2023-03-15 01:59:04 +05:30
nuwang ab5ff9382a Remove file:// handling from http filesource 2023-03-15 01:57:13 +05:30
nuwang 0d7c9611c8 Make sure filesources can handle file:// protocol 2023-03-14 20:39:29 +05:30
nuwang 0c0ed2a4e3 Fix test failures 2023-02-09 18:14:18 +05:30
nuwang c39778224a Fix linting and typing errors 2023-02-09 18:13:33 +05:30
nuwang 269a0ab393 Enhance and add tests for ftp filesources 2023-02-09 18:13:33 +05:30
nuwang 7e11563e5a Make sure ftp files are handled correctly 2023-02-09 18:13:33 +05:30
nuwang 11ccfff186 Add s3fs to test requirements 2023-02-09 18:13:32 +05:30
nuwang b385819fa0 Added more tests for s3 filesources 2023-02-09 18:13:32 +05:30
nuwang 3d07144870 Fix test failures 2023-02-09 18:13:32 +05:30
nuwang c8803f49b2 Add chained resolution of drs access urls 2023-02-09 18:13:32 +05:30
nuwang 744f41ffff Add tests for drs filesource 2023-02-09 18:13:31 +05:30
nuwang 13d682ecad Add tests for base64 filesource 2023-02-09 18:13:31 +05:30
nuwang 129425abb5 Add support for injecting user headers to http file source 2023-02-09 18:13:31 +05:30