Allows describing hierarchical data in JSON or inferring structure from archives or directories.
Datasets or archive sources can be specified via uploads, URLs, paths (if admin && allow_path_paste), library_import_dir/user_library_import_dir, and/or FTP imports. Unlike existing API endpoints, a mix of these on a per file basis is allowed and they work seemlessly between libraries and histories.
Supported "archives" include gzip, zip, bagit directories, bagit achives (with fetching and validations of downloads).
The existing upload API endpoint is quite rough to work with both in terms of adding parameters (e.g. the file type and dbkey hanlding in 4563 was difficult to implement, terribly hacky, and should seemingly have been trivial) and in terms of building requests (one needs to build a tool form - not describe sensible inputs in JSON). This API is built to be intelligable from an API standpoint instead of being constrained to the older style tool form. Additionally it built with hierarchical data in mind in a way that would not be easy at all enhancing the tool form components we don't even render.
This implements 5159 though much simpler YAML descriptions of data libraries should be possible basically as the API descriptions. We can replace the data library script in Ephemeris https://github.com/galaxyproject/ephemeris/blob/master/ephemeris/setup_data_libraries.py with one that converts a simple YAML file into an API call and allows many new options for free.
In future PRs I'll add filtering options to this and it will serve as the backend to 4733.
Need to fix https://github.com/galaxyproject/planemo/issues/750 but I don't want to send along command-line arguments that will break older Galaxies, better just to set an environment variable that will be ignored for older Galaxies I think.
Introduced in commit 45a25a1b8b .
This was causing this error on the `dev` branch on Ubuntu (which uses
`dash`):
```
$ ./run.sh
./scripts/common_startup.sh: 69: [: dev: unexpected operator
Activating virtualenv at .venv
...
```
instead of exiting with the "The Galaxy client build is out of date." message.
These aggregate all the jobs contributing to an implicitly created HistoryDatasetCollectionAssociation.
- Model enhancements that allow this.
- API endpoint for jobs summary of history content (somewhat uniform across HDAs and HDCAs just for consistency).
- Test cases.
- Rework history contents API a bit to eliminate more costly state summary checks by default.
John Chilton