- Fixes numerous problems mostly notably fixes external metadata generation and job metrics collection for recent Pulsar. This was broken when tools were given clean working directory.
- Produce a new message if an older Pulsar version is targetted via REST of library (should help with upgrading to Galaxy 16.04).
- Newer pulsar lib allows more flexible configuration of managers - update documntation to reflect this.
We have a tool_shed directory in lib, so if we enter a test case from some path and test/tool_shed happens to be a valid python top-level module - it blocks out lib/tool_shed and Galaxy doesn't work properly in the test.
A similar problem was encountered with unit tests and fixed with https://github.com/galaxyproject/galaxy/commit/df442a06382f23bdf0401bdd632b9584d3431281 - but I have decided that directory name was a bit log and so this commit unifies the approach to unit and functional test names for tool shed tests.
This change creates a more complete regex for parsing dates used in filters. MySQL in particular errored when the UTC zone character 'Z' was appended to a valid iso 8601 date (because it's MySQL). The FilterParser.parse_date method now:
- strips both the separator 'T' and any ending 'Z'
- one space are allowed as the date/time separator
- milliseconds are permitted
- both time and time with milliseconds are optional
- any added timezone (besides the UTC Z) will cause an error
- feedback was made more clear
- tests were added (and should make the above more clear)
Rather then massively renaming job_directory and job_working_directory, I introduce the concept of the tool execution working directory (tool_working_directory) that is the job working directory + "/working" and that will not be populated with job files.
Some legacy cruft in there for the upgrade, things like galaxy.json and dynamic outputs may be in the job working directory or the tool execution working directory.
- set -e in tool command scripts.
- Disable implicit extra file collection. All dynamic extra file collection requires a <discover_datasets tag.
- Disable format="input" - require explicit metadata targets (metadata_source, format_source).
- Disable interpreter= - hacky and doesn't work as expected in too many circumstances. Use $__tool_directory__.
- Disable $param_file - grepping the command for the string is impercise. If useful we can add flag to match the flag used to produce inputs as a JSON file.
- Disable default version of 1.0.0, the IUC has made it clear they prefer all tools have an explicit version.
Fixes#1609.
Newer tool formats (CWL and YAML just enable this by default), but XML tools can add a strict="true" to the command block to force "set -e" handling on the shell part of tool execution.
Run the framework tests verifying the new and older default behavior using the following commands:
```
./run_tests.sh -framework -id strict_shell_default_off
./run_tests.sh -framework -id strict_shell
```
Newer style <discover_datasets>'s based discovery will default to using input_dbkey the way normal inputs do. Legacy, implicit filename based discovery will continue to use '?' as the default.
This should work for collections and individual extra outputs.
Fixes#1598.
Thanks to @gregvonkuster for finding this issue and documenting it so thoroughly.
John Chilton