- maxseconds on test tag stopped being respected at some point, fix that.
- maxseconds on test tag stopped being parsed at some later point, fix that.
- Revise default behavior so it can be overridden with yet another testing environment variable GALAXY_TEST_DEFAULT_WAIT.
- Add test case to ensure there is no future regression of these behaviors.
This should not be considered the best practice for most text based files, but for binary files we cannot produce diffs anyway so this is probably better than storing the pointless binary blobs in repositories for instance. In particular Dan and I thought this would be useful for testing index files for data managers.
Includes test tool demonstrating this functionality, run test with
./run_tests.sh -framework -id md5sum
... for tool tests. Longer term this functionality should be dropped (i.e. after all the tools are out of Galaxy) and the test-data always lives next to the tool - but for now it decreases the size of the repository ahead of a potential move to github.
By "static" I mean tools such as a FASTQ de-interlacer that would produce a "paired" collection with two datasets everytime. By "determinable" I mean tools that perform N->N operations within the same job - such as a tool that needs to normalize a bunch of datasets all at once and not in separate jobs. (For N->N collection operations that should or can be done in N separate jobs tool authors should just write tools that operate over a dataset and produce a dataset and let the end-user 'map over' that operation.)
There are still large classes of operations where the structure of the output collection cannot be pre-determined - such as splitting files (e.g. bam files by read group) - that are not implemented in this commit.
Model:
The models have been updated to do a more thorough job of tracking collection outputs. Jobs just producing HistoryDatasetCollectionAssociations works fine for simple jobs producing collections - but you don't want to map a list over a tool that produces a pair and produce a bunch of pairs HDCAs and a list:pair HDCA- you just want a bunch of pieces and the one list:pair at that the top.
Workflow:
Workflows containing such operations can be executed - but the workflow editor has not been updated to handle this complexity (and it will require a significant overhaul) so such tools are not available in the workflow editor.
Tool Testing:
This commit also introduces a new tool XML syntax for describing tests on output collections. See files test/functional/tools/collection_creates_list.xml and test/functional/tools/collection_creates_pair.xml for examples.
Tests:
Includes two tools to test this - one that uses explicit pair output names and one that iterates over the structure of input list to produce an output list.
Includes several new tools API tests that test the tools described above via the API and implicit mapping over such tools. Includes two new workflow API tests - one that verifies a simple workflow with output collections works and one that verifies mapping over workflow steps in collections works.
Add tests for new date range and history filtering as well as to ensure only admins get to see external_id and command_line and that users cannot see each other's jobs.
Always allow admins to views all jobs on index (instead of only when user_details is specified) and show. Add user_email and external_id to show (for admins) to bring it inline with index and add command_line to index (for admins) to bring it in line with show.
Show still allow more details including job standard error and output as well as job metrics.
Long term this could allow Galaxy to support - multiple tooling formats (Galaxy-like YAML, CWL http://bit.ly/cwltooldesc, etc...). But I think it is also important from a purely design perspective - this is a core logic class integrating different components - they should not also be doing XML parsing.
To verify the interface for parsing tools is expressive enough to allow multiple useful implementations, I built a test YAML tool description that implements many of the same features as Galaxy but smooths out rough edges (uses exit codes for job failure by default for instance). Loading these tools is disabled by default and it is not documented how to enable them because they are not intended to be part of Galaxy's public API.
Rather than relying solely on exceptions back to nose/test framework - add option (--structured_data_report_file) to run_tests.sh that causes a bunch of detailed data to be dumped to the specified file in a very structured way. Includes full to_dict of the job from the API which in turn includes job metrics, command-line, job's standard error and outputs (instead of the test frameworks), as well as the tool inputs, and exceptions broken out for tool execution versus output checking.
Its all indexed in the file by the test id (without the actual test toolbox depending on knowing the test id) - so one could pair this information with the XUnit output to produce much more detailed breakdowns of the tests.
It wasn't long for this world with a JavaScript tool form anyway - would be nice to implement a casperjs driven interactor though so the UI could be exercised for tests.
Remove duplication related to verifying a single dataset in tests - allows reuse in downstream when verifying the content of explicitly defined tool output collections.
Refactor out waiting for jobs - seems wrong place to put that logic this should allow easier moving.
Nicola Soranzo