This differs from a traditional tool in that its inputs don't need to be in an 'ok' state and instead of creating new datasets and duplicating data on disk, new HDAs are created from the existing datasets.
This special class of tools leverages the infrastructure for tool inputs, tool state tracking, tool module for workflows, tool API, etc... without actually producing command-line jobs. Instead these tools are provided the input model objects and are expected to produce output model objects directly. This provides an oppertunity to copy HDAs without copying the underlying datasets.
The first driving use case for these tools are also included - namely tools that allow zipping and unzipping paired collections. These tools can be mapped over lists (e.g. list:paired to (list, list) or the inverse) using much of the existing infrastructure for tools. Test cases included that validate these work with mapping operations and in workflows.
The most obvious advantage of these versus traditional tools that do the same thing is that the data isn't copied on disk - new HDAs are created directly from the source datasets.
Testing:
This PR includes various API test cases for functionality, these can be run with the following command:
```
./run_tests.sh -api test/api/test_tools.py:ToolsTestCase.test_unzip_collection
./run_tests.sh -api test/api/test_tools.py:ToolsTestCase.test_zip_inputs
./run_tests.sh -api test/api/test_tools.py:ToolsTestCase.test_zip_list_inputs
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_workflow_run_zip_collections
```
Refactor test verification into galaxy-lib-compat module.
- Refactor a bunch class methods in ``test.base.twilltestcase`` into module functions in ``galaxy.tools.verify``.
- Move ``test.base.test_data`` to ``galaxy.tools.verify.test_data``.
- Move ``test.base.asserts`` to ``galaxy.tools.verify.asserts``.
Remove duplication in execution of these methods between composite and normal test outputs. This also entailed reworking the parsing of the composite test outputs to bring them inline with normal outputs. In addition to simplify removing duplication, this means many more tests can be made over composite outputs - such as md5 checks and test assertions. I've added a new framework test tool to verify this.
- Fixes a bug in lib/galaxy/tools/linters/general.py found through galaxy-lib.
- Brings in docstring and import order linting fixes from the gxformat2 project (https://github.com/jmchilton/gxformat2).
I was unable to reproduce #1531 in testing, but if the problem is something to do with stale state the following sledge hammer should fix it.
Runt the new test case with:
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_delete_intermediate_datasets_pja_1
Long term this is just not a solution, but it is the strategy we use for mapped over output HDAs also.
Rebased and fixed based on comments from @nsoranzo indicating the previous attempt did absolutely nothing to fix the problem.
xref https://github.com/galaxyproject/tools-iuc/pull/412/files
Conflicts:
lib/galaxy/tools/actions/__init__.py
carlfeberhard