- Fixes a bug in lib/galaxy/tools/linters/general.py found through galaxy-lib.
- Brings in docstring and import order linting fixes from the gxformat2 project (https://github.com/jmchilton/gxformat2).
I was unable to reproduce #1531 in testing, but if the problem is something to do with stale state the following sledge hammer should fix it.
Runt the new test case with:
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_delete_intermediate_datasets_pja_1
Details:
- Add a new workflow module describing subworkflows.
- Add workflow list to editor side panel - with options to link in a subworkflow module or copy the target workflow into the workflow being editted node for node.
- Update workflow, workflow step, and workflow invocation models to track subworkflow connections and execution.
- Extend workflow outputs with concepts of labels (and UUIDs while I'm there) to match workflow inputs. This allow us to have something to label outputs with in the workflow editor and to reference in the format 2 workflow description language.
- Extend workflow editor UI to allow labeling workflow outputs (and enforce that these are unique across a workflow).
- Extend workflow invocation and progress tracking to allow invoking a subworkflow as part of another workflow invocation.
- Extend workflow import and export code to allow a nested representation of workflows.
- Update format 2 workflow description to allow testing nested workflows.
Most relevant new and modified test cases can be run using the following commands:
```
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_run_subworkflow_simple
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_subworkflow_simple
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_outputs
nosetests test/unit/test_galaxy_mapping.py
nosetests test/unit/workflows/test_workflow_progress.py
```
- Implement a input parameter module that mirrors data and collection input modules but has a type that can currently be one of text, integer, float, color, and boolean.
- Allow connections between these and tool step inputs.
- Extend model to support this.
- Add new input types for format 2 workflow definitions for various types that all map to this kind of step. Typed inputs such as this match well with CWL workflow inputs.
Someday I imagine these will be superior to just marking a tool input "Specify at Runtime" for all the same reasons input steps are superior to leaving inputs unattached.
Copy CWL style of specifying a outputs specification block at the top-level of the workflow specification and within that use the "source" attribute to specify the output. Reuse the specification used by "$link"s to describe this output - namely <label_or_order_index>[#<output_name=output>].
In downstream work this is used for subworkflows and nested tools. Not sure which of these would potentially hit main line Galaxy first so setting this all up in its own commit.
This is a slight tightening up of the ad-hoc, experimental workflow YAML definition used by the test framework (and by Kyle, but lets just admit Kyle is part of Galaxy's test framework).
This format is still defined entirely client side by transcoding the YAML or python object description into real (or format 1) Galaxy workflows - so these cannot realisticaly be declared part of Galaxy's public interface and can remain experimental and subject to change.
In addition to refactoring the code implementing these workflows for upstream modifications to explore new features, the format itself has been made slightly more stringent in two ways:
- 'steps' must now be explicit (previously converted auto-convert list to dict).
- Ensure the workflow declared a 'class' is defined and the class is 'GalaxyWorkflow'.
This will help with discovery of workflow objects with downstream tooling (planemo for workflows?) and brings the Galaxy definition slightly more inline with the CWL definition for workflows (very slightly).
John Chilton