Guess here is that a flush is not occuring on that file because it is happening as part of garbage collection and not explictly as part of an explicit close.
This fixes all unit tests for code exclusively covering the stuff I'm moving into the galaxy-lib that will be used by planemo, pulsar, and future tooling efforts.
This makes the galaxy.tool.parser much more portable, it no longer has a dependency on anything in galaxy.tools or submodules except galaxy.tools.loader (which in turn has no dependencies on galaxy.tools or submodules).
This is part of a long effort to build a galaxy.tools.parser module that has no dependencies outside of say galaxy.util - so I can add it to the code shared with planemo and pulsar. This would allow planemo (and by extension Tool Shed 2.0) to reason about tools in an abstract way without the billion dependencies of galxaxy.tools.Tool.
Details:
- Add a new workflow module describing subworkflows.
- Add workflow list to editor side panel - with options to link in a subworkflow module or copy the target workflow into the workflow being editted node for node.
- Update workflow, workflow step, and workflow invocation models to track subworkflow connections and execution.
- Extend workflow outputs with concepts of labels (and UUIDs while I'm there) to match workflow inputs. This allow us to have something to label outputs with in the workflow editor and to reference in the format 2 workflow description language.
- Extend workflow editor UI to allow labeling workflow outputs (and enforce that these are unique across a workflow).
- Extend workflow invocation and progress tracking to allow invoking a subworkflow as part of another workflow invocation.
- Extend workflow import and export code to allow a nested representation of workflows.
- Update format 2 workflow description to allow testing nested workflows.
Most relevant new and modified test cases can be run using the following commands:
```
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_run_subworkflow_simple
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_subworkflow_simple
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_outputs
nosetests test/unit/test_galaxy_mapping.py
nosetests test/unit/workflows/test_workflow_progress.py
```
- Implement a input parameter module that mirrors data and collection input modules but has a type that can currently be one of text, integer, float, color, and boolean.
- Allow connections between these and tool step inputs.
- Extend model to support this.
- Add new input types for format 2 workflow definitions for various types that all map to this kind of step. Typed inputs such as this match well with CWL workflow inputs.
Someday I imagine these will be superior to just marking a tool input "Specify at Runtime" for all the same reasons input steps are superior to leaving inputs unattached.
* Replace {} and .format with %s and % to preserve Python 2.6 compatibility
* Add a modulepath= paranter to the fake resolver (Bunch object) used to test the modules dependency resolver
- If a step has a label, display it in the step editor side panel title.
- Allow clicking the title to change the label.
- Display the label (if set) as the workflow node box title.
- Add icon to workflow node since the title might not be related to type anymore.
- Enforce unique labels accross the workflow in the editor.
- Qunit test cases for some of this behavior and other recent changes.
- Reduce duplication between initializing generic modules and tool modules.
- Switch tool_id to content_id as variable names throughout the client.
- Rename get_tool_id to get_content_id on workflow modules.
- Add some minimal documentation to the workflow module about get_content_id.
Downstream in the subworkflow commit I switch the over-the-wire communication to use content_id instead of tool_id also and use content_ids to refer to workflow ids in subworkflow moduls.
Allow multiple collections to be fed to a multi data parameter in one reduction step. Fixes#750 and will really simplify certain classes of tools.
Rebased original with fixes for rerun of such reductions.
Manually tested workflow execution and everything seems fine. The workflow editor already thought this was possible, so that is another bug corrected by this enhancement.
To run the associated API test, execute the following command:
./run_tests.sh -with_framework_test_tools -api test/api/test_tools.py:ToolsTestCase.test_reduce_multiple_lists_on_multi_data
Conflicts:
static/maps/mvc/dataset/dataset-choice.js.map
static/maps/mvc/form/form-select-content.js.map
static/scripts/mvc/form/form-select-content.js
Modified this for the workflow export representation but this needed to be updated for the workflow editor because they shared some code in the manager. This was going to be needed downstream when I add labels and uuids to workflow outputs anyway.
Copy CWL style of specifying a outputs specification block at the top-level of the workflow specification and within that use the "source" attribute to specify the output. Reuse the specification used by "$link"s to describe this output - namely <label_or_order_index>[#<output_name=output>].
In downstream work this is used for subworkflows and nested tools. Not sure which of these would potentially hit main line Galaxy first so setting this all up in its own commit.
John Chilton