Previously the job script underwent these checks and I had assumed since that got written later it would never be the case that tool_script.sh wouldn't be synchronized by the job script was - turns out file systems are more creative and less logical than one might expected.
Also increases the count for the number of syncs that will execute because 15 on rare occasion proved not to be enough as well as making both the count and sleep time configurable.
Long term this is just not a solution, but it is the strategy we use for mapped over output HDAs also.
Rebased and fixed based on comments from @nsoranzo indicating the previous attempt did absolutely nothing to fix the problem.
xref https://github.com/galaxyproject/tools-iuc/pull/412/files
Conflicts:
lib/galaxy/tools/actions/__init__.py
- Add monitor tag on toolbox root elements to force Galaxy to watch the toolbox for changes.
- Refactoring and test improvements for existing tool monitoring code.
- Implement toolbox shutdown process to ensure different watchers don't compete with each other.
- Set the default tool conf to be monitored.
- Add configuration option for overriding condarc location used by conda resolve commands.
- Default location of condarc to be in tool_dependency_dir.
- Delay updating the condarc to ensure channels until this would actually be needed (during package installation).
- Add configuration flag to allow Galaxy to *NOT* check the PATH for conda.
- Update unit test that verifies conda's installation to never use conda on PATH, since a precondition of this test is that certain packages haven't been installed previously.
- Change default behavior to not check the PATH for conda, everything else here I feel pretty good about - this I'm ambivalent about - it will reduce error reports and problems with network filesystems, but doesn't seem to be good Unix best practice.
Points 2 and 3 above should address this issue: https://github.com/galaxyproject/ansible-galaxy-extras/pull/33#issuecomment-168384555
This commit also addresses most of #1406.
- Drop version requirement per commends by @bgruening.
- Add examples for conditionals, repeats, sections, colors, and booleans specified truevalue/falsevalue.
- Various bug fixes unearthed by new test cases.
```
<configfiles>
<inputs name="inputs" format="json" version="1" />
</configfiles>
```
Version is specified as 1, because the Galaxy tool plumbing doesn't really distinguish between parameter is '', parameter was explicitly supplied as None, parameter was absent, etc... - everything is a just a string in some ways. This distinction may be more important with JSON and for instance the CWL will require more percision, so we may want to revise Galaxy's tool handling in a backward compatible way someday.
Guess here is that a flush is not occuring on that file because it is happening as part of garbage collection and not explictly as part of an explicit close.
Running a workflow or showing a workflow can both restore the previous behavior by passing legacy=True as an API parameter. By changing these two endpoints in tandem I believe backward compatiblity for most existing code should be maintained unless:
- The external application saved these workflow IDs previously and re-runs workflows without refetching the workflow definition. I could imagine Refinery for instance might do this and will have to update indexed workflows or add legacy=True to workflow requests.
- The external application contacted the database directly after using this API endpoint to fetch more information about the step (seems unlikely).
See conversation:
- http://dev.list.galaxyproject.org/workflow-API-step-order-vs-step-id-in-bioblend-td4668367.html
Rebased with changes suggested by @nsoranzo.
This fixes all unit tests for code exclusively covering the stuff I'm moving into the galaxy-lib that will be used by planemo, pulsar, and future tooling efforts.
This makes the galaxy.tool.parser much more portable, it no longer has a dependency on anything in galaxy.tools or submodules except galaxy.tools.loader (which in turn has no dependencies on galaxy.tools or submodules).
This is part of a long effort to build a galaxy.tools.parser module that has no dependencies outside of say galaxy.util - so I can add it to the code shared with planemo and pulsar. This would allow planemo (and by extension Tool Shed 2.0) to reason about tools in an abstract way without the billion dependencies of galxaxy.tools.Tool.
Details:
- Add a new workflow module describing subworkflows.
- Add workflow list to editor side panel - with options to link in a subworkflow module or copy the target workflow into the workflow being editted node for node.
- Update workflow, workflow step, and workflow invocation models to track subworkflow connections and execution.
- Extend workflow outputs with concepts of labels (and UUIDs while I'm there) to match workflow inputs. This allow us to have something to label outputs with in the workflow editor and to reference in the format 2 workflow description language.
- Extend workflow editor UI to allow labeling workflow outputs (and enforce that these are unique across a workflow).
- Extend workflow invocation and progress tracking to allow invoking a subworkflow as part of another workflow invocation.
- Extend workflow import and export code to allow a nested representation of workflows.
- Update format 2 workflow description to allow testing nested workflows.
Most relevant new and modified test cases can be run using the following commands:
```
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_run_subworkflow_simple
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_subworkflow_simple
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_outputs
nosetests test/unit/test_galaxy_mapping.py
nosetests test/unit/workflows/test_workflow_progress.py
```
- Implement a input parameter module that mirrors data and collection input modules but has a type that can currently be one of text, integer, float, color, and boolean.
- Allow connections between these and tool step inputs.
- Extend model to support this.
- Add new input types for format 2 workflow definitions for various types that all map to this kind of step. Typed inputs such as this match well with CWL workflow inputs.
Someday I imagine these will be superior to just marking a tool input "Specify at Runtime" for all the same reasons input steps are superior to leaving inputs unattached.
* Replace {} and .format with %s and % to preserve Python 2.6 compatibility
* Add a modulepath= paranter to the fake resolver (Bunch object) used to test the modules dependency resolver
John Chilton