At the insistence of @nsoranzo, be more explicit about waiting for workflow test in API tests. While I was in there I also setup new abstractions for waiting on state and updated various workflow cancelling tests to use test_history context for better error reporting.
- Add an API test for datatype-defined composite uploads - including exercising newline conversion and the space_to_tab parameter.
- Add a test decorator skip_without_datatype to mirror skip_without_tool for this test, improve both decorators.
- The ftype parameter in the composite test tools does nothing - drop it and drop it from the XSD spec.
- Slightly improve the documentation for these composite_data elements in the XSD.
A precondition to the real meat of the test is failing to be met sometimes in production - this failure is indicating a problem with the test and not with Galaxy. I previously tried to address this with https://github.com/galaxyproject/galaxy/pull/3988 but that didn't work. Now if the trainsient ok occurs the test will be skipped. Most of the time it won't skip and the rest of the test will execute and ensure there aren't regressions in behaviors related to cleaning up datasets after job completion. I'm placing the skips a couple different places so hopefully we can get a stack trace at somepoint - knowing where the code is when the job state is changing from running to ok will help puzzle out what is handing for the two minutes the job is running in the test framework.
Entire collections can now be downloaded as tarballs. I have used the
StreamBall class from galaxy.utils, so archives start downloading
immediately.
The collection structure is mapped onto a directory struture, where each
level of a collection is a directory in the archive. Collections of BAM
files are included alongside their .bai indices, and composite datatype
are supported as well.
API tests for downloading list, paired and list:paired collections are
included.
Currently collection elements that are not in the OK status will be
skipped, so we should probably hide the download button if not all
collection elements are in a final state.
A couple things need to occur while the jobs are "running" and it seems that 60 seconds wasn't enough time to sleep for all that to happen on Jenkins.
I checked the timing in the raw logs (they are for some reasons being filtered out in the Jenkins display of the XUnit) - and indeed the jobs are properly sleeping the correct amount of time but the client API calls need some more time it seems. My initial thought was maybe the jobs were being interrupted or something - and that does not appear to be the case.
xref #1675
This is of limited utility since we don't really expose the name - and intentionally so. Related open bugs/enhancements that still need to be addressed are:
- Applying rename to the collection (in addition to the elements) - #1680.
- Download of collection elements with element identifier instead of the name: #2023 / #2140.
For instance, fixes#3859 restoring the correct ``element_identifier`` for reduces collections in conditionals. Add tests for combinations of repeats and conditionals.
Currently the logs contain fairly uninformative step delayed messages that don't allow admins to understand why parts of the workflow are being delayed during scheduling.
Here are some examples of before and after.
When running the test:
```
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_workflow_pause
```
Before these lines would show up:
```
galaxy.workflow.run DEBUG 2017-03-30 09:23:47,327 Workflow step 2 of invocation 1 invoked (174.149 ms)
galaxy.workflow.run DEBUG 2017-03-30 09:23:47,347 Workflow step 3 of invocation 1 invoked (19.355 ms)
galaxy.workflow.run DEBUG 2017-03-30 09:23:47,364 Workflow step 4 of invocation 1 delayed (16.262 ms)
```
Now these same lines are as follows:
```
galaxy.workflow.run DEBUG 2017-03-30 09:19:28,601 Workflow step 2 of invocation 1 invoked (172.849 ms)
galaxy.workflow.run DEBUG 2017-03-30 09:19:28,619 Marking step 3 outputs delayed (executing pause step)
galaxy.workflow.run DEBUG 2017-03-30 09:19:28,620 Workflow step 3 of invocation 1 invoked (17.999 ms)
galaxy.workflow.run DEBUG 2017-03-30 09:19:28,633 Workflow step 4 of invocation 1 delayed (dependent step [3] delayed, so this step must be delayed) (13.398 ms)
```
Also, when running the test:
```
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_workflow_run_dynamic_output_collections_3
```
Before these lines would be printed:
```
galaxy.workflow.run DEBUG 2017-03-30 09:25:35,910 Workflow step 4 of invocation 1 invoked (281.479 ms)
galaxy.workflow.run DEBUG 2017-03-30 09:25:35,937 Workflow step 5 of invocation 1 delayed (25.904 ms)
galaxy.workflow.run DEBUG 2017-03-30 09:25:35,970 Workflow step 6 of invocation 1 delayed (32.597 ms)
```
Now these same lines are as follows:
```
galaxy.workflow.run DEBUG 2017-03-30 09:27:54,270 Workflow step 4 of invocation 1 invoked (295.826 ms)
galaxy.workflow.run DEBUG 2017-03-30 09:27:54,315 Workflow step 5 of invocation 1 delayed (dependent collection [1] not yet populated with datasets) (44.581 ms)
galaxy.workflow.run DEBUG 2017-03-30 09:27:54,326 Workflow step 6 of invocation 1 delayed (dependent step [5] delayed, so this step must be delayed) (10.804 ms)
```
When running workflows with steps that depend explicitly on other steps (instead of implicitly between dataset connections), lines such as:
```
galaxy.workflow.run DEBUG 2017-03-30 09:33:52,966 Marking step 3 outputs delayed (workflow paused at this step waiting for review)
galaxy.workflow.run DEBUG 2017-03-30 09:33:53,001 Workflow step 4 of invocation 1 delayed (dependent step [3] delayed, so this step must be delayed) (0.157 ms)
galaxy.workflow.run DEBUG 2017-03-30 09:33:53,001 Workflow step 5 of invocation 1 delayed (depends on step [4] but that step has not been invoked yet) (0.084 ms)
```
and
```
galaxy.workflow.run DEBUG 2017-03-30 09:33:57,090 Workflow step 5 of invocation 1 delayed (depends on step [4] but one or more jobs created from that step have not finished yet) (0.144 ms)
```
now may appear.
John Chilton