This is a somewhat substantial reversal in the typical way that the Conda resolver works. Everything related to caching, copying, linking, and building environments on the fly should now only apply if both of the following conditions are met (1) there is more than one requirement tag in a tool and (2) not all of them are resolvable exactly by the Conda resolver. For recipes that don't meet both of these two criteria - the normal case I would suspect going forward - Galaxy will just look for a hashed environment for these requirements built for all the requirements at once whenever the requirements are installed.
The new 90% case, such environments should be much less buggy for two primary reasons.
- #3299 is solved - in other words Conda is deferred to and if packages have potential conflicts - Conda can choose the right combination of build specifiers to resolve things correctly.
- Environments are no longer built on a per-job basis - this means file system problems related to linking and copying aren't really an issue and complexity related to caching can be safely ignored.
My guess is we should re-write all the Conda docs to make the other use case seem like a corner case - because hopefully it is now.
This commit includes a test tool that wouldn't work without the rewrite I believe.
Expand and then break out workflow management tests into its own file since these aren't related to the workflow editor per se. Includes new tests of workflow "viewing" and "renaming".
Add a new file for testing workflow execution. This currently contains tests for simple execution with a single input and a test for running workflows with tool version upgrades.
This includes a refactoring of some existing tests related to checking various things in the history panel - as part of building up good history panel abstractions. The previous constructs were ported fairly literally from older CasperJS tests. These abstractions therefore referred to everything has "hda"s instead of history items. The newer abstractions therefore allow for collections and are built around HIDs (a visual thing exposed to the user) instead of HDA IDs. I think this is higher-level and more appropriate for a web functional test.
Rebase into workflow GUI tests.
Needed to test workflow tool upgrades when loading workflows, but I think it is good to have a way to preserve the original tool state as well for the sake of absolute reproducibility/tracibility.
- Move api.helpers to base.populators.
- Refactor helper subclasses adapted to use bioblend instead of test framework stuff into base.populators for greater visibility.
- Refactor workflow format 2 testing stuff for reuse by Selenium.
I don't know how this ever worked to be honest - but these changes make a lot of sense. I think there was another bug somewhere that prevented this bug from exhibiting itself.
Don't expose Galaxy's environment to tools unless there is evidence we need to. This should restrict future bugs by limitting what tools are exposed to Galaxy's internals and will fix bugs related to Conda dependencies masking Galaxy - such as #2994.
Should fix#2994.
Rebased with language and tool best practice fixes thanks to @mvdbeek.
Also fixes it for backgrounded workflow invocations (fixes#3268 reported by @guerler).
Includes a new test tool w/test definition that exercises this functionality. This tool fails without these changes.
Tests may bring up multiple Galaxy instances - for integration tests this leads to a longer temp path with each new Galaxy instance brought up. This in turn breaks Conda for tests on Jenkins.
- Test parameters and tables appear on dataset details page after running a tool.
- Test a simple tool execution with an integer field.
- Test re-run a tool.
- Test a tool with a data drop down.
- Improved select2 abstractions.
- Remove broken CasperJS from testing Dockerfile.
- Remove multi-process mode.
Multi-process mode was difficult to maintain and the last time I did testing at scale I found docker-galaxy-stable to be much more useful and easy to use. The test framework now supports GALAXY_TEST_EXTERNAL again so it should be easy enough to rig something up with docker-compose that actually uses a best practice container for scaling tests such as those in ``test/manual``.
So you can backoff re-submission with ``delay="attempt * 30"`` for instance - which will cause a re-submission delay to increase by 30 seconds for each resubmission of a job.
Since we allow dynamic job runners clearly job_conf.xml must be considered a trusted document. Still I know that some are uneasy with eval generally so I only allow a very safe subset of Python expressions in this job_conf document.
John Chilton