... for CWL-style merging of conditional branches in native Galaxy workflows.
I have CWL conditionals working down stream - the simply attach an expression to steps of the workflow and replace all the outputs with null values if the expression does not evaluate to "true". This is the CWL way of "branching" in conditionals, the matching CWL syntax for merging branches is simply to pick non-null values from various branches to continue the workflow with. This tool would is a very natural Galaxy way of doing that.
This tool could be used to build some cool conditional logic with collections without even needing the forthcoming 'when' expressions on workflow steps. Imagine for instance, we added a "Replace Failed" (or "Replace Empty") collection operation to match "Filter Failed" (or "Filter Empty") that could replace collection elements with 'null' values or a supplied default. We could then run two different algorthims across a list (e.g. one that works well for small datasets but fails out for large ones and one that works well with large datasets) and then run "Replace Failed" with null values and then map both results over this tool. Using purely data flow constructs you've picked the right tool for each element of a list.
I haven't added this to the default tool panel yet though, might be worth waiting for either those when branching expressions or more collection operation tool support.
When a tool doesn't specify a `profile` (or sets a `profile` < 18.01) but use
``use_shared_home="false"`` to ask to set a HOME within the job's directory
(normally because the underlying tool writes to the home directory), this
works as expected when resolving dependencies with conda.
But when running the same tool in a Docker container (e.g. via
`planemo test --biocontainers`), the job script didn't export a temporary
HOME into the container (which instead setting `profile="18.01"` correctly
does).
Also add several integration tests for `use_shared_home`, of which
`DockerizedJobsIntegrationTestCase.test_container_job_environment_explicit_isolated_home`
would fail without this fix.
- order of sort_comp and sort_by was wrong for
sort_by in collections
- sort_by was missing for data output
- adds a test (though we can't check the generated
order .. I manually tested with `planemo serve`)
Builds on refactoring the workflow run form landing and orchestration into Vue (#9147).
Implements #9111 - part of https://github.com/galaxyproject/galaxy/projects/15 and outlined in http://bit.ly/simplified-workflow-ui.
What is in the PR:
- Add new Galaxy configuration option - ``simplified_workflow_run_ui`` set to ``off`` by default.
- If instead the admin has set this parameter to ``prefer``, on the workflow run page scan the workflow run request and if three conditions are met show a simplfied tool form. These conditions are:
- No workflow "resource parameters" - pretty niche, not documented, I don't think anyone outside of Canada is using them and even them I'm not sure if they ever got to production.
- No "open" tools (i.e. tools with disconnected runtime inputs).
- No "replacement parameters" defined outside PJAs. I'm calling #{} inside PJA and ${} inside tool steps both "replacement parameters" because the user populates them the same way in the GUI. The API only handles them inside the context of the PJA - it seems the GUI is responsible for replacing them at runtime in the traditional form.
- The simplified workflow form:
- Drops tool and subworkflow steps from rendering all together and instead just renders the inputs. These are not rendered as separate "steps" or forms - but as just different inputs the same clean, simple form (more the like tool GUI). Labels (or step indexes as a fallback) are used at the input name, step annotations are used as the input help text.
- Simplify the workflow request made by the client - send only replacement parameters and inputs - do not serialize tool state for each module. I think this makes what we are tracking more structured and should be more performant as well. Prevents storing HDA IDs in unstructured JSON blobs in the database as well.
- Drops history target option to simplify the UI - simplified_workflow_run_ui_target_history can be set to either 'current' or 'new' to adjust this parameter Galaxy-wide for the simplified form.
- Drops job caching option to simplify the UI - simplified_workflow_run_ui_cache can be set to 'off' or 'on' to change this parameter Galaxy-wide for the simplified form.
- Drops resource parameters - we've already verified there are none for simplified workflows.
the files generated in a test are allowed to differ (+-)
by delta_frac * size_of_the_test_file
use case: I develop a a lare tool collection where some files
differ much (ususally the large files) and some only a bit
(the small files). by applying a large delta to all files
meaningful differences in small files are not detected.
Co-authored-by: Marius van den Beek <m.vandenbeek@gmail.com>
PR #6925 introduced a GUI for connecting non-data (e.g. integer, boolean, color, etc..) workflow input parameter to tool input parameters (the backend for this was originally added in #1306). That ideally was just the beginning of work toward using such values in structured ways in workflows.
This PR extends tool output handling to allow producing of non-data parameters. These can serve as a source for non-data values in workflows the same work workflow input parameters can. To make such values more easy to produce, this PR also introduces Galaxy expression tools - mirroring functionality regularly used in CWL. These small JavaScript-based tools that consume inputs just like a regular Galaxy tool but that produce dictionary of non-data values.
I think these expressions will be maximally useful when paired with format 2 workflows once we allow users to load arbitrary tools (I make the case more in full here https://github.com/galaxyproject/galaxy/pull/7545#issuecomment-473894424), but I outline some potential uses there as well.
Because there is always a checklist in my PR descriptions:
- Tool definition language and plumbing and datatype for expressing expressions as jobs.
- Allow connecting expression tools to parameters in workflows, will delay evaluation of workflow so calculated value
- Example test expression tools for testing and demonstration.
cat-bro