The parent dataset collection now shows the failed datasets,
and that is much easier to deal with when all jobs within
a large collection have failed.
Fixes https://github.com/galaxyproject/galaxy/issues/7492
One downside here is that we don't see anything in the history right
away (except for the failed dataset count) if a hidden collection
contains failed datasets.
PR #6925 introduced a GUI for connecting non-data (e.g. integer, boolean, color, etc..) workflow input parameter to tool input parameters (the backend for this was originally added in #1306). That ideally was just the beginning of work toward using such values in structured ways in workflows.
This PR extends tool output handling to allow producing of non-data parameters. These can serve as a source for non-data values in workflows the same work workflow input parameters can. To make such values more easy to produce, this PR also introduces Galaxy expression tools - mirroring functionality regularly used in CWL. These small JavaScript-based tools that consume inputs just like a regular Galaxy tool but that produce dictionary of non-data values.
I think these expressions will be maximally useful when paired with format 2 workflows once we allow users to load arbitrary tools (I make the case more in full here https://github.com/galaxyproject/galaxy/pull/7545#issuecomment-473894424), but I outline some potential uses there as well.
Because there is always a checklist in my PR descriptions:
- Tool definition language and plumbing and datatype for expressing expressions as jobs.
- Allow connecting expression tools to parameters in workflows, will delay evaluation of workflow so calculated value
- Example test expression tools for testing and demonstration.
This should allow Galaxy to reason about what the tool reports vs. what dependency resolution and metadata reports, allow us to enable logging during metadata collection, and allow us to reason about the tool's I/O from inside the job script during cleanup/metadata collection at the end.
before stdout and stderr were modified while testing for exit codes and
regexes. this had the potential problem that later tests also checked
this output which might lead to confusions.
furthermore a marker line has been introduces which differentiates
Galaxy and original tool output
Björn Grüning