This fixes
```
ERROR galaxy.tools.actions:__init__.py:683 Cannot remap rerun dependencies.
Traceback (most recent call last):
File "/Users/mvandenb/src/galaxy/lib/galaxy/tools/actions/__init__.py", line 664, in _remap_job_on_rerun
self.__remap_parameters(job_to_remap, jtid, jtod, out_data)
File "/Users/mvandenb/src/galaxy/lib/galaxy/tools/actions/__init__.py", line 694, in __remap_parameters
input_values = {p.name: json.loads(p.value) for p in job_to_remap.parameters}
File "/Users/mvandenb/src/galaxy/lib/galaxy/tools/actions/__init__.py", line 694, in <dictcomp>
input_values = {p.name: json.loads(p.value) for p in job_to_remap.parameters}
File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 339, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not
NoneType
```
Optional data inputs or optional selects are stored as `None` (super
inconsistent, since most other parameters are stored as JOSN. We should
create "basic_2.py" using pydantic at one point not too far into the
future ...). This means we can't call `json.loads` on these. Fortunately
this is the only place we do it, and we don't need to consider optional
parameters here anyway.
There used to be a long-standing bug in setting up the MetadataValidator
if default values were used. We fixed this in
https://github.com/galaxyproject/galaxy/pull/13139/commits/e194ef97e49c4947711ad1d20873ddc0aa686a66,
but that means we're now checking for all non-optional values before
running a tool. We have a ton of non-optional MetadatElement items
in datatypes that should maybe be optional (an indication might be if
`default` and `no_value` are specified and set to the same value ... but
I'm not sure that's a 100% thing). So I think that reviewing this
requires domain knowledge of the datatypes and what elements are really required,
and I'm not sure we can do this in a timely fashion, and not break
something that used to work.
So my suggestion is that we add `check_required_metadata=True`
on datatypes for which we have checked that non-optional metadata
elements are really non-optional. For those metadata elements
for which this is not the case we skip the validation as we would
do prior to
https://github.com/galaxyproject/galaxy/pull/13139/commits/e194ef97e49c4947711ad1d20873ddc0aa686a66.
As an example I have marked RDS and RData with check_required_metadata
and added a test for check_required_metadata.
when loading macros from imported files
the base property of the node is
overwritten.
this adds a xml:base property to the node
which is printed in the test
this adds functionality to the linter functions to return the lint
context. without breaking previous behavior.
Planemo seems to use
- lint_tool_source (which returned the fail state) and
- lint_tool_source_with (which returned nothing)
For the later a parameter has been added to make the linter silent
and the lint context is now returned.
For the former a variant has been added which returns the lint context.
The .. and ... and ... and within a lambda gives you
no chance to debug the test without a print statement or
a debugger.
Co-authored-by: Marius van den Beek <m.vandenbeek@gmail.com>
- add fname to LintMessage
- LintContext make lint messages persistent over multiple linter calls
before the messages were reset before the call of the next linter
which is "impractical" if we want to use the lintmessages eg in
planemo
- add test for linting a tool + (internal and external) macros
to ensure that the fname and line number is correct
- add a simple linter test for yaml and json tools
archive asserts:
- check for absent members directly in the assertion
- before the _extract... functions returned `None`
which had no effect if there is no included assertion
(probably this is also the reason why some of the
text assertions check for None)
- so now the assertion can be used without included
sub-assertions to check for member presence
- fix for non-file members: the behavior for zip and tar
was different
hdf5 asserts:
- fix assertion text
- added unit tests for all xml assertions
- restructured xsd:
- grouped general, text, tabular, archive, xml, and h5 assertions
- full definition of all assertions in xsd
- allow for "recursion" in xsd for has_archive_member and element_text
- restructured schema doc: instead of the table its now a
definition list (no more horizontal scrolling in the wide table)
also here assertions are grouped
mvdbeek