This fixes
```
ERROR galaxy.tools.actions:__init__.py:683 Cannot remap rerun dependencies.
Traceback (most recent call last):
File "/Users/mvandenb/src/galaxy/lib/galaxy/tools/actions/__init__.py", line 664, in _remap_job_on_rerun
self.__remap_parameters(job_to_remap, jtid, jtod, out_data)
File "/Users/mvandenb/src/galaxy/lib/galaxy/tools/actions/__init__.py", line 694, in __remap_parameters
input_values = {p.name: json.loads(p.value) for p in job_to_remap.parameters}
File "/Users/mvandenb/src/galaxy/lib/galaxy/tools/actions/__init__.py", line 694, in <dictcomp>
input_values = {p.name: json.loads(p.value) for p in job_to_remap.parameters}
File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 339, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not
NoneType
```
Optional data inputs or optional selects are stored as `None` (super
inconsistent, since most other parameters are stored as JOSN. We should
create "basic_2.py" using pydantic at one point not too far into the
future ...). This means we can't call `json.loads` on these. Fortunately
this is the only place we do it, and we don't need to consider optional
parameters here anyway.
If slurm decided the job is in error. There are configurations of slurm
that error out when cancelling the job monitor script, and others that
don't do this.
defintion.
Fixes
```
2022-02-03 12:40:03,491 ERROR [galaxy.web.framework.decorators] Uncaught exception in exposed API method:
Traceback (most recent call last):
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/web/framework/decorators.py", line 320, in decorator
rval = func(self, trans, *args, **kwargs)
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/webapps/galaxy/api/tools.py", line 563, in create
return self._create(trans, payload, **kwd)
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/webapps/galaxy/api/tools.py", line 631, in _create
vars = tool.handle_input(trans, incoming, history=target_history, use_cached_job=use_cached_job, input_format=input_format)
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/tools/__init__.py", line 1767, in handle_input
all_params, all_errors, rerun_remap_job_id, collection_info = self.expand_incoming(trans=trans, incoming=incoming, request_context=request_context, input_format=input_format)
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/tools/__init__.py", line 1747, in expand_incoming
populate_state(request_context, self.inputs, expanded_incoming, params, errors, simple_errors=False, input_format=input_format)
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/tools/parameters/__init__.py", line 348, in populate_state
_populate_state_legacy(request_context, inputs, incoming, state, errors=errors, context=context, check=check, simple_errors=simple_errors)
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/tools/parameters/__init__.py", line 408, in _populate_state_legacy
state[input.name] = input.get_initial_value(request_context, context)
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/tools/parameters/basic.py", line 1001, in get_initial_value
options = list(self.get_options(trans, other_values))
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/tools/parameters/basic.py", line 890, in get_options
return self.options.get_options(trans, other_values)
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/tools/parameters/dynamic_options.py", line 763, in get_options
rval = filter.filter_options(rval, trans, other_values)
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/tools/parameters/dynamic_options.py", line 207, in filter_options
if not r.metadata.element_is_set(self.key):
File "/tmp/tmp778b108w/galaxy-dev/lib/galaxy/model/metadata.py", line 175, in element_is_set
meta_spec = self.parent.metadata.spec[name]
KeyError: 'groups'
```
which happens when forcing a tabular dataset into a more specialized
input https://github.com/galaxyproject/tools-iuc/blob/e22821f51ae326b3696a9456cbc44f9982512a52/tools/mothur/homova.xml#L48
Arguably this is a tool issue, but we already log a KeyError above and
it seems more of a linting issue than something that should cause a test
failure.
```
Traceback (most recent call last):
File "lib/galaxy/jobs/runners/__init__.py", line 707, in monitor
self.check_watched_items()
File "lib/galaxy/jobs/runners/drmaa.py", line 338, in check_watched_items
if self._complete_terminal_job(ajs, drmaa_state=state) is not None:
File "lib/galaxy/jobs/runners/univa.py", line 85, in _complete_terminal_job
time_granted, mem_granted = _parse_native_specs(ajs.job_id, native_spec)
File "lib/galaxy/jobs/runners/univa.py", line 572, in _parse_native_specs
m = re.search(r"rt=([0-9:]+)[\s,]*", native_spec)
File "/pool/spare/galaxy/galaxy/.venv/lib64/python3.6/re.py", line 182, in search
return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object
```
See https://github.com/galaxyproject/galaxy/issues/13297 reported by @ajs6f fix from @mvdbeek
There used to be a long-standing bug in setting up the MetadataValidator
if default values were used. We fixed this in
https://github.com/galaxyproject/galaxy/pull/13139/commits/e194ef97e49c4947711ad1d20873ddc0aa686a66,
but that means we're now checking for all non-optional values before
running a tool. We have a ton of non-optional MetadatElement items
in datatypes that should maybe be optional (an indication might be if
`default` and `no_value` are specified and set to the same value ... but
I'm not sure that's a 100% thing). So I think that reviewing this
requires domain knowledge of the datatypes and what elements are really required,
and I'm not sure we can do this in a timely fashion, and not break
something that used to work.
So my suggestion is that we add `check_required_metadata=True`
on datatypes for which we have checked that non-optional metadata
elements are really non-optional. For those metadata elements
for which this is not the case we skip the validation as we would
do prior to
https://github.com/galaxyproject/galaxy/pull/13139/commits/e194ef97e49c4947711ad1d20873ddc0aa686a66.
As an example I have marked RDS and RData with check_required_metadata
and added a test for check_required_metadata.
Alexander OSTROVSKY