when tool source is XML. We can't really encode these attributes as JSON
in XML, and while we could make it work it doesn't feel great to force
tool authors to use such a hypothetical syntax.
I think we still keep all the nice validation here, it's just a little
more lenient.
We could also make it an alias, but it seems muich simpler to not refer
to the same attribute with different names. So far this has used
internally only.
The galaxy-tool-shed-schema package is basically pydantic models for the whole tool shed API. It is used by both the backend code for the new tool shed and the testing code that runs on both the old and new tool shed.
The creation of galaxy-tool-util-models package is something I've wanted to do for a while - it should solve some TODOs I've left in the base. I think beyond enabling galaxy-tool-shed-schema package - the other nice thing is that galaxy-schema code can now rely on various tool stuff without requiring dependency on any of the parser or runtime code.
This should also fix things for the new record types in #19377. I've introduced a type that can be used tools and is included schema for collection creation. The galaxy-tool-util-models packages can now be dependended on by galaxy-schema and galaxy-tool-util and these two packages do not need to depend on each other - either direction of that dependency would make me uncomfortable. xref
https://github.com/galaxyproject/galaxy/actions/runs/13704893929/job/38327692654?pr=19377
I previously failed to include a bunch of validation checks I assumed had been covered so fixed that first and got a bunch of new issues detected. Also added a parameter specification test case for https://github.com/galaxyproject/galaxy/issues/19908 and fixed that.
Fix the following mulled unit test:
```
FAILED test/unit/tool_util/mulled/test_get_tests.py::test_hashed_test_search - AssertionError: assert ['samtools --help'] == ['bamtools --help', 'samtools --help']
At index 0 diff: 'samtools --help' != 'bamtools --help'
Right contains one more item: 'samtools --help'
Full diff:
- ['bamtools --help', 'samtools --help']
+ ['samtools --help']
```
that started not finding bamtools-2.4.0 inside
https://anaconda.org/bioconda/bamtools/files any more.
Also:
- Add type annotation to `CondaInDockerContext` and fix ensuing issues.
That's a much better consumer experience. A model generated for
gx_conditional.xml would look like
```typescript
export type ConditionalType =
| WhenTestParameterA
| WhenTestParameterB
| WhenTestParameterC
export type TestParameter = "a"
export type IntegerParameter = number
export type TestParameter1 = "b"
export type BooleanParameter = boolean
export type TestParameter2 = "c"
export type ColorParameter = string
export interface MySchema {
conditional_parameter: ConditionalType
}
export interface WhenTestParameterA {
test_parameter: TestParameter
integer_parameter: IntegerParameter
}
export interface WhenTestParameterB {
test_parameter: TestParameter1
boolean_parameter: BooleanParameter
}
export interface WhenTestParameterC {
test_parameter: TestParameter2
color_parameter: ColorParameter
}
```
instead of
```typescript
export type ConditionalType =
| WhenTestParameterA
| WhenTestParameterB
| WhenTestParameterC
export type TestParameter = string | boolean
export type IntegerParameter = number
export type TestParameter1 = string | boolean
export type BooleanParameter = boolean
export type TestParameter2 = string | boolean
export type ColorParameter = string
export interface MySchema {
conditional_parameter: ConditionalType
}
export interface WhenTestParameterA {
test_parameter: TestParameter
integer_parameter: IntegerParameter
}
export interface WhenTestParameterB {
test_parameter: TestParameter1
boolean_parameter: BooleanParameter
}
export interface WhenTestParameterC {
test_parameter: TestParameter2
color_parameter: ColorParameter
}
```
With the new model I can write an input object without looking at the
tool source, with the previous one I don't know what string I would need
to supply.
(which started failing since ubuntu-latest was moved to 24.04)
by upgrading apptainer to >=1.3.3 to include
https://github.com/apptainer/apptainer/pull/2262 .
Fix:
```
> assert "samtools:1.0--1" in results["passed"], results
E AssertionError:
E {'failed': [{'commands': ['python -c "import pyBigWig; assert(pyBigWig.numpy '
E '== 1); assert(pyBigWig.remote == 1)"'],
E 'container': 'pybigwig:0.3.22--py36h54a71a5_0',
E 'errors': [{'command': 'python -c "import pyBigWig; '
E 'assert(pyBigWig.numpy == 1); '
E 'assert(pyBigWig.remote == 1)"',
E 'output': '\x1b[91mERROR : Could not write info to '
E 'setgroups: Permission denied\n'
E '\x1b[0m\x1b[91mERROR : Error while '
E 'waiting event for user namespace mappings: '
E 'no event received\n'
E '\x1b[0m'},
E {'import': 'pyBigWig',
E 'output': '\x1b[91mERROR : Could not write info to '
E 'setgroups: Permission denied\n'
E '\x1b[0m\x1b[91mERROR : Error while '
E 'waiting event for user namespace mappings: '
E 'no event received\n'
E '\x1b[0m'}],
E 'import_lang': 'python -c',
E 'imports': ['pyBigWig']},
E {'commands': ['samtools view --help 2>&1 | grep Notes > /dev/null'],
E 'container': 'samtools:1.0--1',
E 'errors': [{'command': 'samtools view --help 2>&1 | grep Notes > '
E '/dev/null',
E 'output': '\x1b[91mERROR : Could not write info to '
E 'setgroups: Permission denied\n'
E '\x1b[0m\x1b[91mERROR : Error while '
E 'waiting event for user namespace mappings: '
E 'no event received\n'
E '\x1b[0m'}],
E 'import_lang': 'python -c'}],
E 'notest': ['yasm:1.3.0--0'],
E 'passed': []}
E assert 'samtools:1.0--1' in []
```
Also:
- Improve assertion messages to facilitate debugging
(which started failing since ubuntu-latest was moved to 24.04)
by upgrading apptainer to >=1.3.3 to include
https://github.com/apptainer/apptainer/pull/2262 .
Fix:
```
> assert "samtools:1.0--1" in results["passed"], results
E AssertionError:
E {'failed': [{'commands': ['python -c "import pyBigWig; assert(pyBigWig.numpy '
E '== 1); assert(pyBigWig.remote == 1)"'],
E 'container': 'pybigwig:0.3.22--py36h54a71a5_0',
E 'errors': [{'command': 'python -c "import pyBigWig; '
E 'assert(pyBigWig.numpy == 1); '
E 'assert(pyBigWig.remote == 1)"',
E 'output': '\x1b[91mERROR : Could not write info to '
E 'setgroups: Permission denied\n'
E '\x1b[0m\x1b[91mERROR : Error while '
E 'waiting event for user namespace mappings: '
E 'no event received\n'
E '\x1b[0m'},
E {'import': 'pyBigWig',
E 'output': '\x1b[91mERROR : Could not write info to '
E 'setgroups: Permission denied\n'
E '\x1b[0m\x1b[91mERROR : Error while '
E 'waiting event for user namespace mappings: '
E 'no event received\n'
E '\x1b[0m'}],
E 'import_lang': 'python -c',
E 'imports': ['pyBigWig']},
E {'commands': ['samtools view --help 2>&1 | grep Notes > /dev/null'],
E 'container': 'samtools:1.0--1',
E 'errors': [{'command': 'samtools view --help 2>&1 | grep Notes > '
E '/dev/null',
E 'output': '\x1b[91mERROR : Could not write info to '
E 'setgroups: Permission denied\n'
E '\x1b[0m\x1b[91mERROR : Error while '
E 'waiting event for user namespace mappings: '
E 'no event received\n'
E '\x1b[0m'}],
E 'import_lang': 'python -c'}],
E 'notest': ['yasm:1.3.0--0'],
E 'passed': []}
E assert 'samtools:1.0--1' in []
```
Also:
- Improve assertion messages to facilitate debugging
`__package__` is deprecated since Python 3.13, see
https://docs.python.org/3/reference/datamodel.html#module.__package__
`importlib.resources.files()` can be passed a non-package module
since Python 3.12 (or importlib-resources 5.10.0, see
https://importlib-resources.readthedocs.io/en/latest/history.html#v5-10-0 ),
so we can use `__name__` instead of `__package__`.
Also, I noticed that in lib/galaxy/config/__init__.py we are passing
a directory to `importlib.resources.as_file()`, which is supported
only since Python 3.12 (or importlib-resources 5.9.0).
For the reasons above:
- Update conditional imports in `lib/galaxy/util/resources.py` to
import from `importlib.resources` only for Python 3.12,
otherwise from the importlib-resources package.
- Update importlib-resources requirement in galaxy-util package.
- Drop importlib-resources requirement from galaxy-web-apps as it's
not imported directly from its code any more.
mvdbeek