Commit Graph
564 Commits
Author SHA1 Message Date
Tim Kok 7d21159a28 Fix linting + remove debug statement 2025-09-05 13:08:52 +02:00
Tim Kok 3f029fdf5e Remove debug statement 2025-09-05 12:31:24 +02:00
Tim Kok cc6da148c1 Add support for hierarchical module systems in lmod + modules dependency resolvers 2025-09-05 09:04:31 +02:00
John Chilton d394f76580 Fix unit test names. 2025-08-18 11:08:05 -04:00
Nicola Soranzo 7f89e0eab9 Code cleanups from pyupgrade
Found by running:

```
make pyupgrade
ruff check --fix .
ruff check --preview --unsafe-fixes --select=F401 --fix .
make format
make update-client-api-schema
```

The bulk of the changes are replacements of typing deprecated aliases
such as `Dict` and `List` with the standard library classes. These
aliases became redundant in Python 3.9 when the corresponding
pre-existing classes were enhanced to support `[]`, see
https://docs.python.org/3/library/typing.html#deprecated-aliases

Also:
- Update the `pyupgrade` Makefile target to use `--py38-plus` for the
  packages needed by Pulsar, including the new galaxy-tool-util-models
  package.
2025-07-31 11:17:22 +01:00
Nicola Soranzo e2777065eb Merge branch 'release_25.0' into dev 2025-07-30 17:45:20 +01:00
Nicola Soranzo 21d827f678 Merge branch 'release_24.2' into release_25.0 2025-07-30 17:33:15 +01:00
Nicola Soranzo f28450cb75 Fix `test_base_image_for_targets` mulled test to use mzmine
maker is not using the ``extended-base`` image since
https://github.com/bioconda/bioconda-recipes/commit/ce27c583fdd8fcaa60166fa3b0173083e352fd6b
2025-07-30 14:53:04 +01:00
Nicola Soranzo eabe164b68 Improve type annotations of tool-related code
Also:
- Fix `ToolPanelViewSearch._create_doc()`
- Small refactorings
2025-06-25 14:16:19 +01:00
mvdbeek cafd0c8c62 Merge branch 'release_25.0' into dev 2025-06-19 10:18:46 +02:00
mvdbeek 43feb0efb3 Merge branch 'release_24.2' into release_25.0 2025-06-19 10:17:22 +02:00
mvdbeek fdafb0655d Relax validation of XML test assertion parsing
when tool source is XML. We can't really encode these attributes as JSON
in XML, and while we could make it work it doesn't feel great to force
tool authors to use such a hypothetical syntax.

I think we still keep all the nice validation here, it's just a little
more lenient.
2025-06-18 18:55:10 +02:00
Nicola Soranzo 3bff113bfd Build `DependencyManager only if tool_dependency_dir` is set
as specified by the config file documentation. In fact,
`DependencyManager.__init__()` would crash otherwise.

Also:
- Improve type annotation
2025-06-10 02:30:46 +01:00
Nicola Soranzo 90fad91813 Merge branch 'release_24.2' into release_25.0 2025-06-08 21:47:42 +01:00
Nicola Soranzo 162fe28318 Fix `mull_targets()` with mamba 2.x
Fix mulled unit test test/unit/tool_util/mulled/test_mulled_build.py::test_mulled_build_files_cli[True] :

```
use_mamba = True
tmpdir = local('/tmp/pytest-of-runner/pytest-0/test_mulled_build_files_cli_Tr0')

    @pytest.mark.parametrize("use_mamba", [False, True])
    @external_dependency_management
    def test_mulled_build_files_cli(use_mamba: bool, tmpdir) -> None:
        singularity_image_dir = tmpdir.mkdir("singularity image dir")
        target = build_target("zlib", version="1.2.13", build="h166bdaf_4")
        involucro_context = InvolucroContext(involucro_bin=os.path.join(tmpdir, "involucro"))
        exit_code = mull_targets(
            [target],
            involucro_context=involucro_context,
            command="build-and-test",
            singularity=True,
            use_mamba=use_mamba,
            singularity_image_dir=singularity_image_dir,
        )
>       assert exit_code == 0
E       assert 1 == 0

...

[Jun  7 08:40:09] SERR error    libmamba Expected environment not found at prefix: /usr/local
[Jun  7 08:40:09] SERR critical libmamba Aborting.
```

Upstream issue: https://github.com/mamba-org/mamba/issues/3845 .
Fixed upstream in https://github.com/mamba-org/mamba/pull/3919 ,
released in mamba 2.2.0 .

Also:
- Improve type annotations
- Replace `FALLBACK_LINE_TUPLE` namedtuple with `Target` dataclass
2025-06-07 23:06:31 +01:00
mvdbeek a4bad06a06 Merge branch 'release_24.2' into dev 2025-05-16 16:24:49 +02:00
John Chilton 631e33f5bd Merge pull request #19716 from jmchilton/fix_citation_handling
[24.2] Fix #19515 - invalid citation handling changed with 24.2.
2025-05-13 13:04:22 -04:00
mvdbeek 73c72b8fc7 Fix unit test import 2025-05-12 12:29:43 +02:00
mvdbeek e6b0783194 Enable (Javascript) requirements on frontend/backend 2025-05-12 10:54:42 +02:00
mvdbeek 152bc2ffa5 Add runtime model to API
Like job_internal but replaces dataset and dataset collection with file
and array / object of files.
2025-05-12 10:54:30 +02:00
mvdbeek eb38fd8ec9 Merge branch 'release_24.2' into dev 2025-05-12 09:05:28 +02:00
John Chilton 518fa136d9 [WIP] Implement records - heterogenous dataset collections.
Existing dataset colleciton types are meant to be homogenous - all datasets of the same time. This introduces CWL-style record dataset collections.
2025-05-09 16:38:19 -04:00
Ruben Vorderman 9eb9274cb5 fix tests for singularity 2025-05-09 09:27:53 +02:00
mvdbeek b99112c337 Fix DataCollectionParameterModel factory 2025-05-06 14:23:23 +02:00
Marius van den Beek 2266a09e68 Merge pull request #20158 from mvdbeek/yaml_parser_fixes
Yaml parser fixes and improvements (part 1)
2025-05-06 09:25:58 +02:00
mvdbeek bd05b3d0e1 Settle on type instead of reftyp inside xref
We could also make it an alias, but it seems muich simpler to not refer
to the same attribute with different names. So far this has used
internally only.
2025-05-05 19:07:27 +02:00
mvdbeek 44153c7658 Restore option to specify workflow input with src: ld
Bioblend is testing this, so I guess this might not be dead?
2025-04-22 14:18:50 +02:00
John Chilton f91012244b New packages galaxy-tool-util-models, galaxy-tool-shed-schema.
The galaxy-tool-shed-schema package is basically pydantic models for the whole tool shed API. It is used by both the backend code for the new tool shed and the testing code that runs on both the old and new tool shed.

The creation of galaxy-tool-util-models package is something I've wanted to do for a while - it should solve some TODOs I've left in the base. I think beyond enabling galaxy-tool-shed-schema package - the other nice thing is that galaxy-schema code can now rely on various tool stuff without requiring dependency on any of the parser or runtime code.

This should also fix things for the new record types in #19377. I've introduced a type that can be used tools and is included schema for collection creation. The galaxy-tool-util-models packages can now be dependended on by galaxy-schema and galaxy-tool-util and these two packages do not need to depend on each other - either direction of that dependency would make me uncomfortable. xref
https://github.com/galaxyproject/galaxy/actions/runs/13704893929/job/38327692654?pr=19377
2025-04-16 12:47:37 +02:00
mvdbeek 29d3c23cf2 Merge branch 'release_24.2' into dev 2025-04-15 17:53:08 +02:00
Nicola Soranzo e9861aa672 Enable `warn_redundant_casts` mypy option and drop redundant casts 2025-04-14 16:33:35 +01:00
John Chilton f37075c1a5 Fix various parameter validation issues.
I previously failed to include a bunch of validation checks I assumed had been covered so fixed that first and got a bunch of new issues detected. Also added a parameter specification test case for https://github.com/galaxyproject/galaxy/issues/19908 and fixed that.
2025-04-02 12:49:09 -04:00
Nicola Soranzo 1c8fbbb958 Fix mypy issues
```
test/unit/tool_util/test_parsing.py:954: error: Argument 1 to "get_tool_source" has incompatible type "Traversable"; expected "str | PathLike[str] | None"  [arg-type]
        tool_source = get_tool_source(tool_path)
                                      ^~~~~~~~~
test/unit/tool_util/test_parsing.py:960: error: Argument 1 to "get_tool_source" has incompatible type "Traversable"; expected "str | PathLike[str] | None"  [arg-type]
        tool_source = get_tool_source(tool_path)
                                      ^~~~~~~~~
```
2025-03-26 15:18:55 +00:00
mvdbeek 3c6b6c5971 Merge branch 'release_24.2' into dev 2025-03-26 09:51:08 +01:00
mvdbeek 648239a31f Merge branch 'release_24.1' into release_24.2 2025-03-26 09:48:18 +01:00
mvdbeek 953a972349 Merge branch 'release_24.0' into release_24.1 2025-03-26 09:44:07 +01:00
Nicola Soranzo 8f0e11b929 Use `best_search_result()` function instead of parsing anaconda.org web pages
Fix the following mulled unit test:

```
FAILED test/unit/tool_util/mulled/test_get_tests.py::test_hashed_test_search - AssertionError: assert ['samtools --help'] == ['bamtools --help', 'samtools --help']
  At index 0 diff: 'samtools --help' != 'bamtools --help'
  Right contains one more item: 'samtools --help'
  Full diff:
  - ['bamtools --help', 'samtools --help']
  + ['samtools --help']
```

that started not finding bamtools-2.4.0 inside
https://anaconda.org/bioconda/bamtools/files any more.

Also:
- Add type annotation to `CondaInDockerContext` and fix ensuing issues.
2025-03-25 19:45:07 +00:00
John Chilton ef5ee033f3 Fix #19515 - invalid citation handling changed with 24.2. 2025-03-20 10:39:31 -04:00
mvdbeek eb1709fcbb Merge branch 'release_24.2' into dev 2025-03-08 11:20:03 +01:00
Marius van den Beek 5fcc4585fe Merge pull request #19715 from jmchilton/assertion_linter_fixes
[24.2] Assertion linter fixes
2025-03-08 11:09:51 +01:00
Matthias Bernt 9f86a7db62 linter allow datatype auto in output tags 2025-03-06 17:25:05 +01:00
Matthias Bernt 6f485e8533 speedup test
determining base image is not needed, but takes most of the time
2025-03-01 11:33:13 +01:00
John Chilton e6aee8b4cd Fix assertion model linter. 2025-02-27 10:48:29 -05:00
Matthias Bernt f245770fb1 add forgotten linter test
the tool source was added in https://github.com/galaxyproject/galaxy/pull/15493
but the actual test was missing
2025-02-25 11:49:27 +01:00
Nicola Soranzo 7ce76af796 Format code with black 25.1.0 2025-02-16 18:07:51 +00:00
Nicola Soranzo 0aca5fd7e0 Merge branch 'release_24.2' into dev 2025-02-16 17:34:32 +00:00
Nicola Soranzo e3942053f4 Merge branch 'release_24.1' into release_24.2 2025-02-16 17:26:50 +00:00
Nicola Soranzo ca9bf294f4 Merge branch 'release_24.0' into release_24.1 2025-02-16 17:19:15 +00:00
Nicola Soranzo 18312cfc69 Fix CondaSearch.get_json() to skip all header lines
Fix the following `test_conda_search` failure:

```
______________________________ test_conda_search _______________________________

    @external_dependency_management
    @skip_unless_executable("conda")
    def test_conda_search():
        t = CondaSearch("bioconda")
        search1 = t.get_json("asdfasdf")
        search2 = t.get_json("bioconductor-gosemsim")
        assert search1 == []
>       assert all(r["package"] == "bioconductor-gosemsim" for r in search2)
E       assert False
E        +  where False = all(<generator object test_conda_search.<locals>.<genexpr> at 0x7f5be7fc34a0>)

search1    = []
search2    = [{'build': 'Version', 'package': '#', 'version': 'Name'},
 {'build': '0', 'package': 'bioconductor-gosemsim', 'version': '1.28.2'},
 {'build': 'r3.3.1_0', 'package': 'bioconductor-gosemsim', 'version': '1.29.0'},
 {'build': 'r3.3.1_0', 'package': 'bioconductor-gosemsim', 'version': '1.30.3'},
 ...
 {'build': 'r43hf17093f_0', 'package': 'bioconductor-gosemsim', 'version': '2.28.0'},
 {'build': 'r43hf17093f_1', 'package': 'bioconductor-gosemsim', 'version': '2.28.0'},
 {'build': 'r44he5774e6_0', 'package': 'bioconductor-gosemsim', 'version': '2.32.0'}]
```
2025-02-15 13:34:44 +00:00
Nicola Soranzo 4d15cb5bd8 Type annotation improvements
Also:
- Small refactorings
2025-01-29 18:01:43 +00:00
John Davis 16e18d404e Merge pull request #19395 from mvdbeek/alternative_format_source_fix
Alternative `format_source` fix
2025-01-23 10:54:03 -05:00