Commit Graph
397 Commits
Author SHA1 Message Date
Nicola Soranzo 8f9583f4ac Disallow pebble 5.2.1
due to https://github.com/noxdafox/pebble/issues/164 .
2026-07-25 18:58:46 +01:00
mvdbeek a5f60f2001 Merge branch 'release_26.1' into dev 2026-07-14 19:13:50 +02:00
Marius van den Beek 1cf3cb78d1 Merge pull request #23108 from mvdbeek/upgrade-pulsar-client
[26.1] Update pulsar-galaxy-client dependency to 0.15.15
2026-07-13 20:59:06 +02:00
mvdbeek 962d9f7378 Bump minimum pulsar-galaxy-lib version to 0.15.15 2026-07-13 19:03:47 +02:00
Nicola Soranzo 26009807a2 Make cloudbridge a conditional/test requirement
It's not used by pre-installed tools any more, just for the cloud
object store.
2026-07-11 15:47:35 +01:00
Nicola Soranzo 9472e889cb Type annotation fixes for mypy 2.2.0
and cloudbridge 4.2.0 .
2026-07-11 14:27:00 +01:00
Nicola Soranzo dd6ef0aaed Sync OpenAPI code with FastAPI 0.137.2 2026-07-10 01:49:04 +01:00
Nicola Soranzo 0889af7dfc Always install `fastmcp`
in preparation for pydantic-ai 2.0.0, which replaced the
dependency on `fastmcp` with `fastmcp-slim[client]`.

Otherwise Galaxy fails to start with:

```
Traceback (most recent call last):
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/server/__init__.py", line 6, in <module>
    from .context import Context
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/server/context.py", line 28, in <module>
    from uncalled_for import SharedContext
ModuleNotFoundError: No module named 'uncalled_for'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/__init__.py", line 57, in __getattr__
    from fastmcp.server.context import Context
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/server/__init__.py", line 9, in <module>
    raise ImportError(_install_hints.SERVER_SUPPORT) from exc
ImportError: FastMCP server support is not installed. Install `fastmcp` or `fastmcp-slim[server]`.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/bin/gunicorn", line 10, in <module>
    sys.exit(run())
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 66, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]", prog=prog).run()
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 235, in run
    super().run()
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 71, in run
    Arbiter(self).run()
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/arbiter.py", line 63, in __init__
    self.setup(app)
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/arbiter.py", line 139, in setup
    self.app.wsgi()
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 66, in wsgi
    self.callable = self.load()
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 57, in load
    return self.load_wsgiapp()
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 47, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/util.py", line 464, in import_app
    app = app(*args, **kwargs)
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/fast_factory.py", line 63, in factory
    gx_wsgi_webapp, gx_app = app_pair(
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 178, in app_pair
    populate_api_routes(webapp, app)
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 362, in populate_api_routes
    webapp.add_api_controllers("galaxy.webapps.galaxy.api", app)
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/base/webapp.py", line 252, in add_api_controllers
    for name, module in base.walk_controller_modules(package_name):
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/web/framework/base.py", line 624, in walk_controller_modules
    module = import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/api/mcp.py", line 15, in <module>
    from fastmcp import (
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/__init__.py", line 59, in __getattr__
    raise ImportError(_install_hints.SERVER_SUPPORT) from exc
ImportError: FastMCP server support is not installed. Install `fastmcp` or `fastmcp-slim[server]`.
```
2026-07-10 01:49:03 +01:00
Yakubov, Sergey 6241893238 Bump Rucio clients to 40.2.0
Update Galaxy's Rucio test and optional dependency declarations to require rucio-clients 40.2.0.

Adjust the Rucio object store for client 40 behavior, including checksum import changes, ingest checksum handling, Rucio config cache invalidation, and isolated download directories.

Run Rucio integration tests against the Savannah Rucio 40.2.0 container by default while keeping an environment override for the image.
2026-06-26 12:48:43 -04:00
paperbenni bec3d3ad9c Drop support for Python 3.8
Drop Python 3.8 support in 5 Pulsar-compatible packages
(job_metrics, tool_util, tool_util_models, util, objectstore)
and run pyupgrade --py310-plus on their source code.

- Bump requires-python from >=3.8 to >=3.10 in all 5 packages
- Remove Python 3.8 and 3.9 classifiers
- Remove ruff per-file-ignores for UP rules on these paths
- Remove backports.zoneinfo conditional dependency
- Remove pydyf<0.11 pin from conditional-requirements.txt
- Update Makefile pyupgrade target (remove PY38_PYUPGRADE_PATHS)
- Update CI workflow to test with Python 3.10 instead of 3.8

Clean up unused deprecated typing imports after pyupgrade

Remove now-unused typing imports (Dict, List, Optional, Set, Tuple,
Type, Union) that became dead after pyupgrade --py310-plus converted
annotations to use built-in types and | syntax.

Also run ruff check --fix --select=UP007,UP045 across the entire
codebase to convert remaining Optional[X] -> X | None and
Union[X, Y] -> X | Y patterns.

Enable ruff UP007/UP045 for Python 3.10 union syntax

Remove UP007 (Union[X,Y] -> X | Y) and UP045 (Optional[X] -> X | None)
from the ruff ignore list and convert all type aliases across the
codebase. These rules were deferred while Python 3.9 was supported;
requires-python is now >=3.10.

A custom script was used because neither ruff --fix nor
pyupgrade --py310-plus converts Optional[X]/Union[X,Y] in type alias
positions (e.g. X = Union[A, B]) — they only handle annotation
positions (e.g. def f(x: Optional[int])). All 167 violations were
module-level type aliases. A few edge cases were fixed manually:
single-element Union[X,], typing.Union qualified refs, runtime
Optional[type] calls, and Annotated[Optional[...]] pydantic fields.

Fix UP007 autofix regression with string forward reference type aliases

Commit fa6bd955a0 enabled ruff UP007/UP045 and auto-fixed
module-level type aliases using Union with string forward
references, producing invalid 'str | str' expressions.

This was a known ruff bug (charliermarsh/ruff#826) that has since
been fixed in later ruff versions, but this codebase was
converted before the fix was in place.

Revert to Union syntax and restore TYPE_CHECKING imports that
ruff's TCH rule cleaned up as a side effect when it thought the
forward references were unused.
2026-06-25 15:12:05 +01:00
paperbenni e94193d059 Adjust ruff settings 2026-06-25 14:42:34 +01:00
mvdbeek 41272995ea Merge branch 'release_26.1' into dev 2026-06-20 22:49:25 +02:00
Marius van den Beek 86b3042679 Merge pull request #22869 from guerler/fixes.009
[26.1] Add fixed color set from webcolors package
2026-06-13 19:36:42 +02:00
guerler 5953703358 Use pydantic_extra_types_color for color input validation 2026-06-11 17:46:01 +03:00
mvdbeek 5f7fc94d6a Bump up gravity to 1.2.3 2026-06-11 10:23:27 +02:00
mvdbeek a25a180ab0 Merge branch 'release_26.1' into dev 2026-06-03 15:56:20 +02:00
mvdbeek aa645069d8 Merge branch 'release_26.0' into release_26.1
# Conflicts:
#	lib/galaxy/dependencies/pinned-requirements.txt
#	lib/galaxy/schema/generics.py
#	lib/galaxy/webapps/galaxy/fast_app.py
#	packages/web_apps/setup.cfg
2026-06-03 15:55:14 +02:00
mvdbeek a8fa956fc1 Merge branch 'release_25.1' into release_26.0 2026-05-29 16:44:48 +02:00
mvdbeekandNicola Soranzo 6b07f5492f [25.1] Backport FastAPI/Starlette upgrade for BadHost (CVE-2026-48710)
Updates the pinned FastAPI/Starlette versions on release_25.1 to match
upstream/dev, closing CVE-2026-48710 ("BadHost"). The vulnerability lets
an attacker inject a path into the HTTP Host header and have
``request.url.path`` reflect that path, bypassing path-based access
control in middleware. Starlette 1.0.1+ rejects Host headers with
invalid characters; we pin starlette==1.1.0 to match dev.

Drops Python 3.9 support, matching upstream/dev (commit eb9fde1dcd).
Starlette dropped 3.9 in 0.50.0, so the BadHost fix (starlette >= 1.0.1)
cannot be installed on 3.9. Bumping the floor to 3.10 is the only way
to ship the security fix; the alternative -- marker-splitting the pins
to keep 3.9 on starlette 0.49.x -- would leave 3.9 users exposed to the
CVE we are trying to close.

Actual exposure on release_25.1 (pre-fix)
-----------------------------------------

Galaxy was running a vulnerable starlette but is **not exploitable for
the headline auth-bypass scenario**. Auth runs through FastAPI
dependencies (``get_user``, ``get_trans``, ``AdminUserRequired``), not
through path-string checks in middleware. ``AccessLoggingMiddleware``
already uses ``scope["path"]``; ``add_galaxy_middleware`` /
``GalaxyCORSMiddleware`` / ``RawContextMiddleware`` /
``SentryAsgiMiddleware`` do not branch on ``request.url.path``.

Lower-severity findings that this upgrade also closes:

- ``lib/galaxy/webapps/base/api.py`` ``get_error_response_for_request``
  picks an error schema with ``"ga4gh"/"drs"/"trs" in
  request.url.path`` -- spoofable Host nudges error payload shape
  (info-disclosure / shape confusion only).
- ``lib/galaxy/webapps/galaxy/api/drs.py`` and
  ``lib/galaxy/webapps/galaxy/services/datasets.py`` derive DRS
  ``service_info`` / ``self_uri`` from ``request.url`` -- spoofable
  Host poisons the advertised DRS identity and client-visible URIs,
  not access.
- ``lib/tool_shed/webapp/api2/tools.py`` -- same shape, TRS service
  info.

Bumping the pin closes the parser flaw at source and downgrades all of
the above to non-issues, so no separate ``request.url.path ->
scope["path"]`` sweep is needed.

Pin bumps (cherry-picks the relevant ranges from PRs #21526, #22206,
#22754):

- fastapi 0.118.0 -> 0.136.3
- starlette 0.48.0 -> 1.1.0
- starlette-context 0.4.0 -> 0.5.1
- python-multipart 0.0.20 -> 0.0.29
- anyio 4.11.0 -> 4.13.0

Required code changes backported from upstream/dev:

- ``lib/galaxy/webapps/openapi/_compat/v2.py``: import
  ``GenerateJsonSchema`` and ``get_flat_models_from_fields`` from
  ``fastapi._compat.v2`` and adopt the new ``get_definitions()``
  implementation for FastAPI 0.128.8+ (PRs #21384, dev commits
  0800c025ce, c8ccc7f44d, b3bfb45884).
- ``lib/galaxy/webapps/openapi/utils.py``: route ``GenerateJsonSchema``
  through ``_compat.v2`` and drop the now-unreachable
  ``get_compat_model_name_map`` fallback (dev commit b3bfb45884).
- ``lib/galaxy/schema/generics.py``: drop ``CustomJsonSchema`` here
  (moved to fast_app.py so it can use the patched ``GenerateJsonSchema``
  from ``_compat.v2``).
- ``lib/galaxy/webapps/galaxy/fast_app.py``: relocate
  ``CustomJsonSchema`` and switch its base to the ``_compat.v2``
  ``GenerateJsonSchema``; replace the @app.middleware("http") X-Frame
  Options handler with a pure ASGI ``XFrameOptionsMiddleware`` class
  (dev commit 67eea395ab) since ``BaseHTTPMiddleware`` semantics
  changed in starlette 1.0.
- ``lib/galaxy/webapps/base/api.py`` and
  ``lib/galaxy/webapps/galaxy/api/datasets.py``: drop the ``method``
  argument of ``FileResponse`` which starlette 1.0 removed (dev commit
  63954cb42e).

Also bumps the minimum FastAPI requirement to ``>=0.133.0`` (first
version compatible with starlette>=1.0.0) and adds an explicit
``starlette>=1.0.1`` floor in ``pyproject.toml`` and the
``packages/web_apps`` / ``packages/tool_shed`` setup.cfg files so
source installs cannot resolve to a vulnerable combo.

Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2026-05-28 18:56:59 +02:00
John ChiltonandClaude Opus 4.7 4233f9f6da Bump gxformat2 to 0.27.0 for UDT support
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 14:11:10 -04:00
Nicola Soranzo e6a19f1a2d Remove htcondor from test dependencies 2026-05-23 20:08:49 +01:00
John Chilton 78fc4a33ad Merge pull request #21528 from bgruening/htcondor2
Add a new "htcondor" runner that utlizes the python htcondor version 2 bindings
2026-05-16 10:31:24 -04:00
Nicola Soranzo 607d2fc1df Disallow gxformat2 0.25.0 and 0.26.0
due to https://github.com/galaxyproject/gxformat2/issues/204 .
2026-05-10 16:57:22 +01:00
Bjoern Gruening 5b19460ad8 add htcondor as test requirements 2026-05-08 00:00:08 +02:00
mvdbeek da97ee6191 Add types-cachetools to typecheck deps 2026-04-28 17:18:08 +02:00
Nicola Soranzo c40e09084e Merge branch 'release_26.0' into dev 2026-04-22 03:54:23 +01:00
Nicola Soranzo e0681411fa Pin rocrate<0.15.0 until roc-validator supports RO-Crate Metadata Specification 1.2
rocrate 0.15.0 produces crates conforming to 1.2 spec by default, which
causes roc-validator validation errors like:

```
The RO-Crate metadata file descriptor MUST have a `conformsTo` property with the RO-Crate specification version
```

xref https://github.com/crs4/rocrate-validator/issues/107

Also:
- Remove unused `BaseWorkflowPopulator.validate_invocation_crate_directory()` method.
2026-04-21 16:45:52 +01:00
mvdbeek 9629f2f6fa Add aiocop middleware for blocking-I/O detection
Integrate aiocop (https://github.com/Feverup/aiocop), which uses
sys.audit hooks to catch specific blocking syscalls (socket.connect,
getaddrinfo, subprocess, open, etc.) from inside async tasks and report
the exact call site.

aiocop is pinned to the event-loop thread explicitly because Galaxy's
test harness runs uvicorn in a non-main thread; activation happens on
the ASGI lifespan startup event so the first request is monitored. An
AiocopMiddleware surfaces captured events per-request via an
X-Aiocop-Violations header (count/max-severity/first) so the test
interactor can fail requests on high-severity blocking I/O.

Integration tests spin up a fresh event loop on a new thread per test
module via driver_util.uvicorn_serve, and aiocop has process-global
state (audit hook registration, patch_audit_functions side effects,
detect_slow_tasks's _detect_slow_tasks_configured guard) that must not
be repeated. install_aiocop() is therefore split into a once-per-process
block (audit patching, audit hook registration) and a per-Galaxy-instance
re-arm (main-thread rebinding, callback clear+register, detect_slow_tasks
reset) so each new Galaxy instance actually gets its loop monitored.

Enabled by default under GALAXY_TEST_AIOCOP=1 in run_tests.sh; set to 0
to disable.
2026-04-21 10:55:55 +02:00
Michael R. Crusoe dbfcdb8bc9 python packages: convert to pure namespace packages
only directories in `lib/galaxy` & `packages/*/galaxy`
All other files removed or moved to their own module
All references to those files updated, especially galaxy/version.py
2026-04-15 15:16:34 +02:00
Nicola Soranzo 35dcde758c Pin rocrate<0.15.0 until roc-validator supports RO-Crate Metadata Specification 1.2
rocrate 0.15.0 produces crates conforming to 1.2 spec by default, which
causes roc-validator validation errors like:

```
The RO-Crate metadata file descriptor MUST have a `conformsTo` property with the RO-Crate specification version
```

xref https://github.com/crs4/rocrate-validator/issues/107

Also:
- Remove unused `BaseWorkflowPopulator.validate_invocation_crate_directory()` method.
2026-04-13 17:21:08 +01:00
mvdbeek 74b6e24bea Add Hashicorp Vault token renewal support
Add a celery beat for renewal of Hashicorp Vault tokens.
This eliminates the need for manual token rotation when
using short-lived renewable tokens.

The approach:
- HashicorpVault checks token renewable status on startup (warning
  if not renewable)
- New `renew_vault_token` Celery Beat task calls renew-self
- Configured via `vault_token_renewal_interval` in galaxy.yml
  (default 0 = disabled)
- Requires Celery Beat to be running

Fixes https://github.com/galaxyproject/galaxy/issues/22187
2026-03-30 12:27:04 +02:00
Nicola Soranzo 0db1cce97c Merge branch 'release_26.0' into dev 2026-03-23 12:29:06 +00:00
John ChiltonandClaude Opus 4.6 ecc5d1c41f Update gxformat2 dependency to 0.23.0 release.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 06:15:40 -04:00
Marius van den Beek 8bb0116006 Merge pull request #21953 from jmchilton/toolshed_cleanup_iteration
Shed Cleanup - Eliminate Twill
2026-03-02 10:54:49 +01:00
Nicola Soranzo 0b410de484 Support refgenconf 0.13.0 2026-02-28 19:21:16 +00:00
John ChiltonandClaude Opus 4.6 0596b82a58 Remove twill dependency from pyproject.toml and pinned requirements
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 09:55:29 -05:00
mvdbeek 894e697a2d Upgrade gxformat2 to 0.22.0, remove format2 workarounds
gxformat2 0.22.0 natively handles content_source/content_id in
from_galaxy_native export and URL strings in run: fields during
import. This removes the _to_format2_with_preserved_links placeholder
workaround and the client_convert=False requirement for URL subworkflow
imports.
2026-02-24 13:49:57 +01:00
Nicola Soranzo 5b9e03825a Unify minimum mercurial version 2026-02-23 17:14:01 +00:00
Nicola Soranzo 508f6d8135 Disallow gunicorn 25.1.0
The new gunicornc Control Interface breaks workers, see:

https://github.com/benoitc/gunicorn/discussions/3509
https://github.com/galaxyproject/gravity/pull/141
2026-02-23 17:11:50 +00:00
mvdbeek 6f0873ccdc Merge branch 'release_26.0' into dev 2026-02-12 09:16:51 +01:00
mvdbeek d1025f36bc Merge branch 'release_25.1' into release_26.0 2026-02-12 09:14:23 +01:00
Nicola Soranzo a75074e003 Pin setuptools<82 in uv constraint-dependencies 2026-02-11 19:34:11 +00:00
Nicola Soranzo 138b61526a Update dependencies of pydantic-ai 2026-02-06 20:05:44 +00:00
Nicola Soranzo d333b43e44 Improve ruff configuration to mimic our isort settings 2026-01-27 12:43:26 +00:00
Leonid Kostrykin 1d7a944b31 Merge branch 'dev' into feat/dicom 2026-01-20 11:30:52 +01:00
Nicola Soranzo 5aded9505f Merge pull request #21583 from nsoranzo/drop_py3.9
Drop support for Python 3.9
2026-01-20 09:40:39 +00:00
mvdbeek d3bad8ed1f Require Mercurial 7.2rc0 for Python 3.14 support
Mercurial 7.2rc0 is the first release with Python 3.14 support.
Running 7.1.2 on Python 3.14 causes significant performance issues
that lead to test timeouts. Use version markers to specify 7.2rc0
for Python 3.14+ while keeping 7.1.2 for earlier Python versions.
2026-01-20 08:54:52 +01:00
Nicola Soranzo eb9fde1dcd Drop support for Python 3.9 2026-01-19 18:39:16 +00:00
Nicola Soranzo e7f2a9d154 Merge pull request #21608 from nsoranzo/gravity_1.2.0
Update to gravity 1.2.0
2026-01-19 17:47:23 +00:00
Leonid KostrykinandBjörn Grüning 6e0d82a735 Restrict pydicom to Python >=3.10 in pyproject.toml
Co-authored-by: Björn Grüning <bjoern@gruenings.eu>
2026-01-19 17:42:16 +01:00