The package is app-coupled (galaxy.model, galaxy.config, galaxy.queues,
galaxy.datatypes) and only consumed from app-side code, so it belongs
under galaxy.tools rather than a new top-level package. Riding the
existing packages/app tools symlink also removes the need for a
dedicated package symlink.
Claude-Session: https://claude.ai/code/session_018L7ZmCv2ubKA3JNeSL8Pkr
The packages/app tree links each galaxy subpackage explicitly and
tool_source_store was never added, so the built wheel lacked the package
and per-package mypy resolved its imports as Any (surfacing as
'Returning Any' in LazyToolboxSearch.search).
The package is app-coupled (galaxy.model, galaxy.config, galaxy.queues,
galaxy.datatypes) and only consumed from app-side code, so it belongs
under galaxy.tools rather than a new top-level package. Riding the
existing packages/app tools symlink also removes the need for a
dedicated package symlink. Also drop an unused back-compat alias and
fix stale doc references.
Claude-Session: https://claude.ai/code/session_018L7ZmCv2ubKA3JNeSL8Pkr
Introduces galaxy.tool_source_store: a queryable store of parsed,
macro-expanded tool sources plus a pre-computed ToolIndex of the metadata
batch consumers need (panel section, labels, EDAM, requirements, test
counts, shed provenance).
- Backends: database (store-owned tool_source_record + tool_index tables,
migration f5a73c8b9d12), sqlalchemy (read-only SQLite bundles, e.g.
CVMFS-shipped), and a composite that layers named stores over the
default with read fall-through.
- scripts/tool_source/populate_store.py walks tool_conf files and writes
sources + index + whoosh search index; --watch updates the store on
file changes and broadcasts reload_tool_source_cache. The populator is
the single writer; consumers are read-only.
- Per-conf opt-in via store="name" on a toolbox conf root, resolved
against the tool_source_stores catalog in galaxy.yml.
- The tool_source_record table is deliberately separate from tool_source,
whose rows belong to the job-request path and carry a raw-string
payload contract.
Nothing reads from the store yet — an on-demand-loading toolbox consuming
it is follow-up work (#22633). Extracted from that PR.
in preparation for pydantic-ai 2.0.0, which replaced the
dependency on `fastmcp` with `fastmcp-slim[client]`.
Otherwise Galaxy fails to start with:
```
Traceback (most recent call last):
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/server/__init__.py", line 6, in <module>
from .context import Context
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/server/context.py", line 28, in <module>
from uncalled_for import SharedContext
ModuleNotFoundError: No module named 'uncalled_for'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/__init__.py", line 57, in __getattr__
from fastmcp.server.context import Context
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/server/__init__.py", line 9, in <module>
raise ImportError(_install_hints.SERVER_SUPPORT) from exc
ImportError: FastMCP server support is not installed. Install `fastmcp` or `fastmcp-slim[server]`.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/bin/gunicorn", line 10, in <module>
sys.exit(run())
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 66, in run
WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]", prog=prog).run()
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 235, in run
super().run()
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 71, in run
Arbiter(self).run()
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/arbiter.py", line 63, in __init__
self.setup(app)
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/arbiter.py", line 139, in setup
self.app.wsgi()
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 66, in wsgi
self.callable = self.load()
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 57, in load
return self.load_wsgiapp()
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 47, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/gunicorn/util.py", line 464, in import_app
app = app(*args, **kwargs)
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/fast_factory.py", line 63, in factory
gx_wsgi_webapp, gx_app = app_pair(
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 178, in app_pair
populate_api_routes(webapp, app)
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 362, in populate_api_routes
webapp.add_api_controllers("galaxy.webapps.galaxy.api", app)
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/base/webapp.py", line 252, in add_api_controllers
for name, module in base.walk_controller_modules(package_name):
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/web/framework/base.py", line 624, in walk_controller_modules
module = import_module(module_name)
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/api/mcp.py", line 15, in <module>
from fastmcp import (
File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.10/site-packages/fastmcp/__init__.py", line 59, in __getattr__
raise ImportError(_install_hints.SERVER_SUPPORT) from exc
ImportError: FastMCP server support is not installed. Install `fastmcp` or `fastmcp-slim[server]`.
```
A reusable resolver (galaxy.tool_util.deps.mulled.recommend) that picks the best
quay.io/biocontainers image for a set of conda packages, verified against quay.io
rather than guessed:
- recommend_container() resolves single-package and mulled-v2 (multi-package)
images, distinguishing an exact-version hit from a name-only fallback and
resolving unpinned versions against tags that are actually built.
- biocontainer_tag_built() verifies whether a specific image tag exists.
- Shared lookup helpers move into mulled/util.py and are reused by the existing
container resolver (container_resolvers/mulled.py) to avoid duplication.
Exposed as the mulled-recommend console script:
mulled-recommend samtools=1.17
mulled-recommend bwa,samtools
mulled-recommend --json "samtools=1.3.1,bedtools=2.26.0"
Missing containers and transient network errors degrade gracefully (no exception).
Documented under admin special topics (mulled containers).
Drop Python 3.8 support in 5 Pulsar-compatible packages
(job_metrics, tool_util, tool_util_models, util, objectstore)
and run pyupgrade --py310-plus on their source code.
- Bump requires-python from >=3.8 to >=3.10 in all 5 packages
- Remove Python 3.8 and 3.9 classifiers
- Remove ruff per-file-ignores for UP rules on these paths
- Remove backports.zoneinfo conditional dependency
- Remove pydyf<0.11 pin from conditional-requirements.txt
- Update Makefile pyupgrade target (remove PY38_PYUPGRADE_PATHS)
- Update CI workflow to test with Python 3.10 instead of 3.8
Clean up unused deprecated typing imports after pyupgrade
Remove now-unused typing imports (Dict, List, Optional, Set, Tuple,
Type, Union) that became dead after pyupgrade --py310-plus converted
annotations to use built-in types and | syntax.
Also run ruff check --fix --select=UP007,UP045 across the entire
codebase to convert remaining Optional[X] -> X | None and
Union[X, Y] -> X | Y patterns.
Enable ruff UP007/UP045 for Python 3.10 union syntax
Remove UP007 (Union[X,Y] -> X | Y) and UP045 (Optional[X] -> X | None)
from the ruff ignore list and convert all type aliases across the
codebase. These rules were deferred while Python 3.9 was supported;
requires-python is now >=3.10.
A custom script was used because neither ruff --fix nor
pyupgrade --py310-plus converts Optional[X]/Union[X,Y] in type alias
positions (e.g. X = Union[A, B]) — they only handle annotation
positions (e.g. def f(x: Optional[int])). All 167 violations were
module-level type aliases. A few edge cases were fixed manually:
single-element Union[X,], typing.Union qualified refs, runtime
Optional[type] calls, and Annotated[Optional[...]] pydantic fields.
Fix UP007 autofix regression with string forward reference type aliases
Commit fa6bd955a0 enabled ruff UP007/UP045 and auto-fixed
module-level type aliases using Union with string forward
references, producing invalid 'str | str' expressions.
This was a known ruff bug (charliermarsh/ruff#826) that has since
been fixed in later ruff versions, but this codebase was
converted before the fix was in place.
Revert to Union syntax and restore TYPE_CHECKING imports that
ruff's TCH rule cleaned up as a side effect when it thought the
forward references were unused.
The package declared pydantic-ai>=0.1.16, so CI could resolve 1.44.0
where Agent._max_output_retries was still named _max_result_retries,
breaking test_configured_retries_wired_into_agent. Pin to >=1.99.0 to
match what requirements.txt already pins.
galaxy-tool-util-models was listed as an unversioned dependency in
galaxy-schema, galaxy-tool-util, and galaxy-tool-shed-schema. This
caused pip to leave it at a stale version when upgrading an existing
virtualenv, because the already-installed version satisfied the loose
(unconstrained) requirement.
The symptom was ImportError failures at job execution time — e.g.:
ImportError: cannot import name 'AnySafeValidatorModel' from
'galaxy.tool_util_models.parameter_validators'
— where galaxy-util (at the new version) referenced a symbol that
didn't exist in the older galaxy-tool-util-models still present in
the venv.
Fix: add a >=26.0 lower bound in each dependent package so pip is
forced to upgrade galaxy-tool-util-models when installing any of
these packages at 26.0 or later.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a [project] table is present in pyproject.toml, setuptools
ignores fields in setup.cfg unless they are listed in the dynamic
array. This applies to classifiers and keywords: the build was
silently dropping them from the wheel metadata.
Fix: define classifiers and keywords as static values directly in the
[project] section of each package's pyproject.toml, and remove them from
setup.cfg.
Alternate fix: add both to the dynamic list, keeping the definitions in
setup.cfg. However, I think the first is better since that keeps
pyproject.toml as the single source of truth.
When [project] is defined in pyproject.toml, setuptools does not include
[options.entry_points] from setup.cfg in the built wheel.
Solution: move console script definitions to [project.scripts] in
pyproject.toml for all affected packages.
Updates the pinned FastAPI/Starlette versions on release_25.1 to match
upstream/dev, closing CVE-2026-48710 ("BadHost"). The vulnerability lets
an attacker inject a path into the HTTP Host header and have
``request.url.path`` reflect that path, bypassing path-based access
control in middleware. Starlette 1.0.1+ rejects Host headers with
invalid characters; we pin starlette==1.1.0 to match dev.
Drops Python 3.9 support, matching upstream/dev (commit eb9fde1dcd).
Starlette dropped 3.9 in 0.50.0, so the BadHost fix (starlette >= 1.0.1)
cannot be installed on 3.9. Bumping the floor to 3.10 is the only way
to ship the security fix; the alternative -- marker-splitting the pins
to keep 3.9 on starlette 0.49.x -- would leave 3.9 users exposed to the
CVE we are trying to close.
Actual exposure on release_25.1 (pre-fix)
-----------------------------------------
Galaxy was running a vulnerable starlette but is **not exploitable for
the headline auth-bypass scenario**. Auth runs through FastAPI
dependencies (``get_user``, ``get_trans``, ``AdminUserRequired``), not
through path-string checks in middleware. ``AccessLoggingMiddleware``
already uses ``scope["path"]``; ``add_galaxy_middleware`` /
``GalaxyCORSMiddleware`` / ``RawContextMiddleware`` /
``SentryAsgiMiddleware`` do not branch on ``request.url.path``.
Lower-severity findings that this upgrade also closes:
- ``lib/galaxy/webapps/base/api.py`` ``get_error_response_for_request``
picks an error schema with ``"ga4gh"/"drs"/"trs" in
request.url.path`` -- spoofable Host nudges error payload shape
(info-disclosure / shape confusion only).
- ``lib/galaxy/webapps/galaxy/api/drs.py`` and
``lib/galaxy/webapps/galaxy/services/datasets.py`` derive DRS
``service_info`` / ``self_uri`` from ``request.url`` -- spoofable
Host poisons the advertised DRS identity and client-visible URIs,
not access.
- ``lib/tool_shed/webapp/api2/tools.py`` -- same shape, TRS service
info.
Bumping the pin closes the parser flaw at source and downgrades all of
the above to non-issues, so no separate ``request.url.path ->
scope["path"]`` sweep is needed.
Pin bumps (cherry-picks the relevant ranges from PRs #21526, #22206,
#22754):
- fastapi 0.118.0 -> 0.136.3
- starlette 0.48.0 -> 1.1.0
- starlette-context 0.4.0 -> 0.5.1
- python-multipart 0.0.20 -> 0.0.29
- anyio 4.11.0 -> 4.13.0
Required code changes backported from upstream/dev:
- ``lib/galaxy/webapps/openapi/_compat/v2.py``: import
``GenerateJsonSchema`` and ``get_flat_models_from_fields`` from
``fastapi._compat.v2`` and adopt the new ``get_definitions()``
implementation for FastAPI 0.128.8+ (PRs #21384, dev commits
0800c025ce, c8ccc7f44d, b3bfb45884).
- ``lib/galaxy/webapps/openapi/utils.py``: route ``GenerateJsonSchema``
through ``_compat.v2`` and drop the now-unreachable
``get_compat_model_name_map`` fallback (dev commit b3bfb45884).
- ``lib/galaxy/schema/generics.py``: drop ``CustomJsonSchema`` here
(moved to fast_app.py so it can use the patched ``GenerateJsonSchema``
from ``_compat.v2``).
- ``lib/galaxy/webapps/galaxy/fast_app.py``: relocate
``CustomJsonSchema`` and switch its base to the ``_compat.v2``
``GenerateJsonSchema``; replace the @app.middleware("http") X-Frame
Options handler with a pure ASGI ``XFrameOptionsMiddleware`` class
(dev commit 67eea395ab) since ``BaseHTTPMiddleware`` semantics
changed in starlette 1.0.
- ``lib/galaxy/webapps/base/api.py`` and
``lib/galaxy/webapps/galaxy/api/datasets.py``: drop the ``method``
argument of ``FileResponse`` which starlette 1.0 removed (dev commit
63954cb42e).
Also bumps the minimum FastAPI requirement to ``>=0.133.0`` (first
version compatible with starlette>=1.0.0) and adds an explicit
``starlette>=1.0.1`` floor in ``pyproject.toml`` and the
``packages/web_apps`` / ``packages/tool_shed`` setup.cfg files so
source installs cannot resolve to a vulnerable combo.
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
Every webapp process was running its own HistoryAuditMonitor (and the
per-process Postgres LISTEN that comes with it). Move to a single elected
producer via a new is_history_audit_monitor role on DatabaseHeartbeat.
The election prefers any WorkerProcess with app_type=sse_monitor, falling
back to the max-server_name webapp when no dedicated daemon is running,
so the monitor migrates cleanly when the leader dies.
Add a standalone galaxy-sse-monitor process (new lib/galaxy/sse_monitor
package, galaxy-sse-monitor console script) that builds a minimal
GalaxyManagerApplication, registers itself as sse_monitor.<host>.<pid>,
and blocks on SIGTERM. Running this daemon moves SSE event production
out of the webapp so its dispatch can't stall behind a webapp's GIL.
Gate HistoryAuditMonitor.start/stop on the role callback rather than a
postfork hook; make start() restart-safe (clear _exit) and shutdown()
idempotent so role transitions can cycle the monitor.
rocrate 0.15.0 produces crates conforming to 1.2 spec by default, which
causes roc-validator validation errors like:
```
The RO-Crate metadata file descriptor MUST have a `conformsTo` property with the RO-Crate specification version
```
xref https://github.com/crs4/rocrate-validator/issues/107
Also:
- Remove unused `BaseWorkflowPopulator.validate_invocation_crate_directory()` method.
The packaged galaxy-data test suite imports `hvac` (via the vault
tests), so installing the `test` extra needs to pull it in for the
package tests to pass in CI.
The src/ layout conversion in b7f83c8469 updated where the recipe
writes client_build_hash.txt but left the _dist prereq and the hash
target declared at the old galaxy/web_client/ path. Make never sees
the hash file as "made", so `make _dist` re-runs the full
client-production build every time, even when src/galaxy/web_client/
is already populated.
Verified via `make -n _dist`: with the fix, an existing hash file
correctly short-circuits the rebuild and only runs `uv build`.
mvdbeek