Commit Graph
1155 Commits
Author SHA1 Message Date
Marius van den Beek 1cf3cb78d1 Merge pull request #23108 from mvdbeek/upgrade-pulsar-client
[26.1] Update pulsar-galaxy-client dependency to 0.15.15
2026-07-13 20:59:06 +02:00
mvdbeek 962d9f7378 Bump minimum pulsar-galaxy-lib version to 0.15.15 2026-07-13 19:03:47 +02:00
mvdbeek 5589ec98e8 Add a mulled biocontainer recommender with the mulled-recommend CLI
A reusable resolver (galaxy.tool_util.deps.mulled.recommend) that picks the best
quay.io/biocontainers image for a set of conda packages, verified against quay.io
rather than guessed:

- recommend_container() resolves single-package and mulled-v2 (multi-package)
  images, distinguishing an exact-version hit from a name-only fallback and
  resolving unpinned versions against tags that are actually built.
- biocontainer_tag_built() verifies whether a specific image tag exists.
- Shared lookup helpers move into mulled/util.py and are reused by the existing
  container resolver (container_resolvers/mulled.py) to avoid duplication.

Exposed as the mulled-recommend console script:

    mulled-recommend samtools=1.17
    mulled-recommend bwa,samtools
    mulled-recommend --json "samtools=1.3.1,bedtools=2.26.0"

Missing containers and transient network errors degrade gracefully (no exception).
Documented under admin special topics (mulled containers).
2026-07-07 16:39:16 +02:00
mvdbeek a3750b0326 Merge branch 'release_26.0' into release_26.1 2026-07-06 15:49:42 +02:00
Marius van den Beek 024d55ff08 Merge pull request #22896 from dannon/chatgxy-agent-retries-config
[26.1] Make GalaxyAI agent retry budget configurable (fix "Exceeded maximum output retries")
2026-06-14 12:28:33 +02:00
mvdbeek 5efb77b18a Tighten pydantic-ai lower bound to >=1.99.0 in galaxy-app package
The package declared pydantic-ai>=0.1.16, so CI could resolve 1.44.0
where Agent._max_output_retries was still named _max_result_retries,
breaking test_configured_retries_wired_into_agent. Pin to >=1.99.0 to
match what requirements.txt already pins.
2026-06-14 08:59:29 +02:00
Marius van den Beek 86b3042679 Merge pull request #22869 from guerler/fixes.009
[26.1] Add fixed color set from webcolors package
2026-06-13 19:36:42 +02:00
guerler 5953703358 Use pydantic_extra_types_color for color input validation 2026-06-11 17:46:01 +03:00
mvdbeek 5f7fc94d6a Bump up gravity to 1.2.3 2026-06-11 10:23:27 +02:00
John Davis ad3ea80e74 Pin galaxy-tool-util-models to current minor version in dependents
galaxy-tool-util-models was listed as an unversioned dependency in
galaxy-schema, galaxy-tool-util, and galaxy-tool-shed-schema. This
caused pip to leave it at a stale version when upgrading an existing
virtualenv, because the already-installed version satisfied the loose
(unconstrained) requirement.

The symptom was ImportError failures at job execution time — e.g.:

  ImportError: cannot import name 'AnySafeValidatorModel' from
    'galaxy.tool_util_models.parameter_validators'

    — where galaxy-util (at the new version) referenced a symbol that
    didn't exist in the older galaxy-tool-util-models still present in
    the venv.

    Fix: add a >=26.0 lower bound in each dependent package so pip is
    forced to upgrade galaxy-tool-util-models when installing any of
    these packages at 26.0 or later.

    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 13:30:34 -04:00
Nicola Soranzo 015ab515a2 Merge branch 'release_26.0' into release_26.1 2026-06-07 23:40:06 +01:00
Nicola Soranzo eec86f0b99 Merge branch 'release_26.0' into release_26.1 2026-06-05 15:08:38 +01:00
John Davis b884af499b Move classifiers and keywords from setup.cfg to pyproject.toml
When a [project] table is present in pyproject.toml, setuptools
ignores fields in setup.cfg unless they are listed in the dynamic
array.  This applies to classifiers and keywords: the build was
silently dropping them from the wheel metadata.

Fix: define classifiers and keywords as static values directly in the
[project] section of each package's pyproject.toml, and remove them from
setup.cfg.

Alternate fix: add both to the dynamic list, keeping the definitions in
setup.cfg. However, I think the first is better since that keeps
pyproject.toml as the single source of truth.
2026-06-05 09:32:41 -04:00
Nicola Soranzo 6b0d47911b Drop console scripts from packages' `setup.cfg` files
Follow-up on https://github.com/galaxyproject/galaxy/pull/22825#issuecomment-4625193313
2026-06-05 12:15:43 +01:00
John Davis 3e9e1ab6f9 Merge branch 'release_26.0' into release_26.1
# Conflicts:
#	client/package.json
#	lib/galaxy/version/__init__.py
#	package.json
#	packages/app/HISTORY.rst
#	packages/app/setup.cfg
#	packages/auth/HISTORY.rst
#	packages/auth/setup.cfg
#	packages/config/HISTORY.rst
#	packages/config/setup.cfg
#	packages/data/HISTORY.rst
#	packages/data/setup.cfg
#	packages/files/HISTORY.rst
#	packages/files/setup.cfg
#	packages/job_execution/HISTORY.rst
#	packages/job_execution/setup.cfg
#	packages/job_metrics/HISTORY.rst
#	packages/job_metrics/setup.cfg
#	packages/meta/HISTORY.rst
#	packages/meta/setup.cfg
#	packages/navigation/HISTORY.rst
#	packages/navigation/setup.cfg
#	packages/objectstore/HISTORY.rst
#	packages/objectstore/setup.cfg
#	packages/schema/HISTORY.rst
#	packages/schema/setup.cfg
#	packages/selenium/HISTORY.rst
#	packages/selenium/setup.cfg
#	packages/test_api/HISTORY.rst
#	packages/test_api/setup.cfg
#	packages/test_base/HISTORY.rst
#	packages/test_base/setup.cfg
#	packages/test_driver/HISTORY.rst
#	packages/test_driver/setup.cfg
#	packages/test_selenium/HISTORY.rst
#	packages/test_selenium/setup.cfg
#	packages/tool_shed/HISTORY.rst
#	packages/tool_shed/setup.cfg
#	packages/tool_shed_schema/HISTORY.rst
#	packages/tool_shed_schema/setup.cfg
#	packages/tool_util/HISTORY.rst
#	packages/tool_util/setup.cfg
#	packages/tool_util_models/HISTORY.rst
#	packages/tool_util_models/setup.cfg
#	packages/tours/HISTORY.rst
#	packages/tours/setup.cfg
#	packages/util/HISTORY.rst
#	packages/util/setup.cfg
#	packages/web_apps/HISTORY.rst
#	packages/web_apps/setup.cfg
#	packages/web_client/HISTORY.rst
#	packages/web_client/setup.cfg
#	packages/web_framework/HISTORY.rst
#	packages/web_framework/setup.cfg
#	packages/web_stack/HISTORY.rst
#	packages/web_stack/setup.cfg
2026-06-04 23:25:03 -04:00
John Davis 09b9b3da46 Start work on 26.0.2.dev0 2026-06-04 23:24:59 -04:00
John Davis 57b0c3040d Create version 26.0.1 2026-06-04 23:24:54 -04:00
John Davis be90bf05cb Fix missing console scripts in built wheels
When [project] is defined in pyproject.toml, setuptools does not include
[options.entry_points] from setup.cfg in the built wheel.

Solution: move console script definitions to [project.scripts] in
pyproject.toml for all affected packages.
2026-06-04 13:49:08 -04:00
mvdbeek aa645069d8 Merge branch 'release_26.0' into release_26.1
# Conflicts:
#	lib/galaxy/dependencies/pinned-requirements.txt
#	lib/galaxy/schema/generics.py
#	lib/galaxy/webapps/galaxy/fast_app.py
#	packages/web_apps/setup.cfg
2026-06-03 15:55:14 +02:00
mvdbeek a8fa956fc1 Merge branch 'release_25.1' into release_26.0 2026-05-29 16:44:48 +02:00
mvdbeekandNicola Soranzo 6b07f5492f [25.1] Backport FastAPI/Starlette upgrade for BadHost (CVE-2026-48710)
Updates the pinned FastAPI/Starlette versions on release_25.1 to match
upstream/dev, closing CVE-2026-48710 ("BadHost"). The vulnerability lets
an attacker inject a path into the HTTP Host header and have
``request.url.path`` reflect that path, bypassing path-based access
control in middleware. Starlette 1.0.1+ rejects Host headers with
invalid characters; we pin starlette==1.1.0 to match dev.

Drops Python 3.9 support, matching upstream/dev (commit eb9fde1dcd).
Starlette dropped 3.9 in 0.50.0, so the BadHost fix (starlette >= 1.0.1)
cannot be installed on 3.9. Bumping the floor to 3.10 is the only way
to ship the security fix; the alternative -- marker-splitting the pins
to keep 3.9 on starlette 0.49.x -- would leave 3.9 users exposed to the
CVE we are trying to close.

Actual exposure on release_25.1 (pre-fix)
-----------------------------------------

Galaxy was running a vulnerable starlette but is **not exploitable for
the headline auth-bypass scenario**. Auth runs through FastAPI
dependencies (``get_user``, ``get_trans``, ``AdminUserRequired``), not
through path-string checks in middleware. ``AccessLoggingMiddleware``
already uses ``scope["path"]``; ``add_galaxy_middleware`` /
``GalaxyCORSMiddleware`` / ``RawContextMiddleware`` /
``SentryAsgiMiddleware`` do not branch on ``request.url.path``.

Lower-severity findings that this upgrade also closes:

- ``lib/galaxy/webapps/base/api.py`` ``get_error_response_for_request``
  picks an error schema with ``"ga4gh"/"drs"/"trs" in
  request.url.path`` -- spoofable Host nudges error payload shape
  (info-disclosure / shape confusion only).
- ``lib/galaxy/webapps/galaxy/api/drs.py`` and
  ``lib/galaxy/webapps/galaxy/services/datasets.py`` derive DRS
  ``service_info`` / ``self_uri`` from ``request.url`` -- spoofable
  Host poisons the advertised DRS identity and client-visible URIs,
  not access.
- ``lib/tool_shed/webapp/api2/tools.py`` -- same shape, TRS service
  info.

Bumping the pin closes the parser flaw at source and downgrades all of
the above to non-issues, so no separate ``request.url.path ->
scope["path"]`` sweep is needed.

Pin bumps (cherry-picks the relevant ranges from PRs #21526, #22206,
#22754):

- fastapi 0.118.0 -> 0.136.3
- starlette 0.48.0 -> 1.1.0
- starlette-context 0.4.0 -> 0.5.1
- python-multipart 0.0.20 -> 0.0.29
- anyio 4.11.0 -> 4.13.0

Required code changes backported from upstream/dev:

- ``lib/galaxy/webapps/openapi/_compat/v2.py``: import
  ``GenerateJsonSchema`` and ``get_flat_models_from_fields`` from
  ``fastapi._compat.v2`` and adopt the new ``get_definitions()``
  implementation for FastAPI 0.128.8+ (PRs #21384, dev commits
  0800c025ce, c8ccc7f44d, b3bfb45884).
- ``lib/galaxy/webapps/openapi/utils.py``: route ``GenerateJsonSchema``
  through ``_compat.v2`` and drop the now-unreachable
  ``get_compat_model_name_map`` fallback (dev commit b3bfb45884).
- ``lib/galaxy/schema/generics.py``: drop ``CustomJsonSchema`` here
  (moved to fast_app.py so it can use the patched ``GenerateJsonSchema``
  from ``_compat.v2``).
- ``lib/galaxy/webapps/galaxy/fast_app.py``: relocate
  ``CustomJsonSchema`` and switch its base to the ``_compat.v2``
  ``GenerateJsonSchema``; replace the @app.middleware("http") X-Frame
  Options handler with a pure ASGI ``XFrameOptionsMiddleware`` class
  (dev commit 67eea395ab) since ``BaseHTTPMiddleware`` semantics
  changed in starlette 1.0.
- ``lib/galaxy/webapps/base/api.py`` and
  ``lib/galaxy/webapps/galaxy/api/datasets.py``: drop the ``method``
  argument of ``FileResponse`` which starlette 1.0 removed (dev commit
  63954cb42e).

Also bumps the minimum FastAPI requirement to ``>=0.133.0`` (first
version compatible with starlette>=1.0.0) and adds an explicit
``starlette>=1.0.1`` floor in ``pyproject.toml`` and the
``packages/web_apps`` / ``packages/tool_shed`` setup.cfg files so
source installs cannot resolve to a vulnerable combo.

Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2026-05-28 18:56:59 +02:00
John ChiltonandClaude Opus 4.7 4233f9f6da Bump gxformat2 to 0.27.0 for UDT support
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 14:11:10 -04:00
Nicola Soranzo 607d2fc1df Disallow gxformat2 0.25.0 and 0.26.0
due to https://github.com/galaxyproject/gxformat2/issues/204 .
2026-05-10 16:57:22 +01:00
mvdbeek ce884dc3da Elect single history-audit monitor; add standalone daemon
Every webapp process was running its own HistoryAuditMonitor (and the
per-process Postgres LISTEN that comes with it). Move to a single elected
producer via a new is_history_audit_monitor role on DatabaseHeartbeat.
The election prefers any WorkerProcess with app_type=sse_monitor, falling
back to the max-server_name webapp when no dedicated daemon is running,
so the monitor migrates cleanly when the leader dies.

Add a standalone galaxy-sse-monitor process (new lib/galaxy/sse_monitor
package, galaxy-sse-monitor console script) that builds a minimal
GalaxyManagerApplication, registers itself as sse_monitor.<host>.<pid>,
and blocks on SIGTERM. Running this daemon moves SSE event production
out of the webapp so its dispatch can't stall behind a webapp's GIL.

Gate HistoryAuditMonitor.start/stop on the role callback rather than a
postfork hook; make start() restart-safe (clear _exit) and shutdown()
idempotent so role transitions can cycle the monitor.
2026-04-28 17:20:22 +02:00
Nicola Soranzo c40e09084e Merge branch 'release_26.0' into dev 2026-04-22 03:54:23 +01:00
Nicola Soranzo b7a51d75ba Add missing dependency on starlette
Introduced in commit 9629f2f6fa .
2026-04-21 16:48:52 +01:00
Nicola Soranzo e0681411fa Pin rocrate<0.15.0 until roc-validator supports RO-Crate Metadata Specification 1.2
rocrate 0.15.0 produces crates conforming to 1.2 spec by default, which
causes roc-validator validation errors like:

```
The RO-Crate metadata file descriptor MUST have a `conformsTo` property with the RO-Crate specification version
```

xref https://github.com/crs4/rocrate-validator/issues/107

Also:
- Remove unused `BaseWorkflowPopulator.validate_invocation_crate_directory()` method.
2026-04-21 16:45:52 +01:00
mvdbeek 8e5801c95f Add hvac to galaxy-data test extras
The packaged galaxy-data test suite imports `hvac` (via the vault
tests), so installing the `test` extra needs to pull it in for the
package tests to pass in CI.
2026-04-21 11:24:21 +02:00
Dannon Baker aab15ab70a Fix stale client_build_hash.txt paths in web_client Makefile
The src/ layout conversion in b7f83c8469 updated where the recipe
writes client_build_hash.txt but left the _dist prereq and the hash
target declared at the old galaxy/web_client/ path.  Make never sees
the hash file as "made", so `make _dist` re-runs the full
client-production build every time, even when src/galaxy/web_client/
is already populated.

Verified via `make -n _dist`: with the fix, an existing hash file
correctly short-circuits the rebuild and only runs `uv build`.
2026-04-17 13:03:52 -04:00
Nicola Soranzo b4436f45a1 Simplify mypy testing for packages 2026-04-16 15:48:34 +01:00
Nicola Soranzo 61bec48cdb Ignore and clean up `.egg-info dirs in src/` subdir of packages 2026-04-16 15:46:38 +01:00
Nicola Soranzo b888927ff3 Add README.rst for galaxy-tool-shed-schema package 2026-04-16 09:56:01 +01:00
Michael R. Crusoe 9d34f9659f convert galaxy_test to a pure namespace package 2026-04-15 15:27:31 +02:00
Michael R. Crusoe b7f83c8469 python packages: convert to "src" layout
As suggested at https://packaging.python.org/en/latest/guides/packaging-namespace-packages/#native-namespace-packages

Use setuptools-scm to automatically adds all the SCM-managed files to the sdist
instead of manually listing them in packages/*/MANIFEST.in
2026-04-15 15:16:36 +02:00
Michael R. Crusoe dbfcdb8bc9 python packages: convert to pure namespace packages
only directories in `lib/galaxy` & `packages/*/galaxy`
All other files removed or moved to their own module
All references to those files updated, especially galaxy/version.py
2026-04-15 15:16:34 +02:00
Nicola Soranzo 35dcde758c Pin rocrate<0.15.0 until roc-validator supports RO-Crate Metadata Specification 1.2
rocrate 0.15.0 produces crates conforming to 1.2 spec by default, which
causes roc-validator validation errors like:

```
The RO-Crate metadata file descriptor MUST have a `conformsTo` property with the RO-Crate specification version
```

xref https://github.com/crs4/rocrate-validator/issues/107

Also:
- Remove unused `BaseWorkflowPopulator.validate_invocation_crate_directory()` method.
2026-04-13 17:21:08 +01:00
Nicola Soranzo f16c986e70 Move various requirements to test extra 2026-04-13 17:15:45 +01:00
Nate Coraor 99af84d1bd Merge remote-tracking branch 'upstream/release_26.0' into dev 2026-04-13 11:34:46 -04:00
Nicola Soranzo 59b0074b16 Fix missing package dependencies 2026-04-12 22:44:45 +01:00
Nicola Soranzo 1e9758cd1a Use a separate virtualenv to test each package 2026-04-12 19:10:55 +01:00
Nicola Soranzo c160194ed9 Test and publish galaxy-test-selenium package 2026-04-12 19:10:55 +01:00
Nicola Soranzo 349a5daec1 Merge branch 'release_26.0' into dev 2026-04-10 18:08:40 +01:00
guerler 4211127ac2 Bump dev cycle from 26.0.1.dev0 to 26.0.1.dev1
The 26.0.1.dev0 PyPI packages were already published during the
earlier broken release attempt, so the version string is permanently
burned: PyPI does not allow re-uploading the same version, even for
pre-releases.  Bump the dev counter to .dev1 so the release branch
ends at a version that has not been published yet and future dev
builds from this branch can be uploaded.

This commit only changes the four dev0 version strings introduced by
the previous commit:
  - lib/galaxy/version.py:        VERSION_MINOR = "1.dev0" -> "1.dev1"
  - client/package.json:          "26.0.1.dev0" -> "26.0.1.dev1"
  - packages/*/setup.cfg:         version = 26.0.1.dev0 -> 26.0.1.dev1
  - packages/*/HISTORY.rst:       26.0.1.dev0 header -> 26.0.1.dev1

The pulsar-galaxy-lib>=0.15.0.dev0 dependency pin in
packages/app/setup.cfg and the historical 22.9.0.dev0 changelog
entries in packages/navigation/HISTORY.rst and packages/tours/HISTORY.rst
are deliberately left untouched — they refer to unrelated versions.
2026-04-09 23:59:50 +03:00
guerler 08f930027b Re-apply version bump to 26.0.1.dev0 after v26.0.0 retag
Companion to the previous commit.  After v26.0.0 is retagged onto the
restoration commit and the PyPI publish workflow produces the correct
26.0.0 artifacts, this commit returns the release branch to
26.0.1.dev0 so future development on the 26.0 line continues from a
post-release dev version, matching the state established by
f4478f5d09 ("Start work on 26.0.1.dev0").

This is a fresh commit with the same content as f4478f5d09; it is not
a cherry-pick and does not reuse the original hash or author metadata.

After the v26.0.0 tag has been moved and a successful publish is
verified, the branch HEAD is once again at 26.0.1.dev0 just like
before this two-commit sequence — but now the v26.0.0 tag points at a
commit that actually contains version = 26.0.0.
2026-04-09 23:41:25 +03:00
guerler 854899453b Restore version to 26.0.0 for v26.0.0 retag
The original v26.0.0 tag was retagged from 52e970855b to HEAD because
of bugs in the GitHub Actions release workflow.  Unfortunately HEAD
already contained the post-release version bump from f4478f5d09
("Start work on 26.0.1.dev0"), so PyPI builds from the retagged HEAD
publish wheels labeled 26.0.1.dev0 instead of 26.0.0.

This commit restores the version files to their 26.0.0 state so the
v26.0.0 tag can be moved to point at this commit and the publish
workflow can produce correctly labeled artifacts.

The next commit re-applies the dev0 bump so the release branch ends
back at 26.0.1.dev0 and future development continues normally.

This is purely additive — no history is rewritten and the existing
26.0.1.dev0 commit (f4478f5d09) is left untouched.
2026-04-09 23:40:45 +03:00
Nicola Soranzo 979759398e Merge pull request #22412 from ahmedhamidawan/add_twine_to_package_ci_tests
Add twine check to package test script to lint RST long descriptions
2026-04-09 14:17:42 +01:00
Nicola Soranzo 27c0617202 Merge branch 'release_26.0' into dev 2026-04-08 17:08:26 +01:00
Nicola Soranzo 07d9e5d474 Remove release-test-artifacts and release-artifacts Makefile targets
Superseded by Trusted Publishing via CI.
2026-04-08 14:49:26 +01:00
Nicola Soranzo afaa2515fe Pin twine version
Also:
- Drop unused `sphinx` from `package-dev-requirements.txt`
- Sync `client/.node_version` with `nodejs-wheel` version in
  `lib/galaxy/dependencies/pinned-requirements.txt`
- Sync `packages/web_client/Makefile` with `packages/package.Makefile`
- Rewrite the Makefile target to buid the client in `packages/web_client/Makefile`
2026-04-08 14:47:07 +01:00
Nicola Soranzo 94c9f6f1b2 Don't run `twine check` for test_galaxy_packages_for_pulsar
Also:
- Run twine via uvx when using uv
2026-04-08 03:18:41 +01:00