paperbenni
bec3d3ad9c
Drop Python 3.8 support in 5 Pulsar-compatible packages (job_metrics, tool_util, tool_util_models, util, objectstore) and run pyupgrade --py310-plus on their source code. - Bump requires-python from >=3.8 to >=3.10 in all 5 packages - Remove Python 3.8 and 3.9 classifiers - Remove ruff per-file-ignores for UP rules on these paths - Remove backports.zoneinfo conditional dependency - Remove pydyf<0.11 pin from conditional-requirements.txt - Update Makefile pyupgrade target (remove PY38_PYUPGRADE_PATHS) - Update CI workflow to test with Python 3.10 instead of 3.8 Clean up unused deprecated typing imports after pyupgrade Remove now-unused typing imports (Dict, List, Optional, Set, Tuple, Type, Union) that became dead after pyupgrade --py310-plus converted annotations to use built-in types and | syntax. Also run ruff check --fix --select=UP007,UP045 across the entire codebase to convert remaining Optional[X] -> X | None and Union[X, Y] -> X | Y patterns. Enable ruff UP007/UP045 for Python 3.10 union syntax Remove UP007 (Union[X,Y] -> X | Y) and UP045 (Optional[X] -> X | None) from the ruff ignore list and convert all type aliases across the codebase. These rules were deferred while Python 3.9 was supported; requires-python is now >=3.10. A custom script was used because neither ruff --fix nor pyupgrade --py310-plus converts Optional[X]/Union[X,Y] in type alias positions (e.g. X = Union[A, B]) — they only handle annotation positions (e.g. def f(x: Optional[int])). All 167 violations were module-level type aliases. A few edge cases were fixed manually: single-element Union[X,], typing.Union qualified refs, runtime Optional[type] calls, and Annotated[Optional[...]] pydantic fields. Fix UP007 autofix regression with string forward reference type aliases Commit fa6bd955a0 enabled ruff UP007/UP045 and auto-fixed module-level type aliases using Union with string forward references, producing invalid 'str | str' expressions. This was a known ruff bug (charliermarsh/ruff#826) that has since been fixed in later ruff versions, but this codebase was converted before the fix was in place. Revert to Union syntax and restore TYPE_CHECKING imports that ruff's TCH rule cleaned up as a side effect when it thought the forward references were unused.
Package-centric Galaxy
You clone Galaxy as a Monorepo for its Python backend. This packages directory allows us to distribute smaller connected repositories out of individual Galaxy components. CI ensures these repositories have valid dependencies and such defined by running each components tests in isolation.
Development
Typically Galaxy developers develop against the monorepo but there are times when developing against a small unit of Galaxy might make sense - such as to keep an AI (or human cognitive) context as small as possible.
This can be done by treating Galaxy as a uv workspace. The pyproject.toml file in this directory defines a workspace for Galaxy and allows development and testing of individual packages with uv.
Here is an example workflow that will run tests for a specific package after setting up a virtual environment for that package.
$ git clone https://github.com/galaxyproject/galaxy.git
$ cd galaxy/packages
$ cd auth
$ uv sync --all-extras
$ uv run pytest tests
When switching between projects it is just important to remember to rerun the sync command for that project.
The package Makefile has been updated to use ux also, so the above can
be simplified to just:
$ git clone https://github.com/galaxyproject/galaxy.git
$ cd galaxy/packages
$ cd auth
$ make test
If the environment has already been setup and you wish to just re-run
the tests. The make _test command be used to just do the testing in
the existing environment without messing with sync and adjusting
dependencies.
The Makefile can also be used for typechecking.
$ git clone https://github.com/galaxyproject/galaxy.git
$ cd galaxy/packages
$ cd auth
$ make mypy
$ make _mypy # a shortcut to just run mypy on an existing updated environment
This is equivalent to syncing the uv environment and then adding Galaxy's type checking dependencies to it.
$ uv sync --all-extras
$ uv pip install -r ../../lib/galaxy/dependencies/pinned-typecheck-requirements.txt
$ uv run mypy .
The Makefile can also be used for linting.
$ git clone https://github.com/galaxyproject/galaxy.git
$ cd galaxy/packages
$ cd auth
$ make lint
$ make _lint # a shortcut to just run ruff on an existing updated environment
This is equivalent to syncing the uv environment and then adding Galaxy's linting dependencies to it.
$ uv sync --all-extras
$ uv pip install -r ../../lib/galaxy/dependencies/pinned-lint-requirements.txt
$ uv run ruff check .