The docs job takes ~83m on a pull request, ~82m of which is a single serial sphinx-build. Making `-j auto` the default in doc/Makefile speeds up every caller — CI, `make docs`, and a local build — and `SPHINXOPTS=` still gives a serial build. `docs-develop` no longer needs to pass `-j 4` of its own.
sphinx.ext.viewcode is not parallelized (see 45ed317832, which added GALAXY_DOCS_SKIP_VIEW_CODE for exactly that reason), so it is skipped for pull request builds, which only need to prove the documentation still builds. Pushes, which are what gets deployed to docs.galaxyproject.org, keep the embedded highlighted source.
Measured on a pull request run: 83m to 15m.
Flip the relationship between directives.yml and the Python validator: the YAML
is now authoritative and scripts/markdown_directives_doc.py generates everything
downstream of it:
- lib/galaxy/managers/_markdown_directives.py - generated VALID_ARGUMENTS,
EMBED_CAPABLE_DIRECTIVES, SHARED_ARGUMENTS, DynamicArguments; markdown_parse
re-exports these, staying self-contained (no YAML at runtime) so it remains
reusable outside Galaxy (e.g. gxformat2).
- client .../Utilities/requirements.yml - generated from each directive's
'requires', removing the hand-maintained duplicate.
- directives.md - unchanged generator, now fed by the same source.
Add _shared_arguments to directives.yml (homes 'collapse' as data). Replace the
now-tautological yml-vs-parser cross-check with internal yml validation plus a
dispatch-coverage check against markdown_util.py (the gap that hid the dead
instance_citation_bibtex directive). Guard all three generated artifacts in
test_markdown_directives_doc.py and via make client-gen-markdown-directives.
Emit the directives.md tables pre-formatted to match prettier (column padding),
so the generated reference is prettier-stable and needs no ignore entry; drop the
ineffective client/.prettierignore line for it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enrich directives.yml into the source of truth for directive docs: per-param
type/context/default/description (via shared _parameter_sets), embeddable,
requires, category, renders; add missing instance_*_link and visualization
entries.
Add scripts/markdown_directives_doc.py to render directives.md and cross-check
the yml against markdown_parse.py (VALID_ARGUMENTS, EMBED_CAPABLE_DIRECTIVES,
SHARED_ARGUMENTS) + requirements.yml. Guard via test_markdown_directives_doc.py
and `make client-gen-markdown-directives`, mirroring config-rebuild. directives.md
is generated, so prettier-ignore it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drop Python 3.8 support in 5 Pulsar-compatible packages
(job_metrics, tool_util, tool_util_models, util, objectstore)
and run pyupgrade --py310-plus on their source code.
- Bump requires-python from >=3.8 to >=3.10 in all 5 packages
- Remove Python 3.8 and 3.9 classifiers
- Remove ruff per-file-ignores for UP rules on these paths
- Remove backports.zoneinfo conditional dependency
- Remove pydyf<0.11 pin from conditional-requirements.txt
- Update Makefile pyupgrade target (remove PY38_PYUPGRADE_PATHS)
- Update CI workflow to test with Python 3.10 instead of 3.8
Clean up unused deprecated typing imports after pyupgrade
Remove now-unused typing imports (Dict, List, Optional, Set, Tuple,
Type, Union) that became dead after pyupgrade --py310-plus converted
annotations to use built-in types and | syntax.
Also run ruff check --fix --select=UP007,UP045 across the entire
codebase to convert remaining Optional[X] -> X | None and
Union[X, Y] -> X | Y patterns.
Enable ruff UP007/UP045 for Python 3.10 union syntax
Remove UP007 (Union[X,Y] -> X | Y) and UP045 (Optional[X] -> X | None)
from the ruff ignore list and convert all type aliases across the
codebase. These rules were deferred while Python 3.9 was supported;
requires-python is now >=3.10.
A custom script was used because neither ruff --fix nor
pyupgrade --py310-plus converts Optional[X]/Union[X,Y] in type alias
positions (e.g. X = Union[A, B]) — they only handle annotation
positions (e.g. def f(x: Optional[int])). All 167 violations were
module-level type aliases. A few edge cases were fixed manually:
single-element Union[X,], typing.Union qualified refs, runtime
Optional[type] calls, and Annotated[Optional[...]] pydantic fields.
Fix UP007 autofix regression with string forward reference type aliases
Commit fa6bd955a0 enabled ruff UP007/UP045 and auto-fixed
module-level type aliases using Union with string forward
references, producing invalid 'str | str' expressions.
This was a known ruff bug (charliermarsh/ruff#826) that has since
been fixed in later ruff versions, but this codebase was
converted before the fix was in place.
Revert to Union syntax and restore TYPE_CHECKING imports that
ruff's TCH rule cleaned up as a side effect when it thought the
forward references were unused.
config_schema.yml is the source of truth for all 460 config attributes, but
only ~50 had explicit Python type annotations in GalaxyAppConfiguration. The
remaining attributes were typed as Any via HasDynamicProperties.__getattr__,
requiring consumers to use # type: ignore or defensive getattr() patterns.
Add a `build_config_types` action to config_manage.py that reads the schema and
emits a *AppConfigurationAttributes mixin class with type annotations for every
schema-defined attribute. Schema types map to Python types (str/bool/int/float
→ direct, any → Any, seq → list[Any]). Attributes with a null schema default
get `T | None`. A per-app override dict encodes cases where post-processing in
_process_config changes the runtime type (listified filters, float cache size,
optional interactivetools_map, etc.).
GalaxyAppConfiguration and ToolShedAppConfiguration now inherit from their
respective generated mixin, making all schema attributes statically typed.
Manual overrides in GalaxyAppConfiguration continue to refine specific
attributes (HashFunctionNameEnum, timedelta, themes dicts).
Wire the new action into the Makefile config-rebuild target so generated files
stay in sync when the schema changes.
Also fix two newly-exposed type errors in the codebase:
- galaxy/dependencies/__init__.py: guard None before passing file_source_templates_config_file to exists()
- galaxy/config/__init__.py: cast use_remote_user or single_user to bool
With the workspace in place, flip the client<->api-client relationship.
The OpenAPI schema becomes canonical in the api-client package, and the
main client re-exports its types from @galaxyproject/galaxy-api-client
as a workspace dependency. The old symlink from api-client back into
client/src/api goes away -- the library no longer reaches into the app.
Changes:
- Move client/src/api/schema/schema.ts (48K lines) into
client/packages/api-client/src/schema/schema.ts as the canonical copy.
- Add client/packages/api-client/src/schema/index.ts matching the old
barrel.
- Update api-client's api-types.ts to import from the new local path;
remove the no-longer-needed client/packages/api-client/src/api symlink.
- Replace client/src/api/schema/index.ts with a re-export from
@galaxyproject/galaxy-api-client.
- Add @galaxyproject/galaxy-api-client as a workspace:* dep in client.
- Wire a postinstall hook in client/package.json to build api-client on
pnpm install so its .d.ts is available to vue-tsc.
- Update make update-client-api-schema to generate into the new canonical
location.
- Fix the one file still importing from @/api/schema/schema (the index
barrel re-exports components already).
This PR now demonstrates the workspace end-to-end: a real library
package, consumed by the app, building through the workspace toolchain.
Further extractions (galaxy-utils, galaxy-ui, galaxy-composables) follow
in subsequent PRs using the same pattern.
The install-client target now pulls galaxy-web-client from PyPI at the
version matching the current Galaxy checkout, instead of invoking pnpm
at the repo root. Drops the now-orphaned node-deps target that only
existed to support the old root-level pnpm install.
only directories in `lib/galaxy` & `packages/*/galaxy`
All other files removed or moved to their own module
All references to those files updated, especially galaxy/version.py
Implement full Workflow Execution Service (WES) API support for running Galaxy
workflows via the GA4GH standard. Includes complete core functionality with run
submission, status tracking, cancellation, input/output handling, and task logs.
Core WES Features:
- 6 WES API endpoints (service-info, runs CRUD, cancel, status)
- Support for both workflow_url and workflow_attachment input methods
- Support for gxworkflow:// URI scheme for direct database workflow references
- Automatic history creation with optional custom naming
- Full workflow support: gx_workflow_ga and gx_workflow_format2
- DRS URI generation for all workflow outputs
- State mapping between Galaxy invocation states and WES states
- Cursor-based pagination for run listings
Task Log Features:
- /api/jobs/{job_id}/stdout - Job stdout as plain text
- /api/jobs/{job_id}/stderr - Job stderr as plain text
- /ga4gh/wes/v1/runs/{run_id}/tasks - Paginated task list
- /ga4gh/wes/v1/runs/{run_id}/tasks/{task_id} - Task details
- Build TaskLogs from workflow invocation steps
- Populate RunLog with task_logs list and task_logs_url
Implementation Highlights:
- Auto-generated Pydantic models from GA4GH WES OpenAPI spec
- FastAPI CBV router following Galaxy API patterns
- Service layer with full workflow submission/execution pipeline
- Shared GA4GH utilities for DRS code reuse
- Reduced test duplication with helper functions
Files Added:
- lib/galaxy/schema/wes/__init__.py - Generated Pydantic models
- lib/galaxy/webapps/galaxy/api/wes.py - WES API router
- lib/galaxy/webapps/galaxy/services/wes.py - WesService
- lib/galaxy/webapps/galaxy/services/ga4gh.py - Shared GA4GH utilities
- WES_PLAN.md - Implementation plan
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replaces yarn with pnpm throughout the Makefile and common_startup.sh.
The startup script now installs pnpm via conda, corepack, or npm
depending on the environment.
The Makefile and client README now reference Vite instead of webpack, and the
dev server port is updated from 8081 to 5173 which is Vite's default.
Also removed the webpackChunkName magic comments from a few dynamic imports
since those were webpack-specific and Vite ignores them anyway.
Found by running:
```
make pyupgrade
ruff check --fix .
ruff check --preview --unsafe-fixes --select=F401 --fix .
make format
make update-client-api-schema
```
The bulk of the changes are replacements of typing deprecated aliases
such as `Dict` and `List` with the standard library classes. These
aliases became redundant in Python 3.9 when the corresponding
pre-existing classes were enhanced to support `[]`, see
https://docs.python.org/3/library/typing.html#deprecated-aliases
Also:
- Update the `pyupgrade` Makefile target to use `--py38-plus` for the
packages needed by Pulsar, including the new galaxy-tool-util-models
package.
Alireza Heidari