- Extract FavoritesManager (lib/galaxy/managers/favorites.py): owns
normalize / resolve / persist / order logic; controllers in users.py
are now 2 lines each, and trans.sa_session.commit moves out of the
controller layer. Drops the redundant re-normalize on every write
and replaces FavoriteObjectType.X.value == .value defensive dispatch
with enum-identity comparison. Renames the set_favorite_order
endpoint summary away from the ambiguous "top-level" wording.
- Move curated_tool_tags_by_id off the toolbox into ToolsService;
the controller now routes through the service instead of poking
trans.app.toolbox directly. Raw aggregates (curated_tool_tags,
tool_edam_operations, tool_edam_topics) remain on the toolbox since
they're derived from per-tool data.
- Rename /api/tools/tags -> /api/tags/tool_tags (operation_id
tags__tool_tags) to disambiguate from /api/tools/{id}; update the
client store, tests, and the hand-edited OpenAPI schema entry.
(Run `make update-client-api-schema` before pushing to regenerate
the schema in its canonical alphabetical position.)
- Promote inline imports to module-level (ontology_data.py logging,
app/__init__.py configure_tool_tag_mapping); replace
getattr(self.config, "tool_tag_mappings_file", None) with direct
attribute access now that the option is registered in the schema.
- Convert ontology_data's import-time bundled-mapping reads to
lru_cache so the YAML / TSV files load lazily; fix the
Tool._setup_id comment to point at the actual lowercasing site
(Tool.parse around `self_ids = [self.id.lower()]`).
- doc/source/admin/tool_panel.rst: 26.0 -> 26.1.
- scripts/extract_tool_sections_from_api.py: annotate the public
helpers the test suite imports.
- test_to_dict_cache_drops_when_tool_removed now asserts observable
to_dict() output before/after remove, not private cache membership.
- Split test_tool_discovery_landing into focused tests; extract a
helper that opens the discovery view and types the standard filter.
- Move the Playwright drag helper from test_tool_panel_search.py into
TestWithSeleniumMixin.playwright_drag_item_above so future tests
reuse it.
- Client tests: drop the full-string Whoosh equality in
utilities.test.ts; hard-code expected Whoosh strings in
ToolsList.test.ts (drop createWhooshQuery as both sides of the
assertion) and remove the redundant injectTestRouter mount option
(ToolsList.vue only uses the useRouter composable); drop the
FontAwesome data-icon/iconName assertions in ToolPanel.test.ts in
favor of icon-existence checks; replace three near-identical
add/remove favorite tests in ToolsListCard.test.ts with an it.each;
export quoteToolTagValue from Panels/utilities so
filterConversion.test.js imports the real helper instead of
reconstructing it inline; delete the spy-only "loads the curated
tag mapping" test in ToolBoxSearch.test.ts (the gate is covered by
ToolPanel.test.ts's negative case and the favorite-tag rendering
tests cover the positive case end-to-end).
Two related fixes around the admin-facing helper script and admin docs:
* Make the helper script work from any cwd and against any Galaxy
server: `--api-url`, `--output`, `--timeout` argparse flags; raise
`RuntimeError` on transport / decode failures (was a silent no-op);
anchor the default output path on the script location so
`python scripts/extract_tool_sections_from_api.py` works without a
prior `cd`. Default output now points at `config/tool_tag_mappings.yml`
to match the new admin override path.
* Add a "Curated tool tags" section to `doc/source/admin/tool_panel.rst`
documenting the new `tool_tag_mappings_file` option, the new
`tag:"…"` / `ontology:"…"` search syntax, the `?section=` URL
deprecation, and the reindex requirement after changing the mapping.
Promote the max_tokens fallback in BaseGalaxyAgent from a hardcoded
literal to a class attribute (DEFAULT_MAX_TOKENS = 8192) so individual
agents can declare their own ceiling without overriding the whole
_get_max_tokens method. The history, orchestrator, and custom_tool
agents now bump theirs to 16384 -- those are the ones that produce
long structured output (history listings, multi-step plans, full
UserToolSource JSON schemas) and were the most likely to silently
truncate. Router, tool recommendation, and error_analysis stay on the
8k default; their typical responses sit comfortably under that.
inference_services.<agent>.max_tokens config still wins over the
class default.
Also refreshed the example configs in doc/source/admin/ai_agents.md
so they no longer show 2000 as a representative override value, and
added a parenthetical to the param description noting the new
defaults.
Cover how the SSE pipeline is wired (HistoryAuditMonitor +
galaxy-sse-monitor + control-task fan-out + per-worker connection
manager), how to configure it, what statsd metrics to watch, and
how to set up nginx so /api/events/stream is not buffered.
Wire the new page into the admin toctree and add cross-references
from the nginx and scaling guides where SSE has operational impact.
Replace enable_sse_history_updates and enable_sse_entry_point_updates
with a single enable_sse_updates. The two flags shared the same SSE
connection plumbing (events/stream + dispatcher), and only the
HistoryAuditMonitor lifecycle has real per-process cost — entry-point
dispatch is event-driven and effectively free. One knob admins can flip.
Updates schema, sample config, server (app, configuration manager,
history_audit_monitor, sse_monitor), client stores and their tests,
integration tests, and the regenerated admin docs.
Three related additions to the SSE notification pipeline, bundled here
because they share ``SSEEventDispatcher._send`` as their modification
point:
1. Interactive-tool entry-point SSE channel
- ``entry_point_update`` dispatcher method + queue-worker handler.
- ``InteractiveToolManager.configure_entry_points`` dispatches a
wake-up event after the DB commit; the client refetches
``/api/entry_points`` on receipt (no payload).
- Frontend: new SSE event type, store subscription with XOR polling
fallback via ``enable_sse_entry_point_updates`` config flag.
- Integration + selenium tests.
2. Queue and SSE observability metrics
- Counters, timers, and periodic gauges for SSE dispatch, control-queue
task execution, control-queue depth (via kombu passive declare),
active SSE connections, dropped events, and active WorkerProcess
rows. Flow through the existing ``galaxy_statsd_client`` — no new
infra. Gauges are scheduled via Celery beat at ``queue_metrics_interval``
seconds (default 15). All instrumentation no-ops when statsd isn't
configured.
- Sub-emitter failures log once at WARNING and bump a
``galaxy.queue_metrics.error`` counter tagged by emitter so broken
emitters are visible in metrics without log spam.
3. Active-worker control-queue cache
- 30 s TTL cache on ``all_control_queues_for_declare`` with RLock
stampede protection. At 1000+ events/s this eliminates ~30 DB
round-trips/s per webapp for data that only changes on 60 s
heartbeat cadence. Empty results are not cached — would otherwise
silently drop every SSE event during the startup window.
Add real-time notification delivery via SSE to replace the 30-second
polling interval. The SSE endpoint streams notification_update,
broadcast_update, and notification_status events to connected clients.
Backend:
- New SSEConnectionManager (lib/galaxy/managers/sse.py) maps user IDs
to asyncio queues with thread-safe push via call_soon_threadsafe
- SSE streaming endpoint at GET /api/notifications/stream with
Last-Event-ID catch-up support and 30s keepalive
- Kombu control tasks (notify_users, notify_broadcast) fan out events
across all Galaxy worker processes
- Existing polling API unchanged for backward compatibility
Frontend:
- New useNotificationSSE composable using EventSource with auto-reconnect
- notificationsStore tries SSE first, falls back to polling after 5+
consecutive errors
Tests:
- API integration tests for SSE event delivery, broadcasts, and reconnect
- Selenium E2E tests for notification appearance and bell indicator
Add SSE-based real-time history update notifications
Replace aggressive 3-second history polling with Server-Sent Events
driven by database change detection, configurable via admin setting.
Backend:
- Add pg_notify() to PostgreSQL audit triggers for instant LISTEN/NOTIFY
- New HistoryAuditMonitor: PG LISTEN/NOTIFY with SQLite polling fallback
- New /api/events/stream SSE endpoint (uses StructuredApp, not MinimalManagerApp)
- Kombu control task "history_update" with message TTL (expiration=10s)
- Config: enable_sse_history_updates, history_audit_monitor_poll_interval
Frontend:
- Generalize useNotificationSSE → useSSE composable with event type filtering
- historyStore connects SSE for history_update events, triggers immediate refresh
- notificationsStore updated to use /api/events/stream and useSSE
- Polling kept as fallback at existing intervals
Tests:
- 5 integration tests: endpoint, dataset upload event, history ID in payload,
cross-user isolation, polling backward compatibility
Documents the 7-column `huggingface` data table schema, column
conventions, controlled vocabularies for pipeline_tag/domain, XML
filter patterns, and an example .loc entry.
Announced in galaxyproject/galaxy-hub#3923.
Introduce lib/galaxy/tool_util_models/yaml_parameters.py with a narrow
YamlGalaxyToolParameter union (extra=forbid) used for UserToolSource.inputs
and ToolSourceBase.inputs. v1 supports boolean, integer, float, text, select,
color, data, data_collection, conditional, repeat, section. XML-only fields
(truevalue, falsevalue, argument, is_dynamic, hidden, parameter_type) and
unsupported types (hidden, drill_down, data_column, genomebuild, group_tag,
baseurl, rules, directory) now reject at parse time.
to_internal() maps each YAML model onto the existing internal parameter
metamodel so no downstream pipeline needs to change shape. Production path
keeps re-parsing via YamlToolSource (plan option a).
UserToolSource envelope also gets extra=forbid - stray top-level keys like
argument: at the tool level fail fast. Regenerated ToolSourceSchema.json is
32KB (was 77KB); Monaco no longer autocompletes the leaked fields.
runtimeify() gains yaml_origin flag; evaluation.py passes it based on
tool_source.parse_class(). assert_yaml_v1_parameters() hard-fails on any
out-of-v1 type inside a YAML-origin bundle, recursing through whens/repeat/
section. XML path untouched.
doc/source/admin/user_defined_tools.md documents the supported set and
rejected types. 34 new tests in test_yaml_parameters.py cover red cases,
green round-trips, structural groups, the runtimeify guard, and a snapshot
blacklist on the published schema. test_parsing.py still green (83 tests).
No compat shim for existing beta DB rows that contain XML-only fields -
those will now fail validation on load and need to be recreated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a celery beat for renewal of Hashicorp Vault tokens.
This eliminates the need for manual token rotation when
using short-lived renewable tokens.
The approach:
- HashicorpVault checks token renewable status on startup (warning
if not renewable)
- New `renew_vault_token` Celery Beat task calls renew-self
- Configured via `vault_token_renewal_interval` in galaxy.yml
(default 0 = disabled)
- Requires Celery Beat to be running
Fixes https://github.com/galaxyproject/galaxy/issues/22187
Separate from the server-side sentry_traces_sample_rate so admins can
independently control browser performance trace sampling. Defaults to 0
(no client performance traces).
Dead option - nothing reads it, pause module always available regardless of setting. Add deprecation handler so existing configs don't error on upgrade.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The deprecated ai_api_key/ai_model/ai_api_base_url keys were front and
center in the original doc. This flips the emphasis to inference_services
as the recommended config path, demoting the deprecated keys to a note.
Also removes dataset_analyzer (no implementation exists) and the agents
enabled/disabled config section (not wired up yet).
More portable and idiomatic.
Also:
- Replace `os.path.abspath(f"{dir}/../..")` with `os.path.dirname()`
calls.
- Add test for `get_alternative_versions()` with `recipes_path`
argument specified.
Document configuration options needed for handling large history exports:
- Celery result_expires setting to prevent task results from expiring
- Short-term storage duration settings for export file retention
- Combined configuration example for large history export scenarios
Fixes: https://github.com/galaxyproject/galaxy/issues/21701
Implement full Workflow Execution Service (WES) API support for running Galaxy
workflows via the GA4GH standard. Includes complete core functionality with run
submission, status tracking, cancellation, input/output handling, and task logs.
Core WES Features:
- 6 WES API endpoints (service-info, runs CRUD, cancel, status)
- Support for both workflow_url and workflow_attachment input methods
- Support for gxworkflow:// URI scheme for direct database workflow references
- Automatic history creation with optional custom naming
- Full workflow support: gx_workflow_ga and gx_workflow_format2
- DRS URI generation for all workflow outputs
- State mapping between Galaxy invocation states and WES states
- Cursor-based pagination for run listings
Task Log Features:
- /api/jobs/{job_id}/stdout - Job stdout as plain text
- /api/jobs/{job_id}/stderr - Job stderr as plain text
- /ga4gh/wes/v1/runs/{run_id}/tasks - Paginated task list
- /ga4gh/wes/v1/runs/{run_id}/tasks/{task_id} - Task details
- Build TaskLogs from workflow invocation steps
- Populate RunLog with task_logs list and task_logs_url
Implementation Highlights:
- Auto-generated Pydantic models from GA4GH WES OpenAPI spec
- FastAPI CBV router following Galaxy API patterns
- Service layer with full workflow submission/execution pipeline
- Shared GA4GH utilities for DRS code reuse
- Reduced test duplication with helper functions
Files Added:
- lib/galaxy/schema/wes/__init__.py - Generated Pydantic models
- lib/galaxy/webapps/galaxy/api/wes.py - WES API router
- lib/galaxy/webapps/galaxy/services/wes.py - WesService
- lib/galaxy/webapps/galaxy/services/ga4gh.py - Shared GA4GH utilities
- WES_PLAN.md - Implementation plan
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
mvdbeek