Round out the index discovery so the lazy toolbox boots against the full
shipping tool set:
- ``scripts/tool_source/_discover.py`` now handles ``<tool_dir>`` panel
entries and reads YAML tool ids alongside XML, so multi-version tool
directories and YAML tools land in the index.
- ``LazyToolBox`` extends the bootstrap path to register every variant
discovered for a single tool id (was previously dropping siblings) and
uses the shared YAML id reader when chasing direct ``<tool file=...>``
references from the panel config.
- ``DatabaseToolSourceStore.store_index`` accepts the richer per-id
payload from the discovery script.
Test coverage is extended in ``test_tool_source_storage`` to assert that
the bootstrapped index sees both directory-based and YAML-defined tools.
End-to-end test that boots Galaxy through the composite store +
LazyToolBox path and verifies tools are discoverable via /api/tools,
covering the wiring exercised by all of the preceding commits.
_remap_container saved the directive line while looping over the
container's lines, but then re-matched GALAXY_MARKDOWN_FUNCTION_CALL_LINE
against the loop variable holding the container's *last* line. For stored
report markdown that was never validated (e.g. an unclosed ```galaxy
fence swallowing following prose), the re-match returned None and the
subsequent assert produced an unhandled 500 on
GET /api/invocations/{invocation_id}/report.
Keep the match object found in the loop instead, and turn the assert on
a second directive within one block into MalformedContents so malformed
user content yields a 4xx rather than an AssertionError.
Fixes#23175
Fixes USEGALAXY-EU-MAIN-61HFG00001W43
Add test_output_to_cwl_json_sample_sheet_paired, which exercises the
nested sample_sheet:paired shape that actually triggered the bug report.
The flat sample_sheet test alone did not cover the recursion through
element_to_cwl_json into the sub-collection's own collection_type.
Add test_output_to_cwl_json_paired_or_unpaired_paired so both arms of
paired_or_unpaired (1 element and 2 elements) are covered.
Assert the full CWL File shape (class, basename, checksum, size) rather
than only the basename. The basename alone traces back to the fake's own
input; checking class and checksum verifies the elements were really
converted to CWL File records by output_properties.
output_to_cwl_json keyed only on the outer segment of the collection type
and silently returned None for anything that wasn't list, paired or record.
For a promoted sample_sheet:paired workflow output that None propagates to
consumers such as planemo, which crashes with 'object of type NoneType has
no len()' three frames away from the actual problem, naming neither the
collection nor its type.
Both sample_sheet and paired_or_unpaired are flat ordered lists of elements,
so the existing list/paired branch already has the right shape. Nested types
need no extra handling: element_to_cwl_json recurses through
output_to_cwl_json with the sub-collection's own collection_type.
Replace the bare 'return None' with a NotImplementedError naming the
offending collection type, mirroring the else branch immediately below that
already does this for an unknown history_content_type.
Replace `rules: list[dict[str, Any]]` with 18 typed Pydantic rule models keyed
on a `type` discriminator (1:1 with the runtime rules_dsl rule set). Constrain
RulesMapping.type to the known mapping Literals. Validate regex `expression`
fields (AfterValidator). Fix add_filter_compare value int->float. RulesMapping/
RulesModel become BaseModel (extra ignored) so model_dump emits DSL-only fields.
Give RulesParameterModel a workflow-aware pydantic_template mirroring the other
inline params (optional in unlinked workflow_step, allow_connected_value when
linked) so gx_rules validates across state representations.
Adds gx_rules.xml framework tool + gx_rules parameter_specification.yml cases.
Pairs with editor UI-metadata leak fix#22823 and IWC cleanup galaxyproject/iwc#1278.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The casts pulled GalaxyWebTransaction in at module level, and importing
galaxy.webapps.base.webapp pulls in apispec, which the galaxy-app package
does not depend on, so the package's own test run failed to collect the
module.
Import it under TYPE_CHECKING and cast against the name as a string, the
same way galaxy.managers.users refers to it. cast does not evaluate a
string first argument, so nothing reaches for the module at runtime.
ProvidesFileSourcesUserContext took an untyped trans; annotating it as
ProvidesUserContext pointed galaxy.files at galaxy.managers, and the drs
test then imported that name at runtime for a cast. galaxy-files depends
on galaxy-util alone, so the package's own test run could not find
galaxy.managers and failed to collect.
Describe the transaction where it is used instead: the adapter reads user,
anonymous, user_is_admin, user_ftp_dir, user_vault and app, so state that
as a protocol in galaxy.files. Transactions satisfy it structurally, and
the test casts to something its package actually ships.
The agents stack declared trans as ProvidesUserContext everywhere, so
reaching the toolbox panel views needed a cast back to
SessionRequestContext. Nothing was ever narrower: DependsOnTrans is
declared to return a SessionRequestContext, and the MCP entry point builds
one explicitly, so the declarations were widening what callers hand over.
Declare SessionRequestContext along that chain instead, from the endpoints
through AgentService and GalaxyAgentDependencies to
AgentOperationsManager, and drop the cast. This also makes the toolbox
search attribute resolve, so its type: ignore goes too.
Conditional case inference in visit_input_values calls
test_param.get_initial_value(None, input_values), and that walker has no
transaction in scope to offer. Asserting a transaction there raised
AssertionError, which get_current_case does not catch, so any conditional
whose test parameter is a select would fail instead of inferring its case.
Resolve options from the statically declared ones when no transaction is
supplied, which is all that can be resolved without one and what these
parameters carry in practice.
Type every function signature that took an untyped `trans`, using the
narrowest context each function actually needs: ProvidesAppContext,
ProvidesUserContext or ProvidesHistoryContext in manager and service
code, SessionRequestContext for FastAPI endpoints, and
GalaxyWebTransaction for legacy WSGI controllers. Lower-level packages
(model, security, datatypes, tool_util, util) use TYPE_CHECKING imports
with quoted annotations so no runtime cycle is introduced.
Annotating a previously untyped function turns on mypy body checking for
it, which surfaced several latent bugs, fixed here:
- webapps/base/controller.py passed three arguments to
can_modify_library_item(), which accepts two.
- api/provenance.py BaseProvenanceController was missing
SharableItemSecurityMixin, so security_check() did not exist on it.
- managers/users.py raised exceptions.Message, which does not exist, and
is_admin() returned None rather than False.
- Unguarded toolbox.get_tool() results in tools, workflow modules and
library upload now raise ToolMissingException instead of failing with
AttributeError.
- FTP upload paths raised TypeError when no FTP directory was configured
instead of reporting a request error.
Where an annotation exposed a gap in an interface, the interface is
fixed rather than suppressed: error_codes gained a PEP 562 __getattr__
for its JSON-loaded members, vault moved from StructuredApp to
MinimalManagerApp, dataset_matcher_factory is declared on
ProvidesHistoryContext, and the toolbox panel view methods accept both a
web transaction and a request context.
The adapter appended one list per HDA instead of extending, returning a nested
list-of-lists while every consumer unpacks flat (action, role_id) tuples. The
tuple unpack only happened to succeed when the first dataset carried exactly
two permission rows, which is why test_adapting_two_datasets_to_paired_collection
failed sporadically on CI with 'not enough values to unpack (expected 2, got 1)'.
docutils publishers are stateful and not thread-safe: all threads shared the
functools.cache'd Publisher, so concurrent /api/tools/{id}/build requests
corrupted each other's document state and raised. Tool.help_html swallowed the
exception and returned empty help, which is why the tools list intermittently
rendered cards without their help toggle
(test_tool_discovery_help_toggle_shows_and_hides, the most frequent flaky
selenium/playwright test on dev: 10 failed CI runs 07-06..07-18).
Guard publishing with a lock (results are memoized, so only first-time
conversions contend) and log the swallowed exception with a traceback. The new
concurrency test fails 3/3 without the lock.
Two failure modes in workflow/tool test output verification:
- element_tests on a single-dataset (HDA) output raised a bare
KeyError('elements') that aborted the entire planemo run rather than
being collected as an ordinary test failure. Guard the missing
'elements' key and raise an AssertionError naming the output and
pointing at 'asserts'.
- Nested element_tests (e.g. list:list) were only read under the
'elements' key, so writing 'element_tests' at a nested level was
silently ignored and the assertion never ran. Resolve the preferred
'element_tests' key and its 'elements' alias through a single shared
helper (resolve_element_tests) used by both the top-level and nested
levels, making the two symmetric.
Data managers can record absolute paths below $out_file.extra_files_path, while bundle writing runs on the job handler after that directory has been collected into the object store. Relativizing against dataset.extra_files_path then escapes with '..' and leaves the compute-side path in the bundle index.
The job working directory is already known. For outputs_to_working_directory, reconstruct the exact compute-side extra-files root with the same uuid-keyed output-path rule used during command construction, then pass that per-output root to bundle writing. For other outputs, retain the dataset extra-files path.
This restores ordinary os.path.relpath containment checks, including '.' for the extra-files root itself, instead of inferring the root from a matching directory component. It handles split job/object-store prefixes and id-keyed object stores without changing the bundle format or sending anything to the compute node.
Regression tests cover the split-prefix case, uuid-keyed compute paths on id-keyed stores, the root directory, and nested dataset-like directory names.
Improves database session management during user registration by moving flushes into the token generation logic. This ensures user records are visible to queries in sessions where autoflush is disabled.
Shifts the commit operation to occur only after activation emails are successfully sent and introduces a rollback if email delivery fails, ensuring better data consistency and atomicity.
mvdbeek