Pure refactor: the conf walker now lives at
``lib/galaxy/tool_source_store/discover.py`` and is importable via
``from galaxy.tool_source_store.discover import discover_tools``. No
sys.path injection needed for in-process callers (cold-start
auto-populate, shed-install reroute, test driver in upcoming commits).
``populate_store.py`` updates its two import sites; behaviour unchanged.
Round out the index discovery so the lazy toolbox boots against the full
shipping tool set:
- ``scripts/tool_source/_discover.py`` now handles ``<tool_dir>`` panel
entries and reads YAML tool ids alongside XML, so multi-version tool
directories and YAML tools land in the index.
- ``LazyToolBox`` extends the bootstrap path to register every variant
discovered for a single tool id (was previously dropping siblings) and
uses the shared YAML id reader when chasing direct ``<tool file=...>``
references from the panel config.
- ``DatabaseToolSourceStore.store_index`` accepts the richer per-id
payload from the discovery script.
Test coverage is extended in ``test_tool_source_storage`` to assert that
the bootstrapped index sees both directory-based and YAML-defined tools.
The package is app-coupled (galaxy.model, galaxy.config, galaxy.queues,
galaxy.datatypes) and only consumed from app-side code, so it belongs
under galaxy.tools rather than a new top-level package. Riding the
existing packages/app tools symlink also removes the need for a
dedicated package symlink. Also drop an unused back-compat alias and
fix stale doc references.
Claude-Session: https://claude.ai/code/session_018L7ZmCv2ubKA3JNeSL8Pkr
Introduces galaxy.tool_source_store: a queryable store of parsed,
macro-expanded tool sources plus a pre-computed ToolIndex of the metadata
batch consumers need (panel section, labels, EDAM, requirements, test
counts, shed provenance).
- Backends: database (store-owned tool_source_record + tool_index tables,
migration f5a73c8b9d12), sqlalchemy (read-only SQLite bundles, e.g.
CVMFS-shipped), and a composite that layers named stores over the
default with read fall-through.
- scripts/tool_source/populate_store.py walks tool_conf files and writes
sources + index + whoosh search index; --watch updates the store on
file changes and broadcasts reload_tool_source_cache. The populator is
the single writer; consumers are read-only.
- Per-conf opt-in via store="name" on a toolbox conf root, resolved
against the tool_source_stores catalog in galaxy.yml.
- The tool_source_record table is deliberately separate from tool_source,
whose rows belong to the job-request path and carry a raw-string
payload contract.
Nothing reads from the store yet — an on-demand-loading toolbox consuming
it is follow-up work (#22633). Extracted from that PR.
mvdbeek