Any request to GET /api/datasets/{id}/content/{content_type} could read
an entire HDF5 dataset into memory: h5grove's DatasetContent.data() with
no selection executes dataset[()], selections are unbounded, data_stats
reads everything, attributes() reads all attribute values and group
metadata grows with child count.
Cap in-memory reads at 1 MB:
- data (format=bin/npy/csv) and stats now stream/accumulate in <= 1 MB
slabs, so datasets of any size remain fully usable (h5web requests
numeric data as format=bin)
- data (format=json/tiff), attr and group meta reject oversized requests
with a 400 suggesting the selection parameter, computed pre-read from
shape, dtype itemsize and the parsed selection
- variable-length dtypes cannot be pre-sized (h5py exposes no
H5Dvlen_get_buf_size), so they are capped at ~1000 elements
- invalid selection strings now return 400 instead of 500
MCP clients can already create user tools via create_user_tool, but with no container
verification -- its docstring even suggests bare images like python:3.12-slim, reproducing the
hallucinated/non-biocontainer problem the custom-tool agent now guards against. Add a
recommend_biocontainer MCP tool that wraps recommend_container + biocontainer_tag_built (pure
library functions, no app dependency) so a client can resolve and verify a real
quay.io/biocontainers image -- returning image/found/match_quality/source/notes/verified -- and
use it as the container for create_user_tool. The parse+shape logic is a module-level helper with
injectable collaborators so it is unit-testable offline; auth via the existing api-key path.
--judge-model values are provider-qualified (e.g. `openai:gpt-5`), but the OpenAI API
expects the bare model id. build_judge_model passed the qualified name straight to
OpenAIChatModel, so any judge model failed. Strip the `provider:` prefix the same way the
agents under test already do.
parse_profile is annotated -> str but returned the raw root_dict value, which is a
number for YAML/JSON tool sources (and UserToolSource) that carry a numeric `profile`
(e.g. 24.2). The linters compare it with `Version(profile)`, which raises
"TypeError: expected string or bytes-like object, got 'float'" -- crashing
lint_user_tool_source on any such tool (notably user tools authored by the custom-tool
agent). Coerce to str so the annotation holds and all six Version(profile) call sites work.
Three configfile + multiple-input + real-biocontainer cases (R/ggplot2 scatter,
pandas group-summary, Biopython FASTA filter) to broaden the signal for the
container-resolution / script-materialization / required-inputs comparison.
ToolYamlContains can confirm the container is in the quay.io/biocontainers
namespace, but a substring match can't distinguish a real image from a hallucinated
tag (an invented build suffix). Adds a ContainerVerified evaluator that checks the
produced tool's container against quay.io via biocontainer_tag_built:
- verified-present biocontainer -> 1.0
- verifiably-absent tag (made up) -> 0.0
- not a biocontainers ref, or a transient lookup failure -> 1.0 (no false failure)
Wired into build_custom_tool, so every custom_tool case is scored. Touches the
network like the recommender; degrades to 1.0 when outbound calls aren't available.
make_deps now sets, on the inference_services.custom_tool block:
- container_recommendation_enabled: so the custom_tool eval exercises the full
container-resolution path (dedicated container critic + quay.io recommender +
deterministic override) rather than only the model's raw container choice.
- max_tokens (16384, mirroring the agent's DEFAULT_MAX_TOKENS): the producer emits
a full tool definition (inputs/outputs/configfiles) that overruns the 4000-token
default block and would otherwise fail with a token-limit error before any
response is generated.
Config resolves per-key, so model/api still come from the default block. This adds
an extra model call plus outbound quay.io lookups to custom_tool cases.
Exercises the agent on "use ggplot2 to make a boxplot, let the user pick the
grouping and numeric columns, put the script in a configfile" -- the request
class that previously produced a dropped configfile (Rscript run by name with no
file to create it) and a non-biocontainer image (rocker/tidyverse).
The case asserts, deterministically (ToolYamlContains) and via the LLM-judge
rubric: the R script lives in a configfile (not dropped, not inlined), the
container is an existing quay.io/biocontainers R+ggplot2 image, and the grouping
and numeric columns are user-selectable inputs rather than hardcoded.
The custom tool authoring agent (galaxy.agents.custom_tool) gains container
resolution and producer-output safeguards. It consumes the mulled recommender
(galaxy.tool_util.deps.mulled.recommend):
- Container resolution: a dedicated container critic infers the conda packages
from the tool's shell_command/configfiles, the recommender resolves a verified
quay.io image, and a deterministic gate applies it -- overriding an
exact-version match, a verifiably-absent biocontainer tag, or a non-biocontainer
image, while leaving a present pinned biocontainer alone. Runs independently of
the quality critic.
- Producer reliability: inputs/outputs are required in the LLM authoring view so
grammar-constrained models can't drop them; UserToolSource stays lenient for the
API and stored tools.
- The quality critic supplies deterministic field edits (labels/help/...) applied
in place, falling back to a full re-roll only for structural changes.
- Reject a shell_command that runs a script by name with no configfile to create
it, driving a producer retry.
- Render saved tool YAML with exclude_defaults so it carries only authored fields.
Admin config docs (config_schema.yml / galaxy.yml.sample) describe the
quality_critic_enabled and container_recommendation_enabled flags.
Tests cover the authoring-view required fields, container override paths,
deterministic critic edits, and the script check.
A reusable resolver (galaxy.tool_util.deps.mulled.recommend) that picks the best
quay.io/biocontainers image for a set of conda packages, verified against quay.io
rather than guessed:
- recommend_container() resolves single-package and mulled-v2 (multi-package)
images, distinguishing an exact-version hit from a name-only fallback and
resolving unpinned versions against tags that are actually built.
- biocontainer_tag_built() verifies whether a specific image tag exists.
- Shared lookup helpers move into mulled/util.py and are reused by the existing
container resolver (container_resolvers/mulled.py) to avoid duplication.
Exposed as the mulled-recommend console script:
mulled-recommend samtools=1.17
mulled-recommend bwa,samtools
mulled-recommend --json "samtools=1.3.1,bedtools=2.26.0"
Missing containers and transient network errors degrade gracefully (no exception).
Documented under admin special topics (mulled containers).
ToolPanelElements._section_by_tool was a class-level mutable dict shared
by the tool panel, the integrated panel, every section's elems and every
rendered panel view. EDAM view rendering records ontology buckets through
the same dict the real panel's get_section_for_tool_id reads, and records
survived toolbox reloads. Scope it to the instance.
The HelpInvalidRST/HelpValidRST linters read the <help> text straight off
the XML tree and always validated it as reStructuredText. Markdown help
(and YAML tool sources, which default to Markdown) was therefore flagged
with spurious "Invalid reStructuredText" warnings.
Route both linters through a shared _help_rst() helper that uses
tool_source.parse_help() to obtain the help content and its declared
format, and skip RST validation unless the format is restructuredtext.
Fixes#23071
Marius van den Beek