- pip doesn't support the `--index-strategy` option.
- The best practice for pip is to specify additional indexes in
`PIP_EXTRA_INDEX_URL` and don't define `PIP_INDEX_URL` .
`39597b3366` added the Type Compatibility Algebra section to
`collection_semantics.yml` without regenerating the doc it is the source of, so
those 102 lines have been missing from `doc/source/dev/collection_semantics.md`
since April.
Pure output of `python lib/galaxy/model/dataset_collections/types/semantics.py`
against the unchanged YAML - no prose is written here. Nothing catches this
drift: `--check` only validates the test references, and the generated file is
excluded from prettier because prettier fights the generator.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ra8S7P5FFZSmJChT3rvUgN
GHeading was new in this branch, so unlike the other old-path shims
nothing ever imported client/src/components/BaseComponents/GHeading.vue
-- Common/Heading.vue is the old path that matters and it already
re-exports GHeading. The package's own type-check script from the review
follow-ups checks the source against the tsconfig an external consumer
sees, but nothing ran it; the client lint workflow now does.
The size check asserted more than 50 tokens against an actual 67, so a
third of the contract could be deleted with both tests still green.
Replaces it with two assertions that name what the package actually
promises: every shade of every color family, because GButton builds
selectors as var(--color-#{$color}-600) and a missing shade renders an
unstyled variant; and an exact list of the non-palette tokens.
custom_theme_variables.scss already computes eight masthead tokens
through #{$scss-variable}. Nothing in tokens.css uses interpolation
today, but if it ever does the value comparison comes out as
package="2.5rem" client="#{$masthead-height}", which reads like a token
mismatch rather than a parser limit. Interpolated values are now called
out for what they are.
The package arrived with a tsconfig.json but nothing that runs it, no
entry in vitest's include globs, and inside eslint's "packages" ignore
pattern -- so roughly 3,000 lines of moved component code had no check of
its own. Every guarantee it had came from the main client happening to
compile it through a Vite alias.
- eslint ignores packages/api-client specifically instead of all of
packages/, and the eslint scripts lint packages/ui/src alongside src.
The five errors this surfaced were import-sort autofixes.
- vitest's include picks up packages/*/src, which also starts running
packages/api-client's client.test.ts and integration.test.ts. Those
two files and their 9 tests had never run.
- packages/ui gets a type-check script so `vue-tsc -p` can be invoked
against the package on its own, not only via the client's tsconfig.
Porting GTip off SCSS variables replaced $border-radius-base with a
hardcoded 0.25rem, but $border-radius-base is 0.1875rem, so every variant
gained a pixel of corner radius. The code block lower in the same file
still rounds to 3px, which is what 0.1875rem resolves to.
The color changes in that port are a different matter and are left as
they are: GTip's success/warning/danger swatches now come from the token
palette rather than the $brand-* variables, which is what makes GTip
consistent with GButton and is presumably the point of the move.
GHeading landed as a copy of Common/Heading with icon narrowed from
IconLike to IconDefinition, which drops Galaxy's own icons (galaxyLogo and
friends) from what the prop accepts. That left two Heading implementations
in the tree -- Common/Heading with 94 importers, GHeading used only by
GModal -- and no way to migrate from one to the other, because the
replacement typed a strictly smaller prop.
Widening icon back to IconLike makes GHeading a superset of
Common/Heading: the two are otherwise identical apart from cosmetic
computed-property style and $brand-secondary vs var(--color-grey-200),
which are the same #dee2e6. Common/Heading becomes a shim like the other
15 components, so the 94 call sites keep working untouched and there is
one Heading again.
useClickableElement and useCurrentTitle are exported from the barrel but
ClickableProps and HasTitleProps are not, so a consumer can name the
composables and not the shape of the props they take. Both types were
reachable from the old client paths before the move; re-export them from
the barrel and from the two shims that dropped them.
Adds iconTypes with CustomIconDefinition and IconLike. FontAwesomeIcon
accepts icon definitions generated outside FontAwesome's registry as long
as they match its structure, and Galaxy ships a set of them, so the
package needs a name for "FA icon or a lookalike" that does not reach
back into the app for it.
Cloud tee-streaming integration failed in two setup paths before a download could be exercised.
First, fetch metadata preparation synchronously resolved a new job output. At that point the caching store contained only the empty file the job would write and no remote object. Cache synchronization rejected the empty file, failed its remote pull, removed the file, and raised ObjectNotFound, leaving the __DATA_FETCH__ job queued. Resolve the association with sync_cache=False, matching the output path that metadata setup already receives. This only selects the job's destination path; normal read-time validation of zero-byte datasets is unchanged.
Once fetch jobs could finish, the tests exposed a second configuration error: the Cloud object store discarded the MinIO endpoint and CloudBridge sent requests to AWS, where the test credentials produced InvalidAccessKeyId and dataset display returned 404. Preserve endpoint_url through Cloud configuration serialization and map it to CloudBridge's s3_endpoint_url setting.
Cover metadata setup with a real caching-base fake containing an empty local output and no remote object, scoped to one dataset instance. Also verify XML and YAML endpoint parsing, serialization, and the configuration passed across the optional CloudBridge factory boundary.
The integration assertion parsed raw_value with int(). raw_value is the numeric
metric column rendered as a string, so its shape follows the database and
PostgreSQL returns "1.0000000"; compare with Decimal instead. This is the two
PR-specific integration failures.
CorePlugin took its class-level fallback formatter from whichever instance was
constructed first, so a destination's timezone or show_zero_resubmissions leaked
into how metrics from an unconfigured core plugin rendered, and with core
disabled entirely the fallback stayed None and zero counts rendered as the raw
key. Give the class a stable unconfigured formatter and stop mutating it.
Docs: tightened per review, and say plainly that older jobs do not show the
metric.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5F4LLXH1Rpt2MCLQCgwgU
Hiding the zero is right for almost everyone and wrong for anyone who wants to see the
metric on every job, so put it behind a core option: show_zero_resubmissions, default
false. Parsed with asbool, since XML attributes arrive as strings and "false" is truthy.
Getting it to the rendered metric needed one fix. Formatting looked up
plugin_classes[plugin].formatter -- the class attribute, which CorePlugin assigns exactly
once, so whichever core plugin happened to be constructed first fixed the formatter for
the whole process and no configured option could reach it. That is the same reason the
existing timezone option only ever honours one configuration.
Ask the configured plugin first, mirroring the safety lookup a few lines below, and keep
the class attribute as the fallback for metrics whose plugin is no longer configured. The
default instrumenter is the one consulted, because metrics render without knowing which
destination the job ran on.
Recording resubmission_count only when non-zero kept the metrics panel clean at the cost of
making the metric mean two things: absent could be "never resubmitted" or "this Galaxy
predates the metric". Record it on every job instead, and decide displaying separately.
There was no way to say that. Safety levels already keep a metric out of the UI, but they
decide per metric name, not per value, so they cannot express "show this one only when it
has something to say". The formatter is where a metric is rendered for human consumption,
so let format() return None there to mean recorded-but-not-shown, and have
dictifiable_metrics drop those alongside the ones the safety filter removes.
Core uses it for resubmission_count at zero, which is almost every job. The stored value is
unchanged, so reports and queries see a uniform metric.
CpuInfoFormatter delegates to the base formatter, so its return type widens to match.
Resubmission count is the kind of thing you want to have been collecting before you go
looking for it: it answers "which tools keep getting resubmitted, and are their TPV rules
allocating badly", and an admin only asks that question after the interesting jobs have
already run. An opt-in plugin means the answer is empty on every instance that had not
already guessed it would want one.
Folding it into core costs nothing to justify that. The count reads the state history the
session already holds, since set_final_state loads the collection just before metrics are
collected, so there is no extra query on the path every job takes.
Recorded only when non-zero. Almost no job is ever resubmitted, and writing a zero for
each one is a job_metric_numeric row per job that carries no information; a missing value
reads as zero. That is the same shape core already uses for container details.
Drops the separate plugin, which needed galaxy.model and SQLAlchemy inside a package that
ships to Pulsar compute nodes with only galaxy-util, and whose app-is-None path silently
collected nothing for per-destination metrics configurations.
Keeps Gabriel Saudade's tests and documentation, retargeted at core.
Metrics plugins have only ever seen a job id and a directory, so anything wanting a
property of the job itself has had to reach for the app -- and galaxy-job-metrics is one
of the five packages Pulsar installs on compute nodes, so it cannot import galaxy.model
or SQLAlchemy to do that.
Declare the slice a plugin is allowed to see as a Protocol in the package itself, the way
galaxy.objectstore does with UserObjectStoreResolver and galaxy.files with
UserDefinedFileSources, and have the caller pass the Job it already holds. Job satisfies
it structurally; no import crosses the boundary and no dependency is added.
The framework now calls a new collect() hook whose default delegates to job_properties,
so in-tree and out-of-tree plugins that only need the directory are untouched. Pulsar
only ever calls pre_execute_commands/post_execute_commands, so widening the internal
collect_properties signature does not reach it.
The resubmit state handler already counted RESUBMITTED rows in job_state_history to
build the `attempt` variable admins write resubmit conditions against. Anything else
wanting the same number has had to hand-roll its own reading of the same rows.
Move the definition onto Job so `attempt` and any future caller agree by construction
rather than by coincidence. The state handler keeps its loop for the running/queued
timestamps, which is a different question about the same rows.
No new query: set_final_state appends to state_history before metrics are collected,
which loads the collection, so the property reads what the session already holds.
The loop over a collection's dataset_instances ends in an else with no break
anywhere, so the else runs on every pass rather than only when the collection is
empty. Every job with an output collection therefore gets its element datatypes
changed and a redundant PostJobActionAssociation persisted.
Check whether the collection has elements instead, which is what the else comment
describes.
The deferral itself is load-bearing and stays. Job.get_change_datatype_actions
filters on action_type only, not immediate_actions, so the association reaches
discovery as ext_override - that is how a dynamic collection's elements get the
new extension. Those collections are precreated with ELEMENTS_UNINITIALIZED and so
still defer. Only the redundant association for an already populated collection
goes away.
That association is inert and invisible to the API: both job finish paths skip
immediate_actions, ext_override is keyed by output name so a structured
collection's entry never matches a discovered output, and no API exposes a job's
actions. So nothing here goes red on this change.
test_change_datatype_static_collection_output passes either side of it. It fills a
neighbouring gap - it covers the branch this restructures, a collection structured
up front whose elements already exist, which nothing exercised before. The
deferral half already had test_change_datatype_discovered_outputs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
TES submission returns a provider-assigned task id that Galaxy stores as the job runner external id. Status polling rebuilds the Pulsar client with the Galaxy job id because that id must continue to drive the files and OIDC token endpoints, so the TES client has had no way to recover the provider task id and polls the wrong task.
Pass the recorded external id beside the Galaxy job id when constructing Pulsar clients for polling and cancellation. Older pulsar-galaxy-lib releases safely ignore the extra destination parameter; a paired Pulsar change consumes it for TES get_task and cancel_task operations.
Read the value from the persisted job rather than job_state.job_id, whose fallback to the Galaxy job id would invent a provider id when none was recorded.
The same stderr reached the model at four different sizes depending on who asked
for it: 2000 in get_job_details, 1000 for stdout, 4000 in the get_job_errors
tool, and 500 in _format_job_context. That last one re-sliced a value
get_job_details had already capped, so the 2000 was dead and only 500 characters
of head ever made it into the prompt -- for a tool that prints a banner before
dying, that's 500 characters of banner and none of the error.
They now share truncate_middle and one named budget, JOB_LOG_EXCERPT_CHARS, set
to the largest of the values it replaces so nothing regresses. Middle-trimming
already buys more per character than the head slices it supersedes. Job.info goes
through it too and counts toward the `truncated` flag: TrimmedString(255) only
trims on the way to the database, so a live ORM object can still be holding a
full exception message.
Covers the three call sites that show a job stream to a model: get_job_details,
_format_job_context (which re-slices an already-excerpted value down to 500
characters of head), and the get_job_errors tool result.
mypy binds the loop variable from the first iterable, so reusing `bad` for the
string/None/list cases and then for bools and ints failed `make mypy` with an
incompatible-assignment error -- which I missed by running mypy over the changed
lib/ files but not the changed test.
One list annotated as list[Any] covers every nonsense value, and the assert
carries the offending value so a failure still names which one broke.
CodeQL flags the warning as clear-text logging of sensitive information. It is a
false positive for this call -- the value read is max_query_length -- but the
dataflow is right about the shape: _get_agent_config is a generic accessor over
inference_services and the same function serves api_key a few lines down, so
echoing whatever it returned into a log is a habit worth not having.
The type name is enough for an admin to find the offending YAML line, and no
value from the config dict reaches the log now.
A tool that writes tens of kilobytes to stderr made "Let our Help Wizard figure
it out" fail with "Query too long (32768 chars). Maximum is 10000 characters",
which reads like the user's tool is at fault rather than the log being too big.
error_analysis now trims the query to its configured budget before validating it,
keeping a third of the budget from the head and the rest from the tail -- the
failure is almost always in the last few lines, so a plain head slice would throw
away the useful part. The trim comes back in the response metadata and the wizard
shows a note, so a partial diagnosis is not presented as if it had seen the whole
log.
Two things the fix needed to actually work. Because the budget now reaches a slice
index, max_query_length is resolved through int() with a fallback: inference_services
is a free-form dict, so a bool (YAML reads `yes` as one), an `.inf`, or a
non-positive value would otherwise crash the request or silently trim every query
to nothing. And the role-marker blacklist had to stop firing on logs -- an ordinary
banner printing "Operating System: Linux" matched `system:` and returned "please
rephrase your question" for a log the user never wrote, so the scan is now a class
attribute that error_analysis turns off. Every other agent keeps it.
The wizard posts a job's raw stderr as the query, so a tool that dumps 32kb of
warnings gets a "Query too long" validation error instead of a diagnosis, and a
banner line like "Operating System: Linux" trips the prompt-injection blacklist
on the way past (issue #23190). These cover the truncation helper, the agent
trimming rather than rejecting, the config guard around the cap, and the client
surfacing that only part of the log was read.
Treat action-specific URI discovery as complete only when the concrete action explicitly declares it. Unknown custom actions retain the legacy behavior of serializing every file source.
Keep legacy upload paramfile parsing fail-fast and avoid constructing JobIO during failure cleanup so a secondary error cannot mask the original exception.
prettier rewrites doc/source/dev/collection_semantics.md, which is generated
from collection_semantics.yml by semantics.py. Two problems: it fights the
generator, and it corrupts the content. Its markdown parser does not
recognize $...$ math spans, so it strips the \_ escapes LaTeX needs there -
\text{paired\_or\_unpaired} becomes \text{paired_or_unpaired}. Running it
over the current file drops 36 escaped underscores to 5.
Only bites people who opt into this sample config; there is no
.pre-commit-config.yaml on dev, so CI never ran prettier here and the
committed doc has never been prettier-formatted.
exclude_types already notes .prettierignore is ignored, so this uses exclude.
Other markdown is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RuVLSKbSts49EWpNMcz7gP
Every job currently serializes every configured and user-defined file source into JobIO. Resolving those configurations can read vault secrets and mint OAuth access tokens, so unrelated jobs can contact every OAuth provider available to a user.
Derive the required URIs at dispatch from persisted tool inputs, action-owned request data, and deferred input dataset origins. Directory URI parameters are collected generically, while tool actions report URLs embedded in data-fetch requests, legacy upload paramfiles, and history-import archive parameters. Keep the legacy upload paramfile path stable across job preparation and retries. This avoids duplicating execution metadata in new private job parameters.
Serialize only the highest-scoring configured source for each URI and only the user sources addressed by gxuserfiles UUIDs. Filtering happens before user source properties are resolved, preventing credential work for unrelated sources while preserving the complete configuration needed by external runners.
Addresses #17811.
Nicola Soranzo