From fd044c987792ad465a140915ce3a29f6035cb1c1 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Wed, 26 Aug 2026 11:17:45 -0400 Subject: [PATCH] Regenerate collection_semantics.md `39597b3366` added the Type Compatibility Algebra section to `collection_semantics.yml` without regenerating the doc it is the source of, so those 102 lines have been missing from `doc/source/dev/collection_semantics.md` since April. Pure output of `python lib/galaxy/model/dataset_collections/types/semantics.py` against the unchanged YAML - no prose is written here. Nothing catches this drift: `--check` only validates the test references, and the generated file is excluded from prettier because prettier fights the generator. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Ra8S7P5FFZSmJChT3rvUgN --- doc/source/dev/collection_semantics.md | 102 +++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/doc/source/dev/collection_semantics.md b/doc/source/dev/collection_semantics.md index 55ef137a190..0b655f941f1 100644 --- a/doc/source/dev/collection_semantics.md +++ b/doc/source/dev/collection_semantics.md @@ -1118,3 +1118,105 @@ $$tool(i=C) \rightarrow \left\{o: \text{dataset}\right\}$$ +## Type Compatibility Algebra + +This section is for implementers. It describes the three operations that +answer "do these collection types fit together?" — used at workflow +editor connection time and at runtime when sibling inputs are matched +under a common map-over. + +### The lattice + +The base types form a small subtype lattice: + +``` +list paired_or_unpaired + | | +sample_sheet paired +``` + +Edges are subtype relations. A `sample_sheet` value carries column +metadata that a `list` does not, so it can be substituted where a +`list` is required (information is preserved); the reverse is not safe. +A `paired` value always has two elements; `paired_or_unpaired` admits +1 or 2, so a `paired` can be substituted where `paired_or_unpaired` is +required, but not the reverse. + +Nesting composes. `list:paired_or_unpaired` is a supertype of both +`list:paired` and `list` (the latter via the "single-dataset wrapped +as unpaired" interpretation), so it has two incomparable subtypes. + +### Three operations + +| Operation | Symmetry | Question | Used at | +|---|---|---|---| +| `accepts(other)` | Asymmetric | Does an input slot of type `self` accept an output of type `other`? | Workflow-editor edge validation | +| `compatible(other)` | Symmetric | Do `self` and `other` match such that they could drive a common map-over over sibling inputs of one tool? | Sibling-matching: matching sibling HDCAs / sibling map-over states under a common mapping | +| `can_map_over(other)` | Asymmetric | Does `self` have proper subcollections of type `other` — i.e. can `self` be mapped over to feed an `other` slot? | Connection-time map-over decisions; runtime `effective_collection_type` arithmetic | + +Conventions: `input_type.accepts(output_type)` for direct edges; +`output_type.can_map_over(input_type)` for map-over. `accepts` and +`can_map_over` differ in that `accepts` is the direct-edge case +where ranks already align, while `can_map_over` is the strict nesting +case where the output has *more* rank than the input. `compatible` +is symmetric and either side may go first. + +`compatible(a, b)` is implemented as `a.accepts(b) or b.accepts(a)`. + +### Where each is used + +- `accepts` is called at single-edge validation: connecting one output + to one input. The asymmetry between input and output sides matters + here. Examples: `connection_types.can_match`, + `query.HistoryQuery.direct_match`, and the workflow-editor input + attachment paths in `terminals.ts`. + +- `compatible` is called when two collections must drive a common + map-over as siblings. Neither side is the input slot; both are + concrete shapes (observed HDCA shapes at runtime, sibling map-over + states at connection time). Order of arrival must not change the + answer. Examples: Python `Tree.compatible_shape` (matching.py, + execute.py) and the `mappingConstraints` checks in `terminals.ts`. + +- `can_map_over` is called when deciding whether an output of higher + rank can drive a map-over into a lower-rank input — for instance, a + `list:paired` output feeding a `paired` input by iterating the outer + list. The Python and TypeScript names match + (`can_map_over` / `canMapOver`) because it is the same operational + question in both layers. + +Routing a sibling-matching question through `accepts` instead of +`compatible` produces order-dependent behavior — which sibling input +arrived first changes whether the workflow validates. This was a real +bug in earlier revisions of both the Python and TypeScript code. + +### Worked examples + +- `paired.accepts(paired_or_unpaired)` is `False`. A 1-element + paired_or_unpaired output cannot be connected to an input slot + that strictly requires a pair. Test: `test_paired_accepts_relation`. + +- `paired.compatible(paired_or_unpaired)` is `True`. If both observed + sibling collections happen to align in cardinality, they match for + sibling iteration; cardinality checking happens later at the + children level. Test: + `test_paired_and_paired_or_unpaired_match_symmetric`. + +- `sample_sheet.accepts(list)` is `False`; `list.accepts(sample_sheet)` + is `True`. Tests: + `test_sample_sheet_accepts_relation`, workflow editor + `"rejects list -> sample_sheet connection (asymmetry)"`. + +- `sample_sheet.compatible(list)` and `list.compatible(sample_sheet)` + are both `True`. Tests: `test_compatible`, + `test_tree_compatible_shape_sample_sheet_list_symmetric`. + +### Cross-language synchronization + +The Python implementation lives in +`lib/galaxy/model/dataset_collections/type_description.py`. The +TypeScript implementation lives in +`client/src/components/Workflow/Editor/modules/collectionTypeDescription.ts`. +Both must stay in sync; method names and conventions are identical. + +