diff --git a/lib/galaxy/config/sample/tool_conf.xml.sample b/lib/galaxy/config/sample/tool_conf.xml.sample index 5f6368a262e..80d349011ef 100644 --- a/lib/galaxy/config/sample/tool_conf.xml.sample +++ b/lib/galaxy/config/sample/tool_conf.xml.sample @@ -29,8 +29,19 @@
- - +
diff --git a/lib/galaxy/tools/apply_rules.xml b/lib/galaxy/tools/apply_rules.xml index 1df0063d770..a0677fe7d59 100644 --- a/lib/galaxy/tools/apply_rules.xml +++ b/lib/galaxy/tools/apply_rules.xml @@ -2,7 +2,11 @@ name="Apply rules" version="1.1.0" tool_type="apply_rules_to_collection"> + to reorganize a collection using the rule builder + + model_operation_macros.xml + operation_2409 operation_3695 @@ -23,28 +27,53 @@ Synopsis ======== -This tool allows one to process an existing Galaxy dataset collection's metadata as tabular data, -apply a series of rules to it, and generate a new collection. +Reorganizes a collection by applying a chain of rules (filter, sort, rename, nest, flatten, tag) through an interactive rule builder. =========== Description =========== -When used interactively in the tool form, a dynamic preview of the processing will be available in a tabular data viewer but this tool -may be used in workflows as well where no such preview can be generated. +This tool opens an interactive rule builder that lets you manipulate the element identifiers (and nesting structure) of a collection as if they were columns in a spreadsheet. You can chain multiple operations together — for example, split an identifier on a delimiter, use one part as a new nesting level and another as the element name, filter rows by a pattern, and sort the result — all in a single step. -This tool is an advanced feature but has a lot of flexibility - it can be used to process collections with arbitrary nesting and can do many kinds of filtering, re-sorting, nesting, flattening, and arbitrary combinations thereof not possible with Galaxy's other, more simple -collection operation tools. +This is an advanced tool that combines the capabilities of many simpler collection operations (sort, relabel, nest, flatten, filter) into one flexible interface. Use it when the simpler tools cannot achieve the reorganization you need. -More information about the rule processor in general can be found at `our training site`_. +======== +Examples +======== -.. _our training site: https://training.galaxyproject.org/training-material/search?query=rule+builder +**Flatten and re-nest by a different criterion** +Given a ``list:list`` collection where the outer identifiers are sequencing lanes and inner identifiers are sample names, you could use the rule builder to re-nest so that the outer level is sample name and the inner level is lane — effectively transposing the collection structure. +------- + +**Extract metadata from identifiers** + +If your element identifiers encode multiple pieces of information (e.g. ``patient01_treatment_rep2``), you can split on ``_`` to create columns, then use those columns to define new nesting levels, group tags, or filters. + +------- + +**Assign tags based on identifier patterns** + +You can add a column derived from an identifier (e.g. extract "treatment" or "control" from the name) and assign it as a group tag, all within the rule builder. + +------- + +.. image:: ${static_path}/images/tools/collection_ops/apply_rules.svg + :alt: Apply rules to reorganize a collection + :width: 500 + +------- + +For detailed tutorials on the rule builder, visit `the Galaxy Training Network`_. + +.. _the Galaxy Training Network: https://training.galaxyproject.org/training-material/search?query=rule+builder + +------- .. class:: infomark -This tool will create new history datasets from your collection but your quota usage will not increase. +@QUOTA_USAGE_NOTE@ - ]]> + ]]> diff --git a/lib/galaxy/tools/build_list_1.2.0.xml b/lib/galaxy/tools/build_list_1.2.0.xml index 457ae2b471d..ab839d2773b 100644 --- a/lib/galaxy/tools/build_list_1.2.0.xml +++ b/lib/galaxy/tools/build_list_1.2.0.xml @@ -2,16 +2,19 @@ name="Build list" version="1.2.0" tool_type="build_list"> - + from individual datasets or existing collections + + model_operation_macros.xml + operation_2409 - - + + @@ -21,7 +24,7 @@ - + @@ -42,7 +45,7 @@ Builds a new list collection from individual datasets or collections. Description =========== -This tool combines individual datasets or collections into a new collection. The simplest scenario is building a new colection from individual datasets (case **A** in the image below). You can merge a collection with individual dataset(s). In this case (see **B** in the image below) the individual dataset(s) will be merged with each element of the input collection to create a nested collection. Finally, two or more collection can be merged together creating a nested collection (case **C** in the image below). +This tool combines individual datasets or collections into a new collection. The simplest scenario is building a new collection from individual datasets (case **A** in the image below). You can merge a collection with individual dataset(s). In this case (see **B** in the image below) the individual dataset(s) will be merged with each element of the input collection to create a nested collection. Finally, two or more collections can be merged together creating a nested collection (case **C** in the image below). .. class:: warningmark @@ -51,15 +54,14 @@ This tool combines individual datasets or collections into a new collection. The ------ .. image:: ${static_path}/images/tools/collection_ops/build_list.svg + :alt: Build list operation :width: 800 - :alt: Unzipping operation ------- .. class:: infomark - -This tool will create a new collection from your history datasets but your quota usage will not increase. +@QUOTA_USAGE_NOTE@ ]]> diff --git a/lib/galaxy/tools/convert_sample_sheet.xml b/lib/galaxy/tools/convert_sample_sheet.xml index 0ed0a43a700..44477b900e3 100644 --- a/lib/galaxy/tools/convert_sample_sheet.xml +++ b/lib/galaxy/tools/convert_sample_sheet.xml @@ -26,21 +26,83 @@ Synopsis ======== -Converts a sample sheet collection back to its corresponding list collection type. +Strips sample sheet metadata and produces a regular list collection. + +======================== +What is a sample sheet? +======================== + +A sample sheet is a special type of Galaxy collection that pairs each dataset with structured metadata — such as replicate number, treatment condition, or tissue type. It works like a regular list collection but with an attached spreadsheet of per-sample information. =========== Description =========== -This tool takes a sample sheet collection and produces a regular list collection, removing all sample sheet metadata (column definitions and row values). +This tool removes the metadata layer from a sample sheet and converts it to the corresponding regular list collection type. The datasets themselves are unchanged — only the column definitions and row values are discarded. The conversion follows this mapping: -- ``sample_sheet`` becomes ``list`` -- ``sample_sheet:paired`` becomes ``list:paired`` -- ``sample_sheet:paired_or_unpaired`` becomes ``list:paired_or_unpaired`` +- ``sample_sheet`` → ``list`` +- ``sample_sheet:paired`` → ``list:paired`` +- ``sample_sheet:paired_or_unpaired`` → ``list:paired_or_unpaired`` -Use this tool when you need to pass a sample sheet to a tool that expects a regular list collection, or when you want to discard the sample sheet metadata. +Use this when you need to pass a sample sheet to a tool that only accepts regular list collections. + +======== +Examples +======== + +**Flat sample sheet** + +Suppose you have an RNA-seq experiment uploaded as a sample sheet collection with metadata about each sample:: + + Sample Sheet: "RNA-seq experiment" + Column definitions: replicate (int), treatment (string) + + sample1.fastq | replicate=1, treatment="control" + sample2.fastq | replicate=2, treatment="treated" + +Running **Convert sample sheet** strips the metadata and produces a plain list:: + + List: "RNA-seq experiment (converted)" + + sample1.fastq + sample2.fastq + +Same datasets, same identifiers, but now it is a regular list that any tool can consume. + +------- + +**Paired sample sheet** + +If your sample sheet contains paired-end reads (``sample_sheet:paired``), the tool converts it to ``list:paired``:: + + Sample Sheet (paired): "WGS samples" + Column definitions: replicate (int) + + sample1 -> [forward, reverse] | replicate=42 + sample2 -> [forward, reverse] | replicate=43 + +After conversion:: + + List (paired): "WGS samples (converted)" + + sample1 -> [forward, reverse] + sample2 -> [forward, reverse] + +The paired structure is preserved; only the metadata is removed. + +------- + +.. image:: ${static_path}/images/tools/collection_ops/convert_sample_sheet.svg + :alt: Convert sample sheet to list collection + :width: 500 + +------- + +.. class:: warningmark + +If you need to preserve the metadata, use the **Sample sheet to tabular** tool first to export it as a tab-separated table before converting. ---- diff --git a/lib/galaxy/tools/cross_product_flat.xml b/lib/galaxy/tools/cross_product_flat.xml index 8653b3ba802..6f31505b439 100644 --- a/lib/galaxy/tools/cross_product_flat.xml +++ b/lib/galaxy/tools/cross_product_flat.xml @@ -1,7 +1,7 @@ - + of two collections into flat lists for all-vs-all comparison model_operation_macros.xml @@ -72,72 +72,72 @@ Synopsis ======== -@CROSS_PRODUCT_INTRO@ +Creates two flat lists that, when processed element-wise by a downstream tool, produce every pairwise combination (all-vs-all) of the two input collections. -==================== -How to use this tool -==================== +=========== +Description +=========== -@GALAXY_DOT_PRODUCT_SEMANTICS@ +This tool takes two list collections and produces two output lists that encode the **Cartesian product** (cross product) of the inputs. Both output lists have the same length (``n * m`` elements, where ``n`` and ``m`` are the sizes of the inputs) and the same element identifiers. -Running input lists through this tool produces new dataset lists (described in detail below) that when using -the same natural element-wise matching "map over" semantics described above produce every combination of the -elements of the two lists compared against each other. Running a tool with these two outputs instead of the initial -two inputs produces a list of the comparison of each combination of pairs from the respective inputs. +When you feed these two outputs into a tool that operates on pairs of datasets, Galaxy's element-wise matching ensures that every combination of one element from Input A and one element from Input B is compared exactly once. -.. image:: ${static_path}/images/tools/collection_ops/flat_crossproduct_output.png - :alt: The Flat Cartesian Product of Two Collections +======== +Examples +======== + +**All-vs-all comparison** + +Given:: + + Input A: [a1] [a2] + Input B: [b1] [b2] + +The tool produces two flat lists of length 4:: + + Output A: [a1_b1 → a1] [a1_b2 → a1] [a2_b1 → a2] [a2_b2 → a2] + Output B: [a1_b1 → b1] [a1_b2 → b2] [a2_b1 → b1] [a2_b2 → b2] + +Feeding both outputs to a comparison tool produces:: + + Result: [a1_b1 → compare(a1,b1)] + [a1_b2 → compare(a1,b2)] + [a2_b1 → compare(a2,b1)] + [a2_b2 → compare(a2,b2)] + +The identifier separator (default ``_``) can be changed to ``:`` or ``-``. + +------- + +.. image:: ${static_path}/images/tools/collection_ops/cross_product_flat.svg + :alt: Flat cross product of two collections + :width: 620 + +------- + +================================== +How element-wise matching works +================================== + +Normally, when you connect two lists to a tool that takes two inputs, Galaxy matches them element-wise (dot product): the first element of list 1 is paired with the first element of list 2, and so on. This requires both lists to have the same identifiers. + +.. image:: ${static_path}/images/tools/collection_ops/dot_product.png + :alt: Element-wise (dot product) matching :width: 500 -The result of running a subsequent tool with the outputs produced by this tool will be a much larger list -whose element identifiers are the concatenation of the combinations of the elements identifiers from the -two input lists. +This tool rearranges the inputs so that the same element-wise matching mechanism produces every combination instead. -.. image:: ${static_path}/images/tools/collection_ops/flat_crossproduct_separator.png - :alt: Flat Cross Product Identifier Separator - :width: 500 +------- -============================================ -What this tool does (technical details) -============================================ +.. class:: infomark -This tool consumes two lists - we will call them ``input_a`` and ``input_b``. If ``input_a`` -has length ``n`` and dataset elements identified as ``a1``, ``a2``, ... ``an`` and ``input_b`` -has length ``m`` and dataset elements identified as ``b1``, ``b2``, ... ``bm``, then this tool -produces a pair of larger lists - each of size ``n*m``. +For a nested (``list:list``) output structure instead of a flat list, use **Nested Cross Product**. -Both output lists will be the same length and contain the same set of element identifiers in the -same order. If the kth input can be described as ``(i-1)*n + (j-1)`` where ``1 <= i <= m`` and ``1 <= j <= n`` -then the element identifier for this kth element is the concatenation of the element identifier for -the ith item of ``input_a`` and the jth item of ``input_b``. - -In the first output list, this kth element will be the ith element of ``input_a``. In the second -output list, the kth element will be the jth element of ``input_b``. - -.. image:: ${static_path}/images/tools/collection_ops/flat_cross_product_outputs.png - :alt: Flat Cross Product Outputs - :width: 500 - -These list structures might appear to be a little odd, but they have the very useful property -that if you match up corresponding elements of the lists the result is each combination of -elements in ``input_a`` and ``input_b`` are matched up once. - -.. image:: ${static_path}/images/tools/collection_ops/flat_cross_product_matched.png - :alt: Flat Cross Product Matching Datasets - :width: 500 - -Running a downstream comparison tool that compares two datasets with these two lists produces a -new list with every combination of comparisons. - -.. image:: ${static_path}/images/tools/collection_ops/flat_cross_product_downstream.png - :alt: Flat Cross Product All-vs-All Result - :width: 500 - ----- +------- .. class:: infomark @QUOTA_USAGE_NOTE@ - ]]> + ]]> diff --git a/lib/galaxy/tools/cross_product_nested.xml b/lib/galaxy/tools/cross_product_nested.xml index 12a52f403b4..f389d9a18ff 100644 --- a/lib/galaxy/tools/cross_product_nested.xml +++ b/lib/galaxy/tools/cross_product_nested.xml @@ -1,7 +1,7 @@ - + of two collections into nested lists for all-vs-all comparison model_operation_macros.xml @@ -76,75 +76,89 @@ Synopsis ======== -@CROSS_PRODUCT_INTRO@ +Creates two nested lists (``list:list``) that, when processed element-wise by a downstream tool, produce every pairwise combination (all-vs-all) of the two input collections, organized by the first input. -==================== -How to use this tool -==================== +=========== +Description +=========== -@GALAXY_DOT_PRODUCT_SEMANTICS@ +This tool takes two flat list collections and produces two ``list:list`` output collections that encode the **Cartesian product** (cross product) of the inputs. The outer list has one element per element of Input A, and each inner list has one element per element of Input B. -Running input lists through this tool produces new list structures (described in detail below) that when using -the same natural element-wise matching "map over" semantics described above produce every combination of the -elements of the two lists compared against each other. Running a tool with these two outputs instead of the initial -two inputs produces a nested list structure where the jth element of the inner list of the ith element of the outer -list is a comparison of the ith element of the first list to the jth element of the second list. -Put more simply, the result is a nested list where the identifiers of an element describe which inputs were -matched to produce the comparison output found at that element. +When you feed these two outputs into a tool that takes two inputs, Galaxy matches elements at each nesting level. The result is a nested list where the outer identifier tells you which element from Input A was used, and the inner identifier tells you which element from Input B was used. -.. image:: ${static_path}/images/tools/collection_ops/nested_crossproduct_output.png - :alt: The Cartesian Product of Two Collections +This is similar to **Flat Cross Product**, but the output preserves structure: results are grouped by the first input, making it easier to see all comparisons for a given element of Input A. + +======== +Examples +======== + +**All-vs-all comparison with nested output** + +Given:: + + Input A: [a1] [a2] + Input B: [b1] [b2] + +The tool produces two ``list:list`` collections:: + + Output A (list:list): + + a1/ + b1 → a1 + b2 → a1 + a2/ + b1 → a2 + b2 → a2 + + Output B (list:list): + + a1/ + b1 → b1 + b2 → b2 + a2/ + b1 → b1 + b2 → b2 + +Feeding both outputs to a comparison tool produces:: + + Result (list:list): + + a1/ + b1 → compare(a1, b1) + b2 → compare(a1, b2) + a2/ + b1 → compare(a2, b1) + b2 → compare(a2, b2) + +------- + +.. image:: ${static_path}/images/tools/collection_ops/cross_product_nested.svg + :alt: Nested cross product of two collections + :width: 620 + +------- + +================================== +How element-wise matching works +================================== + +Normally, when you connect two lists to a tool, Galaxy matches them element-wise (dot product). This tool rearranges the inputs so that the same matching mechanism produces every combination instead, with results grouped into a nested structure. + +.. image:: ${static_path}/images/tools/collection_ops/dot_product.png + :alt: Element-wise (dot product) matching :width: 500 -============================================ -What this tool does (technical details) -============================================ +------- -This tool consumes two flat lists. We will call the input collections ``input_a`` and ``input_b``. If ``input_a`` -has length ``n`` and dataset elements identified as ``a1``, ``a2``, ... ``an`` and ``input_b`` -has length ``m`` and dataset elements identified as ``b1``, ``b2``, ... ``bm``, then this tool -produces a pair of output nested lists (specifically of the ``list:list`` collection type) where -the outer list is of length ``n`` and each inner list has a length of ``m`` (a ``n X m`` nested list). The jth element -inside the outer list's ith element is a pseudo copy of the ith dataset of ``input_a``. One -way to think about the output nested lists is as matrices. Here is a diagram of the first output -showing the element identifiers of the outer and inner lists along with dataset identifiers being -"copied" into this new collection. +.. class:: infomark -.. image:: ${static_path}/images/tools/collection_ops/nested_cross_product_out_1.png - :alt: Nested Cross Product First Output - :width: 500 +For a flat output list instead of nested, use **Flat Cross Product**. -The second output is a nested list of pseudo copies of the elements of ``input_b`` instead of -``input_a``. In particular the outer list is again of length ``n`` and each inner list is again -of length ``m`` but this time the jth element inside the outer list's ith element is a pseudo copy -of the jth dataset of ``input_b``. Here is the matrix of these outputs. - -.. image:: ${static_path}/images/tools/collection_ops/nested_cross_product_out_2.png - :alt: Nested Cross Product Second Output - :width: 500 - -These nested list structures might appear to be a little odd, but they have the very useful property -that if you match up corresponding elements of the nested lists the result is each combination of -elements in ``input_a`` and ``input_b`` are matched up once. The following diagram describes these matching -datasets. - -.. image:: ${static_path}/images/tools/collection_ops/nested_cross_product_matching.png - :alt: Matching Inputs - :width: 500 - -Running a tool that compares two datasets with these two nested lists produces a new nested list -as described above. The following diagram shows the structure of this output and how the element -identifiers are preserved and indicate what comparison was performed. - -.. image:: ${static_path}/images/tools/collection_ops/nested_cross_product_output.png - :alt: Matching Inputs - :width: 500 - ----- +------- .. class:: infomark @QUOTA_USAGE_NOTE@ - ]]> + ]]> diff --git a/lib/galaxy/tools/duplicate_file_to_collection.xml b/lib/galaxy/tools/duplicate_file_to_collection.xml index e52aeb4bbd5..bfdaada2e16 100644 --- a/lib/galaxy/tools/duplicate_file_to_collection.xml +++ b/lib/galaxy/tools/duplicate_file_to_collection.xml @@ -2,8 +2,11 @@ name="Duplicate file to collection" version="1.0.0" tool_type="duplicate_file_to_collection"> - + by duplicating a dataset N times + + model_operation_macros.xml + + + List + + + patientA_treat_r1 + + + patientA_ctrl_r1 + + + patientB_treat_r1 + + + + Rule Builder + + + + Apply rules + split, nest, tag + + + + Nested list (re-organized) + + + + patientA + + + treat_r1 + + + ctrl_r1 + + + + patientB + + + treat_r1 + + split on "_", nest by patient + + diff --git a/static/images/tools/collection_ops/build_list.svg b/static/images/tools/collection_ops/build_list.svg index f05d7265fdb..75f2bac9891 100644 --- a/static/images/tools/collection_ops/build_list.svg +++ b/static/images/tools/collection_ops/build_list.svg @@ -1,2062 +1,163 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + A + + + + Dataset 1 + + + Dataset 2 + + + Dataset 3 + + + + Build list + + + + List + + + Dataset 1 + + + Dataset 2 + + + Dataset 3 + + + + + + + + + + B + + + + List + + + Dataset 1 + + + Dataset 2 + + + + + + + + Dataset A + + + + Build list + + + + Nested list + + + + + Dataset 1 + + Dataset A + + + + + Dataset 2 + + Dataset A + + + each element paired with individual dataset + + + + + + + + + + C + + + + List 1 + + + Dataset 1 + + + Dataset 2 + + + + + + + + List 2 + + + Dataset A + + + Dataset B + + + + Build list + + + + Nested list + + + + + Dataset 1 + + Dataset A + + + + + Dataset 2 + + Dataset B + + + collections matched element-wise + diff --git a/static/images/tools/collection_ops/convert_sample_sheet.svg b/static/images/tools/collection_ops/convert_sample_sheet.svg new file mode 100644 index 00000000000..85fd273895b --- /dev/null +++ b/static/images/tools/collection_ops/convert_sample_sheet.svg @@ -0,0 +1,82 @@ + + + + + + + + + + + + + + + + + + + + + Sample Sheet: "RNA-seq" + + + + + + + Name + rep + treatment + + + + + + sample1 + 1 + control + + + + + + sample2 + 2 + treated + + + + + + sample3 + 3 + control + + + metadata columns + + + + + + + Convert + + + + List: "RNA-seq" + + + + sample1.fastq + + + sample2.fastq + + + sample3.fastq + + + same datasets, no metadata + diff --git a/static/images/tools/collection_ops/cross_product_flat.svg b/static/images/tools/collection_ops/cross_product_flat.svg new file mode 100644 index 00000000000..485ebcc7714 --- /dev/null +++ b/static/images/tools/collection_ops/cross_product_flat.svg @@ -0,0 +1,91 @@ + + + + + + + + + + + + + + + + + + + + + Input A + + + a1 + + + a2 + + + + Input B + + + b1 + + + b2 + + + + Cross + product + + + + Output A + + + a1_b1 → a1 + + + a1_b2 → a1 + + + a2_b1 → a2 + + + a2_b2 → a2 + + + + Output B + + + a1_b1 → b1 + + + a1_b2 → b2 + + + a2_b1 → b1 + + + a2_b2 → b2 + + + Match element-wise + to get all-vs-all + + + + + + + + a1_b1: compare(a1,b1) + a1_b2: compare(a1,b2) + a2_b1: compare(a2,b1) + a2_b2: compare(a2,b2) + + diff --git a/static/images/tools/collection_ops/cross_product_nested.svg b/static/images/tools/collection_ops/cross_product_nested.svg new file mode 100644 index 00000000000..bed7697efa6 --- /dev/null +++ b/static/images/tools/collection_ops/cross_product_nested.svg @@ -0,0 +1,102 @@ + + + + + + + + + + + + + + + + + + + + + Input A + + + a1 + + + a2 + + + + Input B + + + b1 + + + b2 + + + + Cross + product + + + + Output A (list:list) + + + + a1 + + + b1 → a1 + + + b2 → a1 + + + + a2 + + + b1 → a2 + + + b2 → a2 + + + + Output B (list:list) + + + + a1 + + + b1 → b1 + + + b2 → b2 + + + + a2 + + + b1 → b1 + + + b2 → b2 + + + Results grouped + by Input A element + + + a1/b1: compare(a1,b1) + a1/b2: compare(a1,b2) + a2/b1: compare(a2,b1) + a2/b2: compare(a2,b2) + + diff --git a/static/images/tools/collection_ops/duplicate_file_to_collection.svg b/static/images/tools/collection_ops/duplicate_file_to_collection.svg new file mode 100644 index 00000000000..243d2bb9f70 --- /dev/null +++ b/static/images/tools/collection_ops/duplicate_file_to_collection.svg @@ -0,0 +1,47 @@ + + + + + + + + + + + + + + + + + + + + + reference.fa + + + N = 3, identifier = "ref" + + + + Duplicate + + + + List + + + + ref 1 + + + ref 2 + + + ref 3 + + + all copies are identical + + diff --git a/static/images/tools/collection_ops/extract_dataset.svg b/static/images/tools/collection_ops/extract_dataset.svg new file mode 100644 index 00000000000..f0d1e776c8a --- /dev/null +++ b/static/images/tools/collection_ops/extract_dataset.svg @@ -0,0 +1,124 @@ + + + + + + + + + + + + + + + + + + + + + + + + A + + + + List + + + 0 + control + + + + 1 + treated + + + 2 + reference + + identifier = "treated" + + + + Extract + + + + treated + + standalone dataset in history + + + + + + + + + + B + + + + Nested list + + + + sampleA + + + 0 + Dataset 1 + + + + 1 + Dataset 2 + + + 2 + Dataset 3 + + + + sampleB + + + 0 + Dataset A + + + + 1 + Dataset B + + + 2 + Dataset C + + + index = 1 + + + + Extract + + + + List + + + Dataset 2 + sampleA + + + Dataset B + sampleB + + one nesting level removed + + diff --git a/static/images/tools/collection_ops/filter_empty.svg b/static/images/tools/collection_ops/filter_empty.svg index 6e4be27b030..3d4b8dbcb05 100644 --- a/static/images/tools/collection_ops/filter_empty.svg +++ b/static/images/tools/collection_ops/filter_empty.svg @@ -1,542 +1,54 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + List + + + sample1 + 1.2 MB + + + + sample2 + 0 bytes + + + sample3 + 800 KB + + sample2 is empty + + + + Filter + + + + List (filtered) + + + sample1 + + + sample3 + + empty element removed + diff --git a/static/images/tools/collection_ops/filter_error.svg b/static/images/tools/collection_ops/filter_error.svg index fb4d01ad199..114dbaed524 100644 --- a/static/images/tools/collection_ops/filter_error.svg +++ b/static/images/tools/collection_ops/filter_error.svg @@ -1,927 +1,54 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + List + + + sample1 + ok + + + + sample2 + error + + + sample3 + ok + + sample2 failed + + + + Filter + + + + List (filtered) + + + sample1 + + + sample3 + + failed element removed + diff --git a/static/images/tools/collection_ops/filter_from_file.svg b/static/images/tools/collection_ops/filter_from_file.svg new file mode 100644 index 00000000000..128244ec61e --- /dev/null +++ b/static/images/tools/collection_ops/filter_from_file.svg @@ -0,0 +1,130 @@ + + + + + + + + + + + + + + + + + + + + + + + + A + + + + List + + + Dataset A + + + Dataset B + + + Dataset X + + + + identifiers.txt + A + B + Z + + + remove if ABSENT + + + + Filter + + + + (filtered) + + + Dataset A + + + Dataset B + + + + (discarded) + + + Dataset X + + + Z has no match — ignored + + + + + + + + + + B + + + + List + + + Dataset A + + + Dataset B + + + Dataset X + + + + identifiers.txt + A + B + Z + + + remove if PRESENT + + + + Filter + + + + (filtered) + + + Dataset X + + + + (discarded) + + + Dataset A + + + Dataset B + + + Z has no match — ignored + + diff --git a/static/images/tools/collection_ops/filter_null.svg b/static/images/tools/collection_ops/filter_null.svg new file mode 100644 index 00000000000..aa63f945f21 --- /dev/null +++ b/static/images/tools/collection_ops/filter_null.svg @@ -0,0 +1,54 @@ + + + + + + + + + + + + + + + + + + + + + List + + + sample1 + ok + + + + sample2 + null + + + sample3 + ok + + sample2 is null + + + + Filter + + + + List (filtered) + + + sample1 + + + sample3 + + null element removed + + diff --git a/static/images/tools/collection_ops/flatten.svg b/static/images/tools/collection_ops/flatten.svg index 664dff4f552..0d4b5de0ada 100644 --- a/static/images/tools/collection_ops/flatten.svg +++ b/static/images/tools/collection_ops/flatten.svg @@ -1,978 +1,66 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + list : paired + + + + sample1 + + + forward + + + reverse + + + + sample2 + + + forward + + + reverse + + + + Flatten + join with "_" + + + + List (flattened) + + + sample1_forward + + + sample1_reverse + + + sample2_forward + + + sample2_reverse + + hierarchy removed, identifiers joined + diff --git a/static/images/tools/collection_ops/harmonize.svg b/static/images/tools/collection_ops/harmonize.svg new file mode 100644 index 00000000000..0f5724eec23 --- /dev/null +++ b/static/images/tools/collection_ops/harmonize.svg @@ -0,0 +1,75 @@ + + + + + + + + + + + + + + + + + + + + + Collection 1 + + + sampleA + + + sampleB + only here + + + sampleC + + + + Collection 2 + + + sampleD + only here + + + sampleC + + + sampleA + + + + Harmonize + keep shared, match order + + + + Output 1 + + + sampleA + + + sampleC + + + + Output 2 + + + sampleA + + + sampleC + + + same identifiers, same order + + diff --git a/static/images/tools/collection_ops/keep_success.svg b/static/images/tools/collection_ops/keep_success.svg new file mode 100644 index 00000000000..c5c4b9466fc --- /dev/null +++ b/static/images/tools/collection_ops/keep_success.svg @@ -0,0 +1,60 @@ + + + + + + + + + + + + + + + + + + + + + List + + + sample1 + ok + + + + sample2 + error + + + sample3 + ok + + + + sample4 + paused + + only ok elements kept + + + + Keep + success only + + + + List (success only) + + + sample1 + + + sample3 + + failed and paused removed + + diff --git a/static/images/tools/collection_ops/merge.svg b/static/images/tools/collection_ops/merge.svg new file mode 100644 index 00000000000..30a14be3b6a --- /dev/null +++ b/static/images/tools/collection_ops/merge.svg @@ -0,0 +1,61 @@ + + + + + + + + + + + + + + + + + + + + + Collection 1 + + + sampleA + + + sampleB + + + + Collection 2 + + + sampleC + + + sampleD + + + + Merge + + + + Merged collection + + + sampleA + + + sampleB + + + sampleC + + + sampleD + + elements concatenated in order + + diff --git a/static/images/tools/collection_ops/nest.svg b/static/images/tools/collection_ops/nest.svg index 4f98aac6a97..1edc7f73fe9 100644 --- a/static/images/tools/collection_ops/nest.svg +++ b/static/images/tools/collection_ops/nest.svg @@ -1,963 +1,63 @@ - - - - - + + + + + + + + + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ... - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ... - + + + + List + + + sample1 + + + sample2 + + + sample3 + + + + Nest + + + + Nested list + + + + sample1 + + + sample1 + + + + sample2 + + + sample2 + + + + sample3 + + + sample3 + + each element wrapped in its own list + diff --git a/static/images/tools/collection_ops/relabel.svg b/static/images/tools/collection_ops/relabel.svg new file mode 100644 index 00000000000..bdec46fa689 --- /dev/null +++ b/static/images/tools/collection_ops/relabel.svg @@ -0,0 +1,55 @@ + + + + + + + + + + + + + + + + + + + + + List + + + sampleA + + + sampleB + + + sampleC + + + + sampleA Alpha + sampleB Beta + sampleC Gamma + + + + Relabel + + + + List (relabelled) + + + Alpha + + + Beta + + + Gamma + + diff --git a/static/images/tools/collection_ops/sample_sheet_to_tabular.svg b/static/images/tools/collection_ops/sample_sheet_to_tabular.svg new file mode 100644 index 00000000000..2e10fa6d3be --- /dev/null +++ b/static/images/tools/collection_ops/sample_sheet_to_tabular.svg @@ -0,0 +1,105 @@ + + + + + + + + + + + + + + + + + + + + + Sample Sheet: "RNA-seq" + + + + + + + Name + rep + treatment + + + + + + sample1 + 1 + control + + + + + + sample2 + 2 + treated + + + + + + sample3 + 3 + control + + + metadata columns + + + + + + + Extract + metadata + + + + Tabular dataset + + + + + + element_id + rep + treatment + + + + + + sample1 + 1 + control + + + + + + sample2 + 2 + treated + + + + + + sample3 + 3 + control + + + tab-separated metadata export + diff --git a/static/images/tools/collection_ops/sort.svg b/static/images/tools/collection_ops/sort.svg new file mode 100644 index 00000000000..e189ee649b6 --- /dev/null +++ b/static/images/tools/collection_ops/sort.svg @@ -0,0 +1,54 @@ + + + + + + + + + + + + + + + + + + + + + List + + + sampleC + + + sampleA + + + sampleB + + unsorted + + + + Sort + alphabetical + + + + List (sorted) + + + sampleA + + + sampleB + + + sampleC + + A, B, C order + + diff --git a/static/images/tools/collection_ops/split_paired_unpaired.svg b/static/images/tools/collection_ops/split_paired_unpaired.svg new file mode 100644 index 00000000000..63de001f9ea --- /dev/null +++ b/static/images/tools/collection_ops/split_paired_unpaired.svg @@ -0,0 +1,70 @@ + + + + + + + + + + + + + + + + + + + + + list : paired_or_unpaired + + + + sample1 (paired) + + + forward + + + reverse + + + + sample2 (unpaired) + + + + sample3 (unpaired) + + mixed paired + unpaired + + + + Split + + + + Paired output + + + sample1 + + + forward + + + reverse + + + + Unpaired output + + + sample2 + + + sample3 + + diff --git a/static/images/tools/collection_ops/tag_from_file.svg b/static/images/tools/collection_ops/tag_from_file.svg new file mode 100644 index 00000000000..6d14379b82a --- /dev/null +++ b/static/images/tools/collection_ops/tag_from_file.svg @@ -0,0 +1,61 @@ + + + + + + + + + + + + + + + + + + + + + List + + + sample1 + + + sample2 + + + sample3 + + + + sample1 group:treat + sample2 group:ctrl + sample3 group:treat + + + + Tag + + + + List (tagged) + + + sample1 + + treat + + + sample2 + + ctrl + + + sample3 + + treat + + diff --git a/static/images/tools/collection_ops/unzip.svg b/static/images/tools/collection_ops/unzip.svg index 25047d07b37..6f93499de90 100644 --- a/static/images/tools/collection_ops/unzip.svg +++ b/static/images/tools/collection_ops/unzip.svg @@ -1,1081 +1,85 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + Paired collection + + + + sample1 + + + forward + + + reverse + + + + sample2 + + + forward + + + reverse + + + + sample3 + + + forward + + + reverse + + 3 paired elements + + + + Unzip + + + + Forward datasets + + + sample1_R1.fastq + + + sample2_R1.fastq + + + sample3_R1.fastq + + + + Reverse datasets + + + sample1_R2.fastq + + + sample2_R2.fastq + + + sample3_R2.fastq + diff --git a/static/images/tools/collection_ops/zip.svg b/static/images/tools/collection_ops/zip.svg index 1b037278606..b2a49b8fdc1 100644 --- a/static/images/tools/collection_ops/zip.svg +++ b/static/images/tools/collection_ops/zip.svg @@ -1,1081 +1,85 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + Forward datasets + + + sample1_R1.fastq + + + sample2_R1.fastq + + + sample3_R1.fastq + + + + Reverse datasets + + + sample1_R2.fastq + + + sample2_R2.fastq + + + sample3_R2.fastq + + + + Zip + + + + Paired collection + + + + sample1 + + + forward + + + reverse + + + + sample2 + + + forward + + + reverse + + + + sample3 + + + forward + + + reverse + + matched by position +