This was an oversight in https://github.com/galaxyproject/galaxy/pull/7745.
This is the quick fix, but I think it's feasible to implement the
accessible logic as a query, which means we could push this down to
a Mixin, and treat it as a regular fiter (so admins could for instance
search/sort for the largest datasets, which is already possible in the
reports app).
In practice we use ParsedFiler.filter_type to discriminate
the different filter types. Filters that require the correct
model class for a specific query are now functions that take
a model_class and generate the correct filter.
Report if empty inputs were used or if the same input was used more than once, these were identified as common sources of potential errors across many tools. Neither of these are definite problems, but both could be depending on the tool and both happen frequently. Outline a few more ideas as TODOs in the code.
If the user is not an admin.
Fixes https://github.com/galaxyproject/galaxy/issues/7710
An alternative would be to skip Accessibility / Ownership
exceptions, but given that we can just restrict the query
this seems like the better approach to me.
The parent dataset collection now shows the failed datasets,
and that is much easier to deal with when all jobs within
a large collection have failed.
Fixes https://github.com/galaxyproject/galaxy/issues/7492
One downside here is that we don't see anything in the history right
away (except for the failed dataset count) if a hidden collection
contains failed datasets.
- Allow import and export collections.
- Fix job I/O for separated tool and job script streams (merged in #7095 during 19.05)
- Fix job state handling so that all imported jobs have terminal state.
- Fixes to eliminate HID handling for associations (objects were previously tracked by HID but inputs/outputs in history may not be in history - so HID is not at all appropriate, problem was amplified by collections).
- Fix job output mapping - was using the HDA name instead of job output name during import (old exports don't have this info - only new ones do).
- Fixes assigning HIDs to datasets during import and fixes to hid_counter - in many ways this was very broken for imported histories.
- Introduce cohesive model object import/export abstractions including a lot of new, well tested functionality.
- Allow import/export of datasets and collections stand-alone regardless of history.
- Allow export and re-import of datasets and collections (so for instance datasets can be created for a job and read in at the end of a job and merged back into the database). This is needed for extended metadata collection.
- Separate implementation for import abstraction that allow import of legacy archives without complicating logic for current archive format.
- Implementation of import and export abstraction for [BagIt](https://datatracker.ietf.org/doc/rfc8493/) using [bdbag](https://github.com/fair-research/bdbag).
- Context manager support and clean APIs for all of this including import and export of directories, archives, and bags.
- A lot more unit testing.
self.dataset_populator._get_contents_request(history_id=history_id).json()
returns the dataset collection as the first element.
We do need a HDA here though, so we'd just randomly pick
a HDA with the HDCA ID. So if that dataset was deleted by chance the
test would fail. Ouch.
PR #6925 introduced a GUI for connecting non-data (e.g. integer, boolean, color, etc..) workflow input parameter to tool input parameters (the backend for this was originally added in #1306). That ideally was just the beginning of work toward using such values in structured ways in workflows.
This PR extends tool output handling to allow producing of non-data parameters. These can serve as a source for non-data values in workflows the same work workflow input parameters can. To make such values more easy to produce, this PR also introduces Galaxy expression tools - mirroring functionality regularly used in CWL. These small JavaScript-based tools that consume inputs just like a regular Galaxy tool but that produce dictionary of non-data values.
I think these expressions will be maximally useful when paired with format 2 workflows once we allow users to load arbitrary tools (I make the case more in full here https://github.com/galaxyproject/galaxy/pull/7545#issuecomment-473894424), but I outline some potential uses there as well.
Because there is always a checklist in my PR descriptions:
- Tool definition language and plumbing and datatype for expressing expressions as jobs.
- Allow connecting expression tools to parameters in workflows, will delay evaluation of workflow so calculated value
- Example test expression tools for testing and demonstration.
Test case for indexing dynamic tools and deactivating dynamic tools. Verify deactivation via attributes in resulting object but also by trying to run a deactivated dynamic tool and getting a 404.
- Introduce models and a API for creating tools dynamically.
- Use Galaxy's testing-only YAML based representation of tools to prototype this.
- Extend Format 2 workflow definitions to allow embedding tools directly into workflows, either directly or using a CWL-style @import syntax.
Testing:
Test cases demonstrating tools can be imported (only by admins) and are runnable are included with this commit. More test cases regarding workflow use of dynamic tools and Format 2 workflow definition extensions are also included.
These tests can be run with the following commands:
```
./run_tests.sh -api test/api/test_tools.py:ToolsTestCase.test_nonadmin_users_cannot_create_tools
./run_tests.sh -api test/api/test_tools.py:ToolsTestCase.test_dynamic_tool_1
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_import_export_dynamic
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_workflow_embed_tool
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_workflow_import_tool
```
Nate Coraor