If the user is not an admin.
Fixes https://github.com/galaxyproject/galaxy/issues/7710
An alternative would be to skip Accessibility / Ownership
exceptions, but given that we can just restrict the query
this seems like the better approach to me.
self.dataset_populator._get_contents_request(history_id=history_id).json()
returns the dataset collection as the first element.
We do need a HDA here though, so we'd just randomly pick
a HDA with the HDCA ID. So if that dataset was deleted by chance the
test would fail. Ouch.
PR #6925 introduced a GUI for connecting non-data (e.g. integer, boolean, color, etc..) workflow input parameter to tool input parameters (the backend for this was originally added in #1306). That ideally was just the beginning of work toward using such values in structured ways in workflows.
This PR extends tool output handling to allow producing of non-data parameters. These can serve as a source for non-data values in workflows the same work workflow input parameters can. To make such values more easy to produce, this PR also introduces Galaxy expression tools - mirroring functionality regularly used in CWL. These small JavaScript-based tools that consume inputs just like a regular Galaxy tool but that produce dictionary of non-data values.
I think these expressions will be maximally useful when paired with format 2 workflows once we allow users to load arbitrary tools (I make the case more in full here https://github.com/galaxyproject/galaxy/pull/7545#issuecomment-473894424), but I outline some potential uses there as well.
Because there is always a checklist in my PR descriptions:
- Tool definition language and plumbing and datatype for expressing expressions as jobs.
- Allow connecting expression tools to parameters in workflows, will delay evaluation of workflow so calculated value
- Example test expression tools for testing and demonstration.
- Introduce models and a API for creating tools dynamically.
- Use Galaxy's testing-only YAML based representation of tools to prototype this.
- Extend Format 2 workflow definitions to allow embedding tools directly into workflows, either directly or using a CWL-style @import syntax.
Testing:
Test cases demonstrating tools can be imported (only by admins) and are runnable are included with this commit. More test cases regarding workflow use of dynamic tools and Format 2 workflow definition extensions are also included.
These tests can be run with the following commands:
```
./run_tests.sh -api test/api/test_tools.py:ToolsTestCase.test_nonadmin_users_cannot_create_tools
./run_tests.sh -api test/api/test_tools.py:ToolsTestCase.test_dynamic_tool_1
./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_import_export_dynamic
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_workflow_embed_tool
./run_tests.sh -api test/api/test_workflows_from_yaml.py:WorkflowsFromYamlApiTestCase.test_workflow_import_tool
```
- Allow workflows to be uploaded via path using the GUI and and API for admins.
- Track the path and resync workflows on save (both .ga and format 2 workflows) - this will allow Planemo to leverage the Galaxy workflow editor to interactively save workflows.
We don't track workflow step inputs in any formal way in our model currently. This has resulted in some current hacks and prevents future enhancements. This commit splits WorkflowStepConnection into two models WorkflowStepInput and WorkflowStepConnection - normalizing the previous table workflow_step_connection on input step and input name.
In terms of current hacks forced on it by restricting all of tool state to be confined to a big JSON blob in the database - we have problems distinguishing keys and values when walking tool state. As we store more and more JSON blobs inside of the giant tool state blob - the worse this problem gets. Take for instance checking for runtime parameters or the rules parameter values - these both use JSON blobs that aren't simple values, so it is hard to tell looking at the tool state blob in the database or the workflow export to tell what is a key or what is a value. Tracking state as normalized inputs with default values and explicit attributes runtime values should allow much more percise state definition and construction.
This variant of the models would also potentially allow defining runtime values with non-tool default values (so default values defined for the workflow but still explicitly settable at runtime). The combinations of overriding defaults and defining runtime values were not representable before.
In terms of future enhancements, there is a lot we cannot track with the current models - such as map/reduce options for collection operations (https://github.com/galaxyproject/galaxy/issues/4623#issuecomment-389544980). This should enable a lot of that. Obviously there are a lot of attributes defined here that are not yet utilized, but I'm using most (all?) of them downstream in the CWL branch. I'd rather populate this table fully realized and fill in the implementation around it as work continues to stream in from the CWL branch - to keep things simple and avoid extra database migrations. But I understand if this feels like speculative complexity we want to avoid despite the implementation being readily available for inspection downstream.
Don't treat data and non-data inputs/outputs different at the API boundary or at the workflow-manager.js level. Keep differences at the level of walking inputs/outputs in the workflow module on the backend and at the terminal definition (models and views) level on the frontend. Make the code and interfaces more consistent in my opinion and is more compatible with future directions I'd like to take things with mix-mode inputs/outputs for CWL.
mvdbeek