There are some repeated operations on an odict in Toolbox that could be cleaned up (fewer lines of code, more readable) with new abstraction in place. Starting with __add_tool_to_tool_panel but will work its way to other methods in Toolbox eventually.
Abstraction reduces cyclomatic complexity of Toolbox.__add_tool_to_tool_panel from 23 down to 18.
How to use:
1.) Place multiple tools with different IDs in your tool conf.
2.) ... ummm ... no step 2 - just use the tools.
Implementation:
The Tool Shed allows tool lineages by assigning each tool version a GUID and tracking versions in a database. This
implementation works by simply allowing the ToolBox to contain multiple tools with the same ID and orders them by the version specified by the tool author.
To track enable this a second tool lineage has been introduced that just uses tool versions instead of a database (non-toolshed installed tools are not longer placed into the Tool Shed install database). The ToolBox has been updated to allow multiple versions per tool id (defaulting to the 'latest' version for all operations which do not specify a version). Both jobs and workflow steps would track tool versions but did not use that version when fetching tools from the Toolbox - these components have been updated to try to use the tool version.
Unit tests working through most of the ToolBox and tool panel have been added, as well as functional tests exercising the tools API and to ensure workflows now at least attempt to respect tool versions (still kind of silently switches versions in some cases). Manual tests against the new tool form seem to demonstrate the tool switching and tool re-running work with only minor changes to the tools API and the job handler.
Leave empty module in galaxy.tools.filters and modify config to ensure backward compatibility (filters in this old directory will continue to work for now).
Update config/galaxy.ini.sample with more discussion of ToolBox filter.
- Allow overriding the base module location for ToolBox filters (needed for OS packages, etc...).
- Allow scanning multiple base modules for filters.
- Unit tests for module loading functionality, custom tool, label, and section filters, default hidden and require_login filters.
Using ToolBox._xxx instead of ToolBox.__xxx because realistically ToolBox is still much to large to grok and so I imagine it will need to be broken up even more (base class focused on just the panel details perhaps - or mixins - etc...).
Bug originally reported by Eric here - http://dev.list.galaxyproject.org/Bug-Toolbox-filters-not-applied-in-workflows-td4662879.html (though this only fixes the superficial bug of them showing up in the workflow editor - and does not yet implement "strong" toolbox filters as requested).
This is actually a broader change that centralizes logic for listing out tool panel contents when displaying components to the user and the added filtering comes about from that. It also eliminates the last places Galaxy was accessing toolbox.tool_panel directly (in those mako templates).
... to shield ToolBox.shed_tool_confs from other components including Tool Shed install models, shed_util_common.py, and install_manager.py.
After this commit - shed_tool_confs is only accessed directly by the ToolBox itself.
None of the components using this method are using the index any more - so eliminate it. Also introduce a new ToolBox method to hide the details of ToolBox.shed_tool_confs from this method.
... in other words to note where index in the resulting tuple is not actually used. Doing this to enable hiding this implementation detail (index) from components in subsequent commits.
May seem sort of like Tool Shed specific stuff - but I think it can be reused elsewhere down the road. Regardless of this however as of this commit no external components directly access ToolBox.tool_panel or ToolBox.integrated_tool_panel - which I think is important - these should be internal details of the ToolBox (or newer modules it leverages).
Start a gradual process of referring to these as dynamic tool confs instead of shed tool confs. I would like like to make them more generally Galaxy-managed tool configurations and distinguish them from deployer-managed configurations - rather then having overtly ToolShed-only logic in ToolBox.
Create one public method load_item on ToolBox that should be used externally instead of load_tool_tag_set, load_section_tag_set, etc.... This reduces the duplication within the toolbox and between the toolbox and the Tool Shed's ToolPanelManager. Probably more importantly it also is another step down the road toward hiding implementation details such as toolbox.tool_panel and toolbox.intergrated_tool_panel from the ToolPanelManager.
Already had unit test coverage for ToolPanelManager stuff here - but added test coverage for loading labels, workflows, and tool directories.
... objects (directly anyway). I would like to refactor all the tool shed components to not deal directly with ToolSection objects but instead just pass around IDs.
Instead of reloading upload actions at this level and exposing tools_by_id - move this logic into the ToolBox in a new method called handle_datatypes_changed().
Big conditional in remove_guids switched on sectioned versus un-sectioned with very little difference in what was actually being done. This reduces the cyclomatic complexity of that method from 35 down to 26 (though it is still the most 'complex' method in that file by a wide margin).
John Chilton