Like the reductions - was previously constrained by sequeezing these values into simple strings - now the tool form will target the API I think this expanded version is a little more straight-forward (though verbose). Adds consistency with rest of the tool form API changes.
Old tool form needed to encode every value as a string so I had done "__collection_reduction__|<hdca_id>" to distinguish that value from an "<hda_id>" - since hdca and hdas can have the same encoded ids. The new tool form API is going to use the API which allows for richer object representations - so {"src": "hda", "id": "<hda_id>"} versus {"src": "hdca", "id": "<hdca_id>"} should be enough to distinguish between passing an HDA and an HDCA to a multiple input data parameter.
Adding consistency allowing each parameter to be wrapped in a object describing the meta-properties of the submitting value - this was requested by Sam to make the new tool form easier to manage, it makes multi-running properties work for non-data parameters, and allows linked/unlinked specification of parameters.
(Lot easier now that I understand what all of the module methods are doing and have an example of a 4th module downstream.) Now with even more unit tests.
Including same test with workflow parameter substition - gotta admit I thought that workflow test was going to fail - so this week is looking pretty good :).
There may now be multiple WorkflowInvocationSteps for each WorkflowStep for steps that are mapped over collections - so to_dict creating a dictionary of this information indexed on order step is problematic because only one WorkflowInvocationStep will be represented per step. Instead now just returning a big list of all of the invocations - which contains all of the same information. This is a backward incompatible API change for the workflow invocation API.
Also update the input mapping stuff with logic for dealing with data collection inputs.
Sharing a workflow with a user was previously sufficient to grant access to all invocations of that workflow. This isn't a huge problem since the information potentially leaking out was limitted to invocation counts, various encoded ids, and update times. Still I think no information about invocations should be avaialble as a result of sharing a workflow - and upcoming changes to Galaxy will result in much more information being made available via the workflow invocation API.
Add new example based on Anton's bamtools split work and label all the outputs as visible - this should likely be the default but until it is might as well demonstrate them as visible since that is how they are most useful.
It was stubbing out connection stuff which worked fine when running test on its own - but after any test sets up the sql alchemy mappings this breaks down because events are trying to fire. This just uses actual model classes - which is just as easy anyway really.
With refactoring to reduce cyclomatic complexity. Also renaming 'input_ext' what it actually is 'random_input_ext'. We should fix that or at least issue a huge warning if we detect 'input' could have reasonable been different things.
Directly setting format attribute, setting to format to 'input' (ambigious, non-deterministic and should be deprecated IMO), using format_source, and using change_format actions.
Add data labels so the tools works in the workflow editor and added a conditional switches to some with collection params and multiple input data parameters to test some state-y logic in workflow editor.
Detail bug report from Michael Crusoe here : https://trello.com/c/0mdGCx4P.
This also fixes a test case added in 289e48b which was both attempting to assert something wrong and was incorrectly implemented. Augmenting the workflow editor test suite with some actually valid test cases that assert the correct behaviors.
For a longer explaination - workflow input terminals have two related concepts 'canAccept' and 'attachable'. An output terminal is 'attachable' if in the abstract it could be attached to the input regardless of whether the input is already filled or not. 'canAccept' is more stateful in that an output terminal is 'canAccept'able if the input terminal is not filled ('_inputFilled') and it is 'attachable'.
So - the problem was 'attachable' was not correctly defined for a multiple input data parameters. 'attachable' was asserting that any connected input terminal could not be 'attachable' by an collection output terminal - so on these asynchronous node state changes collections attached to multiple input data parameters were being wiped out. The more percise/correct distinction is that if a multiple input data parameter has single inputs connected to it - it cannot also have a collection connected to it (yet anyway). This fixes the mentioned bug.
The problematic test case was conflating 'attachable' and 'canAccept'able - I have fixed the test case to verify the correct 'attachable' logic and added newer, higher level test cases to test the canAccept logic and the actual behavior the end user would observe (of the connector being destroy).
Dave Bouvier