Rather than depend on something in ``functional`` test module from ``base.driver_util``, just set these global hacks up in ``driver_util`` itself. This eliminates a dependency of ``functional`` on ``base`` - ideally ``base`` would not depend on any other test modules.
- Add an API test for datatype-defined composite uploads - including exercising newline conversion and the space_to_tab parameter.
- Add a test decorator skip_without_datatype to mirror skip_without_tool for this test, improve both decorators.
- The ftype parameter in the composite test tools does nothing - drop it and drop it from the XSD spec.
- Slightly improve the documentation for these composite_data elements in the XSD.
In https://github.com/galaxyproject/galaxy/pull/4254 `vcf_bgzip` was made
a proper datatype with the introduction of the `VcfGz` class, but was
still described as a subclass in `datatypes_conf.xml` .
Also fix `file_ext` attribute in `BaseVcf` and subclasses.
Allow override by specifying "provided_metadata_style=" attribute on outputs in tool XML - this should allow newer profile tools to use the older style JSON providing a clear path for upgrades.
without needing to specify a discovered dataset pattern. If the ``discovered_datasets`` tag includes ``from_tool_provided_metadata="true"`` the datsets listed in galaxy.json will just be used directly without needing to be "discovered" using a dataset pattern. Metadata and such can continue to be specified either in that file or on the discovered_dataset element except things like pattern (which is not used) and sort_by (since the json should describe the order I suppose).
- Add another test to demonstrate referencing entries in galaxy.json for datasets by dataset path basename instead of ID.
- Add a bit more documentation to test tools.
- Redo the structure of galaxy.json for profile >= 17.09 tools with two new test tools to demonstrate the new structure - for both output dataset metadata and discovered dataset metadata.
- Improved abstractions for interaction between tools and jobs to enable different tool provided metadata collection depending on tool profile version.
Without this fix, the Cheetah expression:
$dataset.is_of_type('unknown_ext')
in a tool command would be equivalent to:
$dataset.is_of_type('txt')
meaning that if the dataset datatype is a subclass of Text, the expression
would evaluate to True without any warning.
xref. https://github.com/galaxyproject/tools-iuc/issues/1373
Also add missing `xml` datatype to
`test/functional/tools/sample_datatypes_conf.xml` which is needed by 3 test
tools.
Without this fix, the Cheetah expression:
$dataset.is_of_type('unknown_ext')
in a tool command would be equivalent to:
$dataset.is_of_type('txt')
meaning that if the dataset datatype is a subclass of Text, the expression
would evaluate to True without any warning.
xref. https://github.com/galaxyproject/tools-iuc/issues/1373
Also add missing `xml` datatype to
`test/functional/tools/sample_datatypes_conf.xml` which is needed by 3 test
tools.
xref #4032
These at least verify that jobs are properly failing whether they use dynamic dataset collection or not. In the case of pre-determinable list structures - they properly go red and expose the problems to the user (try running test/functional/tools/collection_creates_list_fail.xml manually). In the case of dynamic dataset collections - this does not occur though - all the resulting datasets are indeed green.
I think it needs to be touched up but the basic operation seems to work so far. I think what remains to be done is:
- Validate uniqueness of identifiers and provide nice messages if they are not unique.
- Validate that at least the required number of lines are present in the file and provide a nice message if not.
- Add strict mode to ensure exactly the correct number of lines is added.
- Find where validation of identifiers happens in the API and apply same validation here - try not to let unsafe identifiers be created.
- Consider more advanced modes - selecting a column, apply a regex replace, pick two columns for nested lists, etc.... None of this may need to be needed in the first iteration.
- Consider another mode where a collection is labelled against an existing collection - should that be a separate tool of the same tool.
For instance, fixes#3859 restoring the correct ``element_identifier`` for reduces collections in conditionals. Add tests for combinations of repeats and conditionals.
There are two basic behavior changes here and bunch of testing to verify the new behaviors.
1) Previously purging a dataset would cause a creating job to be cancelled but all outputs would need to be "deleted" in order to cause the creating job to be cancelled. After this change the behavior is the same for both deletion and purging - the job will run until all outputs are deleted or purged.
2) Previously Galaxy jobs wouldn't attempt to cleanup purged datasets that were created by a running tool - this cleanup now occurs.
These are tackled as a pair because I assume the difference in the behaviors between deletion and purging was a hack around tools populating datasets that had been purged while running. Cancelling the job wasn't a sure fix - but it would reduce the likelihood of such problems. I think the new approach is a bit more robust and explicit.
Conflicts:
client/galaxy/scripts/mvc/tool/tool-form-composite.js
preserved the dev version
static/maps/mvc/tool/tool-form-composite.js.map
static/scripts/bundled/analysis.bundled.js
static/scripts/bundled/analysis.bundled.js.map
static/scripts/bundled/libs.bundled.js.map
static/scripts/mvc/tool/tool-form-composite.js
Incorporate package extraction fixes thanks to @nsoranzo, the fix for version 4.1 and 4.0 thanks to @mvdbeek, a comment for @bgruening and a new test tool that makes it a bit easier to test this.
Martin Cech