Given a collection with element_identifiers:
```yml
element_1
element_2
```
we can arbitrarily re-order collection elements using a dataset
that specifies the new order, e.g
```
element_2
element_1
```
I think this nicely complements numeric and alphanumeric sorts that are
already possible with this tool.
It gets added anyway when the file is uploaded to Galaxy due to the
default "Use POSIX standard" (to_posix_lines) option. Many tests which
simply copy them with cat and compare them would fail a strict diff test.
- Better error handling (check for bad characters when creating collections).
- Implement a strict mode parameter to do even more validation.
- Rework tabular vs txt mode to be explicit user choice.
- Mirror fix in release_17.01 for datasets not having a history,
I think it needs to be touched up but the basic operation seems to work so far. I think what remains to be done is:
- Validate uniqueness of identifiers and provide nice messages if they are not unique.
- Validate that at least the required number of lines are present in the file and provide a nice message if not.
- Add strict mode to ensure exactly the correct number of lines is added.
- Find where validation of identifiers happens in the API and apply same validation here - try not to let unsafe identifiers be created.
- Consider more advanced modes - selecting a column, apply a regex replace, pick two columns for nested lists, etc.... None of this may need to be needed in the first iteration.
- Consider another mode where a collection is labelled against an existing collection - should that be a separate tool of the same tool.
Having a BaseFastq class allows Fastq and FastqGz classes to inherit from
BaseFastq. FastqGz is therfor not a Fastq datatype and goes through the
converter.
Also add 2 tools to demonstrate that compressed fastq will be converted to
uncompressed fastq if the tool specifies `format="fastq"`, while
`format="fastq.gz"` leaves files compressed.
The tool test can be run with:
```
planemo test --galaxy_root . test/functional/tools/compressed_fastq_no_conversion.xml
planemo test --galaxy_root . test/functional/tools/compressed_fastq_conversion.xml
```
Note that fastq conversion appears to required a galaxy user session, and
planemo testing fails. Interactive testing with `planemo serve` is not affected
and works fine.
- Rename expected output Log from input Log. This relieves a concern I had that the tests were just working because the inputs and outputs match.
- Add another "unregistered" output file to the output directory and add a test for it. This verifies that such files are infact tested.
- Add a nested "unregistered" output file to the output directory and add a test for it. This verifies that nested files are tested.
These files are being moved to https://github.com/galaxyproject/galaxy-test-data.
Files used by framework functional tests remain. This drops the size of the test-data directory from 73M down to less than 1.3M.
Tool test is mildly useful to verify this works for single execution - but more useful as basis for future changesets testing mixed collection and subcollection mapping tool executions.
-framework will still cause just a small set of non-default, non-user facing tools to be tested and exerise various aspects of the tool and test frameworks, but now an argument can be passed in -with_framework_test_tools to allow these tools to be used as part of the normal test framework (i.e. when not using -framework). This serves two purposes - to ensure the feature coverage of existing tests remains high as the tests are migrated out of the distribution and to allow a set of tests to be available for functional (e.g. API tests) that users don't need to see.
Björn Grüning