The new flake8-bugbear 22.7.1 warns if a function defined inside a loop uses
a variable redefined in the loop, due to the late-binding closure gotcha:
https://docs.python-guide.org/writing/gotchas/#late-binding-closures
When possible/sensible, I've moved the function definition before the loop
(adding the variable(s) as parameters) which is clearly also a speed-up.
Most other cases are false alarms, e.g. if the function is used and
discarded within the loop iteration that defines it. In such cases, I've
annotated the offending lines with a `# noqa: B023`.
The only file where I've applied the suggested workaround of immediately
binding arguments is in `lib/galaxy/visualization/plugins/config_parser.py`
Also:
- Small refactorings, in particular to `scripts/apply_tags.py`
Upgrade syntax using `pyupgrade --py36-plus` .
Manually drop several `six` imports.
Also:
- Remove broken pr_cache in scripts/bootstrap_history.py
- Fix broken prefix removal in lib/galaxy/tool_util/deps/mulled/mulled_build.py
I've been refactoring toward this for at least half a decade. Being able to pip install the API and Selenium tests without bringing in the whole app is important to make them usable outside the context of the app - for testing remote, running servers for instance.
If galaxy-test-driver is available the test cases will still spawn a Galaxy instance if needed, but this is not longer required for import or use.
Split test/base into lib/galaxy_test/base and lib/galaxy_test/driver. Dividing things on whether they import galaxy-app code or not - with the goal of a galaxy-test-base package that doesn't depend on galaxy-app and a galaxy-test-driver package that does.
I've been refactoring around this division for a long time. Certain tests (namely the integration tests) always need to bootstrap Galaxy - but the API tests and the Selenium tests should be installable and runnable without Galaxy in the Python environment. So future galaxy-test-api and galaxy-test-selenium packages should have only an optional dependency on galaxy-test-driver.
Properly sturcturing these namespaces is a precursor for real packages on PyPI, clean separation of the tool shed testing code, and cleans up all the flake8 noqa hacks in test/.
This is pretty hackish as fixtures can't be used during test discovery.
To properly shut down the driver I've added a conftest.py file that will
be called when quitting pytest. We might be able to handle this a
little better if we wrote a full plugin, but for now this works.
Also now reports all test indexes, so you can select e.g test 2 of a
tool.
We don't rely on TwillTestCase anymore, that is now exclusively used by
ShedTwillTestCase. Then I've just re-used the infrastructure of
ToolsTestCase for discovering tests, while making sure that we do use an
admin API key during testing.
This gets us a lot closer to being able to construct a ToolTestBuilder from simple dictionaries generated from the API for instance or from any tool really.
- Add flake8-import-order to flake8 Pipfile and remove py27-lint-imports
and py27-lint-imports-include-list tox envs
- Fix most E201 and E202 errors reported by flake8-import-order v0.15,
but pin flake8-import-order to v0.14.3 until
https://github.com/PyCQA/flake8-import-order/issues/123
is fixed
This let us drop 2 jobs on Travis per each job.
Refactor test verification into galaxy-lib-compat module.
- Refactor a bunch class methods in ``test.base.twilltestcase`` into module functions in ``galaxy.tools.verify``.
- Move ``test.base.test_data`` to ``galaxy.tools.verify.test_data``.
- Move ``test.base.asserts`` to ``galaxy.tools.verify.asserts``.
Remove duplication in execution of these methods between composite and normal test outputs. This also entailed reworking the parsing of the composite test outputs to bring them inline with normal outputs. In addition to simplify removing duplication, this means many more tests can be made over composite outputs - such as md5 checks and test assertions. I've added a new framework test tool to verify this.
https://bitbucket.org/galaxy/galaxy-central/pull-request/634/allow-tools-to-explicitly-produce-dataset added the ability for tool to create simple collections (lists and pairs). It also outlined three creation scenarios - fixed collections, pre-determinable collection structures (like output lists based on input lists), and fully dynamic output collections. This pull request allows tool to output nested collections for these first two.
Fully dynamic nested collections (using <discover_datasets> tags) is not implemented in this commit.
Additionally, the tool test syntax has been extended to allow testing nested collection elements and an example tool is included that demonstrates this functionality - test/functional/tools/collection_creates_list_of_pairs.xml.
When finding methods to invoke it seems nose uses the raw function name instead of the class's attribute name (I guess this terminology makes sense), but when reporting results it uses the latter. Synchronizing these names therefore allows calling specific tool test methods from the command-line.
This change is required to implement galaxyproject/planemo#145.
By "static" I mean tools such as a FASTQ de-interlacer that would produce a "paired" collection with two datasets everytime. By "determinable" I mean tools that perform N->N operations within the same job - such as a tool that needs to normalize a bunch of datasets all at once and not in separate jobs. (For N->N collection operations that should or can be done in N separate jobs tool authors should just write tools that operate over a dataset and produce a dataset and let the end-user 'map over' that operation.)
There are still large classes of operations where the structure of the output collection cannot be pre-determined - such as splitting files (e.g. bam files by read group) - that are not implemented in this commit.
Model:
The models have been updated to do a more thorough job of tracking collection outputs. Jobs just producing HistoryDatasetCollectionAssociations works fine for simple jobs producing collections - but you don't want to map a list over a tool that produces a pair and produce a bunch of pairs HDCAs and a list:pair HDCA- you just want a bunch of pieces and the one list:pair at that the top.
Workflow:
Workflows containing such operations can be executed - but the workflow editor has not been updated to handle this complexity (and it will require a significant overhaul) so such tools are not available in the workflow editor.
Tool Testing:
This commit also introduces a new tool XML syntax for describing tests on output collections. See files test/functional/tools/collection_creates_list.xml and test/functional/tools/collection_creates_pair.xml for examples.
Tests:
Includes two tools to test this - one that uses explicit pair output names and one that iterates over the structure of input list to produce an output list.
Includes several new tools API tests that test the tools described above via the API and implicit mapping over such tools. Includes two new workflow API tests - one that verifies a simple workflow with output collections works and one that verifies mapping over workflow steps in collections works.
See test/functional/tools/job_properties.xml for example. Provides full access to assertion based XML tags - tabular and XML based assertions probably not so interesting so these include - has_text, has_line, not_has_text, has_text_matching, has_line_matching.
Rather than relying solely on exceptions back to nose/test framework - add option (--structured_data_report_file) to run_tests.sh that causes a bunch of detailed data to be dumped to the specified file in a very structured way. Includes full to_dict of the job from the API which in turn includes job metrics, command-line, job's standard error and outputs (instead of the test frameworks), as well as the tool inputs, and exceptions broken out for tool execution versus output checking.
Its all indexed in the file by the test id (without the actual test toolbox depending on knowing the test id) - so one could pair this information with the XUnit output to produce much more detailed breakdowns of the tests.
Gives more details to aid testing (especially for tools with lots of outputs) and lets developer collect all correct test data files at once (when GALAXY_TEST_SAVE is set or using `planemo test` with --job_output_files argument) instead of needing to rerun the tool repeatedly.
This will prevent run_tests.sh from picking up the extra test when running full test tool box - the way planemo does (i.e. this fixes the problem where planemo always runs one extra passing test).
Nicola Soranzo