I've been refactoring toward this for at least half a decade. Being able to pip install the API and Selenium tests without bringing in the whole app is important to make them usable outside the context of the app - for testing remote, running servers for instance.
If galaxy-test-driver is available the test cases will still spawn a Galaxy instance if needed, but this is not longer required for import or use.
the files generated in a test are allowed to differ (+-)
by delta_frac * size_of_the_test_file
use case: I develop a a lare tool collection where some files
differ much (ususally the large files) and some only a bit
(the small files). by applying a large delta to all files
meaningful differences in small files are not detected.
Co-authored-by: Marius van den Beek <m.vandenbeek@gmail.com>
dont compare the json file content but rely completely on the assertions on
the loaded json contents.
the json content test was superficial and did not work anymore since the order
of the keys differed. it worked before since the inputs contained only a
single key.
- in particular the case of an empty select did yield "None" before
now its []
Problem was that the ValueWrapper is not None if nothing is selected
- but also non empty selects were "option1,option2" now its
["option1","option2"]
- renamed variable value -> value_wrapper
Split test/base into lib/galaxy_test/base and lib/galaxy_test/driver. Dividing things on whether they import galaxy-app code or not - with the goal of a galaxy-test-base package that doesn't depend on galaxy-app and a galaxy-test-driver package that does.
I've been refactoring around this division for a long time. Certain tests (namely the integration tests) always need to bootstrap Galaxy - but the API tests and the Selenium tests should be installable and runnable without Galaxy in the Python environment. So future galaxy-test-api and galaxy-test-selenium packages should have only an optional dependency on galaxy-test-driver.
Properly sturcturing these namespaces is a precursor for real packages on PyPI, clean separation of the tool shed testing code, and cleans up all the flake8 noqa hacks in test/.
John Chilton