... if you can call 1 new class a framework. Includes a few test cases to exercise/drive it. These examples include a histories API test (a typical API test) and a general test of the API framework itself (mostly just the run_as functionality).
This includes changes to interactor.py to make it more useful outside the context of tool/workflow testing as well as a tweak to test Galaxy that gets started to allow testing of the run_as feature.
Pull logic for how master and user API keys are determinined for testing out of functional_tests.py for reuse elsewhere.
This refactoring will help the creation of an API test framework.
To obtain the behavior (assigning all groups for a given user for each job submission) change the property - 'drmaa_external_runjob_script' from 'scripts/drmaa_external_runner.py' to 'scripts/drmaa_external_runner.py --assign_all_groups'.
Making this optional because my concern is that this could be an expensive operation on some clusters.
create_db and manage_db should now work with install as well.
Migrations are symbolic links into lib/galaxy/model/migrate/versions of migrations affecting these tables. All future migrations to these tables should created in one place like this and linked in the other.
Shared across Tool Shed and Galaxy to reduce code duplication. Previously this was just a bunch, making it a real class allows placing some shared logic in there.
This refactoring also eliminates some global variables galaxy.model.mapping.Session, galaxy.model.mapping.context, and same for tool shed. This may break things, but these are things that should probably be fixed anyway.
In particular small changes to db_shell.py and a unit test that depended on these global variables have been updated. The functional test frameworks likewise needed to be updated to not depend on these - these changes were more substantial.
To fix these functional tests, I essentially replace old references to global variables in Galaxy with references to global variables just defined in the test framework in test/functional/database_contexts.py (a slight improvement).
Namely create_db.py,db_shell.py, and manage_db.py.
Move duplicated code into lib/galaxy/model/orm/scripts.py - add unit tests. PEP-8 clean up of all 4 files. Add incoherent incoherent comment at top of manage_db.py.
As a result of the code de-duplication create_db.py should be usable with the tool shed now.
Will be refactoring lib/galaxy/model/orm/scripts.py to work with new tool shed install database - it will be good to have a place to test these and allow manage_db.py and clean_db.py to work immediately.
- The secondary user groups are not assigned to the user
- When the json file is on a NFS share where root access is not allowed this script fails with a "error: JobTemplate file (/path/to/jsonfile) doesn't exist" error
To fix the first we have to go through all the groups of the user in set_user(uid) and assign them with os.setgroups()
The second one is fixed by removing the check if the json file exists in the function validate_paramters and move that check to a new function which is called after set_user(uid). Because its possible that the new user (the one in uid) has access to that file but root hasn't
... tool shed integration will be more challenging, but this can be used for directly configured Galaxy tools and outlines a syntax that could be shared with a tool shed driven approach.
The one tricky part is how to match workflow outputs to things to check. Right now it is based on the index of the output across the workflow. This is both difficult to determine and very brittle to workflow modifications - how to proceed - require annontation string to be setup? Modify workflow data model to all assigning names to outputs the way inputs havenames? At any rate this current syntax should be considered beta and may change - if it does Galaxy will not continue to support this syntax.
To run the sample test execute the following test command:
sh run_functional_tests.sh -workflow test-data/workflows/1.xml
Middle ground between recreating a completely new database and pointing at existing database with GALAXY_TEST_DBURI. The former requires a lot of setup time, the latter results in test failures in certain cases (namely tool shed tests expecting clean database).
GALAXY_TEST_DB_TEMPLATE can be either a file path (absolute) or URL.
In order to facilitate this, a new Galaxy config option (database_auto_migrate) has been added. If this option is enabled, when Galaxy starts up and points at an existing database, if that database is not at the newest version it will be automatically migrated. This option defaults to False, but is enabled in testing if GALAXY_TEST_DB_TEMPLATE is set.
I think we should go a step further and make this (database_auto_migrate) default to True if database_connection references an sqlite database - unless we believe there are Galaxy instances out there based on sqlite that are REALLY old or that are production enough to warrent requiring admins to do that database migration in a separate step (presumably encouraging them to make a backup pre-migration). Thoughts?
Dave Bouvier