From b8ecf36f2ae52af1cda3eac64f53400e56e895db Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 29 Mar 2016 08:26:21 -0400 Subject: [PATCH] Improve testing environment variable and CLI documentation. Document many more variables and improve documentation of what is there. Include fixes recommended by @nsoranzo. --- run_tests.sh | 129 +++++++++++++++++++++++++++++++++++---------------- 1 file changed, 89 insertions(+), 40 deletions(-) diff --git a/run_tests.sh b/run_tests.sh index 80fadd5cb68..e5c13aa9e20 100755 --- a/run_tests.sh +++ b/run_tests.sh @@ -39,58 +39,107 @@ External Tests: A small subset of tests can be run against an existing Galxy instance. The external Galaxy instance URL can be configured with ---external_url. If this is set eithre --external_master_key or +--external_url. If this is set, either --external_master_key or --external_user_key must be set as well - more tests can be executed with --external_master_key than with a user key. Extra options: + --verbose_errors Force some tests produce more verbose error reporting. - --no_cleanup Do not delete temp files for Python functional tests (-toolshed, -framework, etc...) - --debug On python test error or failure invoke a pdb shell for interactive debugging of the test - --report_file Path of HTML report to produce (for Python Galaxy functional tests). - --xunit_report_file Path of XUnit report to produce (for Python Galaxy functional tests). - --skip-venv Do not create .venv (passes this flag to common_startup.sh) - --dockerize Run tests in a pre-configured Docker container (must be first argument if present). - --db For use with --dockerize, run tests using partially migrated 'postgres', 'mysql', - or 'sqlite' databases. - --external_url External URL to use for Galaxy testing (only certain tests). - --external_master_key Master API key used to configure external tests. - --external_user_key User API used for external tests - not required if + --no_cleanup Do not delete temp files for Python functional tests + (-toolshed, -framework, etc...) + --debug On python test error or failure invoke a pdb shell for + interactive debugging of the test + --report_file Path of HTML report to produce (for Python Galaxy + functional tests). + --xunit_report_file Path of XUnit report to produce (for Python Galaxy + functional tests). + --skip-venv Do not create .venv (passes this flag to + common_startup.sh) + --dockerize Run tests in a pre-configured Docker container (must be + first argument if present). + --db For use with --dockerize, run tests using partially + migrated 'postgres', 'mysql', or 'sqlite' databases. + --external_url External URL to use for Galaxy testing (only certain + tests). + --external_master_key Master API key used to configure external tests. + --external_user_key User API used for external tests - not required if external_master_key is specified. Environment Variables: -In addition to the above command-line options, many environment Variables -can be used to control the Galaxy functional testing processing. +In addition to the above command-line options, many environment variables +can be used to control the Galaxy functional testing processing. Command-line +options above like (--external_url) will set environment variables - in such +cases the command line argument takes precedent over environment variables set +at the time of running this script. Functional Test Environment Variables -GALAXY_TEST_DBURI Database connection string used for functional test - database for Galaxy. -GALAXY_TEST_INSTALL_DBURI Database connection string used for functional test - database for Galaxy's install framework. -GALAXY_TEST_INSTALL_DB_MERGED Set to use same database for Galaxy and install framework, - this defaults to True for Galaxy tests an False for shed tests. -GALAXY_TEST_DB_TEMPLATE If GALAXY_TEST_DBURI is unset, this URL can be retrieved - and should be an sqlite database that will be upgraded - and tested against. -GALAXY_TEST_TMP_DIR Temp directory used for files required by Galaxy server setup - for Galaxy functional tests. -HTTP_ACCEPT_LANGUAGE Defaults to 'en' -GALAXY_TEST_NO_CLEANUP Do not cleanup main test directory after tests, - the deprecated option TOOL_SHED_TEST_NO_CLEANUP does the - same thing. -GALAXY_TEST_HOST Host to use for Galaxy server setup for testing. -GALAXY_TEST_PORT Port to use for Galaxy server setup for testing. -GALAXY_TEST_TOOL_PATH Defaults to 'tools' -GALAXY_TEST_SHED_TOOL_CONF Shed toolbox conf (defaults to config/shed_tool_conf.xml) - used when testing installed to tools with -installed. -TOOL_SHED_TEST_HOST Host to use for shed server setup for testing. -TOOL_SHED_TEST_PORT Port to use for shed server setup for testing. -TOOL_SHED_TEST_FILE_DIR Defaults to test/tool_shed/test_data. -TOOL_SHED_TEST_TMP_DIR Defaults to random /tmp directory - place for tool shed - test server files to be placed. -TOOL_SHED_TEST_OMIT_GALAXY Do not launch a Galaxy server for tool shed testing. +GALAXY_TEST_DBURI Database connection string used for functional + test database for Galaxy. +GALAXY_TEST_INSTALL_DBURI Database connection string used for functional + test database for Galaxy's install framework. +GALAXY_TEST_INSTALL_DB_MERGED Set to use same database for Galaxy and install + framework, this defaults to True for Galaxy + tests an False for shed tests. +GALAXY_TEST_DB_TEMPLATE If GALAXY_TEST_DBURI is unset, this URL can be + retrieved and should be an sqlite database that + will be upgraded and tested against. +GALAXY_TEST_TMP_DIR Temp directory used for files required by + Galaxy server setup for Galaxy functional tests. +GALAXY_TEST_SAVE Location to save certain test files (such as + tool outputs). +GALAXY_TEST_EXTERNAL Target an external Galaxy as part of testing. +GALAXY_CONFIG_MASTER_KEY Master or admin API key to use as part of + testing with GALAXY_TEST_EXTERNAL. +GALAXY_TEST_USER_API_KEY User API key to use as part of testing with + GALAXY_TEST_EXTERNAL. +GALAXY_TEST_HISTORY_ID Point casperjs tests at specific external + history for testing. +GALAXY_TEST_WORKFLOW_FILE Point casperjs tests at specific workflow + file for testing. +GALAXY_TEST_VERBOSE_ERRORS Enable more verbose errors during API tests. +GALAXY_TEST_UPLOAD_ASYNC Upload tool test inputs asynchronously (may + overwhelm sqlite database). +GALAXY_TEST_RAW_DIFF Don't slice up tool test diffs to keep output + managable - print all output. (default off) +GALAXY_TEST_DEFAULT_WAIT Max time allowed for a tool test before Galaxy + gives up (default 86400) - tools may define a + maxseconds attribute to extend this. +GALAXY_TEST_TOOL_DEPENDENCY_DIR tool dependency dir to use for Galaxy during + functional tests. +GALAXY_TEST_FILE_DIR Test data sources (default to + test-data,https://github.com/galaxyproject/galaxy-test-data.git) +GALAXY_TEST_DIRECTORY $GALAXY_ROOT/test +GALAXY_TEST_TOOL_DATA_PATH Set to override tool data path during tool + shed tests. +GALAXY_TEST_FETCH_DATA Fetch remote test data to + GALAXY_TEST_DATA_REPO_CACHE as part of tool + tests if it is not available locally (default + to True). Requires git to be available on the + command-line. +GALAXY_TEST_DATA_REPO_CACHE Where to cache remote test data to (default to + test-data-cache). +HTTP_ACCEPT_LANGUAGE Defaults to 'en' +GALAXY_TEST_NO_CLEANUP Do not cleanup main test directory after tests, + the deprecated option TOOL_SHED_TEST_NO_CLEANUP + does the same thing. +GALAXY_TEST_HOST Host to use for Galaxy server setup for + testing. +GALAXY_TEST_PORT Port to use for Galaxy server setup for + testing. +GALAXY_TEST_TOOL_PATH Path defaulting to 'tools'. +GALAXY_TEST_SHED_TOOL_CONF Shed toolbox conf (defaults to + config/shed_tool_conf.xml) used when testing + installed to tools with -installed. +TOOL_SHED_TEST_HOST Host to use for shed server setup for testing. +TOOL_SHED_TEST_PORT Port to use for shed server setup for testing. +TOOL_SHED_TEST_FILE_DIR Defaults to test/tool_shed/test_data. +TOOL_SHED_TEST_TMP_DIR Defaults to random /tmp directory - place for + tool shed test server files to be placed. +TOOL_SHED_TEST_OMIT_GALAXY Do not launch a Galaxy server for tool shed + testing. Unit Test Environment Variables