That change forced $output.extra_files_path to be the same thing as $output.files_path. See rational in changeset.
Some datatypes however consume this path when generated metadata - post d781366 these datatypes would be accessing the wrong directory. This fixes them.
The suggested best practice I would put forward is $input.extra_files_path and $output.files_path. $output.extra_files_path redirects to $output.files_path for newer Galaxy versions (August 2014+) so that Galaxy configurations requiring this directory be in the jobs working directory work. Datatypes consuming dataset.extra_files_path for these outputs will be broken when used with $output.files_path for the August release of Galaxy - but if this changeset is back-ported to the October 2014 release then these types and tools will work going forward (without modification).
It can be verified that the older releases are broken with the following existing test (fixed with this changeset).
./run_functional_tests.sh -framework -id composite_output
Update scripts/check_eggs.py and scripts/fetch_eggs.py with logic to find the correct config file. These files should be refactored so they can share some of this code the way I refactored the database management scripts I guess.
Now appends test tools to sample tools instead of to default tools Galaxy would other wise load (config/tool_conf.xml or tool_conf.xml if these exist instead of sample) - so this is different behavior but I am unsure if it is worse. --with_framework_test_tools was added for API tests and these should probably target the distribution and not random tools someone might have configured so perhaps this new behavior is superior.
Database and tool migrations won't work if you use galaxy.ini.sample. Perhaps it should automatically try appending .sample? Or we should really be encouraging people to use the config file properly.
Fix thanks to Chong Chen at IBM - "When invoking the LSF drmma api to submit a job, the API will print the message like 'Job <78> is submitted to default queue <normal>'. So if the Galaxy want to use this API, need modify the drmaa_external_runner.py script to handle this message. LSF already support a env variable to disable this message print out, "BSUB_QUIET", set this into env can disable this message output."
A second issue was related to subsequent querying of the job state as a user other than the submitter - according to Chong Chen this has been fixed in latest LSF DRMAA changes on github (https://github.com/PlatformLSF/lsf-drmaa/commit/43e62546172b90d9f95e90273d77f4c35a0bb404).
This directory is required to exist but usually unneeded because Galaxy creates this directory on startup. However, when setting metadata remotely with the LWR that remote Galaxy instance may have never been started and this directory may not exist.
-framework will still cause just a small set of non-default, non-user facing tools to be tested and exerise various aspects of the tool and test frameworks, but now an argument can be passed in -with_framework_test_tools to allow these tools to be used as part of the normal test framework (i.e. when not using -framework). This serves two purposes - to ensure the feature coverage of existing tests remains high as the tests are migrated out of the distribution and to allow a set of tests to be available for functional (e.g. API tests) that users don't need to see.
Enable task splitting in functional tests to test this. Specify tool_dir in samples tool_conf.xml so it doesn't need to be explicitly set in scripts/functional_tests.py.
John Chilton