That change forced $output.extra_files_path to be the same thing as $output.files_path. See rational in changeset.
Some datatypes however consume this path when generated metadata - post d781366 these datatypes would be accessing the wrong directory. This fixes them.
The suggested best practice I would put forward is $input.extra_files_path and $output.files_path. $output.extra_files_path redirects to $output.files_path for newer Galaxy versions (August 2014+) so that Galaxy configurations requiring this directory be in the jobs working directory work. Datatypes consuming dataset.extra_files_path for these outputs will be broken when used with $output.files_path for the August release of Galaxy - but if this changeset is back-ported to the October 2014 release then these types and tools will work going forward (without modification).
It can be verified that the older releases are broken with the following existing test (fixed with this changeset).
./run_functional_tests.sh -framework -id composite_output
At least for the local job runner. This variant of property loading does correctly interpolate %(here) so there is no longer a need to catch those exceptions and object store related properties can use %(here).
This directory is required to exist but usually unneeded because Galaxy creates this directory on startup. However, when setting metadata remotely with the LWR that remote Galaxy instance may have never been started and this directory may not exist.
Add support for a single "import_module" to be passed to the new load_datatypes() method in the datatypes registry. This provides the ability to load a single class module from an installed tool shed repository along with a datatypes_conf.xml file included in the installed repository and pass them to the new load_datatypes() method. In the future, multiple imported modules may be allowed. The datatypes_conf.xml file included in the repository must conform to a slightly different definition than the same named file that comes with the distribution. This new definition will be documented in the Galaxy tool shed wiki.
We now have the ability to load new data types into the Galaxy server from an installed tool shed repository without restarting the Galaxy server.
e.g:
Exception exceptions.TypeError: TypeError('<galaxy.datatypes.metadata.MetadataTempFile object at 0x40d9a10> is not JSON serializable',) in <bound method MutableAttrInstanceState._cleanup of <sqlalchemy.orm.state.MutableAttrInstanceState object at 0x40d99d0>> ignored
Currently this is supported in the local runner (long running set_meta() can now be canceled the same as with jobs) and in the non-staging PBS runner.
There is a slight change to how new Metadata FileParameters are to be created in datatype.set_meta(), see the MAF datatype.
TODO:
Prevent editing of metadata on a running job when metadata is to be set externally.
Have datasets which were copied before job completion use the metadata from the original output dataset; currently .set_meta() is called locally on these.
Nicola Soranzo