Put all desired files in the directory specified in a tool XML as $output.files_path.
Example:
<command interpreter="python">script.py $input $output $output.files_path</command>
becomes:
python ./tools/filters/script.py ./database/files/dataset_1.dat ./database/files/dataset_2.dat ./database/tmp/dataset_2_files
Any file put in ./database/tmp/dataset_2_files will be accessable to html files using a relative link.
Directories are moved to a more permanent location only if they contain files.
To have a child dataset harvested automatically, simply name the file:
child_PARENTID_DESIGNATION_VISIBILITY_EXT (To make a new primary dataset, simply use the form: primary_ASSOCIATEDWITHDATASETID_DESIGNATION_VISIBILITY_EXT)
and place this file in the directory specified by $__new_file_path__
For example:
You define the command in the tool XML as:
<command interpreter="python2.4">some_command.py $input1 $out_file1 $out_file1.id $__new_file_path__</command>
Suppose the input dataset is 1 and the output dataset is 2, the commandline becomes:
python2.4 some_command.py ./database/files/dataset_1.dat ./database/files/dataset_2.dat 2 ./database/tmp
In addition to the primary file (a HTML file), this program creates files in the ./database/tmp directory named:
child_2_SomeImage_invisible_jpg
child_2_SomeText_visible_text
These files are discovered and added as children to the appropriate dataset. The text file will appear in the user's history, but the jpg will not.
All files can be viewed however, by making links in the primary (HTML) history item like:
<img src="display_child?parent_id=2&designation=SomeImage" alt="Some Image"/>
<a href="display_child?parent_id=2&designation=SomeText">Some Text</a>
Designations need to be unique for each originally declared output file, simply using a counter can work well.
These new files can be accessed in the exec_after_process hook under param_dict['__collected_datasets__'].
Note the update to the universe_wsgi.ini.sample file.
Some database changes are required:
ALTER TABLE dataset ADD visible boolean;
UPDATE dataset SET visible = true WHERE visible is NULL;
Converters are manually accessed through the edit_data interface (same as metadata).
Converters to load are specified by datatype_converters_conf.xml, see
datatype_converters_conf.xml.sample.
Converters are defined in XML using the same format as tools, and use the same
internal components.
datatype_converters_conf.xml and the converter path can be changed in the
universe_wsgi.ini file.
customized styles can still use the new layout). Changed the config parameter
to enable the new layout to "use_new_layout" (since it has nothing to do with style!!!)
NEXT PERSON TO UPDATE TEST PLEASE ADD
[app:universe]
use_new_layout=true
To add a display app, override the datatypes __init__ method, adding display apps in the form of
self.add_display_app ( app_id, label, file_function, links_function )
where
app_id is a unique id
label is the primary display label, ie display at 'UCSC'
file_function is a string containing the name of the function that returns a properly formated display
links_function is a string containing the name of the function that returns a list of (link_name,link)
See the Interval datatype for an example:
def __init__(self, **kwd):
"""Initialize interval datatype, by adding UCSC display apps"""
Tabular.__init__(self, **kwd)
self.add_display_app ( 'ucsc', 'display at UCSC', 'as_ucsc_display_file', 'ucsc_links' )
Enhance history template to only display at supported site if dataset not empty.
Commented out wormbase and flybase builds until we get the correct data automatically.
They are somewhat independent (you can use the new layout with the old style
and vice versa) but they belong together.
Tested in Firefox, Safari, IE5.5, IE6, but there are still some features
to be implemented before this is ready for 'main'.
To use, first add "new_style_june_2007=true" to the app:universe section of
universe_wsgi.ini (this gives you the new layout). Second change document_root
to "%(here)s/static/june_2007_style/blue" in the app:static_style section.
Style Features: simplifications, fewer 'hatched' images, space savings in some
places, small changes to the color scheme (same general feel, but the colors
are better matched). Main changes are to history, the new style support nested
datasets much better.
Layout features: no framesets (just iframes, some of which can be replaced with divs and ajax down the road). Frames are resizable and hideable. Tools can provide width hints and history will automatically hide (try UCSC main to see this).
Also, the consolidation makes the total file size substantially smaller. In addition, there is a "scripts/packed" directory that contains highly compressed versions of the javascript files. This should be used for production deployment. The initial load size is reduced by ~80%, before considering gzip and caching.
Fixed a bug in the unit test for new gff version 3 format in sniff.py.
Tweaked the biomart tool so that it will not display problm Info: text in history.
This commit includes some stuff for future GBrowse integration, but nothing included should pose any problems with the current environment.
#) An "xy_plot" tool and a new "build_ucsc_custom_track" tool that
demonstrate the various features added here.
#) Grouping constructs for tool parameters:
- "repeat" element allows for a set of parameters to be repeated an
arbitrary number of times
- "conditional" element allows choosing a set of parameters to display
based on the value of another parameter
These constructs can be arbitrarily nested. Their values are structured
and can be used in hooks, validation, et cetera
#) Support for generating arbitrary config files to pass to a tool
#) Command lines are now full Cheetah templates
#) Better job error reporting, includes tracebacks for internal errors
preparing the job (database change required, see below!)
#) Preparation of command line, config files, exec_before_job hook moved
into job execution stage
#) Parameter values now jsonified before storing in database (this is much
more rigorous that before, and restoring from the database now works
properly)
DATABASE CHANGE:
alter table job add column traceback text;
Database changes required:
alter table history add genome_build varchar(40);
There is now a new tool ToolParameter named GenomeBuildParameter which should be useful in several existing tools.
Also took care of some other miscellaneous items.
contents deleted from disk and is done using the admin interface.
Database changes required:
alter table dataset add purged boolean;
update dataset set purged = False where purged is null;
James Taylor