- move / merge all duplicate methods from the library_and library_admin controllers into the library_common controller
- move / merge all duplicate mako templates into the ~/library/common directory
- make the admin view library browser a grid
- all library item ids are now encoded
("Upload files from filesystem paths") to the admin-side library upload pages.
This form contains a textarea that allows Galaxy admins to paste any number of
filesystem paths (files or directories) from which Galaxy will import library
datasets, saving the directory structure (if desired). Since such ability
allows admins access to any file on the Galaxy server which is readable by
Galaxy's system user, this option is disabled by default, and system
administrators should take care in assigning Galaxy administrators when this
feature is enabled. Controls on what files are accessible to this tool based
on ownership or other properties can be added at a later date if there is
sufficient interest for such features.
This commit also includes a checkbox on the "Upload directory of files" page
(as well as the new "Upload files from filesystem paths" page above) that will
prevent Galaxy from copying data to its files directory (by default,
'database/files/'). This is useful for large library datasets that live in
their own managed locations on the filesystem, this will prevent the existence
of duplicate copies of datasets (but means administrators must take care to
manage data - moving or removing the data from its Galaxy-external location
will render these datasets invalid within Galaxy).
One unique feature to be aware of: when using the "Copy data into Galaxy?"
checkbox on the "Upload directory of files" page, any symbolic links
encountered in the chosen import directory will be made absolute and
dereferenced ONCE. This allows administrators to link large datasets to the
import directory, rather than having to make full copies, while being able to
delete such links after importing. Only the first symlink (the one in the
import directory itself) is dereferenced; all others remain. See the following
for an example:
library_import_dir = /galaxy/import
% ls -lR /galaxy/import
/galaxy/import:
total 6
drwxr-xr-x 2 nate nate 512 Oct 1 11:31 link/
/galaxy/import/link:
total 10
lrwxrwxrwx 1 nate nate 71 Oct 1 10:38 1.bed -> ../../../home/nate/galaxy/test-data/1.bed
lrwxrwxrwx 1 nate nate 60 Oct 1 10:38 2.bed -> /home/nate/galaxy/test-data/2.bed
lrwxrwxrwx 1 nate nate 11 Oct 1 10:38 3.bed -> ../../3.bed
lrwxrwxrwx 1 nate nate 35 Oct 1 11:30 4.bed -> ../../galaxy_symlink/test-data/4.bed
lrwxrwxrwx 1 nate nate 41 Oct 1 11:31 5.bed -> /galaxy/galaxy_symlink/test-data/5.bed
% ls -l /galaxy/3.bed
lrwxrwxrwx 1 nate nate 60 Oct 1 10:39 /galaxy/3.bed -> /home/nate/galaxy/test-data/3.bed
% ls -l /galaxy/galaxy_symlink
lrwxrwxrwx 1 nate nate 44 Oct 1 11:30 /galaxy/galaxy_symlink -> /home/nate/galaxy/
In this example,
1.bed is a relative symbolic link to the real 1.bed.
2.bed is an absolute symlink to the real 2.bed.
3.bed is a relative symlink to ../../3.bed, aka /galaxy/3.bed, which itself is
a symlink to the real 3.bed.
4.bed is a relative symlink which follows another symlink
(/galaxy/galaxy_symlink) to the real 4.bed.
5.bed is an absolute symlink in the same fashion as 4.bed
If the 'link' server directory is chosen on the "Upload directory of files"
page, and "Copy data into Galaxy?" is checked "No", the following files will be
referenced by Galaxy:
/home/nate/galaxy/test-data/1.bed
/home/nate/galaxy/test-data/2.bed
/galaxy/3.bed
/galaxy/galaxy_symlink/test-data/4.bed
/galaxy/galaxy_symlink/test-data/5.bed
The Galaxy administrator may now safely delete /galaxy/import/link, but should
take care not to remove the referenced symbolic links (/galaxy/3.bed,
/galaxy/galaxy_symlink).
Not all symbolic links are dereferenced because it is assumed that if an
administrator links to a path in the import directory which itself is (or
contains) links, that is the preferred path for accessing the data.
- The genetrack controller is no longer used ( I'll delete it soon ), so the GeneTrack application is now loosely coupled
- Merge the 2 new_dataset.mako templates into one common code module name library_dataset_common.mako
- Rename all of the library upload form fields to have the same names as the history upload form fields
- Eliminate a 2nd runtool_btn from being rendered on the same form in tool_form.mako
- Fix bug and add functional tests to cover uploading a library dataset that does not include a template
- Fixed bug where exception was thrown when chosing a role in the admin view
- The contents of inherited templates will no longer be displayed in the inherited container
- Old versions of library datasets will not be displayed in the lbrary browser
- Added the job.traceback information to the dataset error report
- Moved the get_form_wigets method the forms.py to the FormDefinition class, renaming it to get_wigets
More work is needed on for this feature, including additional work on the forms. Functional tests are under way, but much more work is needed here as well.
This is needed, i.e. for Rgenetics Datatypes to prevent loss of metadata ('base_name') which occurs when changing to datatypes without this metadata parameter and would render the dataset unusable (composite filenames could be incorrect).
In addition to the config setting "library_import_dir" for the Admin view, the new config setting "user_library_import_dir" for the Libraries view will allow non-amin users to upload a directory of files.
The configured directory must contain sub-directories named the same as the non-admin user's Galaxy login ( email ). The non-admin user is restricted to uploading files or sub-directories of files contained in their directoy.
- Purged library items will not be displayed
- Deleted library items can only be viewed or undeleted
- Differentiate between displaying deleted libraries and displaying deleted items within an undeleted library
1) For datasets in the library browser, display message, uploaded by, date instead of format, db, info
2) Replace "Manage this dataset's versions" with "Upload a new version of this dataset"
3) When a library dataset is clicked in the library browser, display it's information rahter than it's peek.
1) include templates in the upload form for library datasets
2) Add user_id column to library_dataset_dataset_association table
3) Add message column to library_dataset_dataset_association table
4) Several miscellaneous bug fixes related to uploading library datasets
SQL commands required:
ALTER TABLE library_dataset_dataset_association ADD COLUMN user_id INTEGER;
ALTER TABLE library_dataset_dataset_association ADD CONSTRAINT library_dataset_dataset_association_user_id_id_fkey FOREIGN KEY (user_id) REFERENCES galaxy_user(id);
ALTER TABLE library_dataset_dataset_association ADD COLUMN message VARCHAR(255);
Next steps: 1) display closest associated template on the upload form for library datasets 2) add support for requiring template fields, add suppor tof different template field types besides "text"
- information templates can be created for libraries, folders, datasets - templates inherited downward, this still needs a bit of work, but current functionality should be reviewed.
- other new features are now also available on appropriate drop-down menus
- better naming for some templates / methods
Ross Lazarus