- Added an option to remove dataset from disk when purging, or leave them for renaming and later removal
- Moved History.purge() and Dataset.purge() methods to the cleanup_datasets.py script
- Eliminated duplicate functionality from admin controller, moving appropriate components to Galaxy reports.
- Re-implemented the long-term process we will use for deleting / purging histories and datasets
(we will initially write external shell scripts to rename dataset files rather then deleting them)
distribution point and copy them to Galaxy's eggs/ directory.
fetch_eggs_tarball.py and check_python_platform.py are mostly for offline
systems. fetch_eggs_tarball.py fetches eggs for your platform (or a
specified platform) and then tars them up, for copying to another system.
check_python_platform.py returns the string for your platform/interpreter
that will be used to download from the egg distribution site.
For those that may have installed revision 2256, use the following SQL command to evolve your schema:
alter table dataset drop column removed;
We have changed direction again on the process we will use to cleanup datasets and histories.
- When we are comfortable with the purge process, the Dataset.purge() function will be modified to delete the file from disk rather than simply renaming it, and the Dataset.remove_from_disk() function will be eliminated.
- The entire process below must be executed at least 1 time ( prior to making these changes ) to ensure that purged files are properly removed from disk. This process is:
delete_userless_histories() -> purge_histories() -> purge_datasets() -> remove_datasets()
Shell scripts have been added for each of these stages with the exception of remove_datasets(), which will eventually be eliminated.
SQL commands to alter schema:
alter table dataset add column removed boolean default 'f';
alter history add column purged boolean default 'f';
- The lifecycle of a history after creation is now deleted -> purged
- Deleting a history will delete all associated datasets (only the database is update, nothing removed from disk).
- Purging a history will purge all associated datasets (datasets are renamed, nothing removed from disk).
- The lifecycle of a dataset after creation is now deleted -> purged -> removed
- Deleting a dataset will only update the database.
- Purging a dataset will rename the file on disk and update the database.
- Removing a dataset will remove the renamed, purged file from disk and update the database.
- Also reverted the behavior of history_delete() to pre-rev 2250 so that no datasets are deleted at teh time the history was deleted by the user.
- The admin controller and the cleanup_datasets.py script include all of the above functionality.
connections
2) Added a script for packing the javascripts, everything is now repacked
with yui-compressor (the packed versions are slightly large, but they
gzip better resulting in an overall reduction).
will prevent Galaxy from deleting the file when the dataset is purged. Copies of datasets (such as when sharing a
history), no longer copy file contents.
Database changes required:
CREATE TABLE dataset_filename (
id INTEGER NOT NULL,
create_time TIMESTAMP DEFAULT current_timestamp,
update_time TIMESTAMP DEFAULT current_timestamp,
filename TEXT,
extra_files_path TEXT,
readonly BOOLEAN,
PRIMARY KEY (id)
);
ALTER TABLE dataset ADD filename_id INTEGER;
Anton Nekrutenko