Commit Graph
1860 Commits
Author SHA1 Message Date
Nate Coraor 76e128a6ee monitor: Instead of creating a new history with -n, delete the old
history (which creates a new history...).
2008-05-15 17:33:34 +00:00
Greg Von Kuster bf0a8cc462 Added another wrapper around bx to handle exceptions, all gops tools affected. Added 2 new tests to gops_intersect. 2008-05-07 14:07:23 +00:00
Nate Coraor 38cc984df3 Bugfix, save cookies after creating a new history (otherwise the next
time you run the monitor, it uses the old history).
2008-05-05 16:51:26 +00:00
Nate Coraor ba4411e667 Add -n option to check_galaxy to create a new history. 2008-05-05 16:03:40 +00:00
Nate Coraor be224a4a38 dist-scramble, scrambles eggs for distribution. 2008-05-02 17:05:40 +00:00
James Taylor 980e79c669 Upgrade yuicompresor to version 2.3.5 2008-05-01 15:05:30 +00:00
Greg Von Kuster 3f01a6dd2a Necessary cron changes for cleanup dataset scripts for main. 2008-04-24 20:31:27 +00:00
Nate Coraor 1d5f59a3c3 Updated dataset cleanup scripts to use eggs lib. 2008-04-18 13:53:15 +00:00
Nate Coraor 04127ff3d5 DRMAA build scripts, eggs.ini entry.
And a minor bugfix to eggs.require.
2008-04-16 20:53:27 +00:00
Nate Coraor 746fa583d0 Update pkg_resources to 0.6c8 to fix the cygwin non-executable dll
bug.  Also, fix the setup script to create database/tmp and
database/files.
2008-04-14 19:25:35 +00:00
Nate Coraor 7560751dc8 Separate framework and tool environments.
Whether running locally or via PBS, $PATH should now be handled in your
user's environment.

This also fixes problems like PYTHONPATH on the new blast nodes being
wrong (because they are x86_64 and the frontends are i686).

Note that you can actually start Galaxy without having an eggs dir now.
2008-04-11 20:11:21 +00:00
Nate Coraor feff769b6f Build script for Cheetah on Python 2.5. 2008-04-04 13:38:46 +00:00
Greg Von Kuster a9db5f0e43 Moved all dataset related shell and python scripts to new ~/<galaxy root>/scripts/cleanup_datasets directory. 2008-04-03 18:33:09 +00:00
Nate Coraor 480f701764 Update monitor to delete datasets, not histories. 2008-04-02 15:40:42 +00:00
Nate Coraor 83389f54eb pbs_python egg changed from static to dynamic for the reasons explained
here:

http://g2.trac.bx.psu.edu/wiki/ClusteringGalaxy#Anoteonlibtorque

Also includes a rather ridiculous fix for compiling a fat dynamic
libtorque on OS X due to some autoconf/libtool weirdness.
2008-03-28 21:22:10 +00:00
Greg Von Kuster af3b0669aa Eliminated all Python2.4 references, added necessary assert statements to ensure minimum version of Python 2.4. 2008-03-28 15:24:50 +00:00
Nate Coraor 5f189e45c3 The enormous eggs-out-of-svn commit. More details at:
http://g2.trac.bx.psu.edu/wiki/GalaxyEggs
2008-03-27 19:27:05 +00:00
Wen-Yu Chung 78802fa73b add scripts for preparing nt and wgs databases.
update blastdb location file, wgs is available now.
2008-03-27 17:41:40 +00:00
Daniel Blankenberg 254077c4e3 Add scripts required to fetch and process data for the Microbial Genome Datasource Tool. 2008-03-27 15:36:13 +00:00
Anton Nekrutenko a262ac1754 Finalizing taxonomy. Removed taxonomy processing from scripts. This does not require sqlite anymore. Script for fetching taxonomy now uses flat files directly and is much faster 2008-03-20 19:38:31 +00:00
Anton Nekrutenko 447be81be8 Cleanup of taxonomy processing scripts. Now only one is left 2008-03-18 19:29:51 +00:00
Nate Coraor d7172bc329 A couple small tweaks to the monitor. 2008-03-14 16:49:42 +00:00
Nate Coraor f675e37400 The Galaxy Monitor, and its cron wrapper. 2008-03-14 16:07:30 +00:00
Greg Von Kuster 5392fc0ab8 Fix for history item html for taxonomy data type, eliminated the type from the registry-not quite ready for prime time. Fix for cleanup_datasets script. 2008-03-05 13:22:01 +00:00
Anton Nekrutenko d25b346130 Changes to taxonomy fetches and two new datatypes: taxonomy and newick 2008-03-04 20:30:26 +00:00
Greg Von Kuster 512084b543 Some tweaks to the cleanup_dataset script. 2008-03-04 14:49:55 +00:00
Greg Von Kuster 52047c1b09 Use local CleanupDatasetsApplication class rather than galaxy's main UniverseApplication class for cleanup_datasets.py script. 2008-02-25 21:16:34 +00:00
Anton Nekrutenko f607e77b6a Tool for changing case 2008-02-25 15:15:18 +00:00
Greg Von Kuster f2a38411c4 Fix for cleanup_datasets script. 2008-02-23 14:56:55 +00:00
Greg Von Kuster 9565139b39 Some tweaks to the cleanup dataset scripts. 2008-02-21 13:23:39 +00:00
Greg Von Kuster 560e98d7fd Optimized data retrieval / update for cleanup datasets script and system reports. 2008-02-20 18:02:34 +00:00
Greg Von Kuster 6792e273db Added a new script to remove renamed datasets from disk. 2008-02-06 19:16:25 +00:00
Anton Nekrutenko 2596523c4d typo; 2008-02-06 19:00:34 +00:00
Anton Nekrutenko 77c094d2af readme.txt for taxonomy scripts 2008-02-06 18:59:00 +00:00
Anton Nekrutenko b9d3d02303 Scripts for creating taxonomy database 2008-02-06 18:27:10 +00:00
Greg Von Kuster ef249b0e11 Fixes for shell scripts to clean up histories, datasets. 2008-02-02 15:40:58 +00:00
Greg Von Kuster bac7fc59be New script to rename purged datasets. 2008-02-02 01:59:44 +00:00
Greg Von Kuster cdd0f9c1e1 Next pass at cleaning up datasets and histories.
- Added an option to remove dataset from disk when purging, or leave them for renaming and later removal
- Moved History.purge() and Dataset.purge() methods to the cleanup_datasets.py script
2008-02-01 20:07:11 +00:00
Greg Von Kuster 8d07f84003 Another pass at the cleanup datasets stuff.
- Eliminated duplicate functionality from admin controller, moving appropriate components to Galaxy reports.
- Re-implemented the long-term process we will use for deleting / purging histories and datasets
  (we will initially write external shell scripts to rename dataset files rather then deleting them)
2008-01-30 20:12:43 +00:00
Greg Von Kuster c8ee253ba6 Missed committing similar fixes to cleanup_datasets script and admin controller. 2008-01-27 16:41:29 +00:00
Nate Coraor 7d8656b55c First draft of the fetch_eggs.py script, this will download eggs from a
distribution point and copy them to Galaxy's eggs/ directory.

fetch_eggs_tarball.py and check_python_platform.py are mostly for offline
systems.  fetch_eggs_tarball.py fetches eggs for your platform (or a
specified platform) and then tars them up, for copying to another system.
check_python_platform.py returns the string for your platform/interpreter
that will be used to download from the egg distribution site.
2008-01-25 18:39:00 +00:00
Greg Von Kuster be461b0c8a Eliminated "removed" state from dataset lifecycle, require db schema update for those that installed rev 2256.
For those that may have installed revision 2256, use the following SQL command to evolve your schema:
alter table dataset drop column removed;

We have changed direction again on the process we will use to cleanup datasets and histories.
- When we are comfortable with the purge process, the Dataset.purge() function will be modified to delete the file from disk rather than simply renaming it, and the Dataset.remove_from_disk() function will be eliminated.
- The entire process below must be executed at least 1 time ( prior to making these changes ) to ensure that purged files are properly removed from disk.  This process is:

delete_userless_histories() -> purge_histories() -> purge_datasets() -> remove_datasets()

Shell scripts have been added for each of these stages with the exception of remove_datasets(), which will eventually be eliminated.
2008-01-24 13:56:09 +00:00
Greg Von Kuster 5ea5d75da5 Enhancements to history and dataset cleanup (requires db schema change).
SQL commands to alter schema:
alter table dataset add column removed boolean default 'f';
alter history add column purged boolean default 'f';

- The lifecycle of a history after creation is now deleted -> purged
- Deleting a history will delete all associated datasets (only the database is update, nothing removed from disk).
- Purging a history will purge all associated datasets (datasets are renamed, nothing removed from disk).
- The lifecycle of a dataset after creation is now deleted -> purged -> removed
- Deleting a dataset will only update the database.
- Purging a dataset will rename the file on disk and update the database.
- Removing a dataset will remove the renamed, purged file from disk and update the database.
- Also reverted the behavior of history_delete() to pre-rev 2250 so that no datasets are deleted at teh time the history was deleted by the user.
- The admin controller and the cleanup_datasets.py script include all of the above functionality.
2008-01-22 20:47:04 +00:00
Greg Von Kuster a2b76f55f5 Purging datasets will now only rename them ( by appending the string '_purged' to the original file name ) rather than removing them. Purged dataset files can then be manually deleted from the file system at a later time. 2008-01-21 16:11:12 +00:00
Greg Von Kuster 0a1bdbffe9 Added 2 new options to cleanup_datasets that provide info about what will be abandoned or purged.
Also now using Dataset.create_time in the purge() function since Dataset.update_time would have been reset when the abandon() function was executed.
2008-01-17 19:40:51 +00:00
Greg Von Kuster 2dc1c17a2d Cleaned up code for cleaning up histories and datasets, should now be faster and require much less memory. Other miscellanwoud code cleanup. Added sample database config options to config file. 2008-01-16 19:37:30 +00:00
Greg Von Kuster 01378424b4 New reports app server and 1st report showing disk usage for dataset file system - still need to mess with styles a bit.
Also fixed update_dataset_size script and delete deprecatd random_intervals code file.
2008-01-11 21:24:21 +00:00
Greg Von Kuster 6d80665988 Added file_size column to dataset table, requires db schema alteration:
alter table dataset add column file_size numeric(15,0)
Also added new script update_dataset_size.py
2008-01-08 16:19:25 +00:00
James Taylor 4b88be5e25 1) Modified index page so progressive loading works better for slower
connections

2) Added a script for packing the javascripts, everything is now repacked
   with yui-compressor (the packed versions are slightly large, but they
   gzip better resulting in an overall reduction).
2007-12-11 19:30:03 +00:00
James Taylor a50b8007ef Splitting out UCS2 and UCS4 eggs for linux. Run scripts check
waht python is available and use the right directory. Factored
path stuff out of shell scripts into a shared 'setup_paths.sh'.
2007-12-06 01:27:33 +00:00