Make sure to backup your database before updating.
Also fix a long standing bug in the async controller dealing with updating output datasets upon job completion.
Credit for database migration directions go to Greg.
*** This is a significant change, be sure to follow the migration steps below exactly and in order: ***
---------------
0. Stop Galaxy server
---------------
1. Backup database and galaxy_root/database directory
---------------
2. The current validation_error table is not being used, it is left over from Ian's work that was not made functional. However, if we want to keep it around, we should drop the old version of the table so it will get re-created correctly when the Galaxy server is restarted:
any database - sql command(s):
------------------------------
DROP TABLE validation_error;
---------------
3. Stop server, perform svn update to get latest code, start server to create new history_dataset_association and implicitly_converted_dataset_association tables
---------------
4. Add new columns to dataset table:
postgres sql command(s):
------------------------
ALTER TABLE dataset ADD COLUMN purgable boolean DEFAULT 't';
ALTER TABLE dataset ADD COLUMN external_filename text;
ALTER TABLE dataset ADD COLUMN _extra_files_path text;
mysql sql command(s):
---------------------
ALTER TABLE dataset ADD COLUMN purgable boolean DEFAULT TRUE;
ALTER TABLE dataset ADD COLUMN external_filename text;
ALTER TABLE dataset ADD COLUMN _extra_files_path text;
sqlite sql command(s):
----------------------
ALTER TABLE dataset ADD COLUMN purgable boolean DEFAULT 0;
ALTER TABLE dataset ADD COLUMN external_filename text;
ALTER TABLE dataset ADD COLUMN _extra_files_path text;
---------------
5. Populate new columns:
postgres sql command(s):
------------------------
UPDATE
dataset
SET
external_filename = dataset_filename.filename,
_extra_files_path = dataset_filename.extra_files_path
FROM
dataset_filename
WHERE
dataset.filename_id = dataset_filename.id;
mysql or sqlite sql command(s):
-------------------------------
UPDATE
dataset,
dataset_filename
SET
dataset.external_filename = dataset_filename.filename,
dataset._extra_files_path = dataset_filename.extra_files_path
WHERE
dataset.filename_id = dataset_filename.id;
---------------
6. Populate dataset table with info from dataset_child_association table:
postgres sql command(s):
------------------------
UPDATE
dataset
SET
parent_id = dataset_child_association.parent_dataset_id
FROM
dataset_child_association
WHERE
dataset.id = dataset_child_association.child_dataset_id;
mysql or sqlite sql command(s):
-------------------------------
UPDATE
dataset,
dataset_child_association
SET
dataset.parent_id = dataset_child_association.parent_dataset_id
WHERE
dataset.id = dataset_child_association.child_dataset_id;
---------------
7. Drop dataset_child_association table:
any database - sql command(s):
------------------------------
DROP TABLE dataset_child_association;
---------------
8. Copy parts of the current dataset table to the new history_dataset_association table:
postgres sql command(s):
------------------------
INSERT INTO history_dataset_association
SELECT
id,
history_id,
id AS dataset_id,
create_time,
update_time,
hid,
name,
info,
blurb,
peek,
extension,
metadata,
parent_id,
designation,
deleted,
visible
FROM dataset
ORDER BY id;
mysql or sqlite sql command(s):
-------------------------------
INSERT INTO
history_dataset_association
(
history_id,
dataset_id,
create_time,
update_time,
hid,
name,
info,
blurb,
peek,
extension,
metadata,
parent_id,
designation,
deleted,
visible
)
SELECT
history_id,
id,
create_time,
update_time,
hid,
name,
info,
blurb,
peek,
extension,
metadata,
parent_id,
designation,
deleted,
visible
FROM
dataset
ORDER BY
id;
---------------
9. Update the nextval value of the primary key sequence in the history_dataset_association table
NOTE: THIS IS NOT NECESSARY FOR MYSQL OR SQLITE!
postgres sql command(s):
------------------------
SELECT
setval('history_dataset_association_id_seq', max(id))
FROM
history_dataset_association;
---------------
10. Update dataset table to mark dataset_associated_file dataset as deleted:
postgres sql command(s):
------------------------
UPDATE dataset
SET
deleted = 't'
WHERE
id IN
(SELECT dataset_id FROM dataset_associated_file ORDER BY dataset_id DESC);
mysql or sqlite sql command(s):
-------------------------------
UPDATE dataset
SET
deleted = TRUE
WHERE
id IN
(SELECT dataset_id FROM dataset_associated_file ORDER BY dataset_id DESC);
---------------
11. Drop the dataset_associated_file table:
any database - sql command(s):
------------------------------
DROP TABLE dataset_associated_file;
---------------
12. Alter foreign keys on job_to_input_dataset table:
postgres or sqlite sql command(s):
------------------------
ALTER TABLE
job_to_input_dataset
DROP CONSTRAINT
job_to_input_dataset_dataset_id_fkey;
ALTER TABLE
job_to_input_dataset
ADD CONSTRAINT job_to_input_dataset_dataset_id_fkey
FOREIGN KEY
(dataset_id)
REFERENCES
history_dataset_association(id);
mysql sql command(s):
-------------------------------
ALTER TABLE
job_to_input_dataset
DROP FOREIGN KEY
ix_job_to_input_dataset_dataset_id;
ALTER TABLE
job_to_input_dataset
ADD FOREIGN KEY
ix_job_to_input_dataset_dataset_id (dataset_id)
REFERENCES
history_dataset_association(id);
---------------
13. Alter foreign keys on job_to_output_dataset table:
postgres or sqlite sql command(s):
------------------------
ALTER TABLE
job_to_output_dataset
DROP CONSTRAINT
job_to_output_dataset_dataset_id_fkey;
ALTER TABLE
job_to_output_dataset
ADD CONSTRAINT
job_to_output_dataset_dataset_id_fkey
FOREIGN KEY (dataset_id)
REFERENCES history_dataset_association(id);
mysql sql command(s):
-------------------------------
ALTER TABLE
job_to_output_dataset
DROP FOREIGN KEY
ix_job_to_output_dataset_dataset_id;
ALTER TABLE
job_to_output_dataset
ADD FOREIGN KEY
ix_job_to_output_dataset_dataset_id
FOREIGN KEY (dataset_id)
REFERENCES history_dataset_association(id);
---------------
14. Eliminate columns from the dataset table previously copied to history_dataset_association:
any database - sql command(s):
------------------------------
ALTER TABLE dataset DROP COLUMN hid;
ALTER TABLE dataset DROP COLUMN history_id;
ALTER TABLE dataset DROP COLUMN name;
ALTER TABLE dataset DROP COLUMN info;
ALTER TABLE dataset DROP COLUMN blurb;
ALTER TABLE dataset DROP COLUMN peek;
ALTER TABLE dataset DROP COLUMN extension;
ALTER TABLE dataset DROP COLUMN dbkey;
ALTER TABLE dataset DROP COLUMN metadata;
ALTER TABLE dataset DROP COLUMN parent_id;
ALTER TABLE dataset DROP COLUMN designation;
ALTER TABLE dataset DROP COLUMN visible;
ALTER TABLE dataset DROP COLUMN filename_id;
Number of jobs today - displays the total number of jobs for today
Number of jobs per Internet domain - displays the number of jobs that have been submitted per Internet domain
Number of registered users - displays the total number of registered users
Number of jobs per day for current month - displays days of the current month with the total number of jobs for each day
Number of jobs in error per day for current month - displays days of the current month with the total number of jobs in error for each day
All unfinished jobs - displays jobs that are currently in a "new" state, a "queued" state or a "running" state
Number of jobs per month - displays a list of months with the total number of jobs for each month
Number of jobs in error per month - displays a list of months with the total number of jobs in error for each month
Number of jobs per user - displays users sorted in descending order by the number of jobs they have submitted
Number of jobs per tool - displays tools sorted in alphabetical order by tool id and the number of jobs created by the tool
SQL commands for schema changes:
ALTER TABLE galaxy_session ADD COLUMN current_history_id INTEGER;
ALTER TABLE galaxy_session ADD FOREIGN KEY (current_history_id) REFERENCES history(id);
ALTER TABLE galaxy_session ADD COLUMN session_key VARCHAR(255) UNIQUE;
ALTER TABLE galaxy_session ADD COLUMN is_valid BOOLEAN DEFAULT false;
ALTER TABLE galaxy_session ADD COLUMN prev_session_id INTEGER;
Galaxy config change:
In the [app:main] section, add:
# Galaxy session security
id_secret = changethisinproductiontoo
1) The "universe", "universe_user" and "universe_session" cookies are deprecated and replaced with 1 new cookie named galaxysession". The deprecated cookies are still supported and will be for at least 90 days, but when a valid one is found, the value is taken and used to create a new "galaxysession" cookie and the deprecated cookie is immediately expired.
2) The value of the new "galaxysession" cookie is an encrypted unique, high entropy 128 bit random number. The decoded value is the value stored in the new session_key column of the galaxy_session table.
3) Whenever a user logs in or out they get a new GalaxySession, and the old one in the database is marked as invalidated, so a compromised cookie won't survive across login/logout. Also, when creating the new session, a reference to the previous one is saved ( via the new prev_session_id column in the galaxy_session table ) so we have a way to chain them together.
There was a case where a dataset could have been converted multiple times if the conversion took place in a previous input in the same tool.
Also several other miscellaneous bug fixes.
Now we only support FastqSolexa variants.
If the quality scores are presented as characters,
the integer values are obtained by their ascii code subtract 64.
Cron scripts on test and main will need to be updated when the server is.
GOPS complement tool and annotation profiler will now correctly use chromosome lengths files if available.
MAF tool interfaces now support the use of index species and all species existing in cached alignment sets.
Until main is updated and the old maf_location files can be overwritten, these symbolic links are required:
maf_index.loc -> /depot/data2/galaxy/maf_index_new.loc
maf_pairwise.loc -> /depot/data2/galaxy/maf_pairwise_new.loc
James Taylor