Commit Graph
630 Commits
Author SHA1 Message Date
James Taylor e65294e0d5 Fix workflow menu configuration (submitting with not entries works,
orphaned entry rows are deleted). Also prevent users from sharing a workflow
with themselves.
2008-07-21 18:39:18 -04:00
Greg Von Kuster f8c00de9bc Fix for displaying peek columns if a column contains no data. 2008-07-21 10:51:46 -04:00
Daniel Blankenberg 2c38721060 Typo in last commit. 2008-07-18 12:11:58 -04:00
Daniel Blankenberg a19093d158 Fix bug in JobWrapper.get_input_fnames(), used by pbs runner, when an input dataset was optional and left empty. 2008-07-18 12:10:23 -04:00
James Taylor 70c4b8ee23 Workflows (owned and shared) can now be added to the tool menu. 2008-07-17 22:14:40 -04:00
James Taylor a46def3c5c Merge. 2008-07-17 20:42:00 -04:00
James Taylor d6607906d4 Allow loading a specific controller/action in the root middle frame, and use
this to allow the workflow editor to return to the workflow management page.
2008-07-17 20:40:20 -04:00
James Taylor 3fc7bb94ad Eagerload stored -> current workflow and workflow -> steps. 2008-07-17 19:45:56 -04:00
Greg Von Kuster 8ce37ea721 Fix for Tabular.set_meta() - when column contains no values, default column type metadata to string. 2008-07-17 14:58:30 -04:00
Nate Coraor 8e8c1c9cf2 Basic $REMOTE_USER support.
A database change is necessary:

ALTER TABLE galaxy_user ADD COLUMN external boolean;

To use:

http://g2.trac.bx.psu.edu/wiki/HowToInstall/ApacheProxy

I've tried to be as friendly as possible with respect to enabling and
disabling $REMOTE_USER support.  If you have an existing server and
enable use_remote_user, users whose external login matches their Galaxy
username will have access to their existing history/session.  If the
logins don't match, any existing credentials they had are destroyed.
Likewise, when disabled, users who weren't created automatically may
continue to access their histories/sessions.  Automatically created
users are logged out, and can't log in unless enabled by an administrator.
2008-07-17 09:15:38 -04:00
James Taylor 8cb0cc1edd Don't use redirects in workflow to avoid a really annoying safari
history bug. Use programatically generated form in 'rename'.
2008-07-14 21:41:29 -04:00
James Taylor 8337083dcf Reformatting the workflow index page. The "new workflow" form is now just a
button, and the actions are now popup menus (all progressive enhancement).
2008-07-14 19:48:52 -04:00
James Taylor 319c7e9b03 Some helpers in the framework to make it easier to return programatically generated forms. 2008-07-14 19:41:49 -04:00
James Taylor 6906f5a048 Support for sharing workflows (as well as renaming). A shared workflow is
read only for the recipient, however they can always run the latest version of
the workflow. They can also 'clone' the workflow to create their own editable
copy of the current version.
2008-07-14 01:56:50 +00:00
James Taylor 0071b7389e Fix rendering of Select parameters as_display_text when no value is selected. 2008-07-14 01:54:10 +00:00
James Taylor 1e707938a5 Fix documentation comment in topsort.py 2008-07-10 16:21:27 +00:00
Daniel Blankenberg 1e1a3e81bf Update to latest version (June 30, 2008) of GMAJ. 2008-07-09 19:07:03 +00:00
Daniel Blankenberg 28b89d686f Change database schema to separate Dataset and HistoryDatasetAssociation. This is a significant change, be sure to follow the migration notes exactly.
Make sure to backup your database before updating.


Also fix a long standing bug in the async controller dealing with updating output datasets upon job completion.


Credit for database migration directions go to Greg.

*** This is a significant change, be sure to follow the migration steps below exactly and in order: ***

---------------

0. Stop Galaxy server

---------------

1. Backup database and galaxy_root/database directory

---------------

2. The current validation_error table is not being used, it is left over from Ian's work that was not made functional.  However, if we want to keep it around, we should drop the old version of the table so it will get re-created correctly when the Galaxy server is restarted:

any database - sql command(s):
------------------------------
DROP TABLE validation_error;

---------------

3. Stop server, perform svn update to get latest code, start server to create new history_dataset_association and implicitly_converted_dataset_association tables

---------------

4. Add new columns to dataset table:

postgres sql command(s):
------------------------
ALTER TABLE dataset ADD COLUMN purgable boolean DEFAULT 't';
ALTER TABLE dataset ADD COLUMN external_filename text;
ALTER TABLE dataset ADD COLUMN _extra_files_path text;

mysql sql command(s):
---------------------
ALTER TABLE dataset ADD COLUMN purgable boolean DEFAULT TRUE;
ALTER TABLE dataset ADD COLUMN external_filename text;
ALTER TABLE dataset ADD COLUMN _extra_files_path text;

sqlite sql command(s):
----------------------
ALTER TABLE dataset ADD COLUMN purgable boolean DEFAULT 0;
ALTER TABLE dataset ADD COLUMN external_filename text;
ALTER TABLE dataset ADD COLUMN _extra_files_path text;

---------------

5. Populate new columns:

postgres sql command(s):
------------------------
UPDATE
dataset
SET
external_filename = dataset_filename.filename,
_extra_files_path = dataset_filename.extra_files_path
FROM
dataset_filename
WHERE
dataset.filename_id = dataset_filename.id;

mysql or sqlite sql command(s):
-------------------------------
UPDATE
dataset,
dataset_filename
SET
dataset.external_filename = dataset_filename.filename,
dataset._extra_files_path = dataset_filename.extra_files_path
WHERE
dataset.filename_id = dataset_filename.id;

---------------

6. Populate dataset table with info from dataset_child_association table:

postgres sql command(s):
------------------------
UPDATE
dataset
SET
parent_id = dataset_child_association.parent_dataset_id
FROM
dataset_child_association
WHERE
dataset.id = dataset_child_association.child_dataset_id;

mysql or sqlite sql command(s):
-------------------------------
UPDATE
dataset,
dataset_child_association
SET
dataset.parent_id = dataset_child_association.parent_dataset_id
WHERE
dataset.id = dataset_child_association.child_dataset_id;

---------------

7. Drop dataset_child_association table:

any database - sql command(s):
------------------------------
DROP TABLE dataset_child_association;

---------------

8. Copy parts of the current dataset table to the new history_dataset_association table:

postgres sql command(s):
------------------------
INSERT INTO history_dataset_association
SELECT
id,
history_id,
id AS dataset_id,
create_time,
update_time,
hid,
name,
info,
blurb,
peek,
extension,
metadata,
parent_id,
designation,
deleted,
visible
FROM dataset
ORDER BY id;

mysql or sqlite sql command(s):
-------------------------------
INSERT INTO
history_dataset_association
(
history_id,
dataset_id,
create_time,
update_time,
hid,
name,
info,
blurb,
peek,
extension,
metadata,
parent_id,
designation,
deleted,
visible
)
SELECT
history_id,
id,
create_time,
update_time,
hid,
name,
info,
blurb,
peek,
extension,
metadata,
parent_id,
designation,
deleted,
visible
FROM
dataset
ORDER BY
id;

---------------

9. Update the nextval value of the primary key sequence in the history_dataset_association table

NOTE: THIS IS NOT NECESSARY FOR MYSQL OR SQLITE!

postgres sql command(s):
------------------------
SELECT
setval('history_dataset_association_id_seq', max(id))
FROM
history_dataset_association;

---------------

10. Update dataset table to mark dataset_associated_file dataset as deleted:

postgres sql command(s):
------------------------
UPDATE dataset
SET
deleted = 't'
WHERE
id IN
(SELECT dataset_id FROM dataset_associated_file ORDER BY dataset_id DESC);

mysql or sqlite sql command(s):
-------------------------------
UPDATE dataset
SET
deleted = TRUE
WHERE
id IN
(SELECT dataset_id FROM dataset_associated_file ORDER BY dataset_id DESC);

---------------

11. Drop the dataset_associated_file table:

any database - sql command(s):
------------------------------
DROP TABLE dataset_associated_file;

---------------

12. Alter foreign keys on job_to_input_dataset table:

postgres or sqlite sql command(s):
------------------------
ALTER TABLE
job_to_input_dataset
DROP CONSTRAINT
job_to_input_dataset_dataset_id_fkey;

ALTER TABLE
job_to_input_dataset
ADD CONSTRAINT job_to_input_dataset_dataset_id_fkey
FOREIGN KEY
(dataset_id)
REFERENCES
history_dataset_association(id);


mysql sql command(s):
-------------------------------
ALTER TABLE
job_to_input_dataset
DROP FOREIGN KEY
ix_job_to_input_dataset_dataset_id;

ALTER TABLE
job_to_input_dataset
ADD FOREIGN KEY
ix_job_to_input_dataset_dataset_id (dataset_id)
REFERENCES
history_dataset_association(id);

---------------

13. Alter foreign keys on job_to_output_dataset table:

postgres or sqlite sql command(s):
------------------------
ALTER TABLE
job_to_output_dataset
DROP CONSTRAINT
job_to_output_dataset_dataset_id_fkey;

ALTER TABLE
job_to_output_dataset
ADD CONSTRAINT
job_to_output_dataset_dataset_id_fkey
FOREIGN KEY (dataset_id)
REFERENCES history_dataset_association(id);

mysql sql command(s):
-------------------------------
ALTER TABLE
job_to_output_dataset
DROP FOREIGN KEY
ix_job_to_output_dataset_dataset_id;

ALTER TABLE
job_to_output_dataset
ADD FOREIGN KEY
ix_job_to_output_dataset_dataset_id
FOREIGN KEY (dataset_id)
REFERENCES history_dataset_association(id);

---------------

14. Eliminate columns from the dataset table previously copied to history_dataset_association:

any database - sql command(s):
------------------------------
ALTER TABLE dataset DROP COLUMN hid;
ALTER TABLE dataset DROP COLUMN history_id;
ALTER TABLE dataset DROP COLUMN name;
ALTER TABLE dataset DROP COLUMN info;
ALTER TABLE dataset DROP COLUMN blurb;
ALTER TABLE dataset DROP COLUMN peek;
ALTER TABLE dataset DROP COLUMN extension;
ALTER TABLE dataset DROP COLUMN dbkey;
ALTER TABLE dataset DROP COLUMN metadata;
ALTER TABLE dataset DROP COLUMN parent_id;
ALTER TABLE dataset DROP COLUMN designation;
ALTER TABLE dataset DROP COLUMN visible;
ALTER TABLE dataset DROP COLUMN filename_id;
2008-07-09 17:41:08 +00:00
Greg Von Kuster d02838ce8b Missed a couple of order_by's in the job reports. 2008-07-03 12:55:55 +00:00
Greg Von Kuster 8053e0fe0c Fix for the user reports. 2008-07-02 19:12:52 +00:00
Greg Von Kuster 8a4e65b40d Optimized report queries. 2008-07-02 18:13:34 +00:00
James Taylor 69c4444de2 Workflow: create a new python module for 'WorkflowModules'. Tools are now another type of module, which removes a lot of special cases in the controller. Fixed some problems with validation and parameter groups. 2008-06-27 00:22:02 +00:00
James Taylor a662635a38 Workflow validation should work again with empty select lists (broken by changes to validation which caused empty selects to no longer submit a value). 2008-06-26 20:11:20 +00:00
Greg Von Kuster 27a34e9f21 More Galaxy reports - this should suffice for a while...
Number of jobs today - displays the total number of jobs for today
Number of jobs per Internet domain - displays the number of jobs that have been submitted per Internet domain
Number of registered users - displays the total number of registered users
2008-06-26 18:52:38 +00:00
Daniel Blankenberg 3539922e65 Fix typo on last commit 2008-06-26 14:59:41 +00:00
Daniel Blankenberg 7606f2681f Minor fix for dynamically filtered datasettool parameter when no valid datasets exist. 2008-06-26 14:02:41 +00:00
Greg Von Kuster 80ac564669 Galaxy Jobs reports
Number of jobs per day for current month - displays days of the current month with the total number of jobs for each day
Number of jobs in error per day for current month - displays days of the current month with the total number of jobs in error for each day
All unfinished jobs - displays jobs that are currently in a "new" state, a "queued" state or a "running" state
Number of jobs per month - displays a list of months with the total number of jobs for each month
Number of jobs in error per month - displays a list of months with the total number of jobs in error for each month
Number of jobs per user - displays users sorted in descending order by the number of jobs they have submitted
Number of jobs per tool - displays tools sorted in alphabetical order by tool id and the number of jobs created by the tool
2008-06-25 14:45:28 +00:00
Greg Von Kuster fac1a8e2b7 Fix for my last "fix" to SecurityHelper which actually broke it, need more bytes in the RandomPool. 2008-06-19 18:19:09 +00:00
Greg Von Kuster a1d98205e1 Change to SecurityHelper, no need to cache RandomPool, just create a new RandomPool with exactly 128 bits of entropy each time. 2008-06-19 15:22:07 +00:00
Greg Von Kuster 3443e25e06 Introducing basic session security - REQUIRES DATABASE SCHEMA CHANGE and REQUIRES CONFIG CHANGE.
SQL commands for schema changes:

ALTER TABLE galaxy_session ADD COLUMN current_history_id INTEGER;
ALTER TABLE galaxy_session ADD FOREIGN KEY (current_history_id) REFERENCES history(id);
ALTER TABLE galaxy_session ADD COLUMN session_key VARCHAR(255) UNIQUE;
ALTER TABLE galaxy_session ADD COLUMN is_valid BOOLEAN DEFAULT false;
ALTER TABLE galaxy_session ADD COLUMN prev_session_id INTEGER;

Galaxy config change:

In the [app:main] section, add:

# Galaxy session security
id_secret = changethisinproductiontoo

1) The "universe", "universe_user" and "universe_session" cookies are deprecated and replaced with 1 new cookie named galaxysession".  The deprecated cookies are still supported and will be for at least 90 days, but when a valid one is found, the value is taken and used to create a new "galaxysession" cookie and the deprecated cookie is immediately expired.

2) The value of the new "galaxysession" cookie is an encrypted unique, high entropy 128 bit random number.  The decoded value is the value stored in the new session_key column of the galaxy_session table.

3) Whenever a user logs in or out they get a new GalaxySession, and the old one in the database is marked as invalidated, so a compromised cookie won't survive across login/logout. Also, when creating the new session, a reference to the previous one is saved ( via the new prev_session_id column in the galaxy_session table ) so we have a way to chain them together.
2008-06-19 14:13:00 +00:00
Daniel Blankenberg ecdfb6d3c0 Update implicit dataset conversion to work with tools that use config_files.
There was a case where a dataset could have been converted multiple times if the conversion took place in a previous input in the same tool.

Also several other miscellaneous bug fixes.
2008-06-13 17:41:23 +00:00
Greg Von Kuster 2b4df13a23 Grouping tool version updateed due to tool behvioral change. Also do not read entire fatsa / fastq dataset twice to create blurb. 2008-06-12 12:20:41 +00:00
Greg Von Kuster 666de20b4d Cleanup for fastqsolexa data type converters. We are currently supporting only the Solexa variant and will add support for other variants when the formats stabilize. 2008-06-09 20:15:04 +00:00
Daniel Blankenberg a2bb0fe114 Call set_meta() on child datasets when they are created. 2008-06-09 15:58:57 +00:00
Greg Von Kuster 599fd15a00 Requires Galaxy config change - merged former Galaxy config [datatypes] and [sniff order] sections, along with former datatype_converters.xml into 1 xml file, datatypes_conf.xml.
The new datatypes_conf.xml.sample should be copied to datatypes_conf.xml.
2008-06-09 15:20:42 +00:00
James Taylor 33baa1ebec Child datasets can now be deleted, and deleting the parent recursively
deletes the child.
2008-06-07 00:45:20 +00:00
Wen-Yu Chung 782d7db6be Update fastq format.
Now we only support FastqSolexa variants.
If the quality scores are presented as characters,
the integer values are obtained by their ascii code subtract 64.
2008-06-06 18:52:11 +00:00
Greg Von Kuster e767816519 Fix fastq_to_qual_converter to use 33 as default rather than 64 - now the same as fastq_to_fasta_qual tool. Added fastqsolexa as additional format for fastq converters. 2008-06-06 14:53:19 +00:00
Greg Von Kuster f896510573 Use a different directory for report's compiled mako template cache. 2008-06-05 19:47:53 +00:00
Daniel Blankenberg 70b22fbc60 All datasets are now properly wrapped by DatasetFilenameWrapper. Certain cases had some datasets wrapped by InputValueWrapper. 2008-06-05 18:52:26 +00:00
Daniel Blankenberg dac0f0105d More workflow and dynamic options compatibility fixes. 2008-06-05 16:35:51 +00:00
Ross Lazarus a4252908c8 added snpmatrix data type to genetics.py 2008-06-05 15:46:56 +00:00
Greg Von Kuster 928bc214de Make reports functional in Firefox, IE. 2008-06-04 17:44:12 +00:00
Daniel Blankenberg bef6fd40e3 Typo in DataToolParameter.converter_safe() 2008-06-04 17:43:46 +00:00
Daniel Blankenberg 323454c91c Make DataToolParameter.converter_safe() compatable with deprecated <dynamic_options> 2008-06-04 17:00:32 +00:00
Daniel Blankenberg abd78d076a Fix drill down parameter function in workflow. 2008-06-04 16:32:05 +00:00
Greg Von Kuster 04e0834500 Revert 2715. 2008-06-04 13:02:15 +00:00
Greg Von Kuster dedc7278a1 Requires config change, temporary files are now created in a configurable location - affects many tools. 2008-06-03 20:29:54 +00:00
Daniel Blankenberg 86f0862aa4 Move ucsc and gbrowse data from static to tool-data/shared.
Cron scripts on test and main will need to be updated when the server is.

GOPS complement tool and annotation profiler will now correctly use chromosome lengths files if available.
2008-06-03 18:53:26 +00:00
Nate Coraor f28fe5c4f0 Removed non-Bourne syntax from the SGE job script template. 2008-06-03 16:47:03 +00:00