Daniel Blankenberg
8cd2a39cc7
Add metabolicMine datasource tool.
2011-02-09 15:42:42 -05:00
Daniel Blankenberg
6d3d3edf19
Add YeastMine datasource tool. Site is still under development.
2011-02-01 17:29:40 -05:00
Greg Von Kuster
5f9f45a068
Several data library improvements:
...
1. Enhance importing library dataset into the current history by now allowing the user to select from a list of existing histories or create a new named history ( similar to copying datasets ).  This feature is available for a specific library dataset, a folder ( or root folder ), when selecting multiple datasets from in a library,  and on the results page for searched library datasets.
2. Add Brad Chapman's latest patch for fixing the lucene search implementation
3. Eliminate searches on dataset state for both whoosh and lucene searches
4. Clarify the objects being searched in the regular search and advanced search boxes in the data libraries grids.
5. Add the ability to perform certain actions on 1 or more selected library datasets on the results page for searched library datasets.
6. Add a "select all" checkbox to the browse_library.mako template.
7. Fixed library functional tests accordingly.
2011-01-31 16:41:03 -05:00
Nate Coraor
15aa3cba69
Highlight the futility of uploading files >2GB via a browser.
2011-01-14 23:52:31 -05:00
Nate Coraor
82c4a11c88
Handle a minor bug in the upload tool.
2011-01-03 12:05:13 -05:00
Nate Coraor
b140866d0b
Heh heh. Whoops.
2010-11-30 14:37:19 -05:00
Nate Coraor
a07ef2cdd3
Add Job stdout/stderr to library item info page to aid in debugging.
2010-11-30 14:34:17 -05:00
Daniel Blankenberg
365f9d6ad8
Add modENCODE modMine server tool to tool_conf.xml.sample.
2010-11-23 14:50:46 -05:00
Ross Lazarus
d1270499ea
adding hapmapmart.xml to tools/data_source
2010-11-18 10:30:27 -05:00
Nate Coraor
c9a881e114
Add a new "Files uploaded via FTP" grid to the upload form and related parameter types, form fields, etc.
2010-11-05 11:04:50 -04:00
Nate Coraor
8367cdc4a6
"Fix" zip uploads for Python < 2.6. The scare quotes are because in Python < 2.6 you can't read a zip member without loading the entire member into memory...
2010-10-25 16:10:09 -04:00
Nate Coraor
2fb759bd99
Read zipfiles in chunks when uncompressing in the upload tool. And I continue to wish zipfile was a lot more like tarfile...
2010-10-20 17:00:06 -04:00
Nate Coraor
4ca447aa70
Add support for bz2 compressed uploads.
2010-09-10 14:07:16 -04:00
Greg Von Kuster
a7e30c4441
Apply patch from Brad Chapman providing support for detecting, uploading and displaying UCSC bigWig and bigBed. Add new functional tests for uploading and detecting bigbed and bigwig formats, and correct and clean up the test_get_data.py functional test script.
2010-08-19 13:58:10 -04:00
Nate Coraor
8a66864aee
Stop ignoring the sniff order set in the datatypes config file.
2010-08-18 14:56:10 -04:00
Daniel Blankenberg
70a68cd688
Remove UCSC tablebrowser specific parameters from modmine.xml datasource tool.
2010-08-16 09:22:40 -04:00
Kanwei Li
810c2291de
Detect pdf files on upload [Brad Chapman]. Resolves #357
2010-07-02 18:10:45 -04:00
Daniel Blankenberg
3b70b41f7d
Update UCSC table browser datasource tools to treat data marked as 'gff' as 'gtf'.
2010-06-28 11:50:25 -04:00
Nate Coraor
7c71894533
Remove obsolete binseq.zip and txtseq.zip formats, and allow for uploading single files in a zip archive. Adapted from a patch from Pablo Cingolani.
2010-06-23 16:48:13 -04:00
Ross Lazarus
6c39995d19
Patch to upload.py for composite file uploads where the user supplies urls for components
...
At present, the path for each component is passed as a url and there's no upload type for
composite file components, so this
temporary fix searches for :// in the path to figure out if this is an url upload.
Not ideal and in need of a more robust repair.
2010-06-11 21:21:33 -04:00
Daniel Blankenberg
183c2d3d13
Bug fix for handling 'spaces to tab' for non-binary composite datatype files.
...
Add upload tests for testing 'space to tab' when uploading composite and non-composite datatypes.
2010-06-07 15:34:20 -04:00
Greg Von Kuster
15abe33231
Add the CBI Rice Mart data source tool to the distribution.
2010-06-04 15:49:58 -04:00
Daniel Blankenberg
64c68b5a66
Several Fixes for Uploading Composite datatypes. Revert changeset 3769:69e70e588a8e.
2010-05-12 15:57:05 -04:00
Greg Von Kuster
95ac03c352
Remove the hack I just committed for changing modENCODE fly and worm dbkeys, and translate the values in the tool configs.
2010-05-12 15:18:40 -04:00
Greg Von Kuster
de5de2973c
Force modENCODE worm dbkey to be ce6 and modENCODE fly dbkey to be dm2.
2010-05-12 14:50:25 -04:00
Greg Von Kuster
53363c793e
Add the modENCODEfly and modENCODEworm data source tools.
2010-05-06 19:39:27 -04:00
Nate Coraor
ca066cef04
Add a tip to the upload tool about using url paste for big files
2010-05-04 10:18:42 -04:00
Ross Lazarus
84b05c8113
Fixes to rgQC top table ordering and add offset so can use rgManQQ to make plots
...
Fixes to rgManQQ to handle unsorted data - now sorts before manhattan plot
2010-04-05 14:14:09 -04:00
Nate Coraor
933944c12d
Fix bug in upload where source import files could be removed if an error occurs during detection and galaxy owns the source files.
2010-03-23 12:45:08 -04:00
Nate Coraor
482b69b749
Modify the newline conversion methods in sniff so converting in place is optional. This is necessary to fix a bug that occurs if using the 'server_dir' or 'path_paste' library upload methods: previously, they would modify the file to be imported in-place if permissions allowed (probably not what the admin wanted) or fail if permissions did not allow. New functionality is to return the converted tempfile if 'server_dir' or 'path_paste' methods are used. Also, no newline conversion will be done if the administrator uses the symlink checkbox.
2010-03-11 11:56:20 -05:00
Greg Von Kuster
19592b4e39
Update tool_conf.xml.main and the tool conf for eupathdb.
2010-01-27 08:51:23 -05:00
Kanwei Li
0f1d788690
Wiki links now point to bitbucket instead of trac
2010-01-19 18:15:27 -05:00
Greg Von Kuster
a14492c491
Added from __future__ import division to the Filter tool and increased the version, and fixed the URL in the new bx_browser tool config.
2010-01-14 11:26:08 -05:00
Greg Von Kuster
826da3acac
Add a tool config for the new bx_browser data source tool.
2010-01-14 10:24:37 -05:00
Greg Von Kuster
75753f1783
Add error handling to upload's check_zip.
2009-12-09 16:20:12 -05:00
Greg Von Kuster
cf83cecb0b
Better approach to altering initial content of output dataset if necessary. The upload tool will now call the data type's groom_output_dataset() method ( a better name than before_setting_metadata since it is not related to metadata ). This will now also run on the cluster.
2009-12-08 11:46:13 -05:00
Greg Von Kuster
7051ce7dd5
Rename the before_edit() and after_edit() metadata related methods to be more appropriately named before_setting_metadata() and after_setting_metadata(), and add the metthods to all datatype classes. Move sorting of Bam files from the Bam set_meta() method to this new method.
2009-12-08 09:05:35 -05:00
Greg Von Kuster
bcd6751cdf
Fix for uploading a Bam file that has not yet been sorted, the call to samtools sort has been moved from the sam_to_bam tool to the Bam()set_meta() method to ensure all Bam datasets are sorted prior to indexing. Added new functional tests to cover uploaing unsorted Bam files. Also cleaned up code in the upload tool for uploaing various binary data formats.
2009-12-07 16:04:33 -05:00
Daniel Blankenberg
f92b25d2f5
Enhance tool parameter sanitization. Sanitization rules can now be defined on a per input parameter basis. Resolves ticket #38 and #142 .
...
Sanitization now occurs as part of the InputValueWrapper logic. Previously sanitization occurred as soon as parameter values were provided by the user via form submits, causing the sanitized values to be stored in the database. Now the original user specified values are stored in the database and sanitation happens when str is called on the InputValueWrapper, i.e. during command-line generation and when generating configfiles.
Abstract out param name and value translations that are performed on data source tools. This changes some of the names of the tags used for these purposes (e.g. data_type_translation becomes value_translation) to reflect this generalization. The ToolConfigSyntax wiki page has been updated with these changes.
Future Enhancement:
Allow methods with arguments and keywords to be specified for sanitation and translation purposes.
2009-12-07 10:19:41 -05:00
Greg Von Kuster
08e0d5e532
Fixes for Bam data type's set_meta() and sam_to_bam tool, indexes will now be properly created for bam datasets. Fixes also for uploading Bam files, they will no longer be uncompressed on upload.
2009-12-04 19:51:46 -05:00
Greg Von Kuster
f16d27c2d4
Add datasource tool configs for modMine and Ratmine.
2009-12-03 15:33:15 -05:00
Greg Von Kuster
0e807f13f5
Add support for uploading BAM files.
2009-12-02 20:02:05 -05:00
Kanwei Li
82eef1efcf
Data from UCSC table browser now uses hgta_table for table name. Fixes #67
2009-11-20 15:25:13 -05:00
Greg Von Kuster
dd32491716
A bit of code cleanup in genetics.py, and add all rgenetics data types to datatypes_conf.xml.sample. Also include the new Sff data type in the upload config help section.
2009-11-13 16:24:00 -05:00
Greg Von Kuster
e38e78f5e4
Fixes, cleanup and new functional tests for data types and upload:
...
- moved all supported binary data types to the new binary.py
- changed GeneTrack data type to subclass from Text rather than Binary
- added Sff data type to datatypes_conf.xml.sample
- merged test_sniffinad_and_metadata_settings.py test scritp into test_get_data.py
- added several additional functional test for data types to test_get_data.py
- fixed some bugs in upload.py when uploading binary data types
2009-11-13 14:13:03 -05:00
Greg Von Kuster
dc8e7c1039
Eliminate the _monkeypatch_session_method from assignmapper by cleaning up remaining object flushes. The _monkeypatch_query_method remains due to a single object query in ~/datatypes/metadata.py in the FileParameter.wrap() method. The sqlalchemy session is now also passed into the __init__ methods for both HistoryDatasetAssociation and LibraryDatasetDatasetAssociation when the create_dataset param is True to enable methods in the DatasetInstance class to correctly add the created dataset to the sqlalchemy session and flush it.
2009-11-12 15:25:48 -05:00
Ramkrishna Chakrabarty
702bad5d25
Added sff datatype
...
Also, made changes to upload.py to sniff sff binary files
2009-11-11 17:55:08 -05:00
Greg Von Kuster
8f6e8213a0
Fix most of the db flushes to be compatibel with sqlalchemy 05. Add the _monkeypatch_query_method() back into assignmapper due to a single object.get() method in the MetadataCollection class since Metadata has no current hook into mapping.context ( the sqlalchemy session ). There a 4 flushes in metadata,py and 20 flushes in model.__init__.py that still use the _monkeypatch_session_method in assignmapper due to the same issue, but all other flushes are fixed.
2009-11-11 15:59:41 -05:00
Nate Coraor
8d96c8d979
imported patch alchemy05_fixes_02
2009-11-03 12:52:01 -05:00
Nate Coraor
8d24d0aeba
Remove old encode import tools
2009-11-02 16:21:28 -05:00
Daniel Blankenberg