Nicola Soranzo
cc336f61c0
Make some files compatible with Python3
...
Also enlarge the set of files tested with flake8 under Python3
2016-06-03 19:24:46 +01:00
Nicola Soranzo
8a43b2b492
Python 3: use "as" instead of comma in except clause
2016-05-17 23:53:16 +01:00
John Chilton
997b53f3ae
Implement option to prevent FTP uploads from being deleted on import.
...
- Simply set `ftp_upload_purge` to `False` in galaxy.ini to enable this.
- xref http://dev.list.galaxyproject.org/Data-removal-after-ftp-import-td4667111.html
- slight xref http://dev.list.galaxyproject.org/Uploading-files-to-galaxy-from-a-folder-td4664614.html
2016-04-14 11:34:21 -04:00
Nicola Soranzo
501a10c3f1
Do not import dumps and loads from galaxy.util.json .
2016-03-02 19:04:24 +00:00
John Chilton
448aee9f00
Fix for multi-byte char problem in PR #1556...
...
... caught by @nsoranzo of course.
2016-02-08 13:48:43 +00:00
John Chilton
499357e508
Remove unneeded galaxy.model import in upload.py...
...
... and document the ability to remove from other data sources.
2016-01-21 14:31:42 +00:00
John Chilton
50e531aa59
Refactor galaxy.datatypes.checkers into galaxy.util.checkers.
...
- Primary reason is because it is used all over the place in contexts that have nothing to do with Galaxy datatype classes or datatype registery or sniffing, most importantly by the code in the tool shed for determining if something is a tool or not.
- This code has no dependencies on the datatypes module or any hard dependencies on things not in the Python standard library.
2015-12-17 15:41:06 +00:00
John Chilton
83f57c56c1
Refactor is_multi_byte out of galaxy.util.
...
It introduces a fairly esoteric dependency in galaxy.util for just a couple uses in Galaxy. Separating it out like this will allow reuse of the entire galaxy.util with only one external dependency (docutils, which planemo for instance already depends on).
2015-12-16 21:36:16 +00:00
Mark Einon
d6d8e5ef86
upload.py: Remove unused function parameter
2015-11-13 11:33:56 +00:00
Nicola Soranzo
7830bc62e0
Another fix for 3c40f32fa2 .
2015-06-17 19:21:13 +01:00
Nicola Soranzo
3c40f32fa2
PEP-8 and pylint fixes.
2015-06-16 16:53:49 +01:00
Bjoern Gruening
5b96f80b48
Add CompressedArchive as datatype and do not uncomress it during upload.
2015-01-12 22:36:20 +01:00
Daniel Blankenberg
b022e92f5b
Add gemini.sqlite datatype.
2015-01-09 14:48:56 -05:00
Dannon Baker
9d85ca9e23
Switch to_json_string/from_json_string in galaxy/tools.
2014-09-09 10:06:35 -04:00
John Chilton
6d4d6dd91a
Bugfix: Fix shutil.move for converted files in upload.py.
2014-06-10 09:36:28 -05:00
Dannon Baker
c30b7838cc
Fix for when check_binary fails against an actual binary datatype. The logic in upload needs to be revisited per the comments.
2014-06-02 13:16:15 -04:00
Trevor Wennblom
1848cf2e3d
whitespace
2014-02-20 18:14:09 -06:00
Gert Hulselmans
fb01c3cc96
Avoid corruption of binary files embedded in gzip, bz2 and zip archives in the upload tool.
...
Add an option in the upload tool to disable the conversion of universal
line endings to Posix line endings.
This is useful for avoiding corruption of uploaded files when a binary
file is contained inside a gzip, bz2 and zip archive.
This fixes bug report: https://trello.com/card/issue-with-uploaded-2bit-gz-files/506338ce32ae458f6d15e4b3/702
2013-05-27 20:55:52 +02:00
Dannon Baker
8161b70d89
Merged in nsoranzo/galaxy-central (pull request #135 )
...
Add generic ASN.1 text and binary datatypes
2013-04-04 08:41:54 -04:00
Nicola Soranzo
2f650b8866
Fix file extension determination
...
Extension was determined uncorrectly for filenames with more than 1 dot
2013-03-06 12:37:08 +01:00
Kyle Ellrott
4aad0ad15e
Enabling dataset UUID setting through library upload, upload1 tool, and galaxy.json file
2013-01-29 15:58:58 -08:00
John Chilton
886a5c61b9
Merge.
2012-09-03 22:14:36 -05:00
Daniel Blankenberg
48548341fc
Handle non-ascii unicode in upload tool. Significant pre-existing refactoring still needed.
2012-08-28 19:23:39 -04:00
John Chilton
d281e835ef
Remove hard-coding of unsniffable binary types array and manually
...
checking each sniffable type with a seperate function in
upload.py. Information on both types is now stored dynamically as
static variables in the Binary class.
2012-08-15 23:35:28 -05:00
Nate Coraor
c362e6202e
A bit more actual user cleaning.
2012-01-18 14:34:00 -05:00
Nate Coraor
61062a3ae5
Merge galaxy-central.
2012-01-18 10:34:39 -05:00
Nate Coraor
85ba013e6d
Actual User: Fix for newline conversion on upload, tighten file permissions for files in upload. Please make sure you clean your temp directory regularly.
2011-12-08 13:22:39 -05:00
Greg Von Kuster
17c5b27cdc
Re-engineer the datatypes registry so that it is initialized once when the Galaxy server is started, but data types can continue to be loaded throughout the Galaxy server's session (hopefully this doesn't break anything).
...
Add support for a single "import_module" to be passed to the new load_datatypes() method in the datatypes registry. This provides the ability to load a single class module from an installed tool shed repository along with a datatypes_conf.xml file included in the installed repository and pass them to the new load_datatypes() method. In the future, multiple imported modules may be allowed. The datatypes_conf.xml file included in the repository must conform to a slightly different definition than the same named file that comes with the distribution. This new definition will be documented in the Galaxy tool shed wiki.
We now have the ability to load new data types into the Galaxy server from an installed tool shed repository without restarting the Galaxy server.
2011-11-23 16:16:15 -05:00
ichorny
236276b132
add actual user changes to galaxy-central
2011-10-21 14:01:48 -07:00
Nate Coraor
7d8b701df6
Fix uploading sorted BAMs to libraries when using the link method rather than copying.
2011-08-26 15:59:29 -04:00
Greg Von Kuster
d8279f0e5c
Add an altered version of Jelle Scholtalbers' enhancement code to support uploding various image data types. I've moved some of the code components around from Jelle's version, and added some fixes. The cod ein the new image_util.py file enables detections of images types without the need to create a new Image() class.
...
I've also added baseline support of the HDF5 data type in this change set, but it is currently required to be in the unsniffable_data_types list.
2011-07-27 16:14:15 -04:00
Nate Coraor
3959897d3d
Uploading a compressed library dataset using either the server directory or path paste methods was previously overwriting the input file, which should never happen. This changeset also makes it possible to upload compressed files that remain compressed (by choosing to "link files only"), however, Galaxy does not yet open the compressed file to properly set metadata on its contents.
2011-06-08 15:35:36 -04:00
Greg Von Kuster
61c3ed305f
Move duplicate data type checker methods from sniff and upload into a new ~/datatypes/checkers.py.
2011-06-03 15:36:59 -04:00
Greg Von Kuster
7640af8e06
Don't alter the contents of a file while uploading to a data library if using the filesystem_paths option. This partially resolves the issue where a supposedly sorted BAM file was being resorted upon upload to a data library when using this option. A better imlementation of determining whether a BAM file has been sorted (so that it does not get resorted) remains to be done.
2011-03-15 09:38:29 -04:00
Greg Von Kuster
df6f94b1d0
Per request from Peter Cock, turn the "Copy data into Galaxy?" check box into a select list that clarifies the behavior. The default is to copy files into Galaxy.
2011-03-11 14:56:11 -05:00
Nate Coraor
82c4a11c88
Handle a minor bug in the upload tool.
2011-01-03 12:05:13 -05:00
Nate Coraor
b140866d0b
Heh heh. Whoops.
2010-11-30 14:37:19 -05:00
Nate Coraor
a07ef2cdd3
Add Job stdout/stderr to library item info page to aid in debugging.
2010-11-30 14:34:17 -05:00
Nate Coraor
8367cdc4a6
"Fix" zip uploads for Python < 2.6. The scare quotes are because in Python < 2.6 you can't read a zip member without loading the entire member into memory...
2010-10-25 16:10:09 -04:00
Nate Coraor
2fb759bd99
Read zipfiles in chunks when uncompressing in the upload tool. And I continue to wish zipfile was a lot more like tarfile...
2010-10-20 17:00:06 -04:00
Nate Coraor
4ca447aa70
Add support for bz2 compressed uploads.
2010-09-10 14:07:16 -04:00
Greg Von Kuster
a7e30c4441
Apply patch from Brad Chapman providing support for detecting, uploading and displaying UCSC bigWig and bigBed. Add new functional tests for uploading and detecting bigbed and bigwig formats, and correct and clean up the test_get_data.py functional test script.
2010-08-19 13:58:10 -04:00
Nate Coraor
8a66864aee
Stop ignoring the sniff order set in the datatypes config file.
2010-08-18 14:56:10 -04:00
Kanwei Li
810c2291de
Detect pdf files on upload [Brad Chapman]. Resolves #357
2010-07-02 18:10:45 -04:00
Nate Coraor
7c71894533
Remove obsolete binseq.zip and txtseq.zip formats, and allow for uploading single files in a zip archive. Adapted from a patch from Pablo Cingolani.
2010-06-23 16:48:13 -04:00
Ross Lazarus
6c39995d19
Patch to upload.py for composite file uploads where the user supplies urls for components
...
At present, the path for each component is passed as a url and there's no upload type for
composite file components, so this
temporary fix searches for :// in the path to figure out if this is an url upload.
Not ideal and in need of a more robust repair.
2010-06-11 21:21:33 -04:00
Daniel Blankenberg
183c2d3d13
Bug fix for handling 'spaces to tab' for non-binary composite datatype files.
...
Add upload tests for testing 'space to tab' when uploading composite and non-composite datatypes.
2010-06-07 15:34:20 -04:00
Daniel Blankenberg
64c68b5a66
Several Fixes for Uploading Composite datatypes. Revert changeset 3769:69e70e588a8e.
2010-05-12 15:57:05 -04:00
Ross Lazarus
84b05c8113
Fixes to rgQC top table ordering and add offset so can use rgManQQ to make plots
...
Fixes to rgManQQ to handle unsorted data - now sorts before manhattan plot
2010-04-05 14:14:09 -04:00
Nate Coraor
933944c12d
Fix bug in upload where source import files could be removed if an error occurs during detection and galaxy owns the source files.
2010-03-23 12:45:08 -04:00
Nicola Soranzo