Commit Graph
70 Commits
Author SHA1 Message Date
Kanwei Li fc550cdbab Remove array_tree datatype; add tabix 2011-03-31 18:58:02 -04:00
Daniel Blankenberg d95c6b2e52 Add IGV as an external display application. Contributed by Tobias Wohlfrom.
Provides two links for IGV:
1) web - to view with Java web start IGV if the user has no IGV installed
2) local - to view in the user's running IGV instance (requires remote port enabled in IGV)
2011-03-23 09:57:06 -04:00
Daniel Blankenberg d892a301e3 Add MEME tool configuration file. 2011-02-01 11:17:44 -05:00
Jeremy Goecks 9d34f038b2 Add GTF sniffer to datatypes config sample file. 2011-01-31 18:25:33 -05:00
Kanwei Li 5962f0ab2e trackster: Add bigWig display to trackster. Automatically converts wig to bigwig if needed (NOTE: datatypes_conf.xml.sample has been edited to add the new converter, you must update datatypes_conf.xml to use it). The converter requires that wigToBigWig be in the PATH, but no other tools are needed to view bigwig files as they are provided by bx_python.
- Fix track preferences not being applied
- Fix chroms not being selectable when a new track browser is created
- Fix ReferenceTrack not working with filters
- Fix visual analytics error when tool configuration has changed
2010-12-10 14:59:43 -05:00
Daniel Blankenberg db737b3003 Add basic support for bowtie indexes as a datatype (bowtie_base_index, bowtie_color_index), available via datatype conversion. Currently, the indexes need to be converted manually from the FASTA file before use in bowtie, but they can be reused.
More work is required to allow the one-off indexes built by bowtie to become Galaxy datasets; alternatively, the custom genome selection could be limited to the index datatype for input (and not allow fasta directly), which would allow implicit datatype conversion to occur when a fasta file is selected as input, but this would prevent the index tuning that is currently available when currently selecting a fasta file.
2010-10-07 16:59:41 -04:00
Jeremy Goecks 9c7065d898 Make VCF (variant call format) a Galaxy datatype and enable very basic VCF support in trackster. VCF datatype is sniffable and can be converted to summary tree and interval index. In trackster, VCF files are represented as single-base pair feature tracks. 2010-10-06 16:23:55 -04:00
Brad Chapman 8679ac8442 Add support for displaying BAM files at Ensembl 2010-10-06 11:03:28 -04:00
Ramkrishna Chakrabarty 5219ada91e added svg datatype 2010-08-25 12:27:24 -04:00
Greg Von Kuster a7e30c4441 Apply patch from Brad Chapman providing support for detecting, uploading and displaying UCSC bigWig and bigBed. Add new functional tests for uploading and detecting bigbed and bigwig formats, and correct and clean up the test_get_data.py functional test script. 2010-08-19 13:58:10 -04:00
Daniel Blankenberg 063b5c5381 Add new-style display applications for GBrowse for gff, interval, and wig; only interval is enabled by default, the other two use existing methods. Still working on SAM and BAM for GBrowse. 2010-07-23 16:45:41 -04:00
Jeremy Goecks bd1033e0e8 New feature: GFF files can be viewed in trackster. Specific additions: (a) generalized bed-to-summary-tree converter and bed-to-interval-index converter to handle both BED and GFF files and renamed accordingly; (b) augmented trackster to provide payload data from both BED and GFF files. 2010-07-14 09:26:26 -04:00
Kanwei Li 810c2291de Detect pdf files on upload [Brad Chapman]. Resolves #357 2010-07-02 18:10:45 -04:00
Nate Coraor 7c71894533 Remove obsolete binseq.zip and txtseq.zip formats, and allow for uploading single files in a zip archive. Adapted from a patch from Pablo Cingolani. 2010-06-23 16:48:13 -04:00
Daniel Blankenberg 2603f7efc2 Add mafcustomtrack datatype and a UCSC external display application which can be used to view this datatype at UCSC. 2010-06-14 15:05:07 -04:00
Greg Von Kuster ae749436e0 Add support for velvet data types and velveth and velvetg tool wrappers, all contributed by James E Johnson - University of Minnesota. 2010-06-09 11:23:10 -04:00
Daniel Blankenberg 169a4d2f05 First pass at implementing a method for allowing a maximum file size cutoff for setting optional metadata (e.g. line and sequence counts). Currently csFasta, qualsolid, and fastq make use of this option. 2010-05-24 14:15:02 -04:00
Daniel Blankenberg ab53cc6d3b First pass at adding Ensembl browsers as an external display application. Two different URL generation and data attachment methods are used; one for 'old' Ensembl archives older than ~November 2008 and another for Ensembl sites using the current method. The tool-data/shared/ensembl/ensembl_sites.txt file contains the site and build information for using the current method; the tool-data/shared/ensembl/ensembl_sites_data_URL.txt file has the site and build information for when the older method is to be used.
The new method follows: http://www.ensembl.org/info/docs/webcode/linking.html

The old method follows: http://aug2007.archive.ensembl.org/Homo_sapiens/helpview?se=1;kw=urlsource
2010-05-21 15:25:56 -04:00
Ross Lazarus d61fd8c4df Added ldindep datatype to datatypes sample
Added snpwga tool sections to tool_conf.sample.main - need these
2010-05-20 14:16:08 -04:00
Daniel Blankenberg f08c5cd9ec Add eland and elandmulti formats. Enhance MACs peakcaller to properly handle these formats. 2010-04-27 10:59:46 -04:00
Jeremy Goecks e77c308f4f Add GTF file format to Galaxy. GTF format is an extension of GFF format that is used by Tophat and Cufflinks tool suite. 2010-04-19 11:14:19 -04:00
Kanwei Li 34518df336 Add BedGraph datatype (wiggle-like data for intervals)
trackster:
- Fix various UI issues
- BedGraph -> array_tree converter
2010-04-06 19:00:39 -04:00
Kanwei Li 412f24d567 - Simple converter dependency system where converter won't run until its deps finish
- Fix issue with trying to treat binary data as unicode

trackster:
- Make BAM use summary tree
- Refactor converter code
2010-04-01 14:24:47 -04:00
Daniel Blankenberg 6bce16592e Add bed6 and bed12 datatypes, which are subclasses of bedstrict; converters are available to turn any interval datatype into these types. Change GeneTrack Indexer tool and converter to use bed6 as input. 2010-04-01 13:08:48 -04:00
Kelly Vincent 710e9cfd07 Added new tabular/pileup join tool and associated test files. Also added pileup datatype. 2010-03-26 12:00:55 -04:00
Kanwei Li 1bfffa518c trackster:
- New summary tree data structure that specializes in aggregation at defined levels
- BED converter for summary_tree
- Feature tracks now have a summary display when there are a lot of things to be rendered on screen, but will display simple lines and full detailed view as you zoom in and can fit more on screen
- Added option to display number of features in a region of summary view
- Line tracks can now be displayed as intensity
- Pack scripts
2010-03-25 15:21:57 -04:00
Kanwei Li e6215b5551 trackster:
- Use new array_tree summary structure. Feature tracks now display as intensity graphs at higher levels, and switch to detail levels when they fit the screen well
- Create array_tree indices for bed format to support the above (bam still in progress)
- Other fixes and improvements, including new icons
2010-03-16 18:54:23 -04:00
Daniel Blankenberg 5ede7cba83 Display Application framework enhancements.
Add the ability for display applications to be populated dynamically based upon the content of (e.g. tabular) files.
    Display application links can be filtered by various attributes, including e.g. dataset dbkey matching from field in a file or an attribute matching a Galaxy application configuration setting.
    Param and Data URL values can now be generated dynamically, allowing e.g unique base filenames to be created and used.
See updated xml configurations in /display_applications/ for examples of syntax.
2010-03-11 14:35:36 -05:00
Ross Lazarus fa4022a09a Remove old Lmap from datatypes_conf.xml.sample - now deprecated from genetics.py
Causing buildbot to barf.
2010-03-10 20:59:11 -05:00
Daniel Blankenberg a19ae79b85 Change color space FASTA file type from fastqsolid to fastqcssanger.
Cripple accepted tool input formats for many of the FASTQ tools to only allow only fastqsanger and fastqcssanger to be used.
2010-03-02 10:47:23 -05:00
Daniel Blankenberg 8082c6f36f Add a new FASTQ tool suite. Four FASTQ variants are supported: sanger, illumina, solexa and solid.
Tools include:
	FASTQ Groomer convert between various FASTQ quality formats
	Combine FASTA and QUAL into FASTQ
	FASTQ joiner on paired end reads
	FASTQ splitter on joined paired end reads
	FASTQ to FASTA converter
	FASTQ Summary Statistics by column
	Filter FASTQ reads by quality score and length
	FASTQ Trimmer by column
	Manipulate FASTQ reads on various attributes
	Boxplot of quality statistics (Generic, with outliers)
2010-02-23 16:48:07 -05:00
Daniel Blankenberg d1c905f78d Introduce a new style of external display applications. Display applications can now be entirely defined using xml files, similar to how tools are integrated.
Applications are assigned to specific datatypes (i.e. on an extension basis) via the datatypes_conf.xml file.

View the sample display applications at /display_applications/[ucsc/]*.xml for examples of usage.

Provided sample display applications:
	View BAM files (with bai indexes) at UCSC using BigDataUrl support.
	ucsc interval as bed viewer - not enabled by default (the old style display app is still used by default; both can be used simultaneously - but this would likely be confusing)
	GeneTrack viewer - any interval datatype can now be viewed at GeneTrack, if the application is enabled for a particular datatype; also a valid display application for genetrack datatype.

Display applications can make full use of datatype converters, even allowing explicitly defined multi-step conversions, e.g. interval --> bed --> genetrack; the datatype conversion framework will need to be enhanced to natively support multi-step conversions before this can be done implicitly.


A new datatype, bedstrict, has been defined, the only way to have an item with this datatype is to be created by a tool; metadata cannot be edited; and sniffing this datatype would require aggressively parsing the entirety of the file. A bedstrict file must conform exactly to the BED specification (whereas Galaxy allows BED files to have non-standard columns). These files are suitable e.g. for display at the UCSC genome browser and is used by the new ucsc interval display application.



Add a bed to bedstrict converter, this is used by the ucsc interval display application.

Add a bed to genetrack converter, this is used by the new GeneTrack display application. TODO: If the GeneTrack indexer can be enhanced to accept column assignments, this should be an interval to genetrack converter.


Several performance enhancements available for the ucsc tools, such as bigurl support, potential speed improvement when loading a user's history than the old style for certain displays, e.g. ucsc interval display no longer requires the viewport (position) to be calculated for each relevant history item in a users history; this calculation now occurs on a separate page after the user clicks a view link. Non-strict BED files no longer have their content calculated on the fly and then streamed, etc.


Refer to additional comments in code.
2010-02-12 11:01:30 -05:00
Kanwei Li 4ca761e9cc trackster: support BAM visualization with samtools, automatically enable tracks for any datatype with "get_track_type" defined 2010-01-07 15:45:15 -05:00
Greg Von Kuster 0e807f13f5 Add support for uploading BAM files. 2009-12-02 20:02:05 -05:00
Guruprasad Anada 10af4c5ccf Adding back 'wiggle to interval' converter 2009-12-01 11:49:07 -05:00
Greg Von Kuster dd32491716 A bit of code cleanup in genetics.py, and add all rgenetics data types to datatypes_conf.xml.sample. Also include the new Sff data type in the upload config help section. 2009-11-13 16:24:00 -05:00
Greg Von Kuster e38e78f5e4 Fixes, cleanup and new functional tests for data types and upload:
- moved all supported binary data types to the new binary.py
- changed GeneTrack data type to subclass from Text rather than Binary
- added Sff data type to datatypes_conf.xml.sample
- merged test_sniffinad_and_metadata_settings.py test scritp into test_get_data.py
- added several additional functional test for data types to test_get_data.py
- fixed some bugs in upload.py when uploading binary data types
2009-11-13 14:13:03 -05:00
Greg Von Kuster f77e0e03f4 Add new functional tests for uploading composite data types lped and pbed - handles ticket # 173. 2009-10-13 16:37:20 -04:00
Kelly Vincent 4e00bcb315 Modified datatypes_conf so fastqsanger type displays in upload list. 2009-10-08 14:57:36 -04:00
Kelly Vincent 53d6ccf8f3 Removed sniff method from fastqsanger datatype 2009-10-08 10:45:31 -04:00
Kelly Vincent 7263afac9f Added fastq (generic) datatype and deleted fastqsolexa datatype 2009-10-07 12:43:15 -04:00
Kanwei Li c556f857b7 trackster now supports BED files 2009-09-25 18:47:40 -04:00
James Taylor b2c9c284de More work on server side for tracks. Now uses converters instead of indexers. A wiggle to array_tree converter is provided. Still need to purge indexer stuff. This will not work without new bx egg (or likely at all) 2009-09-18 15:15:39 -04:00
Kelly Vincent 6044060890 Added two new datatypes (sam and bam) and an associated test 2009-08-19 09:13:21 -04:00
Kelly Vincent 3bbac86312 Added fastqsanger data format and required bwa_wrapper to take only that format as input 2009-07-30 12:06:24 -04:00
James Taylor 537f8952ef Merging Ian's trackster update with current head 2009-06-11 12:20:03 -04:00
Guruprasad Anada c072bd4b4a Added functional tests for testing sniffing and metadata settings of qual454 and qualsolid datatypes. 2009-04-29 16:32:23 -04:00
Guruprasad Anada ee3cddc5df Replacing \"qual\" datatype with 3 new datatypes: qualsolid, qual454 and qualsolexa. 2009-04-29 11:53:22 -04:00
Ian Schenck feda672ffa - Performance of indexers much improved.
- Indexing for tracks done in background with a visual treatment done to Trackster

- DB builds can be uploaded by a user (chromInfo/len extension).

- TODO: Add ability to change the dbkey of a dataset to any arbitrary string value.
2009-04-23 13:46:52 -04:00
Ian Schenck 985f7c63fa Collapsed changeset: Trackster plugged into galaxy with behind-the-scenes indexing of Wiggle, coverage and interval datasets. 2009-03-11 00:29:01 +00:00