diff --git a/static/formatHelp.html b/static/formatHelp.html index 36bf5e098e7..7bf7523c736 100644 --- a/static/formatHelp.html +++ b/static/formatHelp.html @@ -1,5 +1,13 @@ + -Galaxy Data Formats + +Galaxy Data Formats + + +

Galaxy Data Formats

@@ -18,16 +26,15 @@ data, or even the correct columns needed by the tool, but filtering by format at least makes the list to select from a bit shorter.

Some of the formats are defined hierarchically, going from very -general ones like tabular (which includes any text +general ones like Tabular (which includes any text file with tab-separated columns), to more restrictive sub-formats -like interval (where three of the columns +like Interval (where three of the columns must be the chromosome, start position, and end position), and on -to even more specific ones such as BED or -GFF that have additional requirements. So for -example if a tool's required input format is tabular, then all of -your history items whose format is recorded as tabular will be -listed, along with those in all sub-formats that also qualify as -tabular (interval, BED, GFF, etc.). +to even more specific ones such as BED that have +additional requirements. So for example if a tool's required input +format is Tabular, then all of your history items whose format is +recorded as Tabular will be listed, along with those in all +sub-formats that also qualify as Tabular (Interval, BED, GFF, etc.).

There are two usual methods for changing a dataset's format in Galaxy: if the file contents are already in the required format but @@ -37,7 +44,7 @@ manually by clicking on the pencil icon beside that dataset in your history. Or, if the file contents really are in a different format, Galaxy provides a number of format conversion tools (e.g. in the Text Manipulation and Convert Formats categories). For instance, -if the tool you want to run requires tabular but your columns are +if the tool you want to run requires Tabular but your columns are delimited by spaces or commas, you can use the "Convert delimiters to TAB" tool under Text Manipulation to reformat your data. However if your files are in a completely unsupported format, then you need @@ -47,7 +54,7 @@ to convert them yourself before uploading.

Format Descriptions

+ +

- -Ab1 - +AB1 +

+This is one of the ABIF family of binary sequence formats from +Applied Biosystems Inc. + +Files should have a '.ab1' file extension. You must +manually select this file format when uploading the file.

-A binary sequence file in 'ab1' format with a '.ab1' file extension. -You must manually select this file format when uploading the file. -

+
+
AXT -

Used for pairwise alignment output from BLASTZ, after post-processing. Each alignment block contains three lines: a summary line and two @@ -94,44 +107,53 @@ The summary line contains chromosomal position and size information about the alignment, and consists of nine required fields. More information +

-A binary file compressed in the BGZF format with a '.bam' file -extension. -SAM format -is the human readable text version of these files. + +

+
+BAM +

+A binary alignment file compressed in the BGZF format with a +'.bam' file extension. + +SAM +is the human-readable text version of this format.

Can be converted to:
-
+

+

+
BED -

This tab-separated format describes a genomic interval, but has strict field specifications for use in genome browsers. BED files can have from 3 to 12 columns, but the order of the columns matters, and only the end ones can be omitted. Some groups of columns must -be all present or all absent. +be all present or all absent. As in Interval format (but unlike +GFF and its relatives), the interval endpoints use a 0-based, +half-open numbering system. Field specifications

@@ -142,17 +164,18 @@ chr22 2000 6000 cloneB 900 - 2000 6000 0 2 433,399, 0,3601

Can be converted to:
-
+

+

+
BedGraph -

- -Binseq.zip -

-A zipped archive consisting of binary sequence files in either -'ab1' or 'scf' format. All files in this archive must have the same -file extension which is one of '.ab1' or '.scf'. You must manually -select this file format when uploading the file. -

+
+
+Binseq.zip +

+A zipped archive consisting of binary sequence files in either AB1 +or SCF format. All files in this archive must have the same file +extension which is one of '.ab1' or '.scf'. +You must manually select this file format when uploading the file. +

+ +

+
FASTA -

A sequence in FASTA format consists of a single-line description, followed by lines of sequence data. The first character of the description line is a -greater-than (">") symbol. All lines should be shorter than 80 -characters. +greater-than ('>') symbol. All lines should be +shorter than 80 characters. 

 >sequence1
 atgcgtttgcgtgc
@@ -190,16 +215,17 @@ tggcgcggtga
Can be converted to:
-
+

+

+
FastqSolexa -

FastqSolexa -is the Illumina (Solexa) variant of the Fastq format, which stores +is the Illumina (Solexa) variant of the FASTQ format, which stores sequences and quality scores in a single file. 

 @seq1  
@@ -224,82 +250,97 @@ GAGTTCTCGTCGCCTGTAGGCACCATCAATCGTATG
Can be converted to:
-
+

-fped - +

+
+FPED

Also known as the FBAT format, for use with the FBAT program. It consists of a pedigree file and a phenotype file. -

+

+

+
GFF -

GFF is a tab-separated format somewhat similar to BED, but it has different columns and is more flexible. There are nine required fields. +Note that unlike Interval and BED, GFF and its relatives (GFF3, GTF) +use 1-based inclusive coordinates to specify genomic intervals.
Can be converted to:
-
+

+

+
GFF3 -

The GFF3 -format addresses the most common extensions to GFF, while preserving -backward compatibility with previous formats. -
+format addresses the most common extensions to GFF, while attempting +to preserve compatibility with previous formats. +Note that unlike Interval and BED, GFF and its relatives (GFF3, GTF) +use 1-based inclusive coordinates to specify genomic intervals. +

+

+
GTF -

GTF is a format for describing genes and other features -associated with DNA, RNA, and protein sequences. +>GTF is a format for describing genes and other features associated +with DNA, RNA, and protein sequences. It is a refinement to GFF that +tightens the specification. +Note that unlike Interval and BED, GFF and its relatives (GFF3, GTF) +use 1-based inclusive coordinates to specify genomic intervals. + +

+

+
HTML -

This format is an HTML web page. Click the eye icon next to the dataset to view it in your browser. -

+

+

+
Interval -

This Galaxy format represents genomic intervals. It is tab-separated, but has the added requirement that three of the columns must be the -chromosome name, start position, and end position. An optional +chromosome name, start position, and end position, where the positions +use a 0-based, half-open numbering system (see below). An optional strand column can also be specified, and an initial header row can be used to label the columns, which do not have to be in any special order. Arbitrary additional columns can also be present. @@ -317,7 +358,8 @@ Required fields: Optional: Example: @@ -328,173 +370,202 @@ Example:
Can be converted to:
    -
  • BED
    +
  • BED
    The exact changes needed and tools to run will vary with what fields -are in the interval file and what type of BED you are converting to. -In general you will likely use Text Manipulation→Compute, Cut, +are in the Interval file and what type of BED you are converting to. +In general you will likely use Text Manipulation → Compute, Cut, or Merge Columns.
-
+

+

+
LAV -

LAV is the raw pairwise alignment format that is output by BLASTZ. The first line begins with #:lav. +

-This is the linkage pedigree format, which consists of separate -map and ped files. Together these files -describe SNPs; the map file contains the position and an identifier -for the SNP, while the pedigree file has the alleles. -To upload this format into Galaxy, do not use auto-detect for the -file format; instead select lped. You will then be -given two sections for uploading files, one for the pedigree file -and one for the map file. For more information, see -linkage pedigree, -map, -and/or ped. + +

+
+LPED +

+This is the linkage pedigree format, which consists of separate MAP and PED +files. Together these files describe SNPs; the map file contains the position +and an identifier for the SNP, while the pedigree file has the alleles. To +upload this format into Galaxy, do not use Auto-detect for the file format; +instead select lped. You will then be given two sections for +uploading files, one for the pedigree file and one for the map file. For more +information, see +linkage pedigree, +MAP, +and/or PED.

Can be converted to:
-
+

+

+
MAF -

-Multiple alignment format that is output by TBA and Multiz. The -first line begins with ##maf. This word is followed by -whitespace-separated "variable=value pairs". There should be no -whitespace surrounding the "=". More information +>MAF is the multi-sequence alignment format that is output by TBA +and Multiz. The first line begins with '##maf'. This +word is followed by whitespace-separated "variable=value" +pairs. There should be no whitespace surrounding the '='.

Can be converted to:
-
- -pbed -

-This is the binary version of the lped file format. + +

+
+PBED +

+This is the binary version of the LPED format.

Can be converted to:
-
+

+

+
PSL -

PSL format is used for alignments returned by BLAT. It does not include any sequence. -

- -Scf -

-A binary sequence file in 'scf' format with a '.scf' file extension. -You must manually select this file format when uploading the file. + +

+
+SCF +

+This is a binary sequence format originally designed for the Staden +sequence handling software package. Files should have a +'.scf' file extension. You must manually select this +file format when uploading the file. More information -

- -Sff -

-A binary file in 'Standard Flowgram Format' with a '.sff' file extension. + +

+
+SFF +

+This is a binary sequence format used by the Roche 454 GS FLX +sequencing machine, and is documented on p. 528 of their +software manual. Files should have a '.sff' file +extension. +

Can be converted to:
-
+

+

+
Table -

Text data separated into columns by something other than tabs. -

+

+

+
Tabular (tab-delimited) -

One or more columns of text data separated by tabs.

Can be converted to:
    -
  • FASTA
    -Convert Formats→Tabular-to-FASTA
    -The tabular file must have a title and sequence column. -
  • interval
    -If the tabular file has the chromosome, or is all on one chromosome, -and has a position you can create an interval file (e.g. for SNPs). -If it is all on one chromosome, use Text Manipulation→Add column -to add a chromosome column. If the given position is 1-based, use -Text Manipulation→Compute with the position column minus 1 to -get the start, and use the original given column for the end. -If the given position is 0-based, use it as the start, and compute -that plus 1 to get the end. +
  • FASTA
    +Convert Formats → Tabular-to-FASTA
    +The Tabular file must have a title and sequence column. +
  • FASTQ
    +NGS: QC and manipulation → Generic FASTQ manipulation → Tabular to FASTQ +
  • Interval
    +If the Tabular file has a chromosome column (or is all on one +chromosome) and has a position column, you can create an Interval +file (e.g. for SNPs). If it is all on one chromosome, use +Text Manipulation → Add column to add a CHROM column. +If the given position is 1-based, use +Text Manipulation → Compute with the position column minus 1 to +get the START, and use the original given column for the END. +If the given position is 0-based, use it as the START, and compute +that plus 1 to get the END.
-
+

+

+
Txtseq.zip -

A zipped archive consisting of flat text sequence files. All files -in this archive must have the same file extension of '.txt'. You -must manually select this file format when uploading the file. -

- -Wiggle custom track - +in this archive must have the same file extension of +'.txt'. You must manually select this file format when +uploading the file.

-The wiggle format is line-oriented. Wiggle data is preceded by a -track definition line, which specifies the type of wiggle. There -are three different types, for different uses. + +

+
+Wiggle custom track +

+Wiggle tracks are typically used to display per-nucleotide scores +in a genome browser. The Wiggle format for custom tracks is +line-oriented, and the wiggle data is preceded by a track definition +line that specifies which of three different types is being used. More information

Can be converted to:
-
+

+

+
Other text type -

Any text file.

Can be converted to:
    -
  • tabular
    -If this has fields separated by spaces, commas, or some other -delimiter it can be converted to tabular using -Text Manipulation→Convert delimiters to TAB +
  • Tabular
    +If the text has fields separated by spaces, commas, or some other +delimiter, it can be converted to Tabular by using +Text Manipulation → Convert delimiters to TAB.
+

+ -











+











+











diff --git a/tools/human_genome_variation/linkToDavid.xml b/tools/human_genome_variation/linkToDavid.xml index 451f4eb1439..db34d48a53f 100644 --- a/tools/human_genome_variation/linkToDavid.xml +++ b/tools/human_genome_variation/linkToDavid.xml @@ -72,7 +72,7 @@ The list is limited to 400 IDs. **Dataset formats** -The input dataset is tabular_ format. The output dataset is html_ format with +The input dataset is in tabular_ format. The output dataset is html_ with a link to the DAVID website as described below. (`Dataset missing?`_) diff --git a/tools/human_genome_variation/lps.xml b/tools/human_genome_variation/lps.xml index 093b7c841ff..aff53e2f720 100644 --- a/tools/human_genome_variation/lps.xml +++ b/tools/human_genome_variation/lps.xml @@ -224,9 +224,9 @@ Let **x** be a row from your input dataset and let **b** be a column from the results file. To compute the probability that row **x** has a label value of +1: - Probability(row **x** has label value = +1) = 1 / [1 + exp{**x** \* **b**\[1..n-1\] + **b**\[n\]}] + Probability(row **x** has label value = +1) = 1 / [1 + exp{**x** \* **b**\[1..N-1\] + **b**\[N\]}] -where **x** \* **b**\[1..n-1\] represents matrix multiplication. +where **x** \* **b**\[1..N-1\] represents matrix multiplication. The second output dataset, called the log file, is a text file which contains additional data about the fitted L1-regularized logistic