diff --git a/doc/source/admin/conda_faq.rst b/doc/source/admin/conda_faq.rst index 5e51b10dce4..4fe67d155b6 100644 --- a/doc/source/admin/conda_faq.rst +++ b/doc/source/admin/conda_faq.rst @@ -1,6 +1,6 @@ -================================= +=========================== Conda for Tool Dependencies -================================= +=========================== Galaxy tools (also called wrappers) traditionally use Tool Shed package recipes to install their dependencies. At the tool's installation time @@ -46,7 +46,7 @@ Below we answer some common questions (collected by Lance Parsons): 1. How do I enable Conda dependency resolution for Galaxy jobs? -********************************************************************************* +*************************************************************** Galaxy's dependency job resolution is managed via ``dependency_resolvers_conf.xml`` configuration file. Most Galaxy administrators @@ -103,7 +103,7 @@ See `galaxy.ini.sample`_ for the complete list. 2. How do Conda dependencies work? Where do things get installed? -********************************************************************************* +***************************************************************** In contrast to the TS dependency system, which was used exclusively by Galaxy, Conda is a pre-existing, independent project. With Conda, it is possible for an @@ -154,7 +154,7 @@ be configured in the ``dependency_resolvers_conf.xml`` and the ``galaxy.ini`` fi 3. What is required to make use of this? Any specific packages, Galaxy revision, OS version, etc.? -********************************************************************************* +************************************************************************************************** The minimum required version of Galaxy to use Conda is 16.01, however version 16.07 or greater is recommended. The 16.07 release of Galaxy has @@ -169,7 +169,7 @@ systems newer than 2007. 4. If I have Conda enabled, what do I need to do to install tools using it? For example, how can I install the latest Trinity? And how will I know the dependencies are installed? -********************************************************************************* +********************************************************************************************************************************************************************************** This depends on your ``galaxy.ini`` setting. Starting with release 16.07, Galaxy can automatically install the Conda package manager for you if you have enabled @@ -198,7 +198,7 @@ are no available TS dependencies. 5. Can I mix traditional Galaxy packages and Conda packages? -********************************************************************************* +************************************************************ Yes, the way this works is that Galaxy goes through the list of requirements for a tool, and then determines for each requirement if it @@ -216,7 +216,7 @@ The first system that satisfies a requirement will be used. See 6. How do I know what system is being used by a given tool? -********************************************************************************* +*********************************************************** The Galaxy log will show which dependency resolution system is used to satisfy each tool dependency and you can specify priorities using the @@ -226,7 +226,7 @@ Admin panel. 7. How do I go about specifying Conda dependencies for a tool? All the docs still seem to recommend (or exclusively discuss) the ``tool_dependencies.xml`` method. -********************************************************************************* +****************************************************************************************************************************************************************** The simple answer is: you don't need to do much to make Conda work for a tool. @@ -243,7 +243,7 @@ deprecate it everywhere. 8. During tool installation what if there is no Conda package available for a given requirement? What if the requirement is resolved in a different software than the original wrapper author meant to use? -********************************************************************************* +*********************************************************************************************************************************************************************************************************** If there is no Conda package available during tool installation the tool will install automatically, and can be used if its dependencies are @@ -256,7 +256,7 @@ installed from the Tool Shed. 9. Where can I find a list of existing Conda packages that I can point to, so I don't have to reinvent the wheel for common dependencies? -********************************************************************************* +***************************************************************************************************************************************** With Conda package manager installed on your system, run: @@ -270,7 +270,7 @@ Galaxy. If you find your package, you are ready to go. If not please 10. How can I create a new Conda package for a dependency? -********************************************************************************* +********************************************************** Adding a package to the BioConda or IUC Conda channels will make it available for Galaxy tools to use as a dependency. To learn how, get in @@ -284,7 +284,7 @@ pypi, cran, or cpan for you (mostly) automatically. 11. Is there a way to convert traditional Tool Shed package recipes that are not yet in a Conda channel? -********************************************************************************* +******************************************************************************************************** First, you do not need to do anything to your wrapper as long as the package name in the requirement tag matches the name of correct @@ -297,7 +297,7 @@ leave the old versions as they are – simply because of time. 12. What is the recommendation for existing installations? Will I continue to maintain both systems or migrate to the new Conda system eventually? -********************************************************************************* +************************************************************************************************************************************************** Old tools will use the traditional installation system; this system will stay and will be supported for installing old tools to guarantee sustainability @@ -305,7 +305,7 @@ and reproducibility. New tools from the IUC, may be Conda only. 13. What can I do if Conda doesn't work for me? -********************************************************************************* +*********************************************** There is currently a limitation in the way Conda packages are being built. This limitation will be addressed shortly by the Conda community, diff --git a/doc/source/admin/interactive_environments.rst b/doc/source/admin/interactive_environments.rst index ba2aeabb5bb..c333715e1f0 100644 --- a/doc/source/admin/interactive_environments.rst +++ b/doc/source/admin/interactive_environments.rst @@ -61,7 +61,7 @@ Once Node and npm are ready to go, you'll need to install the dependencies Running ``node lib/main.js --help`` should produce some useful help text -.. code-block:: +.. code-block:: console Usage: main [options] @@ -81,7 +81,7 @@ as of 2014. Alternately, the proxy can be stated manually or via a system such a Supervisord. Assuming that the ``$GALAXY_ROOT`` environment variable refers to the location of the Galaxy installation, the command for launching the proxy is: -.. code-block:: console +.. code-block:: console $ node $GALAXY_ROOT/lib/galaxy/web/proxy/js/lib/main.js --ip 0.0.0.0 \ --port 8800 --sessions $GALAXY_ROOT/database/session_map.sqlite \ diff --git a/doc/source/admin/mulled_containers.rst b/doc/source/admin/mulled_containers.rst index 792a47f94ad..e0a5b9129b5 100644 --- a/doc/source/admin/mulled_containers.rst +++ b/doc/source/admin/mulled_containers.rst @@ -1,6 +1,6 @@ -================================= +================================ Containers for Tool Dependencies -================================= +================================ Galaxy tools (also called wrappers) are able to use Conda packages (see more information in our `Galaxy Conda documentation`_) and Docker containers as dependency resolvers. @@ -59,7 +59,7 @@ This will search for containers in the biocontainers organisation. Build all packages from bioconda from the last 24h -^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The BioConda community is building a container for every package they create with a command similar to this. @@ -70,7 +70,7 @@ The BioConda community is building a container for every package they create wit Building Docker containers for local Conda packages -^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Conda packages can be tested with creating a busybox based container for this particular package in the following way. This also demonstrates how you can build a container locally and on-the-fly. @@ -101,7 +101,7 @@ The ``--0`` indicates the build version of the conda package. It is recommended you will override already existing images. For Python Conda packages this extension might look like this ``--py35_1``. Build, test and push a conda-forge package to biocontainers -^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > You need to have write access to the biocontainers repository diff --git a/doc/source/conf.py b/doc/source/conf.py index 3fc399b5f0a..c9a9f23eb85 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -22,12 +22,10 @@ source_parsers = { '.md': CommonMarkParser, } -####### REQUIRED GALAXY INCLUDES +# REQUIRED GALAXY INCLUDES sys.path.insert(1, os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir, os.pardir, 'lib'))) -####### - # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. @@ -201,8 +199,8 @@ latex_elements = { # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ - ('index', 'Galaxy.tex', u'Galaxy Code Documentation', - u'Galaxy Team', 'manual'), + ('index', 'Galaxy.tex', u'Galaxy Code Documentation', + u'Galaxy Team', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of @@ -245,9 +243,9 @@ man_pages = [ # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ - ('index', 'Galaxy', u'Galaxy Code Documentation', - u'Galaxy Team', 'Galaxy', 'Data intensive biology for everyone.', - 'Miscellaneous'), + ('index', 'Galaxy', u'Galaxy Code Documentation', + u'Galaxy Team', 'Galaxy', 'Data intensive biology for everyone.', + 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. @@ -259,8 +257,8 @@ texinfo_documents = [ # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' -# -- ReadTheDocs.org Settings ------------------------------------------------ +# -- ReadTheDocs.org Settings ------------------------------------------------ class Mock(object): def __init__(self, *args, **kwargs): pass diff --git a/doc/source/index.rst b/doc/source/index.rst index 71065e61849..cfee8c491c6 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -52,13 +52,12 @@ Indices and tables * :ref:`search` Building this Documentation -========================== +=========================== If you have your own copy of the Galaxy source code, you can also generate your own version of this documentation: :: - $ cd doc - $ make html + $ make -C doc/ html The generated documentation will be in ``doc/build/html/`` and can be viewed with a web browser. Note that you will need to install Sphinx and a fair number of module dependencies before this will produce output. diff --git a/doc/source/lib/galaxy.datatypes.converters.rst b/doc/source/lib/galaxy.datatypes.converters.rst index dacdc91828d..3cbddeed611 100644 --- a/doc/source/lib/galaxy.datatypes.converters.rst +++ b/doc/source/lib/galaxy.datatypes.converters.rst @@ -225,18 +225,18 @@ galaxy.datatypes.converters.vcf_to_vcf_bgzip module :undoc-members: :show-inheritance: -galaxy.datatypes.converters.bcf_bgzip_to_bcf module ---------------------------------------------------- +galaxy.datatypes.converters.bcf_bgzip_to_bcf_converter module +------------------------------------------------------------- -.. automodule:: galaxy.datatypes.converters.bcf_bgzip_to_bcf +.. automodule:: galaxy.datatypes.converters.bcf_bgzip_to_bcf_converter :members: :undoc-members: :show-inheritance: -galaxy.datatypes.converters.bcf_to_bcf_bgzip module ---------------------------------------------------- +galaxy.datatypes.converters.bcf_to_bcf_bgzip_converter module +------------------------------------------------------------- -.. automodule:: galaxy.datatypes.converters.bcf_to_bcf_bgzip +.. automodule:: galaxy.datatypes.converters.bcf_to_bcf_bgzip_converter :members: :undoc-members: :show-inheritance: diff --git a/doc/source/lib/galaxy.jobs.rst b/doc/source/lib/galaxy.jobs.rst index 86c56ad5c51..8905e4691ba 100644 --- a/doc/source/lib/galaxy.jobs.rst +++ b/doc/source/lib/galaxy.jobs.rst @@ -14,7 +14,6 @@ Subpackages galaxy.jobs.actions galaxy.jobs.deferred galaxy.jobs.metrics - galaxy.jobs.rules galaxy.jobs.runners galaxy.jobs.splitters diff --git a/doc/source/lib/galaxy.rst b/doc/source/lib/galaxy.rst index f15c406fbe4..aa8c5fe528e 100644 --- a/doc/source/lib/galaxy.rst +++ b/doc/source/lib/galaxy.rst @@ -22,7 +22,6 @@ Subpackages galaxy.forms galaxy.jobs galaxy.managers - galaxy.metadata galaxy.model galaxy.objectstore galaxy.openid diff --git a/doc/source/lib/galaxy.tools.linters.rst b/doc/source/lib/galaxy.tools.linters.rst index a27e6bf42bd..3463dc953fa 100644 --- a/doc/source/lib/galaxy.tools.linters.rst +++ b/doc/source/lib/galaxy.tools.linters.rst @@ -25,6 +25,14 @@ galaxy.tools.linters.command module :undoc-members: :show-inheritance: +galaxy.tools.linters.general module +----------------------------------- + +.. automodule:: galaxy.tools.linters.general + :members: + :undoc-members: + :show-inheritance: + galaxy.tools.linters.help module -------------------------------- @@ -49,6 +57,14 @@ galaxy.tools.linters.outputs module :undoc-members: :show-inheritance: +galaxy.tools.linters.stdio module +--------------------------------- + +.. automodule:: galaxy.tools.linters.stdio + :members: + :undoc-members: + :show-inheritance: + galaxy.tools.linters.tests module --------------------------------- @@ -57,12 +73,10 @@ galaxy.tools.linters.tests module :undoc-members: :show-inheritance: -galaxy.tools.linters.top_level module +galaxy.tools.linters.xml_order module ------------------------------------- -.. automodule:: galaxy.tools.linters.top_level +.. automodule:: galaxy.tools.linters.xml_order :members: :undoc-members: :show-inheritance: - - diff --git a/doc/source/lib/galaxy.tools.parameters.rst b/doc/source/lib/galaxy.tools.parameters.rst index 28309e13f58..8bfd839859a 100644 --- a/doc/source/lib/galaxy.tools.parameters.rst +++ b/doc/source/lib/galaxy.tools.parameters.rst @@ -65,14 +65,6 @@ galaxy.tools.parameters.meta module :undoc-members: :show-inheritance: -galaxy.tools.parameters.output module -------------------------------------- - -.. automodule:: galaxy.tools.parameters.output - :members: - :undoc-members: - :show-inheritance: - galaxy.tools.parameters.output_collect module --------------------------------------------- @@ -105,4 +97,10 @@ galaxy.tools.parameters.wrapped module :undoc-members: :show-inheritance: +galaxy.tools.parameters.wrapped_json module +------------------------------------------- +.. automodule:: galaxy.tools.parameters.wrapped_json + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/lib/galaxy.util.backports.rst b/doc/source/lib/galaxy.util.backports.rst index 69356d80fcd..ca3788ee04b 100644 --- a/doc/source/lib/galaxy.util.backports.rst +++ b/doc/source/lib/galaxy.util.backports.rst @@ -5,11 +5,3 @@ galaxy.util.backports package :members: :undoc-members: :show-inheritance: - -Subpackages ------------ - -.. toctree:: - - galaxy.util.backports.importlib - diff --git a/doc/source/lib/galaxy.util.rst b/doc/source/lib/galaxy.util.rst index dd2eca1c66b..b059a02d85d 100644 --- a/doc/source/lib/galaxy.util.rst +++ b/doc/source/lib/galaxy.util.rst @@ -42,6 +42,14 @@ galaxy.util.bunch module :undoc-members: :show-inheritance: +galaxy.util.checkers module +--------------------------- + +.. automodule:: galaxy.util.checkers + :members: + :undoc-members: + :show-inheritance: + galaxy.util.dbkeys module ------------------------- @@ -50,26 +58,10 @@ galaxy.util.dbkeys module :undoc-members: :show-inheritance: -galaxy.util.debugging module ----------------------------- +galaxy.util.dictifiable module +------------------------------ -.. automodule:: galaxy.util.debugging - :members: - :undoc-members: - :show-inheritance: - -galaxy.util.dictobj module --------------------------- - -.. automodule:: galaxy.util.dictobj - :members: - :undoc-members: - :show-inheritance: - -galaxy.util.directory_hash module ---------------------------------- - -.. automodule:: galaxy.util.directory_hash +.. automodule:: galaxy.util.dictifiable :members: :undoc-members: :show-inheritance: @@ -82,6 +74,14 @@ galaxy.util.expressions module :undoc-members: :show-inheritance: +galaxy.util.filelock module +--------------------------- + +.. automodule:: galaxy.util.filelock + :members: + :undoc-members: + :show-inheritance: + galaxy.util.hash_util module ---------------------------- @@ -98,6 +98,14 @@ galaxy.util.heartbeat module :undoc-members: :show-inheritance: +galaxy.util.image_util module +----------------------------- + +.. automodule:: galaxy.util.image_util + :members: + :undoc-members: + :show-inheritance: + galaxy.util.inflection module ----------------------------- @@ -130,10 +138,10 @@ galaxy.util.lazy_process module :undoc-members: :show-inheritance: -galaxy.util.lrucache module ---------------------------- +galaxy.util.multi_byte module +----------------------------- -.. automodule:: galaxy.util.lrucache +.. automodule:: galaxy.util.multi_byte :members: :undoc-members: :show-inheritance: @@ -178,6 +186,14 @@ galaxy.util.plugin_config module :undoc-members: :show-inheritance: +galaxy.util.postfork module +--------------------------- + +.. automodule:: galaxy.util.postfork + :members: + :undoc-members: + :show-inheritance: + galaxy.util.properties module ----------------------------- @@ -266,6 +282,14 @@ galaxy.util.topsort module :undoc-members: :show-inheritance: +galaxy.util.ucsc module +----------------------- + +.. automodule:: galaxy.util.ucsc + :members: + :undoc-members: + :show-inheritance: + galaxy.util.validation module ----------------------------- diff --git a/doc/source/releases/16.04.rst b/doc/source/releases/16.04.rst index 6f75d92508d..c3bc54c31fc 100644 --- a/doc/source/releases/16.04.rst +++ b/doc/source/releases/16.04.rst @@ -1,9 +1,8 @@ .. to_doc -------------------------------- 16.04 -------------------------------- +=============================== .. announce_start diff --git a/doc/source/releases/16.04_announce.rst b/doc/source/releases/16.04_announce.rst index 779caa179a8..e615714a65a 100644 --- a/doc/source/releases/16.04_announce.rst +++ b/doc/source/releases/16.04_announce.rst @@ -65,6 +65,7 @@ See `our wiki `__ for additio Security =========================================================== + TL;DR **Only Tool Sheds newer than 16.01 should be deployed from now on.** (with commit 449098d8b14b45269be106f6410c0b9145c51d50 from Mar 30 present) @@ -81,7 +82,7 @@ Deprecation Notices =========================================================== API deprecations -~~~~~~~~~~~~~~~~ +---------------- API for history contents, index: * **types**: is no longer a valid parameter but accessible using ``?q=history_content_type&qv=[dataset | dataset_collection]`` @@ -96,7 +97,7 @@ API histories (removed from the available serialized data on all calls): * **state, state_details, state_ids** - can be replaced by specifically requesting a single array of contents, each containing :code:`{ id, state, deleted, visible }` Galaxy no longer on Bitbucket -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +----------------------------- Galaxy moved its code and development activities from Bitbucket to GitHub in early 2015. Since this time, releases have been mirrored back to Bitbucket. However, after this release, no new changes will be pushed to Bitbucket. Anyone still receiving updates to a Galaxy server via Bitbucket and Mercurial should switch to GitHub and Git. This can be done using the following process: diff --git a/doc/source/releases/16.07.rst b/doc/source/releases/16.07.rst index 35a2577dcd7..911f6d0b029 100644 --- a/doc/source/releases/16.07.rst +++ b/doc/source/releases/16.07.rst @@ -1,9 +1,8 @@ .. to_doc -------------------------------- 16.07 -------------------------------- +=============================== .. announce_start @@ -111,7 +110,7 @@ Enhancements * Fix running tool tests if remote user middleware is enabled. `Pull Request 2173`_ * Improve handling of a missing R environment during Tool Shed dependency - installations. + installations. `Pull Request 2215`_ * Quote some parameters in the trim tool command (thanks to `@nsoranzo `__.) diff --git a/doc/source/releases/index.rst b/doc/source/releases/index.rst index d6b9cfd7d41..5d3859fd31a 100644 --- a/doc/source/releases/index.rst +++ b/doc/source/releases/index.rst @@ -4,6 +4,7 @@ Releases .. toctree:: :maxdepth: 1 + 16.10_announce 16.07_announce 16.04_announce 16.01_announce diff --git a/lib/galaxy/datatypes/binary.py b/lib/galaxy/datatypes/binary.py index 17a649877ec..a1893c2b053 100644 --- a/lib/galaxy/datatypes/binary.py +++ b/lib/galaxy/datatypes/binary.py @@ -1139,10 +1139,13 @@ class OxliCountGraph(OxliBinary): """ OxliCountGraph starts with "OXLI" + one byte version number + 8-bit binary '1' - Test file generated via `load-into-counting.py --n_tables 1 \ - --max-tablesize 1 oxli_countgraph.oxlicg \ - khmer/tests/test-data/100-reads.fq.bz2` + Test file generated via:: + + load-into-counting.py --n_tables 1 --max-tablesize 1 \\ + oxli_countgraph.oxlicg khmer/tests/test-data/100-reads.fq.bz2 + using khmer 2.0 + >>> from galaxy.datatypes.sniff import get_test_fname >>> fname = get_test_fname( 'sequence.csfasta' ) >>> OxliCountGraph().sniff( fname ) @@ -1163,10 +1166,13 @@ class OxliNodeGraph(OxliBinary): """ OxliNodeGraph starts with "OXLI" + one byte version number + 8-bit binary '2' - Test file generated via `load-graph.py --n_tables 1 \ - --max-tablesize 1 oxli_nodegraph.oxling \ - khmer/tests/test-data/100-reads.fq.bz2` + Test file generated via:: + + load-graph.py --n_tables 1 --max-tablesize 1 oxli_nodegraph.oxling \\ + khmer/tests/test-data/100-reads.fq.bz2 + using khmer 2.0 + >>> from galaxy.datatypes.sniff import get_test_fname >>> fname = get_test_fname( 'sequence.csfasta' ) >>> OxliNodeGraph().sniff( fname ) @@ -1187,11 +1193,14 @@ class OxliTagSet(OxliBinary): """ OxliTagSet starts with "OXLI" + one byte version number + 8-bit binary '3' - Test file generated via `load-graph.py --n_tables 1 \ - --max-tablesize 1 oxli_nodegraph.oxling \ - khmer/tests/test-data/100-reads.fq.bz2; \ - mv oxli_nodegraph.oxling.tagset oxli_tagset.oxlits` + Test file generated via:: + + load-graph.py --n_tables 1 --max-tablesize 1 oxli_nodegraph.oxling \\ + khmer/tests/test-data/100-reads.fq.bz2; + mv oxli_nodegraph.oxling.tagset oxli_tagset.oxlits + using khmer 2.0 + >>> from galaxy.datatypes.sniff import get_test_fname >>> fname = get_test_fname( 'sequence.csfasta' ) >>> OxliTagSet().sniff( fname ) @@ -1213,6 +1222,7 @@ class OxliStopTags(OxliBinary): 8-bit binary '4' Test file adapted from khmer 2.0's "khmer/tests/test-data/goodversion-k32.stoptags" + >>> from galaxy.datatypes.sniff import get_test_fname >>> fname = get_test_fname( 'sequence.csfasta' ) >>> OxliStopTags().sniff( fname ) @@ -1233,11 +1243,14 @@ class OxliSubset(OxliBinary): """ OxliSubset starts with "OXLI" + one byte version number + 8-bit binary '5' - Test file generated via `load-graph.py -k 20 example \ - tests/test-data/random-20-a.fa; \ - partition-graph.py example; \ - mv example.subset.0.pmap oxli_subset.oxliss` + Test file generated via:: + + load-graph.py -k 20 example tests/test-data/random-20-a.fa; + partition-graph.py example; + mv example.subset.0.pmap oxli_subset.oxliss + using khmer 2.0 + >>> from galaxy.datatypes.sniff import get_test_fname >>> fname = get_test_fname( 'sequence.csfasta' ) >>> OxliSubset().sniff( fname ) @@ -1257,11 +1270,15 @@ class OxliGraphLabels(OxliBinary): """ OxliGraphLabels starts with "OXLI" + one byte version number + 8-bit binary '6' - Test file generated via `python -c "from khmer import GraphLabels; \ - gl = GraphLabels(20, 1e7, 4); gl.consume_fasta_and_tag_with_labels( - 'tests/test-data/test-labels.fa'); \ - gl.save_labels_and_tags('oxli_graphlabels.oxligl')"` + Test file generated via:: + + python -c "from khmer import GraphLabels; \\ + gl = GraphLabels(20, 1e7, 4); \\ + gl.consume_fasta_and_tag_with_labels('tests/test-data/test-labels.fa'); \\ + gl.save_labels_and_tags('oxli_graphlabels.oxligl')" + using khmer 2.0 + >>> from galaxy.datatypes.sniff import get_test_fname >>> fname = get_test_fname( 'sequence.csfasta' ) >>> OxliGraphLabels().sniff( fname ) diff --git a/lib/galaxy/datatypes/checkers.py b/lib/galaxy/datatypes/checkers.py index 9fc5f098586..ab2fce29024 100644 --- a/lib/galaxy/datatypes/checkers.py +++ b/lib/galaxy/datatypes/checkers.py @@ -1,4 +1,4 @@ -"""Module proxies :module:`galaxy.util.checkers` for backward compatibility. +"""Module proxies :mod:`galaxy.util.checkers` for backward compatibility. External datatypes may make use of these functions. """ diff --git a/lib/galaxy/datatypes/constructive_solid_geometry.py b/lib/galaxy/datatypes/constructive_solid_geometry.py index 8e58c9e85d8..43f1747d2e7 100644 --- a/lib/galaxy/datatypes/constructive_solid_geometry.py +++ b/lib/galaxy/datatypes/constructive_solid_geometry.py @@ -134,7 +134,7 @@ Binary.register_sniffable_binary_format("plybinary", "plybinary", PlyBinary) class Vtk(object): - """ + r""" The Visualization Toolkit provides a number of source and writer objects to read and write popular data file formats. The Visualization Toolkit also provides some of its own file formats. @@ -151,8 +151,8 @@ class Vtk(object): i.e., the numbers that define points coordinates, scalars, cell indices, and so forth. - Binary data must be placed into the file immediately after the newline (\n) - character from the previous ASCII keyword and parameter sequence. + Binary data must be placed into the file immediately after the newline + ('\\n') character from the previous ASCII keyword and parameter sequence. TODO: only legacy formats are currently supported and support for XML formats should be added. diff --git a/lib/galaxy/datatypes/data.py b/lib/galaxy/datatypes/data.py index 6bc7e2a7f8b..0fbcaaaf07a 100644 --- a/lib/galaxy/datatypes/data.py +++ b/lib/galaxy/datatypes/data.py @@ -66,14 +66,13 @@ class Data( object ): 'test' >>> type( DataTest.metadata_spec.test.param ) - """ edam_data = "data_0006" edam_format = "format_1915" # Data is not chunkable by default. CHUNKABLE = False - #: dictionary of metadata fields for this datatype:: + #: Dictionary of metadata fields for this datatype metadata_spec = None # Add metadata elements @@ -855,7 +854,7 @@ class Text( Data ): @dataproviders.decorators.dataprovider_factory( 'line', dataproviders.line.FilteredLineDataProvider.settings ) def line_dataprovider( self, dataset, **settings ): """ - Returns an iterator over the dataset's lines (that have been `strip`ed) + Returns an iterator over the dataset's lines (that have been stripped) optionally excluding blank lines and lines that start with a comment character. """ dataset_source = dataproviders.dataset.DatasetDataProvider( dataset ) @@ -962,10 +961,9 @@ def get_file_peek( file_name, is_multi_byte=False, WIDTH=256, LINE_COUNT=5, skip """ Returns the first LINE_COUNT lines wrapped to WIDTH - ## >>> fname = get_test_fname('4.bed') - ## >>> get_file_peek(fname) - ## 'chr22 30128507 31828507 uc003bnx.1_cds_2_0_chr22_29227_f 0 +\n' - + >>> fname = get_test_fname('4.bed') + >>> get_file_peek(fname, LINE_COUNT=1) + u'chr22\\t30128507\\t31828507\\tuc003bnx.1_cds_2_0_chr22_29227_f\\t0\\t+\\n' """ # Set size for file.readline() to a negative number to force it to # read until either a newline or EOF. Needed for datasets with very diff --git a/lib/galaxy/datatypes/dataproviders/__init__.py b/lib/galaxy/datatypes/dataproviders/__init__.py index ab97e39582c..caf9a7e328c 100644 --- a/lib/galaxy/datatypes/dataproviders/__init__.py +++ b/lib/galaxy/datatypes/dataproviders/__init__.py @@ -6,7 +6,6 @@ consumer datum by datum. As well as subclassing and overriding to get the proper data, Dataproviders can be piped from one to the other. -..example:: .. note:: be careful to NOT pipe providers into subclasses of those providers. Subclasses provide all the functionality of their superclasses, diff --git a/lib/galaxy/datatypes/dataproviders/column.py b/lib/galaxy/datatypes/dataproviders/column.py index d57f415026c..bc97fc6f699 100644 --- a/lib/galaxy/datatypes/dataproviders/column.py +++ b/lib/galaxy/datatypes/dataproviders/column.py @@ -78,8 +78,8 @@ class ColumnarDataProvider( line.RegexLineDataProvider ): Optional: defaults to the tab character. :type deliminator: str - .. note: that the subclass constructors are passed kwargs - so they're - params (limit, offset, etc.) are also applicable here. + .. note:: that the subclass constructors are passed kwargs - so they're + params (limit, offset, etc.) are also applicable here. """ # TODO: other columnar formats: csv, etc. super( ColumnarDataProvider, self ).__init__( source, **kwargs ) @@ -141,9 +141,13 @@ class ColumnarDataProvider( line.RegexLineDataProvider ): The function will compare the column at index `column` against `val` using the given op where op is one of: - lt: less than, le: less than or equal to, - eq: equal to, ne: not equal to, - ge: greather than or equal to, gt: greater than + + - lt: less than + - le: less than or equal to + - eq: equal to + - ne: not equal to + - ge: greather than or equal to + - gt: greater than `val` is cast as float here and will return None if there's a parsing error. """ @@ -173,9 +177,10 @@ class ColumnarDataProvider( line.RegexLineDataProvider ): The function will compare the column at index `column` against `val` using the given op where op is one of: - eq: exactly matches, - has: the column contains the substring `val`, - re: the column matches the regular expression in `val` + + - eq: exactly matches + - has: the column contains the substring `val` + - re: the column matches the regular expression in `val` """ if 'eq' == op: return lambda d: d[column] == val @@ -195,8 +200,9 @@ class ColumnarDataProvider( line.RegexLineDataProvider ): The function will compare the column at index `column` against `val` using the given op where op is one of: - eq: the list `val` exactly matches the list in the column, - has: the list in the column contains the sublist `val`, + + - eq: the list `val` exactly matches the list in the column + - has: the list in the column contains the sublist `val` """ if 'eq' == op: val = self.parse_value( val, 'list' ) @@ -210,8 +216,9 @@ class ColumnarDataProvider( line.RegexLineDataProvider ): Return parser dictionary keyed for each columnar type (as defined in datatypes). - .. note: primitives only by default (str, int, float, boolean, None). + .. note:: primitives only by default (str, int, float, boolean, None). Other (more complex) types are retrieved as strings. + :returns: a dictionary of the form: `{ : }` """ @@ -324,8 +331,8 @@ class DictDataProvider( ColumnarDataProvider ): A combination use of both `column_names` and `indeces` allows 'picking' key/value pairs from the source. - .. note: that the subclass constructors are passed kwargs - so they're - params (limit, offset, etc.) are also applicable here. + .. note:: The subclass constructors are passed kwargs - so their + params (limit, offset, etc.) are also applicable here. """ settings = { 'column_names' : 'list:str', diff --git a/lib/galaxy/datatypes/dataproviders/dataset.py b/lib/galaxy/datatypes/dataproviders/dataset.py index 6547854567d..2ed8314fa48 100644 --- a/lib/galaxy/datatypes/dataproviders/dataset.py +++ b/lib/galaxy/datatypes/dataproviders/dataset.py @@ -56,6 +56,7 @@ class DatasetDataProvider( base.DataProvider ): def get_column_metadata_from_dataset( cls, dataset ): """ Convenience class method to get column metadata from a dataset. + :returns: a dictionary of `column_count`, `column_types`, and `column_names` if they're available, setting each to `None` if not. """ @@ -69,6 +70,7 @@ class DatasetDataProvider( base.DataProvider ): def get_metadata_column_types( self, indeces=None ): """ Return the list of `column_types` for this dataset or `None` if unavailable. + :param indeces: the indeces for the columns of which to return the types. Optional: defaults to None (return all types) :type indeces: list of ints @@ -88,6 +90,7 @@ class DatasetDataProvider( base.DataProvider ): def get_metadata_column_names( self, indeces=None ): """ Return the list of `column_names` for this dataset or `None` if unavailable. + :param indeces: the indeces for the columns of which to return the names. Optional: defaults to None (return all names) :type indeces: list of ints @@ -108,8 +111,10 @@ class DatasetDataProvider( base.DataProvider ): def get_indeces_by_column_names( self, list_of_column_names ): """ Return the list of column indeces when given a list of column_names. + :param list_of_column_names: the names of the columns of which to get indeces. :type list_of_column_names: list of strs + :raises KeyError: if column_names are not found :raises ValueError: if an entry in list_of_column_names is not in column_names """ @@ -399,7 +404,8 @@ class IntervalDataProvider( column.ColumnarDataProvider ): # WITHOUT reading the entire seq into memory - possibly apply some version of limit/offset class FastaDataProvider( base.FilteredDataProvider ): """ - Class that returns fasta format data in a list of maps of the form: + Class that returns fasta format data in a list of maps of the form:: + { id: , sequence: @@ -432,7 +438,8 @@ class FastaDataProvider( base.FilteredDataProvider ): class TwoBitFastaDataProvider( DatasetDataProvider ): """ - Class that returns fasta format data in a list of maps of the form: + Class that returns fasta format data in a list of maps of the form:: + { id: , sequence: diff --git a/lib/galaxy/datatypes/dataproviders/decorators.py b/lib/galaxy/datatypes/dataproviders/decorators.py index a7a662621a7..99e01e1934a 100644 --- a/lib/galaxy/datatypes/dataproviders/decorators.py +++ b/lib/galaxy/datatypes/dataproviders/decorators.py @@ -32,25 +32,24 @@ def has_dataproviders( cls ): in the class. This allows a class to maintain a name -> method map, effectively - 'registering' dataprovider factory methods. + 'registering' dataprovider factory methods:: - .. example:: - @has_dataproviders - class MyDtype( data.Data ): + @has_dataproviders + class MyDtype( data.Data ): - @dataprovider_factory( 'bler' ) - def provide_some_bler( self, dataset, **settings ): - '''blerblerbler''' - dataset_source = providers.DatasetDataProvider( dataset ) - # ... chain other, intermidiate providers here - return providers.BlerDataProvider( dataset_source, **settings ) + @dataprovider_factory( 'bler' ) + def provide_some_bler( self, dataset, **settings ): + '''blerblerbler''' + dataset_source = providers.DatasetDataProvider( dataset ) + # ... chain other, intermidiate providers here + return providers.BlerDataProvider( dataset_source, **settings ) - # use the base method in data.Data - provider = dataset.datatype.dataprovider( dataset, 'bler', - my_setting='blah', ... ) - # OR directly from the map - provider = dataset.datatype.dataproviders[ 'bler' ]( dataset, - my_setting='blah', ... ) + # use the base method in data.Data + provider = dataset.datatype.dataprovider( dataset, 'bler', + my_setting='blah', ... ) + # OR directly from the map + provider = dataset.datatype.dataproviders[ 'bler' ]( dataset, + my_setting='blah', ... ) """ # init the class dataproviders map if necc. if not hasattr( cls, _DATAPROVIDER_CLASS_MAP_KEY ): @@ -82,15 +81,15 @@ def dataprovider_factory( name, settings=None ): function to parse query strings to __init__ arguments as the `parse_query_string_settings` attribute of the factory function. - An example use of the `parse_query_string_settings`: - ..example:: - kwargs = dataset.datatype.dataproviders[ provider ].parse_query_string_settings( query_kwargs ) - return list( dataset.datatype.dataprovider( dataset, provider, **kwargs ) ) + An example use of the `parse_query_string_settings`:: + + kwargs = dataset.datatype.dataproviders[ provider ].parse_query_string_settings( query_kwargs ) + return list( dataset.datatype.dataprovider( dataset, provider, **kwargs ) ) :param name: what name/key to register the factory under in `cls.dataproviders` :type name: any hashable var :param settings: dictionary containing key/type pairs for parsing query strings - to __init__ arguments + to __init__ arguments :type settings: dictionary """ # TODO:?? use *args for settings allowing mulitple dictionaries diff --git a/lib/galaxy/datatypes/dataproviders/line.py b/lib/galaxy/datatypes/dataproviders/line.py index 7836bf2863f..0809773450e 100644 --- a/lib/galaxy/datatypes/dataproviders/line.py +++ b/lib/galaxy/datatypes/dataproviders/line.py @@ -88,7 +88,7 @@ class RegexLineDataProvider( FilteredLineDataProvider ): of regexs. .. note:: the regex matches are effectively OR'd (if **any** regex matches - the line it is considered valid and will be provided). + the line it is considered valid and will be provided). """ settings = { 'regex_list' : 'list:escaped', diff --git a/lib/galaxy/datatypes/sequence.py b/lib/galaxy/datatypes/sequence.py index c977d8fd288..d1660145064 100644 --- a/lib/galaxy/datatypes/sequence.py +++ b/lib/galaxy/datatypes/sequence.py @@ -1007,19 +1007,28 @@ class DotBracket ( Sequence ): Galaxy Dbn (Dot-Bracket notation) rules: * The first non-empty line is a header line: no comment lines are allowed. + * A header line starts with a '>' symbol and continues with 0 or multiple symbols until the line ends. + * The second non-empty line is a sequence line. - * A sequence line may only include chars that match the Fasta format (https://en.wikipedia.org/wiki/FASTA_format#Sequence_representation) symbols for nucleotides: ACGTURYKMSWBDHVN, and may thus not include whitespaces. + + * A sequence line may only include chars that match the FASTA format (https://en.wikipedia.org/wiki/FASTA_format#Sequence_representation) symbols for nucleotides: ACGTURYKMSWBDHVN, and may thus not include whitespaces. * A sequence line has no prefix and no suffix. * A sequence line is case insensitive. + * The third non-empty line is a structure (Dot-Bracket) line and only describes the 2D structure of the sequence above it. + * A structure line must consist of the following chars: '.{}[]()'. * A structure line must be of the same length as the sequence line, and each char represents the structure of the nucleotide above it. * A structure line has no prefix and no suffix. * A nucleotide pairs with only 1 or 0 other nucleotides. + * In a structure line, the number of '(' symbols equals the number of ')' symbols, the number of '[' symbols equals the number of ']' symbols and the number of '{' symbols equals the number of '}' symbols. + * The format accepts multiple entries per file, given that each entry is provided as three lines: the header, sequence and structure line. + * Sniffing is only applied on the first entry. + * Empty lines are allowed. """ diff --git a/lib/galaxy/datatypes/tabular.py b/lib/galaxy/datatypes/tabular.py index 2a4c580d111..b52a1e491a0 100644 --- a/lib/galaxy/datatypes/tabular.py +++ b/lib/galaxy/datatypes/tabular.py @@ -1058,31 +1058,31 @@ class ConnectivityTable( Tabular ): The ConnectivityTable (CT) is a file format used for describing RNA 2D structures by tools including MFOLD, UNAFOLD and the RNAStructure package. The tabular file format is defined as - follows: + follows:: -5 energy = -12.3 sequence name -1 G 0 2 0 1 -2 A 1 3 0 2 -3 A 2 4 0 3 -4 A 3 5 0 4 -5 C 4 6 1 5 + 5 energy = -12.3 sequence name + 1 G 0 2 0 1 + 2 A 1 3 0 2 + 3 A 2 4 0 3 + 4 A 3 5 0 4 + 5 C 4 6 1 5 The links given at the edam ontology page do not indicate what type of separator is used (space or tab) while different implementations exist. The implementation that uses spaces as - separator (implemented in RNAStructure) is as follows: + separator (implemented in RNAStructure) is as follows:: - 10 ENERGY = -34.8 seqname - 1 G 0 2 9 1 - 2 G 1 3 8 2 - 3 G 2 4 7 3 - 4 a 3 5 0 4 - 5 a 4 6 0 5 - 6 a 5 7 0 6 - 7 C 6 8 3 7 - 8 C 7 9 2 8 - 9 C 8 10 1 9 - 10 a 9 0 0 10 + 10 ENERGY = -34.8 seqname + 1 G 0 2 9 1 + 2 G 1 3 8 2 + 3 G 2 4 7 3 + 4 a 3 5 0 4 + 5 a 4 6 0 5 + 6 a 5 7 0 6 + 7 C 6 8 3 7 + 8 C 7 9 2 8 + 9 C 8 10 1 9 + 10 a 9 0 0 10 """ i = 0 diff --git a/lib/galaxy/jobs/__init__.py b/lib/galaxy/jobs/__init__.py index f87e6844bbe..4aa18d54c11 100644 --- a/lib/galaxy/jobs/__init__.py +++ b/lib/galaxy/jobs/__init__.py @@ -541,9 +541,8 @@ class JobConfiguration( object ): a list of IDs, the JobToolConfigurations for the first id in ``ids`` matching a tool definition. - .. note:: - - You should not mix tool shed tool IDs, versionless tool shed IDs, and tool config tool IDs that refer to the same tool. + .. note:: You should not mix tool shed tool IDs, versionless tool shed + IDs, and tool config tool IDs that refer to the same tool. :param ids: Tool ID or IDs to fetch the JobToolConfiguration of. :type ids: list or str. diff --git a/lib/galaxy/jobs/metrics/instrumenters/__init__.py b/lib/galaxy/jobs/metrics/instrumenters/__init__.py index 0a81cffb56b..b996daf5c79 100644 --- a/lib/galaxy/jobs/metrics/instrumenters/__init__.py +++ b/lib/galaxy/jobs/metrics/instrumenters/__init__.py @@ -1,4 +1,4 @@ -"""This module describes the abstract interface for :class:`InstrumentPlugin`s. +"""This module describes the abstract interface for :class:`InstrumentPlugin`. These are responsible for collecting and formatting a coherent set of metrics. """ diff --git a/lib/galaxy/managers/__init__.py b/lib/galaxy/managers/__init__.py index d5a7ab9252e..fc897383194 100644 --- a/lib/galaxy/managers/__init__.py +++ b/lib/galaxy/managers/__init__.py @@ -6,26 +6,26 @@ Encapsulates the intersection of trans (or trans.sa_session), models, and Controllers. Responsibilities: - model operations that involve the trans/sa_session (CRUD) - security: - ownership, accessibility - common aspect-oriented operations via new mixins: - sharable, annotatable, tagable, ratable + +- model operations that involve the trans/sa_session (CRUD) +- security: ownership, accessibility +- common aspect-oriented operations via new mixins: sharable, annotatable, + tagable, ratable Not responsible for: - encoding/decoding ids - any http gobblygook - formatting of returned data (always python structures) - formatting of raised errors + +- encoding/decoding ids +- any http gobblygook +- formatting of returned data (always python structures) +- formatting of raised errors The goal is to have Controllers only handle: - query-string/payload parsing and encoding/decoding ids - http - return formatting -and: - control, improve namespacing in Controllers - DRY for Controller ops (define here - use in both UI/API Controllers) +- query-string/payload parsing and encoding/decoding ids +- http +- return formatting +- control, improve namespacing in Controllers +- DRY for Controller ops (define here - use in both UI/API Controllers) In other words, 'Business logic' independent of web transactions/user context (trans) should be pushed into models - but logic that requires the context diff --git a/lib/galaxy/managers/base.py b/lib/galaxy/managers/base.py index 73bf377cd8a..afe2ce2db76 100644 --- a/lib/galaxy/managers/base.py +++ b/lib/galaxy/managers/base.py @@ -4,9 +4,10 @@ defines a base ModelManager, ModelSerializer, and ModelDeserializer. ModelManagers are used for operations on models that occur outside the scope of a single model object, such as: - - object creation - - object lookup - - interactions between 2+ objects of different model classes + +- object creation +- object lookup +- interactions between 2+ objects of different model classes (Since these were to replace model Mixins from web/framework/base/controller.py the rule of thumb used there also generally @@ -903,10 +904,12 @@ class ModelFilterParser( HasAModelManager ): """ Converts string tuples (partially converted query string params) of attr, op, val into either: - - ORM based filters (filters that can be applied by the ORM at the SQL - level) or - - functional filters (filters that use derived values or values not - within the SQL tables) + + - ORM based filters (filters that can be applied by the ORM at the SQL + level) or + - functional filters (filters that use derived values or values not + within the SQL tables) + These filters can then be applied to queries. This abstraction allows 'smarter' application of limit and offset at either the diff --git a/lib/galaxy/managers/datasets.py b/lib/galaxy/managers/datasets.py index 812d809e7bb..6dd1d402f97 100644 --- a/lib/galaxy/managers/datasets.py +++ b/lib/galaxy/managers/datasets.py @@ -236,7 +236,8 @@ class DatasetDeserializer( base.ModelDeserializer, deletable.PurgableDeserialize def deserialize_permissions( self, dataset, key, permissions, user=None, **context ): """ Create permissions for each list of encoded role ids in the (validated) - `permissions` dictionary, where `permissions` is in the form: + `permissions` dictionary, where `permissions` is in the form:: + { 'manage': [ , ... ], 'access': [ , ... ] } """ self.manager.permissions.manage.error_unless_permitted( dataset, user ) @@ -355,7 +356,7 @@ class DatasetAssociationManager( base.ModelManager, """ Return True if this hda/ldda is a composite type dataset. - .. note: see also (whereever we keep information on composite datatypes?) + .. note:: see also (whereever we keep information on composite datatypes?) """ return dataset_assoc.extension in self.app.datatypes_registry.get_composite_extensions() diff --git a/lib/galaxy/managers/libraries.py b/lib/galaxy/managers/libraries.py index 3fb32a2cb54..9bcb4d82fd6 100644 --- a/lib/galaxy/managers/libraries.py +++ b/lib/galaxy/managers/libraries.py @@ -33,7 +33,7 @@ class LibraryManager( object ): :type check_accessible: bool :returns: the requested library - :rtype: Library + :rtype: galaxy.model.Library """ try: library = trans.sa_session.query( trans.app.model.Library ).filter( trans.app.model.Library.table.c.id == decoded_library_id ).one() @@ -145,8 +145,8 @@ class LibraryManager( object ): """ Check if library is accessible to user. - :param folder: library - :type folder: Library + :param library: library + :type library: galaxy.model.Library :param check_accessible: flag whether to check that user can access library :type check_accessible: bool @@ -176,7 +176,7 @@ class LibraryManager( object ): Return library data in the form of a dictionary. :param library: library - :type library: Library + :type library: galaxy.model.Library :returns: dict with data about the library :rtype: dictionary @@ -201,7 +201,7 @@ class LibraryManager( object ): Load all permissions currently related to the given library. :param library: the model object - :type library: Library + :type library: galaxy.model.Library :rtype: dictionary :returns: dict of current roles for all available permission types diff --git a/lib/galaxy/managers/rbac_secured.py b/lib/galaxy/managers/rbac_secured.py index e90a50e2959..0b2883bba78 100644 --- a/lib/galaxy/managers/rbac_secured.py +++ b/lib/galaxy/managers/rbac_secured.py @@ -13,7 +13,7 @@ class RBACPermissionFailedException( galaxy.exceptions.InsufficientPermissionsEx class RBACPermission( object ): """ - Base class for wrangling/controlling the permissions ORM models (*Permissions, Roles) + Base class for wrangling/controlling the permissions ORM models (\*Permissions, Roles) that control which users can perform certain actions on their associated models (Libraries, Datasets). """ @@ -68,8 +68,9 @@ class DatasetRBACPermission( RBACPermission ): DatasetPermissions are typed (but not polymorphic themselves) by a string 'action'. There are two types: - - manage permissions : can a role manage the permissions on a dataset - - access : can a role read/look at/copy a dataset + + - manage permissions : can a role manage the permissions on a dataset + - access : can a role read/look at/copy a dataset """ permissions_class = model.DatasetPermissions action_name = None diff --git a/lib/galaxy/managers/roles.py b/lib/galaxy/managers/roles.py index 3bd298b723a..085594c5851 100644 --- a/lib/galaxy/managers/roles.py +++ b/lib/galaxy/managers/roles.py @@ -33,7 +33,7 @@ class RoleManager( base.ModelManager ): :type decoded_role_id: int :returns: the loaded Role object - :rtype: Role + :rtype: galaxy.model.Role :raises: InconsistentDatabase, RequestParameterInvalidException, InternalServerError """ diff --git a/lib/galaxy/tools/parameters/basic.py b/lib/galaxy/tools/parameters/basic.py index fbb16e2d283..5076dd8bd56 100644 --- a/lib/galaxy/tools/parameters/basic.py +++ b/lib/galaxy/tools/parameters/basic.py @@ -268,6 +268,7 @@ class TextToolParameter( ToolParameter ): class IntegerToolParameter( TextToolParameter ): """ Parameter that takes an integer value. + >>> from galaxy.util.bunch import Bunch >>> trans = Bunch( history=Bunch(), workflow_building_mode=True ) >>> p = IntegerToolParameter( None, XML( '' ) ) @@ -343,6 +344,7 @@ class IntegerToolParameter( TextToolParameter ): class FloatToolParameter( TextToolParameter ): """ Parameter that takes a real number value. + >>> from galaxy.util.bunch import Bunch >>> trans = Bunch( history=Bunch(), workflow_building_mode=True ) >>> p = FloatToolParameter( None, XML( '' ) ) diff --git a/lib/galaxy/util/simplegraph.py b/lib/galaxy/util/simplegraph.py index 88eaed964c3..280eb574086 100644 --- a/lib/galaxy/util/simplegraph.py +++ b/lib/galaxy/util/simplegraph.py @@ -109,7 +109,8 @@ class SimpleGraph( object ): def gen_edge_dicts( self ): """ - Returns a generator that yields node dictionaries in the form: + Returns a generator that yields node dictionaries in the form:: + { 'source': , 'target': , @@ -121,7 +122,8 @@ class SimpleGraph( object ): def as_dict( self ): """ - Returns a dictionary of the form + Returns a dictionary of the form:: + { 'nodes': , 'edges': } """ return { 'nodes': list( self.gen_node_dicts() ), 'edges': list( self.gen_edge_dicts() ) } diff --git a/lib/galaxy/webapps/galaxy/api/history_contents.py b/lib/galaxy/webapps/galaxy/api/history_contents.py index d49a166decf..d7fc43b7664 100644 --- a/lib/galaxy/webapps/galaxy/api/history_contents.py +++ b/lib/galaxy/webapps/galaxy/api/history_contents.py @@ -619,7 +619,7 @@ class HistoryContentsController( BaseAPIController, UsesLibraryMixin, UsesLibrar :returns: archive file for download - .. note: this is a volatile endpoint and settings and behavior may change. + .. note:: this is a volatile endpoint and settings and behavior may change. """ # roughly from: http://stackoverflow.com/a/31976060 (windows, linux) invalid_filename_char_regex = re.compile( r'[:<>|\\\/\?\* "]' ) diff --git a/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py b/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py index 938048f4122..65f083cbcce 100644 --- a/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py +++ b/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py @@ -146,15 +146,15 @@ class Download( object ): def url_download( self, install_dir, downloaded_file_name, download_url, extract=True, checksums={} ): """ - The given download_url can have an extension like #md5#, #sha256#, (or #md5= to support pypi defaults). + The given download_url can have an extension like #md5#, #sha256#, (or #md5= to support pypi defaults). - https://pypi.python.org/packages/source/k/khmer/khmer-1.0.tar.gz#md5#b60639a8b2939836f66495b9a88df757 + https://pypi.python.org/packages/source/k/khmer/khmer-1.0.tar.gz#md5#b60639a8b2939836f66495b9a88df757 - Alternatively, to not break HTTP spec, you can specify md5 and - sha256 as keys in the element. + Alternatively, to not break HTTP spec, you can specify md5 and + sha256 as keys in the element. - This indicates a checksum which will be checked after download. - If the checksum does not match an exception is thrown. + This indicates a checksum which will be checked after download. + If the checksum does not match an exception is thrown. """ file_path = os.path.join( install_dir, downloaded_file_name ) src = None @@ -300,7 +300,8 @@ class AssertDirectoryExists( RecipeStep ): def assert_directory_exists( self, full_path ): """ Return True if a symbolic link or directory exists, but if full_path is a file, - return False. """ + return False. + """ if full_path is None: return False if os.path.isfile( full_path ): @@ -911,20 +912,20 @@ class RegexReplace( RecipeStep ): filtered_actions and dir. This step supports the full range of python's regular expression engine, including backreferences in - the replacement text. + the replacement text. Example:: - Example: - - ^CFLAGS(\s*)=\s*-g\s*-Wall\s*-O2\s*$$ - CFLAGS\1= -g -Wall -O2 -I$$(NCURSES_INCLUDE_PATH)/ncurses/ -I$$(NCURSES_INCLUDE_PATH) -L$$(NCURSES_LIB_PATH) - + + ^CFLAGS(\s*)=\s*-g\s*-Wall\s*-O2\s*$$ + CFLAGS\1= -g -Wall -O2 -I$$(NCURSES_INCLUDE_PATH)/ncurses/ -I$$(NCURSES_INCLUDE_PATH) -L$$(NCURSES_LIB_PATH) + - Before: - CFLAGS = -g -Wall -O2 + Before:: - After: - CFLAGS = -g -Wall -O2 -I$(NCURSES_INCLUDE_PATH)/ncurses/ -I$(NCURSES_INCLUDE_PATH) -L$(NCURSES_LIB_PATH) + CFLAGS = -g -Wall -O2 + After:: + + CFLAGS = -g -Wall -O2 -I$(NCURSES_INCLUDE_PATH)/ncurses/ -I$(NCURSES_INCLUDE_PATH) -L$(NCURSES_LIB_PATH) """ log_file = os.path.join( install_environment.install_dir, basic_util.INSTALLATION_LOG ) if os.path.exists( log_file ): @@ -1011,20 +1012,19 @@ class SetEnvironment( RecipeStep ): The first tag set defines a complex repository dependency like this. This tag set ensures that changeset revision XXX of the repository named package_graphicsmagick_1_3 owned by YYY in the tool shed ZZZ has been - previously installed. + previously installed:: - - - - - ... + + + + + ... - * By the way, there is an env.sh file associated with version 1.3.18 of the graphicsmagick package which looks - something like this (we'll reference this file later in this discussion. - ---- - GRAPHICSMAGICK_ROOT_DIR=//graphicsmagick/1.3.18/YYY/package_graphicsmagick_1_3/XXX/gmagick; - export GRAPHICSMAGICK_ROOT_DIR - ---- + By the way, there is an env.sh file associated with version 1.3.18 of the graphicsmagick package which looks + something like this (we'll reference this file later in this discussion):: + + GRAPHICSMAGICK_ROOT_DIR=//graphicsmagick/1.3.18/YYY/package_graphicsmagick_1_3/XXX/gmagick; + export GRAPHICSMAGICK_ROOT_DIR The second tag set defines a specific package dependency that has been previously installed (guaranteed by the tag set discussed above) and compiled, where the compiled dependency is needed by the tool dependency currently @@ -1033,44 +1033,42 @@ class SetEnvironment( RecipeStep ): version 2.0.0 of the osra package requires version 1.3.18 of the graphicsmagick package in order to successfully compile. When this tag set is handled, one of the effects is that the env.sh file associated with graphicsmagick version 1.3.18 is "sourced", which undoubtedly sets or alters certain environment variables (e.g. PATH, PYTHONPATH, - etc). + etc):: - - - - - - + + + + + + The third tag set enables discovery of the same required package dependency discussed above for correctly compiling the osra version 2.0.0 package, but in this case the package can be discovered at tool execution time. Using the $ENV[] option as shown in this example, the value of the environment variable named GRAPHICSMAGICK_ROOT_DIR (which was set in the environment using the second tag set described above) will be used to automatically alter the env.sh file associated with the osra version 2.0.0 tool dependency when it is installed into Galaxy. * Refer to where we - discussed the env.sh file for version 1.3.18 of the graphicsmagick package above. + discussed the env.sh file for version 1.3.18 of the graphicsmagick package above:: - - $ENV[GRAPHICSMAGICK_ROOT_DIR]/lib/ - $INSTALL_DIR/potrace/build/lib/ - $INSTALL_DIR/bin - - $INSTALL_DIR/share - + + $ENV[GRAPHICSMAGICK_ROOT_DIR]/lib/ + $INSTALL_DIR/potrace/build/lib/ + $INSTALL_DIR/bin + + $INSTALL_DIR/share + The above tag will produce an env.sh file for version 2.0.0 of the osra package when it it installed into Galaxy that looks something like this. Notice that the path to the gmagick binary is included here since it expands the - defined $ENV[GRAPHICSMAGICK_ROOT_DIR] value in the above tag set. + defined $ENV[GRAPHICSMAGICK_ROOT_DIR] value in the above tag set:: - ---- - LD_LIBRARY_PATH=//graphicsmagick/1.3.18/YYY/package_graphicsmagick_1_3/XXX/gmagick/lib/:$LD_LIBRARY_PATH; - export LD_LIBRARY_PATH - LD_LIBRARY_PATH=//osra/1.4.0/YYY/depends_on/XXX/potrace/build/lib/:$LD_LIBRARY_PATH; - export LD_LIBRARY_PATH - PATH=//osra/1.4.0/YYY/depends_on/XXX/bin:$PATH; - export PATH - OSRA_DATA_FILES=//osra/1.4.0/YYY/depends_on/XXX/share; - export OSRA_DATA_FILES - ---- + LD_LIBRARY_PATH=//graphicsmagick/1.3.18/YYY/package_graphicsmagick_1_3/XXX/gmagick/lib/:$LD_LIBRARY_PATH; + export LD_LIBRARY_PATH + LD_LIBRARY_PATH=//osra/1.4.0/YYY/depends_on/XXX/potrace/build/lib/:$LD_LIBRARY_PATH; + export LD_LIBRARY_PATH + PATH=//osra/1.4.0/YYY/depends_on/XXX/bin:$PATH; + export PATH + OSRA_DATA_FILES=//osra/1.4.0/YYY/depends_on/XXX/share; + export OSRA_DATA_FILES """ env_var_value = env_var_dict[ 'value' ] # env_var_value is the text of an environment variable tag like this: @@ -1575,7 +1573,6 @@ class SetupPythonEnvironment( Download, RecipeStep ): although it may never be necessary. If initial_download is True, the recipe steps will be filtered and returned and the installation directory (i.e., dir) will be defined and returned. If we're not in the initial download stage, these actions will not occur, and None values will be returned for them. - """ # # diff --git a/scripts/bootstrap_history.py b/scripts/bootstrap_history.py index b2968881765..fdc74493332 100644 --- a/scripts/bootstrap_history.py +++ b/scripts/bootstrap_history.py @@ -46,9 +46,8 @@ DEVTEAM = [ TEMPLATE = """ .. to_doc -------------------------------- %s -------------------------------- +=============================== .. announce_start diff --git a/scripts/galaxy-main b/scripts/galaxy-main index 8a4796d939b..7523c549fcd 100755 --- a/scripts/galaxy-main +++ b/scripts/galaxy-main @@ -57,11 +57,8 @@ REQUIRES_DAEMONIZE_MESSAGE = "Attempted to use Galaxy in daemon mode, but daemon log = logging.getLogger(__name__) -base_file = os.path.basename(__file__) -if base_file == 'main.py': - GALAXY_ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir, os.pardir)) -elif base_file == 'galaxy-main': - GALAXY_ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir)) +real_file = os.path.realpath(__file__) +GALAXY_ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(real_file), os.pardir)) GALAXY_LIB_DIR = os.path.join(GALAXY_ROOT_DIR, "lib") DEFAULT_INI_APP = "main" DEFAULT_INIS = ["config/galaxy.ini", "universe_wsgi.ini", "config/galaxy.ini.sample"]