mirror of
https://github.com/galaxyproject/galaxy.git
synced 2026-09-21 13:50:20 +08:00
WIP admin documentation updates
This commit is contained in:
Nate Coraor
@@ -0,0 +1,57 @@
|
||||
Application Server
|
||||
========================================
|
||||
|
||||
Overview
|
||||
----------------------------
|
||||
|
||||
It is possible to run the Galaxy server in many different ways, including under different web application frameworks, or
|
||||
as a standalone server with no web stack. For most of its modern life, prior to the 18.01 release, Galaxy used the
|
||||
`Python Paste`_ web stack, and ran in a single process.
|
||||
|
||||
Beginning with Galaxy release 18.01, the default application server for new installations of Galaxy is `uWSGI`_, which
|
||||
has numerous benefits:
|
||||
|
||||
- Written in C and designed to be high performance.
|
||||
- Internal support for serving static content.
|
||||
- Supports WebSockets, which allows for support of Galaxy Interactive Environments out of the box, removing the
|
||||
dependency on Node.js.
|
||||
- ...
|
||||
|
||||
In addition, uWSGI has been the recommended application server for production Galaxy servers for many years, but was not
|
||||
previously provided with Galaxy.
|
||||
|
||||
What server am I running?
|
||||
----------------------------
|
||||
|
||||
**Galaxy releases older than 18.01**
|
||||
|
||||
- If you start your server with the provided ``run.sh`` script, it is using Paste.
|
||||
-
|
||||
|
||||
Unless it has been manually configured to run under uWSGI, your server is using Paste.
|
||||
|
||||
**Galaxy releases 18.01 and newer**
|
||||
|
||||
If you upgraded from
|
||||
|
||||
In any case, ``pgrep(1)`` or ``ps(1)`` should provide the answer, for example:
|
||||
|
||||
```sh-session
|
||||
galaxy@server$ pgrep -af -u $EUID galaxy
|
||||
16232 uwsgi --ini-paste config/galaxy.ini
|
||||
16233 uwsgi --ini-paste config/galaxy.ini
|
||||
17406 /srv/galaxy/venv/bin/python2.7 /srv/galaxy/venv/bin/uwsgi --yaml config/galaxy.yml
|
||||
17419 /srv/galaxy/venv/bin/python2.7 /srv/galaxy/venv/bin/uwsgi --yaml config/galaxy.yml
|
||||
17172 python ./scripts/paster.py serve config/galaxy.ini
|
||||
```
|
||||
|
||||
The first 4 processes listed are uWSGI
|
||||
|
||||
|
||||
|
||||
If you are upgrading to Galaxy 18.01
|
||||
from an older release, still have your ``galaxy.ini`` file in place, and have not manually switched to another stack
|
||||
(such as uWSGI), Galaxy will continue to run under Paste.
|
||||
|
||||
.. _Python Paste: http://paste.readthedocs.io/
|
||||
.. _uWSGI: https://uwsgi-docs.readthedocs.io/
|
||||
@@ -0,0 +1,111 @@
|
||||
Galaxy Configuration
|
||||
========================================
|
||||
|
||||
Overview
|
||||
----------------------------
|
||||
|
||||
Galaxy has a large number of configuration files in an array of formats. Although it is not required to modify *any*
|
||||
Galaxy configuration files to run the server, most Galaxy servers will modify at least the core configuration file.
|
||||
These have grown organically over time as new features with the needs for advanced syntaxes and config isolation have
|
||||
been added. An effort is under way to `standardize and unify configuration formats`_.
|
||||
|
||||
Configuration files can be found underneath the ``config/`` subdirectory, wherein you can find ``<name>.sample`` files
|
||||
corresponding to configuration files that you can modify by copying the ``<name>.sample`` to ``<name>``. In many cases,
|
||||
you will find that the sample configuration provides the most up-to-date and detailed documentation about the features
|
||||
configured therein.
|
||||
|
||||
Most config files are administered by hand, although a few (ones which begin with ``shed_*``) are modified by Galaxy
|
||||
when installing from the Galaxy Tool Shed. When starting Galaxy for the first time, these files are copied from their
|
||||
sample files automatically. You should not need to modify these unless you want to change the directory in to which Tool
|
||||
Shed tools are installed.
|
||||
|
||||
|
||||
Configuration Files
|
||||
----------------------------
|
||||
|
||||
The primary Galaxy configuration file is ``galaxy.yml``. You will need to use this file to modify core functionality
|
||||
such as the port on which Galaxy listens, the directory in which Galaxy datasets are stored, the database connection
|
||||
options, and so forth.
|
||||
|
||||
The most commonly modified configuration files include:
|
||||
|
||||
- ``galaxy.yml``: Core Galaxy configuration file.
|
||||
- ``tool_conf.xml``: Describes the paths to local tool configurations that Galaxy should attempt to load. Tools that
|
||||
are installed via the Tool Shed are configured to load in the ``shed_tool_conf.xml`` file. See the :ref:`Tool Panel
|
||||
Administration <tool_panel>` and `Installing Tools into Galaxy`_ documentation for more.
|
||||
- ``datatypes_conf.xml``: Describes the file formats that are supported in Galaxy. See the `Datatypes documentation`_
|
||||
for more.
|
||||
- ``job_conf.xml``: Controls how Galaxy runs tools, e.g. to run them on a compute cluster. See the `Cluster
|
||||
documentation`_ for more.
|
||||
|
||||
Some configuration files are only used when adding local components, rather than ones installed from the Tool Shed:
|
||||
|
||||
- ``tool_conf.xml``: As described above.
|
||||
- ``tool_data_table_conf.xml``: Describes the mapping between Data Tables - the structured format that allow tools to
|
||||
work with locally cached reference data - and the location files that describe the actual data that is available (e.g.
|
||||
paths, genome builds, etc.). Data table configurations are also provided by tools in the Tool Shed, those are
|
||||
configured in ``shed_tool_data_table_conf.xml``. See the `Data Preparation documentation`_ for more.
|
||||
- ``data_manager_conf.xml``: Describes the paths to local Data Managers, special Galaxy tools that automatically fetch
|
||||
or create data for Data Tables and manage the corresponding data table and location configuration files. See the `Data
|
||||
Managers documentation`_ for more.
|
||||
- ``local_conda_mapping.yml``: Define mappings between the names specified in the tool configuration (``<requirement>``
|
||||
tags) and the conda resolver's names (conda package name).
|
||||
- ``lmod_modules_mapping.yml``: Define mappings between the names specified in the tool configuration (``<requirement>``
|
||||
tags) and the Lmod system.
|
||||
|
||||
Some configuration files are used to control the way that Galaxy resolves tool dependencies. Most Galaxy tools are only
|
||||
descriptions of how to run a particular command line tool, and they do not contain the dependent command line tool. The
|
||||
task of locating and making available these command line tools is performed by the Galaxy tool dependencies system,
|
||||
which has configuration files of its own:
|
||||
|
||||
Additional configuration files and their purposes are:
|
||||
|
||||
- ``auth_conf.xml``: Configures the pluggable authentication service. By default, Galaxy users are created and managed
|
||||
internally.
|
||||
- ``build_sites.yml``: Controls which display applications are available and their configuration paths
|
||||
- ``containers_conf.yml``: Configures the beta Galaxy containers interface, currently only used by Galaxy Interactive
|
||||
Environments, and only neccesary for Docker Swarm support.
|
||||
- ``dependency_resolvers_conf.xml``: Describes how Galaxy tools (which are typically just descriptions of how to run a
|
||||
particular command line tool) should locate their dependencies (the command line tool) that are not part of the tool.
|
||||
See the `Dependency Resolvers documentation <dependency_resolvers>` for more.
|
||||
- ``error_report.yml``: Controls how user-initiated error reporting (e.g. due to tool failure) is performed. See the
|
||||
:ref:`Bug Reports documentation <bug_reports>` for more.
|
||||
- ``job_metrics_conf.xml``: Enables reporting of certain conditions and collection of metrics when jobs run.
|
||||
- ``job_resource_params_conf.xml``: Describes tool form elements that should be inserted into tool forms that can be
|
||||
used by users to control runtime parameters such as memory allocations, cluster selection, and so forth.
|
||||
- ``object_store_conf.xml``: Configures more advanced storage paradigms for Galaxy datasets, including layout across
|
||||
multiple filesystems, or in object storage systems such as Swift or Amazon S3.
|
||||
- ``openid_conf.xml``: Controls which OpenID (if enabled) providers should be presented as options to the user on the
|
||||
login form.
|
||||
- ``swarm_manager_conf.yml``: Configures the experimental Docker Swarm manager.
|
||||
- ``tool_destinations.yml``: Configures dynamic tool destinations, which allow for mapping tools to job destinations
|
||||
based on certain runtime job properties, such as the user submitting it, input sizes, and so forth.
|
||||
- ``tool_sheds_conf.xml``: Defines the list of Tool Shed servers that should appear in the Galaxy Administration
|
||||
interface when searching for new tools.
|
||||
- ``workflow_schedulers_conf.xml``: Similar to the job configuration, controls the scheduling of workflows as jobs.
|
||||
|
||||
.. _standardize and unify configuration formats: https://github.com/galaxyproject/galaxy/issues/...
|
||||
.. _Installing Tools into Galaxy: https://galaxyproject.org/admin/tools/add-tool-from-toolshed-tutorial/
|
||||
.. _Datatypes documentation: https://galaxyproject.org/learn/datatypes/
|
||||
.. _Cluster documentation: https://galaxyproject.org/admin/config/performance/cluster/
|
||||
.. _Data Preparation documentation: https://galaxyproject.org/admin/data-preparation/
|
||||
.. _Data Managers documentation: https://galaxyproject.org/admin/tools/data-managers/
|
||||
|
||||
|
||||
Configuration
|
||||
----------------------------
|
||||
|
||||
- Configure ``config/reports.yml`` in the same manner as your main galaxy instance (i.e., same database connection, but different port). This is a uwsgi YAML configuration file and should contain a reports section with app-specific configuration (options described below).
|
||||
|
||||
- The default port for the reports application is ``9001``, and like Galaxy it only binds to localhost by default.
|
||||
- ``database_connection`` should match the value used in your Galaxy configuration
|
||||
- ``database_connection`` should point at a Postgres database, experimental support for MySQL is available but sqlite is not supported at all.
|
||||
|
||||
- Run reports in a uWSGI server with ``sh run_reports.sh``
|
||||
- Use a web browser and go to the address you configured in ``reports.yml`` (defaults to http://localhost:9001/)
|
||||
|
||||
.. ----------------------------
|
||||
.. Configuration Options
|
||||
.. ----------------------------
|
||||
..
|
||||
.. .. include:: galaxy_options.rst
|
||||
@@ -0,0 +1,99 @@
|
||||
Logging Configuration
|
||||
========================================
|
||||
|
||||
Overview
|
||||
----------------------------
|
||||
|
||||
Logging in Galaxy is performed through the Python `logging`_ module. Traditionally, and on Galaxy releases prior to
|
||||
18.01, customization of the logging could be performed in two ways:
|
||||
|
||||
1. Basic/automatic configuration with control over log level and log destination (standard output or a named log file).
|
||||
2. More complex configuration using `logging.config.fileConfig`_.
|
||||
|
||||
Starting with release 18.01, it's possible to use the more featureful `logging.config.dictConfig`_ format, with a few
|
||||
Galaxy-specific extensions described below.
|
||||
|
||||
By default, Galaxy logs all messages to standard output at the ``DEBUG`` logging level.
|
||||
|
||||
.. .. The way in which logging is configured depends on whether you are using Python Paste or uWSGI. See :ref:`Application
|
||||
.. .. Server <application_server>` for help figuring out what you're using. If using uWSGI, it also depends on whether you are
|
||||
.. .. using an INI or YAML configuration file.
|
||||
|
||||
The way in which logging is configured depends on whether you are using an YAML or INI configuration file. Galaxy
|
||||
servers that were created starting with Galaxy Release 18.01 or later use a YAML configuration file. Galaxy servers that
|
||||
were created with 17.09 or older use an INI configuration file. If you upgrade a pre-18.01 server to 18.01 or later but
|
||||
do not convert your INI config (``galaxy.ini``) to a YAML config (``galaxy.yml``), the INI config will still be used.
|
||||
|
||||
Basic Configuration
|
||||
----------------------------
|
||||
|
||||
YAML
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
**Setting the log level:**
|
||||
|
||||
```yaml
|
||||
galaxy:
|
||||
log_level: LEVEL
|
||||
```
|
||||
|
||||
Where ``LEVEL`` is one of the `logging levels`_ documented in the `logging`_ module.
|
||||
|
||||
**Logging to a file:**
|
||||
|
||||
When using ``run.sh --daemon``, the log is written to ``galaxy.log`` in the current directory. To change this:
|
||||
|
||||
INI
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Setting the log level:
|
||||
|
||||
|
||||
```ini
|
||||
[app:main]
|
||||
log_level = LEVEL
|
||||
```
|
||||
|
||||
**Logging to a file:**
|
||||
|
||||
Where ``LEVEL`` is one of the `logging levels`_ documented in the `logging`_ module.
|
||||
|
||||
When using ``run.sh --daemon``, the log is written to ``paster.log`` in the current directory. To change this:
|
||||
|
||||
Advanced Configuration
|
||||
----------------------------
|
||||
|
||||
YAML
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
INI
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
On Galaxy servers older than 18.01, or newer servers still running under Paste (see :ref:`Application Server
|
||||
<application_server`), the logging level is controlled by the ``log_level`` option in ``galaxy.ini``. The possible
|
||||
values are the `logging levels`_ documented in the `logging`_ module. If not set, the default is ``INFO``, however,
|
||||
``galaxy.ini.sample`` ships with the value set to ``DEBUG`` for development purposes.
|
||||
|
||||
By default, if started in the foreground (e.g. ``run.sh`` with no arguments), log messages are sent to standard output.
|
||||
If running detached with ``run.sh --daemon``, messages are instead logged to ``paster.log`` in the current directory.
|
||||
|
||||
|
||||
|
||||
If not specified,
|
||||
the default l
|
||||
|
||||
|
||||
In the main Galaxy configuration file, ``galaxy.yml``,
|
||||
|
||||
|
||||
.. _logging: https://docs.python.org/2/library/logging.html
|
||||
.. _logging levels: https://docs.python.org/2/library/logging.html#logging-levels
|
||||
.. _logging.config.fileConfig: https://docs.python.org/2/library/logging.config.html#logging.config.fileConfig
|
||||
.. _logging.config.dictConfig: https://docs.python.org/2/library/logging.config.html#logging.config.dictConfig
|
||||
.. _fileConfig file format: https://docs.python.org/2/library/logging.config.html#configuration-file-format
|
||||
Reference in New Issue
Block a user