opensource/galaxy
3
0
Fork 0
mirror of https://github.com/galaxyproject/galaxy.git synced 2026-09-24 16:30:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/release_20.01' into dev

Browse Source
This commit is contained in:
Dannon Baker
2020-02-07 09:07:11 -05:00
parent 81d20839f5 6ebb796c56
commit 226ae08597
14 changed files with 61 additions and 101 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/admin/config_logging.rst
+2 -2
View File
@@ -238,5 +238,5 @@ advanced logging configuration. For example:
While Galaxy's custom log format fields can be used (as seen in the example), the ``filename_template`` handler
configuration extension is only available in the YAML format configuration file.
.. _logging levels: https://docs.python.org/2/library/logging.html#logging-levels
.. _fileConfig file format: https://docs.python.org/2/library/logging.config.html#configuration-file-format
.. _logging levels: https://docs.python.org/library/logging.html#logging-levels
.. _fileConfig file format: https://docs.python.org/library/logging.config.html#configuration-file-format
doc/source/admin/framework_dependencies.rst
+1 -19
View File
@@ -96,18 +96,6 @@ this script yourself to set up Galaxy pip and the dependencies without creating
.. code-block:: console
(venv)$ PYTHONPATH= sh /srv/galaxy/server/scripts/common_startup.sh --no-create-venv
Requirement already satisfied: pip>=8.1 in /home/nate/.virtualenvs/test/lib/python2.7/site-packages
Collecting numpy==1.9.2 (from -r requirements.txt (line 4))
Downloading https://wheels.galaxyproject.org/packages/numpy-1.9.2-cp27-cp27mu-manylinux1_x86_64.whl (10.2MB)
100% |████████████████████████████████| 10.2MB 21.7MB/s
Collecting bx-python==0.7.3 (from -r requirements.txt (line 5))
Downloading https://wheels.galaxyproject.org/packages/bx_python-0.7.3-cp27-cp27mu-manylinux1_x86_64.whl (2.1MB)
100% |████████████████████████████████| 2.2MB 97.2MB/s
...
Installing collected packages: numpy, bx-python, ...
Successfully installed numpy-1.9.2 bx-python-0.7.3 ...
.. warning::
@@ -195,7 +183,7 @@ Galaxy can create a virtualenv using the adapted virtualenv package. Once a vali
A Conda environment named ``_galaxy_`` will be created using python 2 and the appropriate virtualenv package will be installed into this environment.
Using this environment a ``.venv`` is initialized. This is a one-time setup, and all other activation and dependency
management happens exactly as if a system python was used for creating ``.venv``.
management happens exactly as if a system Python was used for creating ``.venv``.
.. _Conda: https://conda.io/
.. _Conda environments: https://conda.io/docs/user-guide/tasks/manage-environments.html
@@ -255,12 +243,6 @@ Adding additional Galaxy dependencies
New packages can be added to Galaxy, or the versions of existing packages can be updated, using `pipenv`_ and `Starforge`_, Galaxy's Docker-based build system.
.. note::
Dependency pinning management is being migrated to pipenv_. As of this release, pinning for packages used for Galaxy
development are managed by pipenv_, but pinning for regular runtime packages are still managed with manual changes
to ``pinned-requirements.txt``. See `Pull Request #4891`_ for details.
The process is still under development and will be streamlined and automated over time. For the time being, please use
the following process to add new packages and have their wheels built:
doc/source/admin/nginx.md
+5 -1
View File
@@ -349,11 +349,15 @@ galaxy:
nginx_upload_path: '/_upload'
```
```eval_rst
.. _protect-reports:
```
### Use Galaxy Authentication to Protect Custom Paths
You may find it useful to require authentication for access to certain paths on your server. For example, Galaxy can
run a separate reports app which gives useful information about your Galaxy instance. See the [Reports Configuration
documentation](reports.md) and [Peter Briggs' blog post on the
documentation](reports) and [Peter Briggs' blog post on the
subject](http://galacticengineer.blogspot.com/2015/06/exposing-galaxy-reports-via-nginx-in.html) for more.
After successfully following the blog post, Galaxy reports should be available at e.g. `https://galaxy.example.org/reports`.
doc/source/admin/reports.md
-21
View File
@@ -1,21 +0,0 @@
# Galaxy Reports
Galaxy includes a report tool that is separate from the main process but which gives lots of potentially useful information about the usage of a Galaxy instance, for example the numbers of jobs that have been run each month, how much disk space each user is currently consuming and so on.
## Setup on localhost
The report tool takes its configuration settings from a file called `reports.yml`, which is located in the ``config/` subdirectory of the Galaxy distribution.
Configuring the reports for your local setup is a case of:
* Making a copy of ``reports.yml.sample`` called `reports.yml`.
* Editing the `database_connection` parameter to match the one in your `galaxy.yml` file
* Optionally, editing the `port` parameter (by default the tool uses port 9001)
You should also set the 'salt' parameter `session_secret` if you intend to expose the reports via the web proxy (see below).
Then you can start the report server using `sh run_reports.sh` and view the reports by pointing a web browser running on the same server to http://127.0.0.1:9001. If you'd like the report tool to persist between sessions then use `sh run_reports.sh --daemon` to run it as a background process. As with Galaxy itself, use `--stop-daemon` to halt the background process. (The log file is written to `reports_webapp.log` if you need to try and debug a problem.)
## Expose Outside
To make your reports available from outside of the localhost using NGINX proxy server you can check out the [blogpost](http://galacticengineer.blogspot.co.uk/2015/06/exposing-galaxy-reports-via-nginx-in.html) by Peter Briggs and the [Protect Galaxy Reports](https://galaxyproject.org/admin/config/nginx-proxy/#protect-galaxy-reports) section at the [Community Hub](https://galaxyproject.org).
doc/source/admin/reports.rst
+11 -1
View File
@@ -45,10 +45,20 @@ Configuration
- The default port for the reports application is ``9001``, and like Galaxy it only binds to localhost by default.
- ``database_connection`` should match the value used in your Galaxy configuration
- ``database_connection`` should point at a Postgres database, experimental support for MySQL is available but sqlite is not supported at all.
- ``database_connection`` should point at a PostgreSQL database,
experimental support for MySQL is available but SQLite is not supported
at all.
- Run reports in a uWSGI server with ``sh run_reports.sh``
- Use a web browser and go to the address you configured in ``reports.yml`` (defaults to http://localhost:9001/)
- If you'd like the report tool to persist between sessions then use
``sh run_reports.sh --daemon`` to run it as a background process. As with
Galaxy itself, use the ``--stop-daemon`` option to halt the background
process. (The process output is written to ``reports_webapp.log`` if you have
to debug a problem.)
- To make your reports available from outside of the localhost using the NGINX
proxy server, you can check out the documentation on how to
:ref:`protect Galaxy reports <protect-reports>` using authentication.
----------------------------
Configuration Options
doc/source/admin/reports_options.rst
+1 -2
View File
@@ -4,8 +4,7 @@
:Description:
Verbosity of console log messages. Acceptable values can be found
here:
https://docs.python.org/2/library/logging.html#logging-levels
here: https://docs.python.org/library/logging.html#logging-levels
:Default: ``DEBUG``
:Type: str
doc/source/admin/special_topics/index.rst
+1
View File
@@ -6,6 +6,7 @@ Special Topics
:maxdepth: 2
ftp
interactivetools
interactive_environments
mulled_containers
grt
doc/source/admin/special_topics/interactivetools.rst
+1 -1
View File
@@ -40,7 +40,7 @@ Some important benefits of using Galaxy InteractiveTools
Server-side configuration of Galaxy InteractiveTools
-------------------------------------------------
----------------------------------------------------
The **galaxy.yml** file will need to be populated as seen in **config/galaxy.yml.interactivetools**.
doc/source/conf.py
+2 -2
View File
@@ -129,8 +129,8 @@ pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# Intersphinx mapping to Python 2.7 documentation
intersphinx_mapping = {'python': ('https://docs.python.org/2.7', None)}
# Intersphinx mapping to Python documentation
intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
# -- Options for HTML output ---------------------------------------------------
doc/source/dev/build_a_job_runner.rst
+23 -23
View File
@@ -234,7 +234,7 @@ Updated result after an iteration (after invocation of ``check_watched_item`` 6
Note: Iterating through the queue is already taken care by the framework.
To inform galaxy about the status of the job:
To inform Galaxy about the status of the job:
- Get the job status from external runner using the ``job_id``.
@@ -249,38 +249,38 @@ To inform galaxy about the status of the job:
.. code-block:: python
def check_watched_item(self, job_state):
!job_status = get_task_from_external_runner(job_state.job_id)
if job_status == "over_with_success":
job_state.running = False
job_state.job_wrapper.change_state(model.Job.states.OK)
!create_log_file()
self.mark_as_finished(job_state)
return None
job_status = get_task_from_external_runner(job_state.job_id)
if job_status == "over_with_success":
job_state.running = False
job_state.job_wrapper.change_state(model.Job.states.OK)
create_log_files()
self.mark_as_finished(job_state)
return None
elif job_status == "running":
job_state.running = True
job_state.job_wrapper.change_state(model.Job.states.RUNNING)
return job_state
elif job_status == "running":
job_state.running = True
job_state.job_wrapper.change_state(model.Job.states.RUNNING)
return job_state
elif job_status == "pending":
return job_state
elif job_status == "pending":
return job_state
elif job_status == "over_with_error":
job_state.running = False
job_state.job_wrapper.change_state(model.Job.states.ERROR)
!create_log_file()
self.mark_as_failed(job_state)
return None
elif job_status == "over_with_error":
job_state.running = False
job_state.job_wrapper.change_state(model.Job.states.ERROR)
create_log_files()
self.mark_as_failed(job_state)
return None
Note:
- Methods prefixed with ! are user-defined methods.
- ``get_task_from_external_runner`` and ``create_log_files`` are user-defined methods.
- Return value is ``job_state`` for running, pending jobs and None for rest of the states of jobs.
``create_log_files()`` are nothing but copying the files (``error_file``,
``output_file``, ``exit_code_file``) from external runner's directory to
working directory of Galaxy.
``output_file``, ``exit_code_file``) from the external runner's directory to
the working directory of Galaxy.
Source of the files are from the output directory of your external
runner. Destination of the files will be:
doc/source/dev/data_managers.rst
+2 -7
View File
@@ -365,9 +365,7 @@ Example 1 JSON Output from Data Manager Tool to Galaxy
}
}
This creates a new entry in the Tool Data Table:
.. code-block::
This creates a new entry in the Tool Data Table::
#<unique_build_id> <dbkey> <display_name> <file_path>
sacCer2 sacCer2 S. cerevisiae June 2008 (SGD/sacCer2) (sacCer2) /Users/dan/galaxy-central/tool-data/sacCer2/seq/sacCer2.fa
@@ -881,10 +879,7 @@ Example JSON Output from tool to galaxy, dbkey is sacCer2
}
}
New Entry in Data Table, dbkey is sacCer2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block::
This creates a new entry in the Tool Data Table::
#<unique_build_id> <dbkey> <display_name> <file_path>
sacCer2 sacCer2 S. cerevisiae June 2008 (SGD/sacCer2) (sacCer2) /Users/dan/galaxy-central/tool-data/sacCer2/seq/sacCer2.fa
doc/source/dev/data_types.md
+10 -20
View File
@@ -35,27 +35,17 @@ tag section of the `datatypes_conf.xml` file. Sample
where
- `extension` - the data type's Dataset file
extension (e.g., `ab1`, `bed`, `gff`, `qual`,
etc.) `type` - the path to the class for that
data type.
- `extension` - the data type's Dataset file extension (e.g., `ab1`, `bed`,
`gff`, `qual`, etc.)
- `type` - the path to the class for that data type.
- `mimetype` - if present (it's optional), the data type's mime type
- `display_in_upload` - if present (it's optional and defaults to False), the
associated file extension will be displayed in the "File Format" select list
in the "Upload File from your computer" tool in the "Get Data" tool section of
the tool panel.
- `mimetype` - if present (it's optional), the data
type's mime type `display_in_upload` - if present
(it's optional and defaults to False), the
associated file extension will be displayed in
the "File Format" select list in the "Upload File
from your computer" tool in the "Get Data" tool
section of the tool panel.
```xml
<datatypes>
<registration converters_path="lib/galaxy/datatypes/converters">
<datatype extension="ab1" type="galaxy.datatypes.images:Ab1" mimetype="application/octet-stream" display_in_upload="true"/>
<datatype extension="foo" type="galaxy.datatypes.tabular:Foobar" display_in_upload="true"/>
...
```
**Note:** If you do not wish to add extended
functionality to for a new datatype, but simply want
functionality to a new datatype, but simply want
to restrict the output of a set of tools to be used
in another set of tools, you can add the flag
`subclass="True"` to the datatype definition line.
@@ -560,7 +550,7 @@ from galaxy.datatypes.metadata import MetadataElement
The use of `<converter>` tags contained within `<datatype>` tags is supported in the same way they are supported within the `datatypes_conf.xml.sample` file in the Galaxy distribution.
```xml
<datatype extension="ref.taxonomy" type="galaxy.datatypes.metagenomics:RefTaxonomy" display_in_upload="true"
<datatype extension="ref.taxonomy" type="galaxy.datatypes.metagenomics:RefTaxonomy" display_in_upload="true">
<converter file="ref_to_seq_taxonomy_converter.xml" target_datatype="seq.taxonomy"/>
</datatype>
```
lib/galaxy/config/sample/reports.yml.sample
+1 -1
View File
@@ -90,7 +90,7 @@ uwsgi:
reports:
# Verbosity of console log messages. Acceptable values can be found
# here: https://docs.python.org/2/library/logging.html#logging-levels
# here: https://docs.python.org/library/logging.html#logging-levels
#log_level: DEBUG
# Database connection. Galaxy Reports are intended for production
lib/galaxy/webapps/reports/config_schema.yml
+1 -1
View File
@@ -13,7 +13,7 @@ mapping:
default: DEBUG
desc: |
Verbosity of console log messages. Acceptable values can be found here:
https://docs.python.org/2/library/logging.html#logging-levels
https://docs.python.org/library/logging.html#logging-levels
database_connection:
type: str