opensource/galaxy
3
0
Fork 0
mirror of https://github.com/galaxyproject/galaxy.git synced 2026-09-21 05:45:37 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #13436 from mvdbeek/document_gunicorn_nginx

Browse Source 
[22.01] Document gunicorn and nginx
This commit is contained in:
John Chilton
2022-02-24 10:43:04 -05:00
committed by GitHub
parent d053cf8f85 80b5bc7c65
commit 897b5c22dc
3 changed files with 46 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/admin/_inc_proxy_serving_root.rst
+5 -4
View File
@@ -1,10 +1,11 @@
This configuration assumes that Galaxy will be the only site on your server using the given hostname (e.g.
``https://galaxy.example.org``).
Beginning with Galaxy Release 18.01, the default application server that Galaxy runs under is uWSGI. Because of this,
the native high performance uWSGI protocol should be used for communication between |PROXY| and Galaxy, rather
than HTTP. Legacy instructions for proxying via HTTP can be found in the `Galaxy Release 17.09 proxy documentation`_.
Beginning with Galaxy Release 22.01, the default application server that Galaxy runs under is Gunicorn. Because of this,
the HTTP protocol should be used for communication between |PROXY| and Galaxy, rather
than the uWSGI protocol used in earlier version of this document.
Legacy instructions for proxying via uWSGI can be found in the `Galaxy Release 21.09 proxy documentation`_.
Since |PROXY| is more efficient than uWSGI at serving static content, it is best to serve it directly, reducing the load
Since |PROXY| is more efficient than Gunicorn at serving static content, it is best to serve it directly, reducing the load
on the Galaxy process and allowing for more effective compression (if enabled), caching, and pipelining. Directives to
do so are included in the example below.
doc/source/admin/nginx.md
+30 -33
View File
@@ -28,9 +28,7 @@ Instructions for [proxying with Apache](apache.md) are also available.
### NGINX Proxy Prerequisities
If you plan to use nginx to handle your file uploads, you will (most likely) not be able to use your package manager's
version of nginx. The [Receiving Files With NGINX](#receiving-files-with-nginx) section explains this in detail and
provides some options for installing *nginx + upload module* packages maintained by the Galaxy Committers Team.
If you are **not** planning to use the recommended [tus.io method](#receiving-files-via-the-tus-protocol) to handle file uploads but want to use nginx to handle uploads, you will (most likely) not be able to use your package manager's version of nginx. The [Receiving Files With NGINX](#receiving-files-with-nginx) section explains this in detail and provides some options for installing *nginx + upload module* packages maintained by the Galaxy Committers Team.
Otherwise, your system package manager's version of nginx should be suitable. Under Debian, the
[nginx-light][nginx-light] package contains all the necessary modules used in this guide. On EL, the [EPEL][epel]
@@ -49,11 +47,9 @@ version of nginx is suitable.
```eval_rst
.. include:: _inc_proxy_serving_root.rst
.. _Galaxy Release 17.09 Proxy Documentation: https://docs.galaxyproject.org/en/release_17.09/admin/special_topics/nginx.html
.. _Galaxy Release 21.09 Proxy Documentation: https://docs.galaxyproject.org/en/release_21.09/admin/nginx.html
```
uWSGI protocol support is built in to nginx, so (unlike Apache) no extra modules or recompiling should be required.
The following configuration is not exhaustive, only the portions most relevant to serving Galaxy are shown, these should
be incorporated with your existing/default nginx config as is appropriate for your server. Notably, the nginx package
you installed most likely has a multi-file config layout. If you are not already familiar with that layout and where
@@ -75,7 +71,7 @@ http {
gzip_buffers 16 8k;
# allow up to 3 minutes for Galaxy to respond to slow requests before timing out
uwsgi_read_timeout 180;
proxy_read_timeout 180;
# maximum file upload size
client_max_body_size 10g;
@@ -122,11 +118,13 @@ http {
# Enable HSTS
add_header Strict-Transport-Security "max-age=15552000; includeSubdomains";
# proxy all requests not matching other locations to uWSGI
# proxy all requests not matching other locations to Gunicorn
location / {
uwsgi_pass unix:///srv/galaxy/var/uwsgi.sock;
uwsgi_param UWSGI_SCHEME $scheme;
include uwsgi_params;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_pass http://unix:/srv/galaxy/var/gunicorn.sock;
}
# serve framework static content
@@ -152,9 +150,8 @@ http {
}
```
Be sure to set `$galaxy_root` to the path to your copy of Galaxy and modify the value of `uwsgi_pass` to match your
uWSGI socket path. With the default configuration, uWSGI will bind to a random TCP socket, so you will need to set it to
a fixed value as described in the [Scaling and Load Balancing](scaling.md) documentation. If using a UNIX domain
Be sure to set `$galaxy_root` to the path to your copy of Galaxy and modify the value of `proxy_pass` to match your
Gunicorn socket path. With the default configuration, gunicorn will bind to a TCP socket, so you will need to Gunicorn to bind to a UNIX domain socket as described in the [Scaling and Load Balancing](scaling.md) documentation. If using a UNIX domain
socket, be sure to pay particular attention to the discussion of users and permissions.
### Additional Notes
@@ -165,9 +162,9 @@ socket, be sure to pay particular attention to the discussion of users and permi
- If your existing nginx configuration contains a line or included config file defining a default server, be sure to
disable it by commenting its `server {}` or preventing its inclusion (under Debian this is done by removing its
symlink from `/etc/nginx/sites-enabled`).
- `uwsgi_read_timeout` can be adjusted as appropriate for your site. This is the amount of time allowed for
communication between nginx and uWSGI to block while waiting for a response from Galaxy, and is useful for holding
client (browser) connections while uWSGI is restarting Galaxy subprocesses or Galaxy is performing a slow operation.
- `proxy_read_timeout` can be adjusted as appropriate for your site. This is the amount of time allowed for
communication between nginx and Gunicorn to block while waiting for a response from Galaxy, and is useful for holding
client (browser) connections while Gunicorn is restarting Galaxy subprocesses or Galaxy is performing a slow operation.
- The parameter `client_max_body_size` specifies the maximum upload size that can be handled by POST requests through
nginx. You should set this to the largest file size that you wish to allow for upload and that could be reasonably
handled by your site. It defaults to 1MB, so it will need to be increased if you are dealing with genome sized
@@ -191,11 +188,13 @@ previous section:
```nginx
#...
# proxy all requests not matching other locations to uWSGI
# proxy all requests not matching other locations to Gunicorn
location /galaxy {
uwsgi_pass unix:///srv/galaxy/var/uwsgi.sock
uwsgi_param UWSGI_SCHEME $scheme;
include uwsgi_params;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_pass http://unix:/srv/galaxy/var/gunicorn.sock:/galaxy;
}
# serve framework static content
@@ -211,16 +210,17 @@ previous section:
```
2. The Galaxy application needs to be aware that it is running with a prefix (for generating URLs in dynamic pages).
This is accomplished by configuring uWSGI (the `uwsgi` section in `config/galaxy.yml`) like so and restarting Galaxy:
This is accomplished by configuring Galaxy in your `config/galaxy.yml` file like so and restarting Galaxy:
```yaml
uwsgi:
#...
socket: /srv/galaxy/var/uwsgi.sock
mount: /galaxy=galaxy.webapps.galaxy.buildapp:uwsgi_app()
manage-script-name: true
# `module` MUST NOT be set when `mount` is in use
#module: galaxy.webapps.galaxy.buildapp:uwsgi_app()
gravity:
gunicorn:
# ...
bind: /srv/galaxy/var/gunicorn.sock
galaxy:
# ...
galaxy_url_prefix: /galaxy
# ...
```
```eval_rst
@@ -229,8 +229,7 @@ previous section:
overrides the automatic setting. If you have this option set, unset it unless you know what you're doing.
```
Be sure to consult the [Scaling and Load Balancing](scaling.md) documentation, other options unrelated to proxying
should also be set in the `uwsgi` section of the config.
Be sure to consult the [Scaling and Load Balancing](scaling.md) documentation.
## Advanced Configuration Topics
@@ -488,8 +487,6 @@ subject](http://galacticengineer.blogspot.com/2015/06/exposing-galaxy-reports-vi
After successfully following the blog post, Galaxy reports should be available at e.g. `https://galaxy.example.org/reports`.
To secure this page to only Galaxy administrators, adjust your nginx config accordingly:
**TODO:** This is not valid for the uWSGI proxy method and needs to be updated. -nate 2018-01-11
```nginx
location /reports {
#...
doc/source/admin/scaling.md
+11 -4
View File
@@ -210,11 +210,14 @@ gravity:
app_server: gunicorn
gunicorn:
# listening options
bind: '/srv/galaxy/var/gunicorn.sock'
bind: 'unix:/srv/galaxy/var/gunicorn.sock'
gunicorn_extra_args: '--forwarded-allow-ips="*"'
```
Here we've used a UNIX domain socket because there's less overhead than a TCP socket and it can be secured by filesystem
permissions, but you can also listen on a port:
permissions. Note that we've added `--forwarded-allow-ips="*"` to ensure that the domain socket is trusted as a source from which to proxy headers.
You can also listen on a port:
```yaml
app_server: gunicorn
@@ -223,6 +226,8 @@ permissions, but you can also listen on a port:
bind: '127.0.0.1:4001'
```
If you are listening on a port do not set `--forwarded-allow-ips="*"`.
The choice of port 4001 is arbitrary, but in both cases, the socket location must match whatever socket the proxy server
is configured to communicate with. If using a UNIX domain socket, be sure that the proxy server's user has read/write
permission on the socket. Because Galaxy and the proxy server most likely run as different users, this is not likely to
@@ -241,10 +246,12 @@ gravity:
app_server: gunicorn
gunicorn:
# listening options
bind: '/srv/galaxy/var/gunicorn.sock'
gunicorn_extra_args: '--bind 127.0.0.1:8080'
bind: 'unix:/srv/galaxy/var/gunicorn.sock'
gunicorn_extra_args: '--forwarded-allow-ips="*" --bind 127.0.0.1:8080'
```
Note that this should only be used for debugging purposes due to `--forwarded-allow-ips="*"`.
**Without a proxy server**:
It is strongly recommended to use a proxy server.