From 3c08896cdbc294eff8a9e2cc18c119a9a4c1df22 Mon Sep 17 00:00:00 2001 From: mvdbeek Date: Thu, 24 Feb 2022 18:19:05 +0100 Subject: [PATCH 1/6] Update performance tracking --- .../special_topics/performance_tracking.rst | 29 ++++++++----------- 1 file changed, 12 insertions(+), 17 deletions(-) diff --git a/doc/source/admin/special_topics/performance_tracking.rst b/doc/source/admin/special_topics/performance_tracking.rst index e500cca698b..59b3b6346c1 100644 --- a/doc/source/admin/special_topics/performance_tracking.rst +++ b/doc/source/admin/special_topics/performance_tracking.rst @@ -5,34 +5,29 @@ Tracking performance over time and identifying slow queries in your Galaxy can b Most performance tracking requires sending metrics to a metrics collection server such as `Graphite `__ or `StatsD `__. This document assumes you have already deployed a metrics server. -uWSGI +Gunicorn ----- -There is some built-in uWSGI support for performance logging. You can send uWSGI's internal metrics to a carbon (Graphite) server by setting the carbon option in your galaxy.yml: +There is some built-in Gunicorn support for performance logging. You can send Gunicorn's internal metrics to a StatsD server by setting the `--statsd-host` and `--statsd-prefix` command line options for Gunicorn in the `gravity` section of `galaxy.yml`: .. code-block:: yaml - uwsgi: - socket: ... - carbon: 127.0.0.1:2003 + gravity: + ... + gunicorn: + gunicorn_extra_args: `--statsd-host 127.0.0.1:8125 --statsd-prefix=galaxy` + ... -Or a StatsD server via: - -.. code-block:: yaml - - uwsgi: - socket: ... - stats-push: statsd:127.0.0.1:8125 - -The `official documentation `__ contains further information on uWSGI and stats servers. In the `uWSGI Stats Server `__ documentation, you can see an example of the sort of information that you will be able to collect. Note that you will need to make sure that the statsd pusher plugin is activated in your uWSGI servers. Alternatively, you can use `gxadmin `__ to generate data ready to load in an InfluxDB database. In this case, you will need to add the stats option to your galaxy.yml: .. code-block:: yaml - uwsgi: - socket: ... - stats: 127.0.0.1:9191 + gravity: + ... + gunicorn: + gunicorn_extra_args: `--statsd-host 127.0.0.1:9191 --statsd-prefix=galaxy` + ... And then run gxadmin like this: From a44e458f6ec1605c7abcec6c4f9503f972f21ee4 Mon Sep 17 00:00:00 2001 From: mvdbeek Date: Thu, 24 Feb 2022 18:23:30 +0100 Subject: [PATCH 2/6] Add gunicorn doc link, drop graphite --- doc/source/admin/special_topics/performance_tracking.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/source/admin/special_topics/performance_tracking.rst b/doc/source/admin/special_topics/performance_tracking.rst index 59b3b6346c1..f5a4a40829b 100644 --- a/doc/source/admin/special_topics/performance_tracking.rst +++ b/doc/source/admin/special_topics/performance_tracking.rst @@ -3,12 +3,12 @@ Galaxy Performance Tracking Tracking performance over time and identifying slow queries in your Galaxy can be an extremely important thing to do, especially for larger Galaxies. -Most performance tracking requires sending metrics to a metrics collection server such as `Graphite `__ or `StatsD `__. This document assumes you have already deployed a metrics server. +Most performance tracking requires sending metrics to a metrics collection server such as `StatsD `__. This document assumes you have already deployed StatsD. Gunicorn ----- -There is some built-in Gunicorn support for performance logging. You can send Gunicorn's internal metrics to a StatsD server by setting the `--statsd-host` and `--statsd-prefix` command line options for Gunicorn in the `gravity` section of `galaxy.yml`: +There is some built-in `Gunicorn `__ support for performance logging. You can send Gunicorn's internal metrics to a StatsD server by setting the `--statsd-host` and `--statsd-prefix` command line options for Gunicorn in the `gravity` section of `galaxy.yml`: .. code-block:: yaml From bbf412e7092d7291122b3dc65c537c5bcebe7cae Mon Sep 17 00:00:00 2001 From: Marius van den Beek Date: Thu, 24 Feb 2022 18:24:26 +0100 Subject: [PATCH 3/6] Use gunicorn prefix for gunicorn metrics Co-authored-by: Helena --- doc/source/admin/special_topics/performance_tracking.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/admin/special_topics/performance_tracking.rst b/doc/source/admin/special_topics/performance_tracking.rst index f5a4a40829b..b2099a4d05f 100644 --- a/doc/source/admin/special_topics/performance_tracking.rst +++ b/doc/source/admin/special_topics/performance_tracking.rst @@ -15,7 +15,7 @@ There is some built-in `Gunicorn Date: Thu, 24 Feb 2022 18:27:36 +0100 Subject: [PATCH 4/6] Drop uwsgi-specific stats setup Co-authored-by: Helena --- .../admin/special_topics/performance_tracking.rst | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/doc/source/admin/special_topics/performance_tracking.rst b/doc/source/admin/special_topics/performance_tracking.rst index b2099a4d05f..b1b166225be 100644 --- a/doc/source/admin/special_topics/performance_tracking.rst +++ b/doc/source/admin/special_topics/performance_tracking.rst @@ -19,18 +19,6 @@ There is some built-in `Gunicorn `__ to generate data ready to load in an InfluxDB database. In this case, you will need to add the stats option to your galaxy.yml: - -.. code-block:: yaml - - gravity: - ... - gunicorn: - gunicorn_extra_args: `--statsd-host 127.0.0.1:9191 --statsd-prefix=galaxy` - ... - -And then run gxadmin like this: - .. code-block:: bash From 9a5406e3faa9a7ff857693af1753b2a3d7fa671e Mon Sep 17 00:00:00 2001 From: mvdbeek Date: Thu, 24 Feb 2022 18:32:51 +0100 Subject: [PATCH 5/6] Mention GTN tutorial --- doc/source/admin/special_topics/performance_tracking.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/source/admin/special_topics/performance_tracking.rst b/doc/source/admin/special_topics/performance_tracking.rst index b1b166225be..ed0fbf9ab08 100644 --- a/doc/source/admin/special_topics/performance_tracking.rst +++ b/doc/source/admin/special_topics/performance_tracking.rst @@ -2,6 +2,7 @@ Galaxy Performance Tracking =========================== Tracking performance over time and identifying slow queries in your Galaxy can be an extremely important thing to do, especially for larger Galaxies. +[A more in-depth tutorial is available at the Galaxy Training Network](https://training.galaxyproject.org/training-material/topics/admin/tutorials/monitoring/tutorial.html). Most performance tracking requires sending metrics to a metrics collection server such as `StatsD `__. This document assumes you have already deployed StatsD. From 6e1dfa3614b24e7ef984fb0ebaee1b689180642c Mon Sep 17 00:00:00 2001 From: mvdbeek Date: Thu, 24 Feb 2022 18:33:08 +0100 Subject: [PATCH 6/6] I think that section was also supposed to be dropped --- doc/source/admin/special_topics/performance_tracking.rst | 6 ------ 1 file changed, 6 deletions(-) diff --git a/doc/source/admin/special_topics/performance_tracking.rst b/doc/source/admin/special_topics/performance_tracking.rst index ed0fbf9ab08..e6a420a7fae 100644 --- a/doc/source/admin/special_topics/performance_tracking.rst +++ b/doc/source/admin/special_topics/performance_tracking.rst @@ -19,12 +19,6 @@ There is some built-in `Gunicorn