opensource/galaxy
3
0
Fork 0
mirror of https://github.com/galaxyproject/galaxy.git synced 2026-09-24 16:30:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

This strips the graphite-specific middleware; statistic functionality is provided by the statsd middleware and we don't need two separate mechanisms to maintain.

Browse Source
This commit is contained in:
Dannon Baker
2018-04-30 13:50:05 -04:00
parent 8c152bda45
commit 02ffdf0be9
10 changed files with 1 additions and 196 deletions
Download Patch File Download Diff File Expand all files Collapse all files
config/galaxy.yml.sample
+1 -26
View File
@@ -52,8 +52,7 @@ uwsgi:
# --daemon and/or production deployments.
master: false
# Path to the application's Python virtual environment. If using Conda for
# Galaxy's framework dependencies (not tools!), do not set this.
# Path to the application's Python virtual environment.
virtualenv: .venv
# Path to the application's Python library.
@@ -1053,30 +1052,6 @@ galaxy:
# prefix with a tag path set to the page url
#statsd_influxdb: false
# Log to graphite Graphite is an external statistics aggregator
# (https://github.com/graphite-project/carbon) Enabling the following
# options will cause galaxy to log request timing and other statistics
# to the configured graphite instance. The graphite_prefix is useful
# if you are running multiple Galaxy instances and want to segment
# statistics between them within the same aggregator.
#graphite_host: null
# Log to graphite Graphite is an external statistics aggregator
# (https://github.com/graphite-project/carbon) Enabling the following
# options will cause galaxy to log request timing and other statistics
# to the configured graphite instance. The graphite_prefix is useful
# if you are running multiple Galaxy instances and want to segment
# statistics between them within the same aggregator.
#graphite_port: 2003
# Log to graphite Graphite is an external statistics aggregator
# (https://github.com/graphite-project/carbon) Enabling the following
# options will cause galaxy to log request timing and other statistics
# to the configured graphite instance. The graphite_prefix is useful
# if you are running multiple Galaxy instances and want to segment
# statistics between them within the same aggregator.
#graphite_prefix: galaxy
# Add an option to the library upload form which allows administrators
# to upload a directory of files.
#library_import_dir: null
doc/source/admin/galaxy_options.rst
-48
View File
@@ -2170,54 +2170,6 @@
:Type: bool
~~~~~~~~~~~~~~~~~
``graphite_host``
~~~~~~~~~~~~~~~~~
:Description:
Log to graphite Graphite is an external statistics aggregator
(https://github.com/graphite-project/carbon) Enabling the
following options will cause galaxy to log request timing and
other statistics to the configured graphite instance. The
graphite_prefix is useful if you are running multiple Galaxy
instances and want to segment statistics between them within the
same aggregator.
:Default: ``None``
:Type: str
~~~~~~~~~~~~~~~~~
``graphite_port``
~~~~~~~~~~~~~~~~~
:Description:
Log to graphite Graphite is an external statistics aggregator
(https://github.com/graphite-project/carbon) Enabling the
following options will cause galaxy to log request timing and
other statistics to the configured graphite instance. The
graphite_prefix is useful if you are running multiple Galaxy
instances and want to segment statistics between them within the
same aggregator.
:Default: ``2003``
:Type: int
~~~~~~~~~~~~~~~~~~~
``graphite_prefix``
~~~~~~~~~~~~~~~~~~~
:Description:
Log to graphite Graphite is an external statistics aggregator
(https://github.com/graphite-project/carbon) Enabling the
following options will cause galaxy to log request timing and
other statistics to the configured graphite instance. The
graphite_prefix is useful if you are running multiple Galaxy
instances and want to segment statistics between them within the
same aggregator.
:Default: ``galaxy``
:Type: str
~~~~~~~~~~~~~~~~~~~~~~
``library_import_dir``
~~~~~~~~~~~~~~~~~~~~~~
doc/source/admin/special_topics/performance_tracking.rst
-10
View File
@@ -41,16 +41,6 @@ The statsD configuration requires setting the following options in the ``galaxy`
statsd_port: 8125
statsd_prefix: galaxy
And the graphite configuration is very similar:
.. code-block:: yaml
galaxy:
# ...
graphite_host: 127.0.0.1
graphite_port: 2003
graphite_prefix: galaxy
Most people visualize the statistics using something like `Grafana <https://grafana.com/>`__:
.. image:: grafana.png
doc/source/lib/galaxy.web.framework.middleware.rst
-8
View File
@@ -25,14 +25,6 @@ galaxy\.web\.framework\.middleware\.error module
:undoc-members:
:show-inheritance:
galaxy\.web\.framework\.middleware\.graphite module
---------------------------------------------------
.. automodule:: galaxy.web.framework.middleware.graphite
:members:
:undoc-members:
:show-inheritance:
galaxy\.web\.framework\.middleware\.profile module
--------------------------------------------------
lib/galaxy/config.py
-4
View File
@@ -643,10 +643,6 @@ class Configuration(object):
self.statsd_port = int(kwargs.get('statsd_port', 8125))
self.statsd_prefix = kwargs.get('statsd_prefix', 'galaxy')
self.statsd_influxdb = string_as_bool(kwargs.get('statsd_influxdb', False))
# Statistics and profiling with graphite
self.graphite_host = kwargs.get('graphite_host', '')
self.graphite_port = int(kwargs.get('graphite_port', 2003))
self.graphite_prefix = kwargs.get('graphite_prefix', 'galaxy')
# Logging with fluentd
self.fluent_log = string_as_bool(kwargs.get('fluent_log', False))
self.fluent_host = kwargs.get('fluent_host', 'localhost')
lib/galaxy/dependencies/__init__.py
-3
View File
@@ -103,9 +103,6 @@ class ConditionalDependencies(object):
def check_statsd(self):
return self.config.get("statsd_host", None) is not None
def check_graphite(self):
return self.config.get("graphite_host", None) is not None
def check_weberror(self):
return (asbool(self.config["debug"]) and
asbool(self.config["use_interactive"]))
lib/galaxy/dependencies/conditional-requirements.txt
-1
View File
@@ -9,7 +9,6 @@ raven
pbs_python
drmaa
statsd
graphitesend
docker
azure-storage==0.32.0
# PyRods not in PyPI
lib/galaxy/web/framework/middleware/graphite.py
-52
View File
@@ -1,52 +0,0 @@
"""
Middleware for sending request statistics to graphite
"""
from __future__ import absolute_import
import logging
import time
log = logging.getLogger(__name__)
try:
import graphitesend
except ImportError:
# This middleware will never be used without graphite. This block allows
# unit tests pass on systems without it.
graphitesend = None
class GraphiteMiddleware(object):
"""
This middleware will log request durations to the configured graphite
instance.
"""
def __init__(self,
application,
graphite_host,
graphite_port,
graphite_prefix):
if not graphitesend:
raise ImportError("graphite middleware configured, but no graphite python module found. "
"Please install the python graphitesend module to use this functionality.")
self.application = application
try:
self.graphite_client = graphitesend.init(graphite_server=graphite_host, graphite_port=int(graphite_port), prefix=graphite_prefix.rstrip('.'))
except graphitesend.graphitesend.GraphiteSendException:
self.graphite_client = None
log.exception("Could not instantiate graphite metrics logger. It will be disabled until Galaxy restart")
def __call__(self, environ, start_response):
start_time = time.time()
req = self.application(environ, start_response)
# If graphite is disabled, exit early.
if not self.graphite_client:
return req
dt = int((time.time() - start_time) * 1000)
try:
self.graphite_client.send(environ.get('controller_action_key', None) or environ.get('PATH_INFO', "NOPATH").strip('/').replace('/', '.'), dt)
except graphitesend.GraphiteSendException:
log.exception("Graphite Error")
return req
lib/galaxy/webapps/galaxy/buildapp.py
-9
View File
@@ -979,15 +979,6 @@ def wrap_in_middleware(app, global_conf, application_stack, **local_conf):
conf.get('statsd_prefix', 'galaxy'),
conf.get('statsd_influxdb', False)))
log.debug("Enabling 'statsd' middleware")
# graphite request timing and profiling
graphite_host = conf.get('graphite_host', None)
if graphite_host:
from galaxy.web.framework.middleware.graphite import GraphiteMiddleware
app = wrap_if_allowed(app, stack, GraphiteMiddleware,
args=(graphite_host,
conf.get('graphite_port', 2003),
conf.get('graphite_prefix', 'galaxy')))
log.debug("Enabling 'graphite' middleware")
# If we're using remote_user authentication, add middleware that
# protects Galaxy from improperly configured authentication in the
# upstream server
lib/galaxy/webapps/galaxy/config_schema.yml
-35
View File
@@ -1620,41 +1620,6 @@ mapping:
Instead of sending prefix + dot-separated-path, Galaxy will send
prefix with a tag path set to the page url
graphite_host:
type: str
required: false
desc: |
Log to graphite
Graphite is an external statistics aggregator (https://github.com/graphite-project/carbon)
Enabling the following options will cause galaxy to log request timing and
other statistics to the configured graphite instance. The graphite_prefix is
useful if you are running multiple Galaxy instances and want to segment
statistics between them within the same aggregator.
graphite_port:
type: int
default: 2003
required: false
desc: |
Log to graphite
Graphite is an external statistics aggregator (https://github.com/graphite-project/carbon)
Enabling the following options will cause galaxy to log request timing and
other statistics to the configured graphite instance. The graphite_prefix is
useful if you are running multiple Galaxy instances and want to segment
statistics between them within the same aggregator.
graphite_prefix:
type: str
default: galaxy
required: false
desc: |
Log to graphite
Graphite is an external statistics aggregator (https://github.com/graphite-project/carbon)
Enabling the following options will cause galaxy to log request timing and
other statistics to the configured graphite instance. The graphite_prefix is
useful if you are running multiple Galaxy instances and want to segment
statistics between them within the same aggregator.
library_import_dir:
type: str
default: null