Commit Graph
24 Commits
Author SHA1 Message Date
Nicola Soranzo 452aba9ee3 Fix remaining issues reported by bugbear; add it to flake8 reqs 2021-01-16 05:58:15 +00:00
mvdbeek ccdb6c3f10 Replace usage of mkstemp with NamedTemporaryFile 2020-12-07 22:31:53 +01:00
Nicola Soranzo 9d74bba7fb Drop support for retired Python 3.5
Upgrade syntax using `pyupgrade --py36-plus` .

Manually drop several `six` imports.

Also:
- Remove broken pr_cache in scripts/bootstrap_history.py
- Fix broken prefix removal in lib/galaxy/tool_util/deps/mulled/mulled_build.py
2020-10-07 11:52:13 +01:00
mvdbeek 3cc5a28006 Fallback for python2 not having math.inf 2019-10-26 18:42:45 +02:00
mvdbeek 9b18f5904d Add unit test that asserts past tasks won't be executed 2019-10-26 18:42:45 +02:00
mvdbeek fa9d1ef29b Fix database_heartbeat unit test 2019-08-18 16:01:32 +02:00
mvdbeek 30027a094b Rename instances of GalaxyQueueWorker from control_worker to queue_worker
It just seems more logical, and I repeatedly was using the queue_worker
instead of control_worker and wondering why things didn't work.
2019-08-18 16:01:32 +02:00
mvdbeek 6b52f62fe4 Adapt unit test to new DatabaseHeartbeat import 2019-08-18 16:01:32 +02:00
mvdbeek 79df4cb4f5 Fix import order 2019-08-18 16:01:32 +02:00
mvdbeek 2598fd3115 Fix test by monkey patching send_control_task 2019-08-18 16:01:32 +02:00
Nate Coraor a0b1d0066f Unit test fixes 2019-07-07 16:29:58 +02:00
John Chilton 584c117df3 Refactor to cleanup dependencies around galaxy.web.stack.
Move galaxy.web.stack.database_heartbeat into galaxy.model - since it was the only thing in galaxy.web.stack that used galaxy.model at all and it didn't import anything else from galaxy.web.stack.

Once that was done, I refactored this whole package (galaxy.web.stack) into galaxy.web_stack. It doesn't use the framework setup or exported by the base galaxy.web package so it wouldn't seem to belong there. It also has very different dependencies than the rest of that. And finally and most importantly, this elimiantes almost all dependency from galaxy.jobs and galaxy.tools and galaxy.workflows to anything in galaxy.web.
2019-06-26 18:54:52 -04:00
John Chilton 55f58cb627 Make heartbeat unit tests a bit more robust. 2019-04-04 15:08:52 -04:00
mvdbeek fc8916525e Allow a little more time for thread to start up in heartbeat unit test 2019-03-30 10:08:04 +01:00
mvdbeek 34eb9468d4 Use ApplicationStack for accessing server name 2019-03-29 21:46:18 +01:00
mvdbeek 22b21fde62 Only run rabbitmq queue worker test if GALAXY_TEST_AMQP_INTERNAL_CONNECTION is set 2019-03-29 21:46:18 +01:00
mvdbeek f4d5ae61f6 Make tests use finalizer instead of yield
and verify tasks get executed the expected number of times.
Since the part after the yield statement runs to early it
was shutting down the app(s) prematurely.
2019-03-29 21:46:18 +01:00
mvdbeek e778c50453 Also test against rabbitmq 2019-03-29 21:46:18 +01:00
mvdbeek 006f42ddd6 Delete remaining messages on startup
So far I don't think we have any messages that should survive
restarts. If we do at some point we should route them differently
(or change the existing worker to use ephemeral messages).
2019-03-29 21:46:18 +01:00
mvdbeek 038db80258 Add test using multiple independent apps
Unfortunately adding a third app doesn't seem to work.
In that case only the last 2 apps receive the control task.

I do know that this isn't a problem with real separate processes,
but I don't know what is causing this.
2019-03-29 21:46:18 +01:00
mvdbeek 75279121a4 Add option to return worker results
Occasionally we want to wait for a toolbox or data table
reload before continuing, so knowing when that's done
(=wait for the result) is helpful.

Getting the result of a task seems like a good idea,
we could use this for a whole lot of things.
One thing is targeting functions to (remote) handlers
that have access to information that web handlers don't,
like how busy a cluster is or which dependencies are
available or whether a handler is dead.
2019-03-29 21:46:18 +01:00
mvdbeek c8392299c8 Make processes and mules report heartbeat in database
It's important to not set the DatabaseHeartbeat `server_name` prior to forking,
which we avoid be getting `server_name` from
`application_stack.app.config.server_name`.
`WorkerProcess.server_name` for uwsgi web processes will look like:
`main.web.1` and mules like `main.job-handlers.1` while webless workers
remain named `handler1`.
2019-03-29 21:46:18 +01:00
mvdbeek 631a9fc359 Also test with postgresql fixture 2019-03-29 21:46:18 +01:00
mvdbeek 950bb4666f Add simple test for queue worker 2019-03-29 21:46:18 +01:00