Commit Graph
14 Commits
Author SHA1 Message Date
Nicola Soranzo 9d74bba7fb Drop support for retired Python 3.5
Upgrade syntax using `pyupgrade --py36-plus` .

Manually drop several `six` imports.

Also:
- Remove broken pr_cache in scripts/bootstrap_history.py
- Fix broken prefix removal in lib/galaxy/tool_util/deps/mulled/mulled_build.py
2020-10-07 11:52:13 +01:00
mvdbeek 3cc5a28006 Fallback for python2 not having math.inf 2019-10-26 18:42:45 +02:00
mvdbeek 9b18f5904d Add unit test that asserts past tasks won't be executed 2019-10-26 18:42:45 +02:00
mvdbeek 30027a094b Rename instances of GalaxyQueueWorker from control_worker to queue_worker
It just seems more logical, and I repeatedly was using the queue_worker
instead of control_worker and wondering why things didn't work.
2019-08-18 16:01:32 +02:00
John Chilton 584c117df3 Refactor to cleanup dependencies around galaxy.web.stack.
Move galaxy.web.stack.database_heartbeat into galaxy.model - since it was the only thing in galaxy.web.stack that used galaxy.model at all and it didn't import anything else from galaxy.web.stack.

Once that was done, I refactored this whole package (galaxy.web.stack) into galaxy.web_stack. It doesn't use the framework setup or exported by the base galaxy.web package so it wouldn't seem to belong there. It also has very different dependencies than the rest of that. And finally and most importantly, this elimiantes almost all dependency from galaxy.jobs and galaxy.tools and galaxy.workflows to anything in galaxy.web.
2019-06-26 18:54:52 -04:00
mvdbeek 34eb9468d4 Use ApplicationStack for accessing server name 2019-03-29 21:46:18 +01:00
mvdbeek f4d5ae61f6 Make tests use finalizer instead of yield
and verify tasks get executed the expected number of times.
Since the part after the yield statement runs to early it
was shutting down the app(s) prematurely.
2019-03-29 21:46:18 +01:00
mvdbeek e778c50453 Also test against rabbitmq 2019-03-29 21:46:18 +01:00
mvdbeek 006f42ddd6 Delete remaining messages on startup
So far I don't think we have any messages that should survive
restarts. If we do at some point we should route them differently
(or change the existing worker to use ephemeral messages).
2019-03-29 21:46:18 +01:00
mvdbeek 038db80258 Add test using multiple independent apps
Unfortunately adding a third app doesn't seem to work.
In that case only the last 2 apps receive the control task.

I do know that this isn't a problem with real separate processes,
but I don't know what is causing this.
2019-03-29 21:46:18 +01:00
mvdbeek 75279121a4 Add option to return worker results
Occasionally we want to wait for a toolbox or data table
reload before continuing, so knowing when that's done
(=wait for the result) is helpful.

Getting the result of a task seems like a good idea,
we could use this for a whole lot of things.
One thing is targeting functions to (remote) handlers
that have access to information that web handlers don't,
like how busy a cluster is or which dependencies are
available or whether a handler is dead.
2019-03-29 21:46:18 +01:00
mvdbeek c8392299c8 Make processes and mules report heartbeat in database
It's important to not set the DatabaseHeartbeat `server_name` prior to forking,
which we avoid be getting `server_name` from
`application_stack.app.config.server_name`.
`WorkerProcess.server_name` for uwsgi web processes will look like:
`main.web.1` and mules like `main.job-handlers.1` while webless workers
remain named `handler1`.
2019-03-29 21:46:18 +01:00
mvdbeek 631a9fc359 Also test with postgresql fixture 2019-03-29 21:46:18 +01:00
mvdbeek 950bb4666f Add simple test for queue worker 2019-03-29 21:46:18 +01:00