Commit Graph
39 Commits
Author SHA1 Message Date
Nate Coraor f474eb2584 Dependency pins after excluding broken docutils versions. 2021-04-28 10:16:42 -04:00
John Chilton cc41f30af6 Better extended metadata configuration for Pulsar (don't copy outputs) 2021-04-15 15:01:28 -04:00
John Chilton d9ca2ed983 Transfer fewer files when using extended + remote metadata. 2021-04-14 13:21:15 -04:00
mvdbeek 66f8ad90a0 Add pyproject.toml pin 2021-04-08 09:52:36 +02:00
qiagu 56f715c59d Merge branch 'dev' into ometiff2_new 2021-04-07 10:45:40 -07:00
John Chilton 071456f891 Merge pull request #11220 from gregvonkuster/mrcfile
Add mrc datatype (and Python dependency)
2021-04-07 09:45:32 -04:00
mvdbeek 8db40adf8a Add celery 2021-03-29 09:29:24 -04:00
Dannon Baker 323e21f5a8 Initial circus addition -- dep update needs to be run still 2021-03-29 09:29:24 -04:00
Sergey Golitsynskiy bb9547a1a3 Merge pull request #11696 from ic4f/dev_no_sqlalchemy_utils
Replace sqlalchemy-utilities with local implementation
2021-03-24 10:36:46 -04:00
Sergey Golitsynskiy 951b9115ce Try lagom pin<1.2.1 2021-03-22 18:45:53 -04:00
Sergey Golitsynskiy 3ac3f6b04c Pin lagom to !=1.2.1 2021-03-22 17:17:24 -04:00
Sergey Golitsynskiy e7a6699e0b Pin pygithub to prevent downgrading
Without the pin, poetry downgraded pygtihub from 1.54.1 to 1.53.
Cause of issue: pygithub (1.54.1) depends on pyjwt (<2.0)
Poetry upgrades pyjwt from 1.7.1 to 2.0.1. This causes subsequent downgrade of pygithub to 1.54, but 1.54 is incompatible with requests (>=2.14.0,<2.25). So Poetry downgrades it to 1.53, which does not have the <2.25 constraint.
2021-03-22 15:11:46 -04:00
Sergey Golitsynskiy cccf0a8ed3 Remove sqlalchemy-utils from pyproject and pinned files 2021-03-21 21:52:54 -04:00
qiagu b8db43011f Merge branch 'dev' into ometiff2 2021-03-17 10:55:43 -07:00
Sergey Golitsynskiy c3744f0fee Set minimum version of fastapi 2021-03-17 00:07:37 -04:00
Sergey GolitsynskiyandNicola Soranzo 667085525c Specify min python version for starlette-context
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2021-03-17 00:07:37 -04:00
Sergey Golitsynskiy 776f09990d Pin min version of starlette-context
Prevent automatic downgrading by Poetry's dependency resolver
(On 3/15/21 suggested downgrade to 0.1.3)
2021-03-17 00:07:37 -04:00
qiagu e06eb44a7f Merge branch 'dev' into ometiff2 2021-03-16 14:28:18 -07:00
Sergey Golitsynskiy 2f0a024915 Pin sqlalchemy in package requirements files 2021-03-16 13:55:21 -04:00
Sergey GolitsynskiyandNicola Soranzo 0babeebe33 Add comment to dependency pin
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2021-03-16 13:55:21 -04:00
Sergey Golitsynskiy 8842e289ad Pin SQAlchemy to ~1.3.22
1.4 released; our codebase not compatible yet.
2021-03-16 13:55:21 -04:00
Nicola Soranzo c4c978014d Pin refgenconf to <0.10.0
Due to breaking changes, see
https://github.com/galaxyproject/galaxy/issues/11601
2021-03-15 20:00:31 +00:00
davelopez 2d5bee6dbc Add fluent-logger==0.9.6 (latest) as dev dependency
For compatibility msgpack has to be pinned down from 1.0.2 to 0.6.2
2021-03-10 12:51:26 +01:00
qiagu 7fde8fe7ea Revise test for ome.tiff datatype 2021-03-03 11:30:58 -08:00
John Chilton 11c1715fc3 Holistic approach to dependency injection.
Dependency injection is cool - it should however not be confined to the top-layer of the application. Inversion of control can help every layer and is a great way to de-couple various parts of the app.

This code brings in a framework (Lagom) to build a dependency injection framework and then adds a small function (``depends``) that adapts it to FastAPI's dependency injection mechanism ("Depends").

The advantages to this approach are numerous.

We don't need to write these little adapters for each component of the backend to adapt it FastAPI. For example look at the change to the roles API controller:

```diff
-def get_role_manager(app: UniverseApplication = Depends(get_app)) -> RoleManager:
-    return app.role_manager
-
-
 @cbv(router)
 class FastAPIRoles:
-    role_manager: RoleManager = Depends(get_role_manager)
+    role_manager: RoleManager = depends(RoleManager)
```

This is much less boilerplate. We don't need to implement & type that function get_role_manager and we don't need to bring in the import on UniverseApplication.

Additionally, we've got a clean abstraction that shields us from ``fastapi`` imports in all of our controllers. It should make it more possible to switch to new frameworks and such as the Python ecosystem matures.

Also, the same DI that is used to inject ``RoleManager`` into this contoller is used to inject app into RoleManager when it is constructed during application initialization. Any component being managed by UniverseApplication can now rely on its constructor arguments to be injected if it wants. It is easy to see the cool examples on FastAPI and think it is just a technology for controllers, but it totally is not.

I don't think there is really a way to use FastAPI's dependency injection outside the context of that framework, but even if one could Lagom is superior. All the auto-wiring is by type and requires zero framework and zero configuration (https://github.com/meadsteve/lagom#auto-wiring-with-zero-configuraton).

Having a web framework provide these framework-bound extension points for injecting stuff into controllers was the state of the art of Java like 15 years ago. Skipping that whole learning process and using the type system and auto-wiring that isn't dependent on framework annotation really jumps out to Java circa 8 years ago!

Why Lagom is an interesting question. When researching DI frameworks, I couldn't find a clear winner but Lagom focus on type annotations versus annotating by name makes it clearly more modern than a lot frameworks by much bigger names (https://github.com/meadsteve/lagom/blob/master/docs/comparison.md). The other type-centric framework that had even a nice interface that I found was punq (https://punq.readthedocs.io/en/latest/). The development just doesn't seem as active on punq. While I didn't land up using the integration Lagom seems to have async frameworks in mind (https://github.com/meadsteve/lagom/blob/master/lagom/integrations/fast_api.py), so that is another plus. Ultimately though I think I can swap between these two with like 10 lines of code switch, they do cool things with very simple interfaces and neither requires a bunch of investment in annotation on your components.
2021-02-23 10:28:46 -05:00
mvdbeek 2d6f479132 Move request session scoping to path dependency
This avoids holding on to sessions while a response is being streamed.
Also adds unit tests that verify sessions are really request or thread
local (in case of background threads / non request actions (e.g job /
workflow handlers)).
2021-02-21 19:37:04 +01:00
mvdbeek 2545965425 Replace starlette-context with BaseHTTPMiddleware
this is still not ideal, as the entire response will be loaded into
memory. This is a problem with streaming responses.
https://github.com/encode/starlette/issues/1012#issuecomment-673461832
is actually enlightening here:

> "This means this class will either load the entirety of streaming requests into memory (this issue) and run the background before returning the response (#919 ), or if we fix those problems, that it will then encourage users to leave resources in a pending or open state, an arguably worse result. In short, it's problematic."

Which is exactly what we'd be doing with a middleware, keeping resources
open for longer than necessary, which we need to avoid if we ever want
to run background / async tasks.  I think the solution here what we
already had, path operation dependencies.
2021-02-20 13:04:59 +01:00
mvdbeek 8a9b213822 Let ModelMapping manage session scope 2021-02-20 09:15:37 +01:00
mvdbeek 2de1676d0a Use request id middleware to scope session
We still need to remove the session at the end of the request (https://docs.sqlalchemy.org/en/13/orm/contextual.html#using-thread-local-scope-with-web-applications), so maybe
a middleware that does this is the right thing.
2021-02-18 16:23:17 +01:00
John Chilton f1ec1b6459 Merge pull request #11342 from mvdbeek/resolve_session_problems
Resolve session problems, enabled pages and tags fastAPI routes
2021-02-17 10:15:43 -05:00
Nate Coraor 7a15718861 Merge remote-tracking branch 'upstream/release_21.01' into dev 2021-02-16 21:59:52 -05:00
John Chilton e3dedd86bc Performance metrics for tests. 2021-02-12 20:44:05 -05:00
mvdbeek 61a2365e56 Add fastAPI dependencies for yield in dependencies on python 3.6 2021-02-11 17:23:17 +01:00
Greg Von Kuster afc4604712 Unpin mrcfile in pyproject.toml 2021-02-09 11:51:17 -05:00
mvdbeek f751811761 Merge branch 'release_21.01' into dev 2021-02-09 16:21:04 +01:00
Greg Von Kuster de897e5fb3 Add the mrcfile entry to pyproject.toml 2021-02-08 15:58:41 -05:00
mvdbeek dd7aa6ec38 Merge branch 'release_21.01' into dev 2021-02-08 16:01:53 +01:00
Dannon Baker 6d5e4412eb Pin pulsar-galaxy-lib to 0.14.1 with merge forward 2021-02-04 12:13:32 -05:00
Nicola Soranzo 70def633f1 Replace pipenv with poetry. Update all dependencies 2021-01-21 20:30:00 +00:00