FilePrefix reads the first Megabytes of a file, which is much more
than the previously used `check_binary` which read only 1KB.
Since we're re-using FilePrefix instance this shouldn't have any impact
on performace. In addition we're now using `python-magic` to detect
the encoding and mime-type. We're using the mime-type detection
for PDF only at this point, but we're also using the detected encoding
if we encounter non-utf8 content.
Disallow python-irodsclient 1.1.2 due to
https://github.com/irods/python-irodsclient/issues/356
which breaks several integration tests with:
```
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/irods/message/__init__.py", line 163, in _recv_message_in_len
host, port = sock.getpeername()
ValueError: too many values to unpack (expected 2)
```
Also:
- Remove redundant `galaxy.schema` module from the `app` package (it's
already in `data`)
- Fix requirements of the `app` and `webapps` packages
- Add missing `PyJWT` and `python-dateutil` to `tool.poetry.dependencies`
(which were already included in the pinned requirements as deps of deps)
Also:
- Remove redundant `galaxy.schema` module from the `app` package (it's
already in `data`)
- Fix requirements of the `app` and `webapps` packages
- Add missing `PyJWT` and `python-dateutil` to `tool.poetry.dependencies`
(which were already included in the pinned requirements as deps of deps)
instead of the deprecated starlette WSGIMiddleware that was vendored by
fastAPI. I'm also hoping this fixes the weird exceptions a la:
```
uvicorn.error ERROR 2022-02-14 11:59:42,141 [pN:main,p:4823,tN:Thread-216] Exception in ASGI application
Traceback (most recent call last):
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 364, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
return await self.app(scope, receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/fastapi/applications.py", line 212, in __call__
await super().__call__(scope, receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 159, in __call__
await self.app(scope, receive, _send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/exceptions.py", line 71, in __call__
await self.app(scope, receive, sender)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/routing.py", line 656, in __call__
await route.handle(scope, receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/routing.py", line 408, in handle
await self.app(scope, receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/fastapi/applications.py", line 212, in __call__
await super().__call__(scope, receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 159, in __call__
await self.app(scope, receive, _send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette_context/middleware/raw_middleware.py", line 96, in __call__
await self.app(scope, receive, send_wrapper)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/middleware/base.py", line 65, in __call__
task_group.cancel_scope.cancel()
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 574, in __aexit__
raise exceptions[0]
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 611, in _run_wrapped_task
await coro
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/middleware/base.py", line 34, in coro
await self.app(scope, request.receive, send_stream.send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/exceptions.py", line 71, in __call__
await self.app(scope, receive, sender)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/routing.py", line 656, in __call__
await route.handle(scope, receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/routing.py", line 408, in handle
await self.app(scope, receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/middleware/wsgi.py", line 64, in __call__
await responder(receive, send)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/middleware/wsgi.py", line 91, in __call__
await anyio.to_thread.run_sync(self.wsgi, environ, self.start_response)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 572, in __aexit__
raise ExceptionGroup(exceptions)
anyio._backends._asyncio.ExceptionGroup: 2 exceptions were raised in the task group:
----------------------------
Traceback (most recent call last):
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 611, in _run_wrapped_task
await coro
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/middleware/wsgi.py", line 98, in sender
await send(message)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/exceptions.py", line 68, in sender
await send(message)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/streams/memory.py", line 193, in send
self.send_nowait(item)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/streams/memory.py", line 177, in send_nowait
raise BrokenResourceError
anyio.BrokenResourceError
----------------------------
Traceback (most recent call last):
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/middleware/wsgi.py", line 91, in __call__
await anyio.to_thread.run_sync(self.wsgi, environ, self.start_response)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/to_thread.py", line 29, in run_sync
limiter=limiter)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread
return await future
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 754, in run
result = context.run(func, *args)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/starlette/middleware/wsgi.py", line 132, in wsgi
self.stream_send.send, {"type": "http.response.body", "body": b""}
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/from_thread.py", line 35, in run
return asynclib.run_async_from_thread(func, *args)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 847, in run_async_from_thread
return f.result()
File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/concurrent/futures/_base.py", line 435, in result
return self.__get_result()
File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/streams/memory.py", line 193, in send
self.send_nowait(item)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/anyio/streams/memory.py", line 177, in send_nowait
raise BrokenResourceError
anyio.BrokenResourceError
```
For VCS references, need to specify the branch if it's not `master`.
Also:
- Update Poetry download link
- Require pip >= 19.3 to support direct references as requirement
specifiers
https://www.python.org/dev/peps/pep-0440/#direct-references
