- Update Python dependencies that didn't have a cp311 wheel: h5py and
psycopg2-binary.
- For packages for which there is no recent version that works on all
Python versions supported by Galaxy (numpy and scipy), replace old
requirement with multiple Python-version-specific requirements.
Fix `pysam.index()` calls which now use samtools 1.16 where
`samtools index input output` doesn't work any more if `output` is the path
of an already existing file, but `samtools index -o output input` works
fine.
Also:
- Pre-create the (empty) index file for mock test datasets as a unit test
reproducer of the above issue.
- Fix `test_cram` unit test that wasn't checking the return value of
`CRAM.set_index_file()`, which was in fact returning False because
`test-data/2.cram` was not sorted (updated now).
- `PROJECT_OWNER`, `PROJECT_USERAME`, `RAW_CONTENT_URL` and `_version_re` are unused.
- `:changelog:` hyperlink target is unnecessary.
- `test_suite` and `tests_require` are deprecated since setuptools v41.5.0
- Improves options and display around discarded datasets.
- Adds new APIs for importing model stores.
- Adds support to model store layer for importing/exporting workflow invocations and APIs and tests to support this.
- Add new DEFERRED dataset state - with APIs for materializing them or the option to defer them on the worker node if using tool_evaluation_strategy=remote on the job destination.
- UI and API options for uploading deferred datasets individually or into collections.
FilePrefix reads the first Megabytes of a file, which is much more
than the previously used `check_binary` which read only 1KB.
Since we're re-using FilePrefix instance this shouldn't have any impact
on performace. In addition we're now using `python-magic` to detect
the encoding and mime-type. We're using the mime-type detection
for PDF only at this point, but we're also using the detected encoding
if we encounter non-utf8 content.
with:
```
find packages/ -name '*.py' | xargs black
```
The `packages/` directory is currently excluded in the `[tool.black]`
section of `pyproject.toml` because black follows symlinks.
to `pyproject.toml` and packages' `requirements.txt` .
We have been importing it explicitly since commit
fc2ce45767 (December 2020).
Also:
- Remove now unnecessary minimum version for pygithub
- Create a new project galaxy-files that does not depend on anything but galaxy-util.
- Add a dependency in galaxy-data on galaxy-files, use typing in galaxy.data.sniff to make this clear.
- Cleanup the galaxy-files unit tests to not depend on galaxy.datatypes.sniff code - this wasn't ideal to start with.
- Add a small integration-ish unit test test_sniff_file_sources.py to test that piece of sniff.py that depends on galaxy-files.
Nate Coraor