mirror of
https://github.com/galaxyproject/galaxy.git
synced 2026-08-30 16:58:03 +08:00
John Chilton
c78de27659
#19255 reports three backends. Two are boto3 and this branch already fixes them. The third, and the one the reporter called unusable, is generic_s3: 4GB+ downloads succeed but arrive truncated to about 3GB, silently. generic_s3 is GenericS3ObjectStore, which extends the boto2 S3ObjectStore -- a different class from the boto3 one that got `_stream_remote` -- so it inherited the no-op and kept pulling into the cache. It is also the case the tee handles best: `_tee_to_cache` compares streamed bytes against the store's size, turning a silent truncation into a loud failure instead of a corrupt cached object. boto2's Key reads in sized chunks like the boto3 body does. Its close reads the rest of the response first unless told not to, hence fast=True: a client that hung up should not make Galaxy drain 3GB before releasing the connection.