mirror of
https://github.com/galaxyproject/galaxy.git
synced 2026-09-19 02:21:32 +08:00
John Chilton
7882a067c8
Setup a hierarchical objectstore with ids - these ids can be zero weight for objectstores that should only be used on a per-job-destination basis. Job destinations can then set object_store_id as a param to force a particular object store from the hierarchical configuration to be used. This can be set in job_conf.xml directly for static mappings or combined with dynamic job destinations for dynamic mapping to object stores. Dynamic job destinations can dispatch on cluster conditions, user, user preferences, etc.. Additionally, job resource parameters can be used with dynamic job destinations to allow user input on the object store to use. An integration test case is included that tests and demonstrates all these different pieces. It sets up a Galaxy with a job and object store configuration that routes one tool to a specific disk store (called "static"), and routes another tool dynamically using job resource parameters that allow the user to pick between "slow, cheap" and "fast, expensive" storage options (for the test it is all just disk stores used - but you can imagine choosing between EBS and S3 or something this way). I'd be happy to see something more formal than job resource parameters or user preferences developed to help users route job outputs to particular object stores - this is just a demonstration of what can be done today after these changes using extension points already being used by admins and users.