Allows asynchornous submission of LWR jobs to remote servers without a remote LWR service running. LWR still needs to be configured there and job updates are should be configured to be sent back to Galaxy through AMQP.
A long running process will be executed on the remote server to monitor the job - while this will always be needed in some cases - for others it would be great to be able to embedded the update steps into the job to be submitted to the DRM itself so that this long running process can be just a submission.
This is one more potential step down the road toward allowing remote execution of Galaxy jobs with ever fewer remote constraints. Extensions to this would include allowing Galaxy job updates via the jobs API (along with staging) so MQ becomes optional, allowing auto-configuration of remote LWR application if unavailable or unconfigured, more transfer options (scp), more CLI shell options (fabric, ansible, etc...).
This is based on work by Hector del Risco at the University of Florida.
Various tool-style parameters can be defined in job_resource_params_conf.xml (see sample) and these define the pool of all possible resource parameters that can be injected into a tool. These can be groupped together and assigned to tools in job_conf.xml (see documented changes to job_conf.xml.sample_advanced).
A conditional is then injected into each tool that gives the end-user the ability to select these parameters. The values of these parameters (as a dict) can then be obtained by dynamic job destination that define a 'resource_params' argument.
Lots here still needs to be tested/verified - can workflows exported from Galaxies without resource parameters (or different ones, be reused in Galaxies with resource parameters) (I think yes?)? can workflows exported with resource parameters be used in other Galaxies (I think yes?), how best to present UI while running workflows? (maybe nothing needs to be changed), should exported job parameter data be swept clean (say when exporting a workflow for instance).
A great extension to this would be to support conditional when defining parameters.
Finally, the Galaxy team will make a reasonable effort to support this syntax for job_conf.xml, job_resource_params_conf.xml, and internal interface for job parameters for sometime once this changeset has reached -stable but until these interfaces may change.
and add a destination_total_concurrent_jobs limit that prevents Galaxy from
queueing *any* jobs at a destination or tagged group of destinations once the
limit is reached.
In particular ampq_publish_retry can now be set to True to retry disconnected connections on publication. Hopefully, this will fix producer-related disconnect problems seen in production.
Add new module - galaxy.util.specs - with utilities for things I predict would be common spec types.
This simplifies some things in terms of defaults and empty values that means other simplifications outside the LWR job runner constructor as well (e.g. for ssl_ parameters).
... or they will not work correctly with the LWR. This is making explicit a convention that was already followed by all devteam tools. If need arises for multiple paths in the same data table or there is an important tool out there that does not label it's paths as "path" let me know and we can try to implement some XML markup in the tool_data_table.xml file to declare these fields as paths.
Use new assumption to cleanup what LWR considers unstructured paths (unstructured paths is no longer even a good name for this concept unfortunately).
Dave Bouvier