James Taylor 399d29cec6 First pass at re-factoring job management: ...

- Creating the command line has been broken up into several methods
  associated with 'Tool'

- Creation of the command line is now done by DefaultToolAction and stored in
  the database (thus if the tool config changes between queueing and running
  of a job things will not break, and if the JobQueue was running on a
  different machine from the web frontend it would not need knowledge of the
  tool config)

- Broke the old JobQueue up quite a bit. New classes are:

  jobs.JobQueue: jobs to be run still get added here. It has a single monitor
  thread that iterates the jobs waiting for one to become runnable, then it
  tries to dispatch the job

  jobs.DefaultJobDispatcher: responsible for deciding how to run a job once
  it is ready to run. Just a trivial implementation for now, as before is PBS
  is enabled and the tool is not a 'data_source' tool, try to run it using
  PBS, otherwise run locally

  jobs.runners.local.LocalJobRunner: the old pool of worker threads that runs
  jobs processes locally

  jobs.runners.local.PBSJobRunner: PBS job runner. Only uses a single thread
  to watch all queued/running jobs. This is TOTALLY UNTESTED. I haven't even
  imported it. I just tried to break up Nate's code to make a state machine
  that could be managed with one thread. Need to know more about the PBS
  setup to test.

Next steps:

- Clean up locally running jobs when the queue is shut down (kill the jobs
  and mark the state change appropriately)

- For all jobs it should be possible to requeue and jobs in the 'waiting'
  state on startup

- For the PBS queue, it should be possible to recover the state of any jobs
  that were queued or running on startup. However we might need more data
  in the database for this

2006-12-14 00:24:38 +00:00

..

cookbook

Cleaning up modules. Still a few stragglers that can't be easily eggified.

2006-11-15 18:27:21 +00:00

galaxy

First pass at re-factoring job management:

2006-12-14 00:24:38 +00:00

fpconst.py

Cleaning up modules. Still a few stragglers that can't be easily eggified.

2006-11-15 18:27:21 +00:00

pkg_resources.py

Cleaning up modules. Still a few stragglers that can't be easily eggified.

2006-11-15 18:27:21 +00:00

psyco_full.py

Cleaning up modules. Still a few stragglers that can't be easily eggified.

2006-11-15 18:27:21 +00:00