opensource/galaxy
3
0
Fork 0
mirror of https://github.com/galaxyproject/galaxy.git synced 2026-10-24 12:13:33 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #14017 from nuwang/misc_typo_fixes_and_cleanup

Browse Source
This commit is contained in:
Nicola Soranzo
2022-06-07 01:10:11 +02:00
committed by GitHub
parent 27b2ee0e9e 846aec1b13
commit fa7a20ed19
3 changed files with 10 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/galaxy/config/sample/job_conf.xml.sample_advanced
+5 -5
View File
@@ -123,7 +123,7 @@
<!-- <param id="pulsar_config">path/to/pulsar/app.yml</param> -->
</plugin>
<plugin id="k8s" type="runner" load="galaxy.jobs.runners.kubernetes:KubernetesJobRunner">
<!-- The Kubernetes (k8s) plugin allows to send jobs to a k8s cluster which shares filesystem with Galaxy.
<!-- The Kubernetes (k8s) plugin allows Galaxy to send jobs to a k8s cluster which it shares a filesystem with.
The shared file system needs to be exposed to k8s through a Persistent Volume (rw) and a Persistent
Volume Claim. An example of a Persistent Volume could be, in yaml (access modes, reclaim policy and
@@ -148,7 +148,7 @@
The path (nfs:path: in the example) set needs to be a parent directory of the directories used for
variables “file_path” and “new_file_path” on the galaxy.yml files. Clearly, for this particular example
to work, there needs to be a NFS server serving that directory on that ip. Please make sure that you
use reasonable storage size for your set up (possibly larger that the 10Gi written).
use a reasonable storage size for your setup (possibly larger that the 10Gi written).
An example of the volume claim should be (this needs to be followed more closely) (pv_claim.yaml):
kind: PersistentVolumeClaim
@@ -164,7 +164,7 @@
storage: 2Gi
The volume claim needs to reference the name of the volume in spec:volumeName. The name of the claim
(metadat:name) is referenced in the plugin definition (see below), through param
(metadata:name) is referenced in the plugin definition (see below), through param
"k8s_persistent_volume_claim_name". These two k8s object need to be created before galaxy can use them:
kubectl create -f <path/to/persistent_volume.yaml>
@@ -182,7 +182,7 @@
<!-- Sets the pull policy to be used for all containers invoked for jobs by the Kubernetes cluster. Can take
any value among the possible Kubernetes container image pull policies: Always, IfNotPresent or Never
(respecting capitalization). Any other value, such as "Default", will not change the setting on k8s and
leave for the containers to run with the default pull policy specified at the cluster (this is normally
leave for the containers to run with the default pull policy of the cluster (this is normally
IfNotPresent). Container images using latest are by default always pulled, so you need to use defined
versions for offline cases to work (and images need to be previously pulled). -->
@@ -968,7 +968,7 @@
- "seconds_running" (the number of seconds the job was in a running state within Galaxy)
The ``handler`` attribute is optional, if not present, the job's original
handler will be reused for the resubmitted job. The ``destination`` attriubte
handler will be reused for the resubmitted job. The ``destination`` attribute
is optional, if not present the job's original destination will be reused for the
re-submission. The ``delay`` attribute is optional, if present it will cause the job to
delay for that number of seconds before being re-submitted.
lib/galaxy/jobs/runners/state_handlers/resubmit.py
-41
View File
@@ -17,47 +17,6 @@ MESSAGES = dict(
)
def eval_condition(condition, job_state):
runner_state = getattr(job_state, "runner_state", None) or JobState.runner_states.UNKNOWN_ERROR
attempt = 1
now = datetime.utcnow()
last_running_state = None
last_queued_state = None
for state in job_state.job_wrapper.get_job().state_history:
if state.state == model.Job.states.RUNNING:
last_running_state = state
elif state.state == model.Job.states.QUEUED:
last_queued_state = state
elif state.state == model.Job.states.RESUBMITTED:
attempt = attempt + 1
seconds_running = 0
seconds_since_queued = 0
if last_running_state:
seconds_running = (now - last_running_state.create_time).total_seconds()
if last_queued_state:
seconds_since_queued = (now - last_queued_state.create_time).total_seconds()
condition_locals = {
"walltime_reached": runner_state == JobState.runner_states.WALLTIME_REACHED,
"memory_limit_reached": runner_state == JobState.runner_states.MEMORY_LIMIT_REACHED,
"tool_detected_failure": runner_state == JobState.runner_states.TOOL_DETECT_ERROR,
"unknown_error": JobState.runner_states.UNKNOWN_ERROR,
"any_failure": True,
"any_potential_job_failure": True, # Add a hook here - later on allow tools to describe things that are definitely input problems.
"attempt": attempt,
"seconds_running": seconds_running,
"seconds_since_queued": seconds_since_queued,
}
# Small optimization to eliminate the need to parse AST and eval for simple variables.
if condition in condition_locals:
return condition_locals[condition]
else:
return safe_eval(condition, condition_locals)
def failure(app, job_runner, job_state):
# Leave handler quickly if no resubmit conditions specified or if the runner state doesn't allow resubmission.
resubmit_definitions = job_state.job_destination.get("resubmit")
test/unit/app/jobs/job_conf.sample_advanced.yml
+5 -5
View File
@@ -33,7 +33,7 @@ runners:
k8s:
load: galaxy.jobs.runners.kubernetes:KubernetesJobRunner
# The Kubernetes (k8s) plugin allows to send jobs to a k8s cluster which shares filesystem with Galaxy.
# The Kubernetes (k8s) plugin allows Galaxy to send jobs to a k8s cluster which it shares a filesystem with.
# The shared file system needs to be exposed to k8s through a Persistent Volume (rw) and a Persistent
# Volume Claim. An example of a Persistent Volume could be, in yaml (access modes, reclaim policy and
@@ -58,7 +58,7 @@ runners:
# The path (nfs:path: in the example) set needs to be a parent directory of the directories used for
# variables “file_path” and “new_file_path” on the galaxy.yml files. Clearly, for this particular example
# to work, there needs to be a NFS server serving that directory on that ip. Please make sure that you
# use reasonable storage size for your set up (possibly larger that the 10Gi written).
# use a reasonable storage size for your setup (possibly larger that the 10Gi written).
# An example of the volume claim should be (this needs to be followed more closely) (pv_claim.yaml):
@@ -75,7 +75,7 @@ runners:
# storage: 2Gi
# The volume claim needs to reference the name of the volume in spec:volumeName. The name of the claim
# (metadat:name) is referenced in the plugin definition (see below), through param
# (metadata:name) is referenced in the plugin definition (see below), through param
# "k8s_persistent_volume_claim_name". These two k8s object need to be created before galaxy can use them:
# kubectl create -f <path/to/persistent_volume.yaml>
@@ -91,7 +91,7 @@ runners:
# Sets the pull policy to be used for all containers invoked for jobs by the Kubernetes cluster. Can take
# any value among the possible Kubernetes container image pull policies: Always, IfNotPresent or Never
# (respecting capitalization). Any other value, such as "Default", will not change the setting on k8s and
# leave for the containers to run with the default pull policy specified at the cluster (this is normally
# leave for the containers to run with the default pull policy of the cluster (this is normally
# IfNotPresent). Container images using latest are by default always pulled, so you need to use defined
# versions for offline cases to work (and images need to be previously pulled).
# k8s_pull_policy: Default
@@ -923,7 +923,7 @@ execution:
# - "seconds_running" (the number of seconds the job was in a running state within Galaxy)
# The ``handler`` attribute is optional, if not present, the job's original
# handler will be reused for the resubmitted job. The ``environment`` attriubte
# handler will be reused for the resubmitted job. The ``environment`` attribute
# is optional, if not present the job's original environment will be reused for the
# re-submission. The ``delay`` attribute is optional, if present it will cause the job to
# delay for that number of seconds before being re-submitted.