mirror of
https://github.com/galaxyproject/galaxy.git
synced 2026-09-24 16:30:27 +08:00
Added instructions for debuuging Galaxy using a Slurm cluster
This commit is contained in:
@@ -14,19 +14,19 @@ playbooks for that).
|
||||
|
||||
1. Install libslurm-dev
|
||||
* `sudo apt install libslurm-dev`
|
||||
* On my VM, the above command puts `slurm/slurm.h` in `/usr/include` and `libslurm.a` in `/usr/lib/x86_64-linux-gnu`
|
||||
* On my VM, the above command puts slurm/slurm.h in /usr/include and libslurm.a in /usr/lib/x86_64-linux-gnu
|
||||
* You need these header and library files to compile slurm-drmaa in the next step
|
||||
|
||||
2. Install slurm-drmaa
|
||||
* Download slurm-drmaa tar file from `https://github.com/natefoo/slurm-drmaa/releases/download/1.1.2/slurm-drmaa-1.1.2.tar.gz`
|
||||
* Download slurm-drmaa tar file from https://github.com/natefoo/slurm-drmaa/releases/download/1.1.2/slurm-drmaa-1.1.2.tar.gz
|
||||
* Extract the downloaded tar file into home directory: `tar -xzvf slurm-drmaa-1.1.2.tar.gz -C ~`
|
||||
* To compile and install slurm-drmaa
|
||||
* `cd ~/slurm-drmaa-1.1.2`
|
||||
* `./configure --with-slurm-inc=/usr/include --with-slurm-lib=/usr/lib/x86_64-linux-gpu`
|
||||
* `sudo make install`
|
||||
* On my VM, the above command puts `libdrmaa.so` in `/usr/local/lib`. You need the path to this file in the next step
|
||||
* On my VM, the above command puts libdrmaa.so in /usr/local/lib. You need the path to this file in the next step
|
||||
|
||||
3. Replace the contents of `~/galaxy/config/job_conf.xml` with the following code snippet (create the file if it does not already exist)
|
||||
3. Replace the contents of ~/galaxy/config/job_conf.xml with the following code snippet (create the file if it does not already exist)
|
||||
```
|
||||
<?xml version="1.0"?>
|
||||
<job_conf>
|
||||
@@ -46,24 +46,27 @@ playbooks for that).
|
||||
</job_conf>
|
||||
```
|
||||
|
||||
4. Install Slurm (Instructions borrowed from `https://gist.github.com/ckandoth/2acef6310041244a690e4c08d2610423`)
|
||||
4. Uncomment the following line in ~/galaxy/config/galaxy.yml
|
||||
* `job_config_file: job_conf.xml`
|
||||
|
||||
5. Install Slurm (Instructions borrowed from https://gist.github.com/ckandoth/2acef6310041244a690e4c08d2610423)
|
||||
* Install the necessary packages: `sudo apt install -y slurm-wlm slurm-wlm-doc`
|
||||
* Load `/usr/share/doc/slurm-wlm/html/configurator.html` in a browser
|
||||
* Load /usr/share/doc/slurm-wlm/html/configurator.html in a browser
|
||||
* Set your VM's hostname in `SlurmctldHost` and `NodeName`
|
||||
* Set `StateSaveLocation` to `/var/spool/slurmctld`
|
||||
* Set `SlurmdSpoolDir` to `/var/spool/slurmd`
|
||||
* Set `ProctrackType` to `LinuxProc`
|
||||
* Set `SelectType` to `Cons_res`
|
||||
* Set `SelectTypeParameters` to `CR_Core_Memory`
|
||||
* Set `SelectType` to `Cons_res`
|
||||
* Set `SelectTypeParameters` to `CR_Core_Memory`
|
||||
* Set `JobAcctGatherType` to `Linux`
|
||||
* Hit `Submit` and save the resulting text into `/etc/slurm-llnl/slurm.conf`
|
||||
* I.e., the configuration file referred to in `/lib/systemd/system/slurmctld.service` and `/lib/systemd/system/slurmd.service`
|
||||
* Create `/var/spool/slurmd` and `/var/spool/slurmctld` and `/var/log/slurm_jobacct.log` file and set ownership appropriately
|
||||
* Hit Submit and save the resulting text into `/etc/slurm-llnl/slurm.conf`
|
||||
* I.e., the configuration file referred to in /lib/systemd/system/slurmctld.service and /lib/systemd/system/slurmd.service
|
||||
* Create /var/spool/slurmd and /var/spool/slurmctld directories and /var/log/slurm_jobacct.log file and set ownership appropriately
|
||||
* `sudo mkdir -p /var/spool/slurmd`
|
||||
* `sudo mkdir -p /var/spool/slurmctld`
|
||||
* `sudo touch /var/log/slurm_jobacct.log`
|
||||
* `sudo chown slurm:slurm /var/spool/slurmd /var/spool/slurmctld /var/log/slurm_jobacct.log`
|
||||
* Install mailutils so that Slurm won't complain about `/bin/mail` missing
|
||||
* Install mailutils so that Slurm won't complain about /bin/mail missing
|
||||
* `sudo apt install -y mailutils`
|
||||
* Make sure munge is installed/running, and a munge.key was created with user-only read-only permissions, owned by munge:munge
|
||||
* `sudo apt install munge`
|
||||
@@ -73,6 +76,6 @@ playbooks for that).
|
||||
* `sudo service slurmd start`
|
||||
* `sudo service slurmctld start`
|
||||
|
||||
5. Follow the instructions [here](./debuging_galaxy.md) to setup VSCode for debugging.
|
||||
6. Follow the instructions [here](./debuging_galaxy.md) to setup VSCode for debugging.
|
||||
|
||||
Enjoy debugging session your Galaxy instance backed by a Slurm cluster!
|
||||
|
||||
@@ -22,5 +22,6 @@ A multi-hour long video playlist covering these slides can be found at
|
||||
writing_tests
|
||||
debugging_tests
|
||||
debugging_galaxy
|
||||
debugging_galaxy_slurm
|
||||
starting_galaxy
|
||||
file_upload
|
||||
|
||||
Reference in New Issue
Block a user