instalación de slurm en servidor multi-core Debian

Hola,

hace un tiempo explicaba aquí cómo montar grid-engine en Debian para gestionar trabajos paralelos en un servidor multicore. Como he tenido algunos problemas he probado slurm, que está activamente desarrollado y documentado. 

 

En concreto lo he hecho sobre un servidor Debian 11. Si vienes de otros sistemas, como LSF, tendrás que convertir tus comandos con tablas como ésta. Los pasos que he seguido los he adaptado de aquí:

sudo apt install slurmd slurmctld -y
sudo chmod 777 /etc/slurm

# creo fichero de configuración /etc/slurm/slurm.conf,
# tendrás que adaptar CPUs= y RealMemory= al final ,
# en mi caso he definido "16 nodos"

sudo cat << EOF > /etc/slurm/slurm.conf
# See the slurm.conf man page for more information.
#
ClusterName=localcluster
SlurmctldHost=localhost
MpiDefault=none
ProctrackType=proctrack/linuxproc
ReturnToService=2
SlurmctldPidFile=/var/run/slurmctld.pid
SlurmctldPort=6817
SlurmdPidFile=/var/run/slurmd.pid
SlurmdPort=6818
SlurmdSpoolDir=/var/lib/slurm/slurmd
SlurmUser=slurm
StateSaveLocation=/var/lib/slurm/slurmctld
SwitchType=switch/none
TaskPlugin=task/none
#
# TIMERS
InactiveLimit=0
KillWait=30
MinJobAge=300
SlurmctldTimeout=120
SlurmdTimeout=300
Waittime=0
# SCHEDULING
SchedulerType=sched/backfill
SelectType=select/cons_tres
SelectTypeParameters=CR_Core
#
#AccountingStoragePort=
AccountingStorageType=accounting_storage/none
JobCompType=jobcomp/none
JobAcctGatherFrequency=30
JobAcctGatherType=jobacct_gather/none
SlurmctldDebug=info
SlurmctldLogFile=/var/log/slurm/slurmctld.log
SlurmdDebug=info
SlurmdLogFile=/var/log/slurm/slurmd.log
#
# COMPUTE NODES
NodeName=localhost CPUs=16 RealMemory=200 State=UNKNOWN
PartitionName=LocalQ Nodes=ALL Default=YES MaxTime=INFINITE State=UP
EOF

# iniciamos el gestor
sudo chmod 755 /etc/slurm
sudo systemctl start slurmctld
sudo systemctl start slurmd

# probamos el cluster
sinfo
sbatch --wrap=date
cat slurm-1.out 

PD Para ver las propiedades de tu nuevo "cluster" puedes probar:

scontrol show node

Hasta pronto, 

Bruno

instalación de Grid Engine en servidor multi-core Debian

Hola,

esta es mi primera entrada tras volver a la EEAD-CSIC.

Si hace unos años explicaba en este blog cómo instalar el gestor de colas de cálculo gridengine en un servidor  Ubuntu, esta semana lo actualizo para un servidor multicore con Debian 11:

Instalación de los diferentes componentes gridengine en la misma máquina:

   sudo apt install gridengine-master gridengine-qmon gridengine-exec

Si tuvieras que empezar de cero puedes hacerlo con:

    sudo apt purge gridengine-common
    sudo rm -rf /var/spool/gridengine/spooldb/sge
    sudo apt install gridengine-master gridengine-qmon gridengine-exec

Tras leer esto edité el fichero /etc/hosts así:

   127.0.0.1 localhost.localdomain localhost
   127.0.1.1 master master
   121.xxx.yyy.zzz myhost
 

Ojo que la instalación crea el usuario sgeadmin. Para que tu propio usuario tenga privilegios de administración, crear y configurar una cola llamada all.q deberás hacer los siguiente:  

    sudo -u sgeadmin qconf -am myuser

    # and to a userlist:
    qconf -au myuser users

   # Add a submission host:
   qconf -as myhost

   # Add an execution host, you will be prompted for information about the execution host
   qconf -ae
 
   # Add a new host group:
   qconf -ahgrp @allhosts

   # Add the exec host to the @allhosts list:
   qconf -aattr hostgroup hostlist myhost @allhosts

   # Add and configure queue, set the slots to CPU/cores, check parallel env (pe_list)
   qconf -aq all.q

   # Add the host group to the queue:
   qconf -aattr queue hostlist @allhosts  all.q

   # Allocate slots in this queue:
   qconf -aattr queue slots "[myhost=12]" all.q

Con esto estaría, y puedes comprobarlo con qstat -f o lanzando un trabajo con qsub

Con los siguientes comandos puedes reiniciar los respectivos servicios en Debian:

   sudo systemctl restart gridengine-master.service
   sudo systemctl restart gridengine-exec.service

Hasta pronto,

Bruno

HOWTO install Grid Engine on multi-core Linux box to run GET_HOMOLOGUES

Hi,
I post this in English so that all users of GET_HOMOLOGUES can benefit. It is inspired on a previous tutorial posted in scidom and the expertise of David Ramírez from our provider SIE.

The aim of this entry is to demonstrate how to set up a Grid Engine queue manager on your local multi-core Linux box or server so that you can get the most of that computing power during your GET_HOMOLOGUES jobs. We will install Grid Engine on its default path, /opt/, which requires superuser permissions. As I'll do this un Ubuntu, I will do 'sudo su' to temporarily get superuser privileges; this should be replaces simply with 'su' in other Linux flavours. Otherwise you can install it elsewhere if you don't have admin rights.

1) Visit http://gridscheduler.sourceforge.net , create a new user and download the latest 64bit binary to /opt:

$ cd /opt
$ sudo su 
$ useradd sgeadmin
$ wget -c http://dl.dropbox.com/u/47200624/respin/ge2011.11.tar.gz $ tar xvfz ge2011.11.tar.gz
$ chown -R sgeadmin ge2011.11/
$ chgrp -R sgeadmin ge2011.11/

$ ln -s ge2011.11 sge

2) Set relevant environment variables in /etc/bash.bashrc  [system-wide, can also be named /etc/basrhc] or alternatively in ~/.bashrc for a given user:

export arch=x86_64
export SGE_ROOT=/opt/sge
export PATH=$PATH:"$SGE_ROOT/bin/linux-x64"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$SGE_ROOT/lib"

And make this changes live:

$ source /etc/bash.bashrc

3) Set your host name to anything but localhost by editing /etc/hosts so that the first line is something like this (localhost or 127.0.x.x IP addresses are not valid):

172.1.1.1   yourhost

4) Install Grid Engine server with all defaults except cluster name, which usually will be 'yourhost':
$ ./install_qmaster

5) Install Grid Engine client with all defaults:
$ ./install_execd

6) Optionally configure default all.q queue:
$  qconf -mq all.q

7) Add your host to list of admitted hosts:
$ qconf -as yourhost
$ exit

You should now be done. A test will confirm this. Please open a new terminal and move to your GET_HOMOLOGUES installation folder:

$ cd get_homologues-x86-20150306
$  ./get_homologues.pl -d sample_buch_fasta -m cluster

If the jobs finishes successfully then you are indeed done. I hope this helps,
Bruno