To use WEXAC, you must register for the service via Internal Services with your lab’s departmental administrator. Please contact us if the lab you are associated with is not a WEXAC member (i.e. does not have WEXAC nodes).
For detailed information on our WEXAC policy, please refer to our WEXAC policy page.
Using the WEXAC Software Stack
with the Environment Modules Package
The Environment Modules package allows you to dynamically modify your WEXAC cluster user environment, so as to run specific software packages.
While Modules implementation is shell dependent, commands are not. The following table lists a number of sample commands.
| Command | Description |
|---|---|
| module avail | List available Modules software |
| module list | List currently loaded Modules packages |
| module load matlab/R2011b load | Load a particular environment for a package |
| module switch matlab/R2012a | Switch the environment of a package |
| module unload matlab/R2012a | Unload a particular environment for a package |
Following is a list of sample installed software packages, which can be loaded using Modules:
- BEDTools/2.16.2
- blast/2.2.26
- blat/3.4
- clustalw/2.1
- comsol/4.2a
- EMBROSS/6.4.0
- fasta/36.3.5a
- IGV/2.1
- jdk/1.7.0
- matlab/R2007b
- matlab/R2010b
- matlab/R2011b
- matlab/R2012a
- mpich/1.2.7
- mysql/5.5.22
- ncbi-blast+/2.2.25
- ncbi-blast+/2.2.26
- perl/5.14.2
- perl/5.8.8
- pgi/11.10
- pgi/12.4
- php/5.3.10
- prank/111130
- pymol/1.5
- python/3.2.2
- R/2.12.1
- R/2.14.2
- rmblast/1.2
- rstudio-server/0.95.265
- samtools/0.1.17
- tcltk/8.5.11
- velvet/1.2.03
- vim-7.3/7.3
- websvn/2.3.3
To learn more about the Environment Modules project, visit http://modules.sourceforge.net.
LSF (Load Sharing Facility)
WEXAC runs Platform LSF—powerful, enterprise-grade IBM middleware with comprehensive resource management and job scheduling functionality.
LSF’s scheduling policy is based largely on the fairshare model.
Running LSF commands
To run a package from tcsh using LSF, enter the following command:
# bsub -o somepath.o -e somepath.e somescript.csh
Scripts will resemble the following:
#!/bin/bash
module load matlab/R2010b
hostname
matlab -nodisplay -nojvm < /myhome/h.m > /myhome/h.out
bash users can modify the .bashrc file to include the following lines:
if [ -f /storage/apps/Modules/default/etc/profile.modules ]; then
. /apps/Modules/default/etc/profile.modules
module load matlab/R2010b R/2.14.2
fi
The following table lists key LSF commands, along with their SGE equivalents.
| Description | LSF Command | SGE Equivalent |
|---|---|---|
Queue Commands |
||
| List all jobs running on the cluster | bjobs -u all | qstat |
| List all jobs running on the cluster, expanded to display nodes |
bjobs -u all -X | qstat -g t |
| List your running jobs (also accepts -X flag to expand to display nodes) |
bjobs | qstat -u User_ID |
| Display detailed information on a job (with JOBID representing the job ID number) |
bjobs -l JOBID | qstat -j JOBID |
| List available queues, including the number of jobs and slots |
bqueues | qstat -g c |
Job Commands |
||
| Submit job named jobname | bsub < jobname.bsub | qsub jobname.job |
| Delete running job (with JOBID representing the job ID number) |
bkill JOBID | qdel JOBID |
Job Flags |
||
| Submit job to queue queuename | #BSUB -q queuename | #$ -q queuename.q |
| Job name | #BSUB -J JobName | #$ -N JobName |
| Array job with 10 elements | #BSUB -J JobName[1-10 | #$ -t 1-10 |
| Request xx number of slots | #BSUB -n xx | #$ -pe mvapich xx |
| Issue e-mail notification upon job initiation | #BSUB -B | #$ -m b |
| Issue e-mail notification upon job completion | #BSUB -N | #$ -m e |
| Write output to output.log (LSF - I for array JOBINDEX) | #BSUB -o output.log | #$ -o output.log |
| Write error to error.log (LSF - I for array JOBINDEX) |
#BSUB -e error.log | #$ -e error.log |
Job Variables |
||
| Job ID | $LSB_JOBID | $JOB_ID |
| Array job index (when using -t in SGE and -J Name[1-X] in LSF) |
$LSB_JOBINDEX | $SGE_TASK_ID |
| Number of parallel slots requested | $LSB_DJOB_NUMPROC | $NSLOTS |
For further information on LSF, refer to the LSF Quick Reference Guide.
For specifics on SGE to LSF migration, feel free to review the WEXAC SGE-to-LSF Migration user training presentation.