How to Run a PBS Batch Job

The following information is available regarding using PBS on the Particle Physics PC farm:

General Information:

General
PBS Resources
PBS Script Files
Submitting Jobs
Examples
Important Notes

General Information and Examples

To run a batch job through PBS, you first create a PBS script. The script specifies, among other things, the number of cpus,
the amount of memory, and the length of time you're requesting. Each system has additional resources that you can request.

   When a batch job starts execution, a number of environment variables are predefined, 
 which include:
   
 Variables defined on the execution host.
 
 Variables exported from the submission host with
      -v (selected variables) and -V (all variables).
   
 Variables defined by PBS.
  
 The following reflect the environment where the user ran qsub:
   PBS_O_HOST    The host where you ran the qsub command.
   PBS_O_LOGNAME Your user ID where you ran qsub.
   PBS_O_HOME    Your home directory where you ran qsub.
   PBS_O_WORKDIR The working directory where you ran qsub.

 These reflect the environment where the job is executing:
   PBS_ENVIRONMENT Set to PBS_BATCH to indicate the job is a batch job, or
                   to PBS_INTERACTIVE to indicate the job is a PBS interactive job.
   PBS_O_QUEUE   The original queue you submitted to.
   PBS_QUEUE     The queue the job is executing from.
   PBS_JOBID     The job's PBS identifier.
   PBS_JOBNAME   The job's name.

PBS Resources

Resource_List.mem = 256mb
Resource_List.cput = 08:00:00
Resource_List.ncpus = 1
Resource_List.neednodes = 1:ppn=1
Resource_List.nice = 10
Resource_List.walltime = 72:00:00

PBS Scripts

PBS directives are specified as comments in the script. In particular, all lines beginning with #PBS are PBS directives. After the PBS directives is the body of the script -- commands which are executed when the script runs. Here are samples PBS script.

script 1
script 2

Submitting Jobs

To submit the above script to PBS, you use the qsub command. For instance, if the script were called myscript you'd submit it using

   physdsk2 560> qsub myscript
   3212.eio09.weizmann.ac.il

The second line is the job identifier returned by PBS, and indicates that the script has been accepted.

PBS is configured at the PC farm so that you are not required to be logged into the system you want to submit a job to. For example, you could submit a job from physdsk1 or physdsk2 . You can submit a job to a specific PBS server by specifying the server host name for that system. For example:

    physdsk2 561> qsub -q M@eio09  myscript
    24073.eio09.weizmann.ac.il

This command submitted a job from physdsk2 to eio33 .

Notes

You can specify the number of CPUs your job needs (-l ncpus=), memory (-l mem=), and the maximum wall clock time (-l walltime=). These numbers should not exceed the queue limits. If you do not specify any values, the default values associated with the queue are used (currently, 1 CPU, 8 hours and nice=10).

By default, PBS executes your job from your home directory. However, you can use the environment variable PBS_O_WORKDIR to change to the directory where you submitted your job, since PBS sets this variable to the directory from which you issued the 'qsub' command.

cd $PBS_O_WORKDIR
mpirun ./myprog

The "-V" option can be used to export all environment variables from the submitting shell to your PBS job.