variables définies à la soumission d'un job

Plusieurs variables sont définis à la soumission du script :

Variable Name Value at qsub Time
PBS_O_HOST Client host name
PBS_O_WORKDIR Current working directory (for the qsub command)

La variable PBS_O_WORKDIR spécifie le répertoire de travail, c'est à dire le répertoire où le script a été lancé. Lors de la soumission, il faut ajouter cd $PBS_O_WORKDIR pour se placer dans le répertoire de travail. Une seconde solution est d'utiliser l'option -d /scratch/homer lors du qsub

You can use the env command (either in a PBS script or on the command line of a PBS interactive session) to find out what environment variables are set within a PBS job.
In addition to the PBS_xxxx variables, the following two are useful to know:

  • NCPUS : Defaults to number of CPUs that you requested for the node.
  • OMP_NUM_THREADS : Defaults to 1 unless you explicitly set it to a different number. If your PBS job runs an OpenMP or MPI/OpenMP application, this variable sets the number of threads in the parallel region.
  • FORT_BUFFERED : Setting this variable to 1 enables records to be accumulated in the buffer and flushed to disk later.

Gestion des ressources

  • Allocation

Resources are allocated to jobs both by explicitly requesting them and by applying
specified defaults. Jobs explicitly request resources either at the host level in
chunks defined in a selection statement, or in job-wide resource requests.

qsub ... -l <resource name>=<value> <--- job-wide request
-l select=<chunks> <--- selection statement

The only resources that can be requested in chunks are host-level resources, such
as mem and ncpus. The only resources that can be in a job-wide request are server-
level or queue-level resources, such as walltime. An explicit resource request can
appear here, with this order of precedence:

  • Requesting Resources in Chunks

A chunk declares the value of each resource in a set of resources which are to
be allocated as a unit to a job. All of a chunk must be taken from a single
vnode. A chunk request is a host-level request, and it must be for a host-
level resource. A chunk is the smallest set of resources that will be allo-
cated to a job. It is one or more resource_name=value statements separated by
a colon.

  • Requesting Job-wide Resources :

A job-wide resource request is for resource(s) at the server or queue level.
This resource must be a server-level or queue-level resource. A job-wide
resource is designed to be used by the entire job, and is available to the
complex, not just one execution host.

  • exemples de ressources in chunks (man pbs_resources)
    mpiprocs  Number of MPI processes for this chunk.  Defaults to 1 if ncpus  >  0,  0
                 otherwise.   Can  be  requested only inside of a select statement.  Type:
                 The number of lines in PBS_NODEFILE is the sum of the values of  mpiprocs
                 for all chunks requested by the job.  For each chunk with mpiprocs=P, the
                 host name for that chunk is written to the PBS_NODEFILE P times.  Can  be
                 requested only inside of a select statement.

   ncpus     Number of processors requested.  Cannot be shared across vnodes.  Can  be
                 requested only inside of a select statement.  Consumable.
                 Type: integer.

   ompthreads   Number of OpenMP threads for this chunk.  Defaults to ncpus if not speci-
                 fied.   Can  be requested only inside of a select statement.  Non-consum-
                 able.  Type: integer.  Python type: int.
                 For the MPI process with rank 0,  the  environment  variables  NCPUS  and
                 OMP_NUM_THREADS  are  set to the value of ompthreads.  For other MPI pro-
                 cesses, behavior is dependent on MPI implementation.  See The PBS Profes-
                 sional Administrator’s Guide.

    mem       Amount of physical memory i.e. workingset allocated to  the  job,  either
                 job-wide  or  host-level.  Consumable.  Can be requested only inside of a
                 select statement.  Type: size.