GNU parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU parallel can then split the input and pipe it into commands in parallel.

Versions and Availability

Module Names for gnuparallel on qb2
Machine Version Module Name
None Available N/A N/A
▶ **FIX-ME** FAQ?


Parallel typical serial and MPI-based applications.

(1) Parallel serial jobs

Example of a blast job on Mike:


#PBS -A hpc_smictest3
#PBS -l nodes=2:ppn=16
#PBS -l walltime=1:00:00
#PBS -q workq

export JOBS_PER_NODE=16

parallel --progress \				# shows progres
         --joblog logfile \			# job logfile
         -j $JOBS_PER_NODE \		# jobs per node
         --slf $PBS_NODEFILE \			# nodes assigned to your job
         --workdir $WDIR \			
         ./ {} {/.} :::: input.lst  #script_to_parallize input output joblist

where: input.lst contains job input list:


where: is the script for running a serial blast job

e.g.: ./ input1.faa input1 -- how to run single serial job


export WDIR=/xxx/xxx
cd $WDIR
blastp -query $1 -db db/img_v400_PROT.00 -out output/$2.out -outfmt 7 -max_target_seqs 100 -num_threads 2

(2) Parallel MPI jobs

Use "mpirun" to run a laplace


#PBS -A your_allocation_name
#PBS -l walltime=2:00:00
#PBS -l nodes=4:ppn=16
#PBS -q checkpt

export JOBS_PER_NODE=8
export NPROCS=2
cd $WDIR
parallel --progress \
         -j $JOBS_PER_NODE \
         --slf $PBS_NODEFILE \
         --workdir $WDIR \
         ./ {} $NPROCS :::: input.lst

where: is the script to run one MPI job


FILE=$(eval echo $1)
param=`cat ${FILE}`
mpirun -ppn $2 $WDIR/lap_mpi $param

where: input.lst contains job input list:


cat input1:
4096 4096 2 2 0.08 20000 0 0


Last modified: January 30 2017 12:37:10.