Using and Administering
Figure 20 shows a sample job command file for PVM
3.3.11+ (SP2MPI architecture). Before using PVM, users
should contact their administrator to determine which PVM architecture has
been installed. The SP2MPI architecture version should be used when
users require that their jobs run in user space.
Figure 20. Sample PVM 3.3.11+ (SP2MPI Architecture) Job Command File
#!/bin/ksh
# @ job_type = parallel
# @ class = PVM3
# @ requirements = (Adapter == "hps_us")
# @ output = my_PVM_program.$(cluster).$(process).out
# @ error = my_PVM_program.$(cluster).$(process).err
# @ node = 3,3
# @ queue
# Set PVM daemon and starter path dictated by LoadLeveler administrator
starter_path=/home/userid/loadl/pvm3/bin/SP2MPI
daemon_path=/home/userid/loadl/pvm3/lib/SP2MPI
# Export "MP_EUILIB" before starting PVM3 (default is "ip")
export MP_EUILIB=us
echo MP_EUILIB=$MP_EUILIB
# Clean up old PVM log and daemon files belonging to user
filelog=/tmp/pvml.id | awk -F'=' '{print $2}' | awk -F'(' '{print $1}'
filedaemon=/tmp/pvmd.id | awk -F'=' '{print $2}' | awk -F'(' '{print $1}'
rm -f $filelog > /dev/null
rm -f $filedaemon > /dev/null
# Start PVM daemon in background
$daemon_path/pvmd3 &
echo "pvm background pid=$!"
echo "Sleep 2 seconds"
sleep 2
echo "PVM daemon started"
# Start parallel executable
llnode_cnt=`echo "$LOADL_PROCESSOR_LIST" | awk '{print NF}'`
actual_cnt=expr "$llnode_cnt" - 1
$starter_path/starter -n $actual_cnt /home/userid/my_PVM_program
echo "Parallel executable starting"
# Check processes running and halt PVM daemon
echo "ps -a" | /home/userid/loadl/pvm3/lib/SP2MPI/pvm
echo "Halt PVM daemon"
echo "halt" | /home/userid/loadl/pvm3/lib/SP2MPI/pvm
wait
echo "PVM daemon completed"
|
Note the following requirements for PVM 3.3.11+ (SP2MPI
architecture) jobs:
- The job must have job_type = parallel.
- You must specify one more processor then you actually need to run the
parallel job. PVM spawns an additional task to relay messages to and
from the PVM daemon. Parallel tasks cannot communicate with PVM daemon
directly. The additional task will be spawned on the last processor in
the LOADL_PROCESSOR_LIST. For more information on this environment
variable set by LoadLeveler see Obtaining Allocated Host Names.
- You must use the PVM daemon and starter path dictated by the LoadLeveler
administrator. The parallel_path keyword is ignored.
- You must export MP_EUILIB as us when running in user space over
the switch. MP_PROCS, MP_RMPOOL and MP_HOSTFILE are ignored when
running under LoadLeveler.
- You should clean up any temporary PVM log or daemon files before starting
the PVM daemon.
- You must start the PVM daemon in the job script, and you must start it in
the background ($daemon_path/pvmd3 &).
- You must compile your parallel program following the PVM guidelines for
PVM 3.3.11+ (SP2MPI architecture).
- You must start the parallel executable through the PVM starter
program. The PVM starter program has no relationship to the LoadLeveler
starter daemon.
- You must specify the parallel executable as an argument to the PVM starter
program.
- You must specify the actual number of parallel tasks to the PVM starter
program. This number must be one less then the number of processors
allocated through LoadLeveler.
- You must halt the PVM daemon when the PVM starter program
completes.
- You can invoke the PVM starter program only once.
This example demonstrates the sequence of events that
occur when you submit the sample job command file shown in Figure 20.
Figure 21 illustrates the following:
- From the job command file, (1) the PVM daemon, pvmd3, and
(2) the PVM starter are started under the LoadLeveler
starter. The PVM starter tells the PVM daemon to start two tasks
(my_PVM_program).
- (3) The PVM daemon starts the POE Partition Manager, which in
turn (4) starts the POE daemons, (represented as pvmd2) on all
three nodes.
- (5) The POE daemons (pvmd2) start the parallel tasks,
my_PVM_program, on all nodes under the LoadLeveler starter.
The last parallel task, my_PVM_program on Node 3, is the additional
task which relays messages between the PVM daemon and the parallel
tasks.
Figure 21. Sequence of Events in a PVM 3.3.11+ Job
View figure.
[ Top of Page | Previous Page | Next Page | Table of Contents | Index ]
View figure.