3. General
• Trace creation and visualization does not have to
happen on the same platform!
– You can look at traces created on Quarry (or your
own desktop) on BigRed
Robert Henschel
4. Quarry – SoftEnv Keys
• Vampir Version 5.2
+vampir-5.2-32 or +vampir-5.2-64
• VampirServer Version 1.10
+vampirserver-1.10-32
or
+vampirserver-1.10-64
• VampirTrace Version 5.6.3
@intel-10.1.022-64
+openmpi-1.3.1-intel-10.1-64
+vampirtrace-5.6.3-intel-64
or
@intel-10.1.022-32
+openmpi-1.3.1-intel-10.1-32
+vampirtrace-5.6.3-intel-32
Robert Henschel
5. Quarry – Running Vampir
• Vampir Version 5.2
– Interactive job in PBS
– On interactive nodes b005-b008
Robert Henschel
6. Quarry – Running Vampir
• Vampir Version 5.2
– Interactive job in PBS
qsub -I -X
soft add +vampir-5.2-64
vampir
– On interactive nodes b005-b008
• You share the node with other people, this is
especially a problem for opening large trace files!
ssh -X b005
soft add +vampir-5.2-64
vampir
Robert Henschel
7. Quarry – Running VampirServer
• Vampir Server Version 1.10
– Server in PBS, viewer on interactive node
– Server and viewer in interactive job in PBS
– Server and viewer on interactive nodes b005-b008
Robert Henschel
8. Quarry – Running VampirServer
• Vampir Server Version 1.10
– Server in PBS, viewer on interactive node
• Server in PBS
– Make sure your “.soft” file contains
+openmpi-1.3.1-intel-10.1-64
+vampirserver-1.10-64
» Logout/Login to active changes in “.soft”
– PBS script
#PBS -l nodes=2:ppn=8,walltime=5:00:00
#PBS -m ae
#PBS -N VampirServer
#PBS
mpirun -np 16 -machinefile $PBS_NODEFILE vngd -n 16 1>$HOME/vampir.out 2>&1
Start job
qsub vampir.pbs
– Wait for the job to start
Robert Henschel
9. Quarry – Running VampirServer
– Lookup host and port information
» It may take a minute or two after the job
started for this file to get updated!!!
cat $HOME/vampir.out
Running...
Server listens on: b052:30000
• Viewer on interactive nodes b005-b008
ssh -X b005
soft add +openmpi-1.3.1-intel-10.1-64
soft add +vampirserver-1.10-64
vng -a b052 -p 30000
Robert Henschel
10. Quarry – Running VampirServer
– Interactive job in PBS
• Only one node
qsub -I -X
soft add +openmpi-1.3.1-intel-10.1-64
soft add +vampirserver-1.10-64
mpirun -np 8 vngd -n 8 1>output 2>&1 &
cat output
Running...
Server listens on: b048:30000
vng -a localhost -p 30000
Robert Henschel
11. Quarry – Running VampirServer
– On interactive nodes b005-b008
• Only one node
• Runtime needs to be less than 24 hours
• You share the node with other people
ssh -X b005
soft add +openmpi-1.3.1-intel-10.1-64
soft add +vampirserver-1.10-64
mpirun -np 4 vngd -n 4 1>output 2>&1 &
cat output
Running...
Server listens on: b005:30000
vng -a localhost -p 30000
Robert Henschel
12. Quarry – Tracing with VampirTrace 5.6.3
• VampirTrace is configured to work with the Intel
compilers version 10.1 and OpenMPI 1.3.1
• Include the following keys in your .soft file
@intel-10.1.022-64
+openmpi-1.3.1-intel-10.1-64
+vampirtrace-5.6.3-intel-64
• The VampirTrace compiler wrappers are now available
• It is possible to use them also with GCC, if you need to
do that, please send a mail to hpahelp@indiana.edu
• We can also provide a version for the PGI compilers.
Robert Henschel
13. BigRed – SoftEnv Keys
• Vampir Version 5.1
+vampir-5.1-64
• VampirServer Version 1.10
+vampirserver-1.10-64
• VampirTrace Version 5.6.3
+openmpi-1.3.1-ibm-64
+vampirtrace-5.6.3-ibm-64
or
+openmpi-1.3.1-ibm-32
+vampirtrace-5.6.3-ibm-32
Robert Henschel
14. BigRed – Running Vampir
• Vampir Version 5.1
– Interactive job in LoadLeveler
Robert Henschel
15. BigRed – Running Vampir
• Vampir Version 5.1
– Interactive job in LoadLeveler
soft add +interactivejob
interactivejob -w 60
soft add +vampir-5.1-64
vampir
• interactivejob can be customize for longer
interactive sessions
– run “interactivejob -?” for more
Robert Henschel
16. BigRed – Running VampirServer
• Vampir Server Version 1.10
– Server in LoadLeveler, viewer on login node
Robert Henschel
17. BigRed – Running VampirServer
• Vampir Server Version 1.10
– Server in LoadLeveler, viewer on login node
• Server in LoadLeveler
– Make sure your “.soft” file contains
+openmpi-1.3.1-ibm-64
+vampirserver-1.10-64
» Logout/Login to active changes in “.soft”
– Submit job to LoadLeveler
run_server.sh 4 DEBUG 15
or
run_server.sh 16 LONG 15
– Wait for the job to start
Robert Henschel
18. BigRed – Running VampirServer
– Lookup host and port information
» It may take a minute or two after the job
started for this file to get updated!!!
cat $HOME/paralleljob.x.0.out
Running...
Server listens on: s10c1b4:30000
• Viewer on login node
soft add +vampirserver-1.10-64
vng -a s10c1b4 -p 30000
Robert Henschel
19. BigRed – Tracing with VampirTrace 5.6.3
• VampirTrace is configured to work with the IBM compilers
and OpenMPI 1.3.1
• Include the following keys in your .soft file
+openmpi-1.3.1-ibm-64
+vampirtrace-5.6.3-ibm-64
Logout/Login to active changes in the .soft file or run
resoft from on command line.
• The VampirTrace compiler wrappers are now available
Robert Henschel
