NERSCPowering Scientific Discovery Since 1974

MILC Medium IPM Profile on Bassi

Step ID b0301.180242.0 Job Name b0301.nersc.gov.180242
Owner hjw Account mpccc Status Completed
LL class debug Submit class debug Job type Parallel
Nodes 8 Wall secs 350 Wall hrs 0.10
MPP secs 134,400 MPP hrs 37.33 Raw Secs 22,400
Submit Jul-17-07 14:02:32 Start Jul-17-07 14:25:18 Wait 00:22:46
Completion Jul-17-07 14:31:09 systime 474 usrtime 12,272
Task Max. Memory (GB) 2.546e-01  
Nodelist b0404, b0701, b0808, b1105, b1111, b0304, b1108, b0803
*Indicates dispatch time

Summary | Hardware Counters | MPI Statistics

  

IPM summary

Executable ./su3_rmd
Number of tasks 64 Aggregate GFlop/sec 68.9597 Average GFlop/sec/task 1.0775
Average wall secs 1.472e+02 Aggregate memory (GB) 16.2387 Average memory/task (GB) 0.2537
Average MPI secs/task 1.147e+01 MPI time % 7.79 Aggregate MPI calls made 2.078e+07

IPM summary statistics - 64 tasks

MetricSum over all tasksAverage (per task)Task CV (%)Task MinimumTask Maxium
Aggregate Floating Point Operations (Flop x 10**9) 1.015e+04 1.586e+02 0.00 1.586e+02 1.586e+02
GFlop/sec 6.896e+01 1.077e+00 0.04 1.077e+00 1.078e+00
Maximum Memory Usage (GBytes) 1.624e+01 2.537e-01 0.12 2.536e-01 2.546e-01
Time Spent in MPI Routines (sec) 7.342e+02 1.147e+01 1.31 1.087e+01 1.172e+01
Wallclock Time (sec) 9.422e+03 1.472e+02 0.04 1.471e+02 1.473e+02
Memory in units of gigabytes; time in seconds.
  

Hardware counter statistics - 64 tasks

Counter NameSum over all tasksAverage (per task)Task CV (%)Task MinimumTask Maxium
PM_FPU_1FLOP 1.181085e+12 1.845445e+10 0.00 1.845329e+10 1.845541e+10
PM_FPU_FMA 4.485362e+12 7.008378e+10 0.00 7.008230e+10 7.008538e+10
PM_INST_CMPL 1.783558e+13 2.786810e+11 0.28 2.768492e+11 2.800510e+11
PM_LD_REF_L1 5.838726e+12 9.123010e+10 0.16 9.070262e+10 9.152717e+10
PM_RUN_CYC 1.758132e+13 2.747082e+11 0.11 2.733865e+11 2.751271e+11
PM_ST_REF_L1 2.540178e+12 3.969029e+10 0.30 3.935492e+10 3.995576e+10
  

MPI time statistics - 64 tasks

CallSum over all tasksAverage (per task)Task CV (%)Task MinimumTask Maxium% of MPI% of wall
MPI_Wait 5.490e+02 8.578e+00 1.95 7.933e+00 8.821e+00 74.771 5.827
MPI_Isend 1.177e+02 1.840e+00 2.40 1.751e+00 1.961e+00 16.035 1.250
MPI_Allreduce 5.423e+01 8.474e-01 4.47 7.419e-01 9.213e-01 7.386 0.576
MPI_Irecv 1.203e+01 1.879e-01 3.43 1.741e-01 2.077e-01 1.638 0.128
MPI_Barrier 1.052e+00 1.644e-02 67.01 4.864e-05 2.663e-02 0.143 0.011
MPI_Comm_size 1.210e-01 1.890e-03 2.23 1.806e-03 1.994e-03 0.016 0.001
MPI_Bcast 6.917e-02 1.081e-03 35.66 2.317e-04 1.928e-03 0.009 0.001
MPI_Comm_rank 4.268e-05 6.668e-07 25.62 4.768e-07 9.537e-07 0.000 0.000

Percent of total MPI Time

Pct. MPI Time by function

MPI call frequency statistics - 64 tasks

CallTasksNumber of Calls (per task)Buffer Size (Bytes)
SumAverageCV (%)MinimumMaxiumSumAvg. per taskAvg. per call
MPI_Wait 64 1.014e+07 1.584e+05 0.00e+00 1.584e+05 1.584e+05 0.000e+00 0.000e+00 0.000e+00
MPI_Isend 64 5.070e+06 7.921e+04 0.00e+00 7.921e+04 7.921e+04 0.000e+00 0.000e+00 0.000e+00
MPI_Allreduce 64 9.485e+04 1.482e+03 0.00e+00 1.482e+03 1.482e+03 0.000e+00 0.000e+00 0.000e+00
MPI_Irecv 64 5.070e+06 7.921e+04 0.00e+00 7.921e+04 7.921e+04 0.000e+00 0.000e+00 0.000e+00
MPI_Barrier 64 1.280e+02 2.000e+00 0.00e+00 2.000e+00 2.000e+00 0.000e+00 0.000e+00 0.000e+00
MPI_Comm_size 64 4.102e+05 6.410e+03 1.95e-03 6.410e+03 6.411e+03 0.000e+00 0.000e+00 0.000e+00
MPI_Bcast 64 1.920e+02 3.000e+00 0.00e+00 3.000e+00 3.000e+00 0.000e+00 0.000e+00 0.000e+00
MPI_Comm_rank 64 6.400e+01 1.000e+00 0.00e+00 1.000e+00 1.000e+00 0.000e+00 0.000e+00 0.000e+00

Memory Snapshots

Memory usage snapshots are taken every 15 minutes on the nodes. This job was running on the nodes listed below when snapshots were taken. A yellow background warns that a node was running very low on small page memory; a pink background indicates a shortage that likely forced memory paging. Memory paging leads to a severe performance penalty and may cause the application to die and perhaps take the node with it.

Node NameTotal GB UsedLarge Page GB UsedLarge Page GB FreeSmall Page GB UsedSmall Page GB FreeTimeNotes
b0304 6.01 3.55 15.38 2.46 5.85 2007-07-17 14:30:01  
b0404 6.87 2.80 16.12 4.07 4.24 2007-07-17 14:30:01  
b0701 3.60 2.30 16.62 1.30 7.01 2007-07-17 14:30:01  
b0803 5.11 2.30 16.62 2.81 5.50 2007-07-17 14:30:01  
b0808 3.29 2.30 16.62 0.99 7.32 2007-07-17 14:30:01  
b1105 11.06 6.80 12.12 4.26 4.05 2007-07-17 14:30:01  
b1108 3.19 2.30 16.62 0.89 7.42 2007-07-17 14:30:01  
b1111 3.19 2.30 16.62 0.89 7.42 2007-07-17 14:30:01