Divide loop timers by the number of threads

For timers within parallel loops, each timed task is being applied on N threads. If each task takes T second, then the timers would return a total time of N*T seconds, which may be misrepresentative in the timing tables. A better representation is to divide the total time of a loop timer by N threads. This gives an "average" time per task across all cores but is more in line with the total time measured by a timer that straddles the parallel loop. Another thing we can do is to try to measure the amount of "real" time instead of CPU time that each task takes (i.e. use CPU_TIME function in Timer_TheTime. That might obviate the need to divide by the number of threads. Signed-off-by: Bob Yantosca <[email protected]>
geoschem · May 20, 2020 · 0cfd663 · 0cfd663
1 parent bc0c69f
commit 0cfd663
Showing 1 changed file with 7 additions and 4 deletions.
diff --git a/GeosUtil/timers_mod.F90 b/GeosUtil/timers_mod.F90
@@ -60,10 +60,11 @@ MODULE Timers_Mod
   INTEGER                                   :: TimerCurrentSize = 0
 
   ! Maximum Supported Timers. Increasing will increase memory footprint.
-  INTEGER, PARAMETER                        :: TimerMaxSize = 30
+  INTEGER, PARAMETER                        :: TimerMaxSize = 35
 
   ! Number of threads for parallel loops
-  INTEGER                                   :: nThreads = 1
+  INTEGER                                   :: nThreads   = 1
+  REAL(f8)                                  :: d_nThreads = 1.0_f8
 
   ! The definition of the GC_Timer type.
   TYPE GC_Timer
@@ -146,7 +147,8 @@ SUBROUTINE Timer_Setup( TheMode )
 
     ! Determine the number of threads available for parallel loops
     !$OMP PARALLEL
-    nThreads = OMP_GET_NUM_THREADS()
+    nThreads   = OMP_GET_NUM_THREADS()
+    d_nThreads = DBLE( nThreads )
     !$OMP END PARALLEL
 
     ! Debug
@@ -575,7 +577,8 @@ SUBROUTINE Timer_Sum_Loop( TimerName, RC )
     ENDIF
 
     ! Sum the total time across threads
-    SavedTimers(TimerLoc)%TOTAL_TIME = SUM(SavedTimers(TimerLoc)%TOTAL_TIME_LOOP)
+    SavedTimers(TimerLoc)%TOTAL_TIME =                                       &
+         SUM(SavedTimers(TimerLoc)%TOTAL_TIME_LOOP) / d_nThreads
 
     ! Debug
     !Print*, 'Timer:', TRIM(TimerName)