Here we evaluate the performance ofWayPoint. We use a subset directory cache implementation with varying sizes and associativities for comparison. We also use an optimistic, albeit unrealizable, full on-die directory to serve as a baseline. The optimistic full directory is infinitely sized, thus removing all but cold-cache misses from the performance overhead of the baseline.
9.4.1
Directory Cache Sizing
Figure 9.5 shows the runtime of the benchmarks normalized to the optimistic co- herence baseline. The data on the left are configurations withoutWayPoint where invalidations occur when the directory cache capacity is reached. The data on the right are for configurations with WayPoint. An eviction with WayPoint re- sults in an entry being placed in an overflow list in cached memory, thus avoiding the overhead of invalidations. Note that the directory sizes, measured in en- tries per L3 bank, are not equal between the two configurations because the non-
0.0 x 1.0 x 2.0 x 3.0 x 4.0 x 5.0 x
cg dmm gjk heat kmeans march mri sobel stencil
Ru nt im e N or m al iz ed t o H W cc O pt i
16384 baseline 8192 baseline 4096 baseline 2048 baseline 2048 Waypoint 1024 WayPoint 512 WayPoint 256 WayPoint
Figure 9.5: The runtime normalized to the optimistic hardware cache coherence implementation is shown for different sizes of directory cache. Configurations without WayPoint are shown on the left and those with it on the right.
WayPoint configurations slow down precipitously with fewer than 2048 directory entries per bank.
The results in Figure 9.5 indicate that at least 8192 entries per bank would be needed to eliminate large slowdowns when using a conventional directory cache. With WayPoint, that number can be 2048 or fewer, thus providing compara- ble performance across all benchmarks with over 4× fewer resources. Moreover, performance degradation for designs with WayPoint follows a much smoother curve leading us to believe that even for pathological workloads that we may not have evaluated,WayPoint should be far more robust to workload variation than configurations without WayPoint. Note that the one benchmark with apprecia- ble overhead for WayPoint is stencil. The accesses for this benchmark tend to have little locality and little reuse since the benchmark streams through every out- put once and each input a small number of times each iteration. Furthermore, the read sharing is limited. Therefore, stencil tends to thrash in the directory cache more than the other workloads. Even so, the overhead for stencil is much less with a directory cache supplemented by WayPoint than without WayPoint.
9.4.2
Directory Cache Associativity
Figures 9.6(a) and 9.6(b) give the execution time with and withoutWayPoint while varying the associativity of a fixed-sized directory. The goal of these experiments is to demonstrate the ability of WayPoint to reduce the associativity demands of the directory cache. All results vary the number of ways and sets inversely to maintain a constant 16384-entry directory cache except for the infinite-sized on-die directory baseline, which has one set and unlimited ways. We show 32– 512 way directory caches withWayPoint and 128–512 way without WayPoint. The discrepancy is due to the greatly increased runtime due to thrashing and sub- sequent invalidations for lower associativity directory caches withoutWayPoint.
The results show that for highly associative directory caches, the configura- tions perform comparably. However, for the configurations without WayPoint, there is a clear performance cliff where runtime increases dramatically below a threshold associativity. The performance versus associativity curve is smoother forWayPoint. With WayPoint, performance varies by less than 10% across all configurations. We find that some benchmarks have critical associativities where performance is constant while reducing associativity until a threshold is reached. The benchmarks cg, gjk, march, sobel, and stencil exhibit critical associativi- ties. The reason for this performance cliff is that these benchmarks have working sets that are spread evenly across caches and have a regular access pattern, which means many lines in lower-level caches map to the same bank of the last-level cache and its slice of the directory cache. When the number of lines per cache times the number of caches exceeds the available directory associativity, thrashing occurs, thus greatly reducing performance.
0.0 x 0.2 x 0.4 x 0.6 x 0.8 x 1.0 x 1.2 x
cg dmm gjk heat kmeans march mri sobel stencil
Ru nt im e N orm al iz ed to H W cc O pt i
HWccOpt 512waysWP 256waysWP 128waysWP 64waysWP 32waysWP
(a) With WayPoint
0.0 x 2.0 x 4.0 x 6.0 x 8.0 x 10.0 x
cg dmm gjk heat kmeans march mri sobel stencil
Ru nt im e N orm al iz ed t o H W cc O pt i
HWccOpt 512ways 256ways 128ways
(b) Without WayPoint
Figure 9.6: Runtime of WayPoint-enabled simulations with different associativ- ities with fixed directory size. Results normalized to optimistic hardware cache coherence. Note that we have a one-to-one correspondence between sets andWay- Point lists in (a), resulting in slightly less contention and thus better performance for less-associative on-die caches.
performed to the software to remove conflicts at the directory. A deeper discus- sion of the software optimizations is given in Chapter 7. What is important to note is that without these optimizations, the results for configurations without WayPoint would have much higher runtimes. Some of the configurations had such a high degree of thrashing that simulation became intractable.
9.4.3
Power and Area Estimates
We evaluate the area required for our directory cache architecture with and with- out WayPoint and show the results in Table 9.5. Each of the 32 L3 banks is assigned a disjoint region of the address space, and has a corresponding directory cache bank that handles the same address space region. We use CACTI 6.5 [110] to estimate area and power of the directory caches. We evaluate a high-performance 45 nm process at 1.2 GHz with two-cycle pipelined accesses and separate read and write ports. We assume a total die area of 300 mm2 based on the analysis performed in [42]. We use average activity factors from our timing simulator to estimate dynamic power usage.
The results show that the power and area efficiency of the structures degrades linearly with the degree of associativity; these results motivate a mechanism such as WayPoint, which provides good performance with a low-associativity direc- tory cache. Moreover, we find that directory cache capacity can be doubled at a much lower marginal power and area cost than can associativity—generally less than a 2× area and power increase for every 2× capacity increase. However, we have found that the marginal utility of more than 2048 entries per bank to be small with WayPoint. Note that while Table 9.5 shows the area overhead of a 64-way directory cache to be sizeable (23.4% of chip die area), Figure 9.6(b) indicates that even at a much higher capacity, four times that associativity would
Table 9.5: Power and area estimates for a 2048-entryWayPoint implementation.
Configuration Bank Dimension (mm) Combined Area Combined Power (W) Sets Ways Width Height mm2 Chip Area Static Dynamic Total
512 4 0.567 0.484 8.78 2.93% 1.48 0.15 1.63 256 8 0.396 0.878 11.14 3.71% 1.64 0.23 1.87 128 16 0.324 1.660 17.23 5.74% 1.99 0.43 2.42 64 32 0.305 3.323 32.43 10.8% 2.74 0.75 3.50 32 64 0.341 6.596 71.93 23.4% 4.94 1.44 6.38
be required to approach the performance of aWayPoint, which requires greatly reduced hardware overheads.