1.6.1 DERECHO ANTIGUO

7.1 SUMMARY

This thesis proposes and studies a software-oriented shared cache management approach for CMPs. The new approach proposes to augment two extra fields to TLB and page table entries so that the address of a last-level cache access can be translated in a controlled way. The cache access address is formed by replacing designated bits from the original physical address by the values from the augmented fields. The main memory access still uses the original physical address. Thus it is decoupled from the last-level cache access. The augmented fields introduce moderate increase in the page table storage, but this minor modification to the hardware wins many opportunities. Instead of distributing data blindly like the classic shared cache scheme, the proposed approach allows software components to manipulate data placement in the last-level cache by supplying proper values for the augmented fields in page table entries. Compared to hardware-based cache schemes that manage data at cache block granularity, the proposed approach raises the data placement granularity to page size. By sacrificing control granularity to control flexibility, the proposed approach brings many optimization opportunities for shared last-level caches of CMPs. For instance, cautious data placement can have a profound impact on a program’s performance, quality of service, power consumption, reliability, fault tolerance, and so on. This thesis studies how to utilize this flexible data placement mechanism to improve program performance by extracting critical hint information through off-line trace analysis.

Since the software-oriented shared cache management approach offloads majority of the management task to the off-line trace analysis, the decision making process is no longer restricted by the response time and hardware resources. Sophisticated trace analysis al-

gorithms can be performed to extract the best available information. Especially, it allows different strategies to be applied based on the characteristics of various programs. This study is broadly split into three parts, each of which mainly targets one of three most important types of modern workloads.

For single-threaded programs, the proposed static 2D page coloring scheme derives an optimal page placement schema through off-line analysis of the trade-off between remote cache access latency and the cache miss rate for a collected L2 cache access trace. The generated hints are then used to guide the program execution with the same input set. The results of this oracle scheme indicate that it effectively removes unnecessary remote cache accesses and mitigates cache contention, leading to superior performance for single- threaded programs running on CMPs. Due to the ideal setting, it provides an upper bound for optimization of this kind. A more practical solution, named dynamic 2D page coloring scheme, is proposed. Following the same philosophy, it utilizes online information to make best data placement decisions. However, the initial page placement decisions can be sub- optimal due to the lack of accurate information regarding the future data usage. To recover the impact caused by improper initial page placement decisions, a page migration scheme is also studied. It is used in conjunction with the dynamic 2D page coloring scheme when enabled. The evaluation results show that the dynamic 2D page coloring scheme approaches the performance of the static scheme very well. The additional speedup achieved by the page migration scheme is moderate, but it rescues the performance in case the initial page coloring scheme fails to make proper decisions.

The second type of workloads is latency-oriented multithreaded programs. The funda- mental difference between this type of workloads and single-threaded programs is represented by the data sharing behavior among parallel threads in latency-oriented multithreaded pro- grams. A block of shared data can be possessed by multiple threads simultaneously. Moving a block of shared data closer to one of its sharers can make it apart from other sharers, effec- tively leading to worse performance for the majority of sharers. On the other side, placing a block of shared data in the middle of all sharers may improve overall fairness. However, every sharer can end up being worse off than the optimal access latency. As a result, determining the optimal location for a block of shared data is not a trivial task. This thesis introduces a

pattern recognition algorithm based on the K-means clustering method to identify commonly seen cache access patterns. Generated hints are used to guide data placement for private data and data replication for highly shared data. Through data replication, the described dilemma for shared data placement is largely avoided. The increased cache pressure incurred by data replication is mitigated nicely through two ways: First, data affinity hints direct private data placement to their owners’ local cache slices. The amount of data replication for private data can be reduced significantly, if they are correctly placed. Second, data repli- cation is mostly limited to shared data, which tends to have small footprint. So replication only increases the cache capacity demand moderately. The evaluation results show that the pattern recognition algorithm is effective in identifying data distribution patterns. The de- rived hints guide data placement and replication properly, improving program performance substantially.

The last type of workloads is throughput-oriented multithreaded programs. Due to their unique characteristics, the hint-guided data placement proven to be effective for latency- oriented programs does not work well. Indeed, throughput-oriented programs are very dif- ficult to optimize by using off-line analysis hints, because they tend to have many non- deterministic activities. It is not possible to capture these dynamic characteristics through static off-line analysis. The dynamics of thread creation and termination also makes static data placement futile. The solution proposed in this thesis tries to recognize data that can receive potential reuses in the future. Through controlled replication of those data, data affinity is improved while cache contention is confined. One important observation that lays the foundation for the proposed controlled replication scheme is: the data reuse property of a memory instruction is determined by the algorithm design of a program. If a block of data accessed by a memory instruction shows good temporal locality, then other data blocks accessed by the same memory instruction can exhibit the same reuse behavior. Memory instructions that access data with high reuse rate can be identified through off-line analysis of a profile trace. As a result, the dynamic information regarding data reuse at run-time can be conveyed using static hints. Moreover, this scheme is also applicable to other mul- tithreaded programs. The experimental results show that the evaluated server programs achieve meaningful performance improvement over the victim replication scheme. However,

the improvement for other latency-oriented multithreaded programs was limited with our current implementation of the idea.