Cache performance analysis of algorithms

TL;DR: The cache performance analysis of algorithms whose primary goal is to determine the number of cache hits and cache misses that an algorithm incurs is developed, view an algorithm as a combination of basic memory access patterns, analyze each pattern's cache performance, and apply the analysis to accurately predict the cache performance of the algorithm.

Abstract: How effectively a program uses the memory hierarchy of a computer system can have a tremendous impact on its performance. The failure of a program to access data in the cache memory, hence the need to access that data in main memory, has a cost of up to 100 cycles on today's systems, a penalty that is expected to rise with future systems. As a result much recent algorithm design research has been conscious of the cache, and has focused on its effective use. Understanding the behavior of the cache when developing algorithms is crucial to such research. We develop the cache performance analysis of algorithms whose primary goal is to determine the number of cache hits and cache misses that an algorithm incurs. We view an algorithm as a combination of basic memory access patterns, analyze each pattern's cache performance, and apply the analysis to accurately predict the cache performance of the algorithm. Our analysis is precise: we pay attention to constant factors and quantify performance for even moderate data sizes. Our caching model is realistic: we determine the performance of our access pattern models for direct-mapped caches and caches with a limited degree of set-associativity. For certain patterns that arise in accessing common data structures, we demonstrate and prove that set-associativity yields little or no benefit over direct-mapped caching.