Caches and algorithms

TL;DR: This thesis focuses on demonstrating the potential performance gains of cache-conscious design, and introduces collective analysis, a framework within which cache performance can be predicted as a function of both cache and algorithm configuration.

Abstract: This thesis investigates the design and analysis of algorithms in the presence of caching. Since the introduction of caches, miss penalties have been steadily increasing relative to cycle times and have grown to the point where good performance cannot be achieved without good cache performance. Unfortunately, many fundamental algorithms were developed without considering caching. Worse still, most new algorithms being written do not take cache performance into account. Despite the complexity that caching adds to the programming and performance models, cache miss penalties have grown to the point that algorithm designers can no longer ignore the interaction between caches and algorithms. To show the importance of this paradigm shift, this thesis focuses on demonstrating the potential performance gains of cache-conscious design. Efficient implementations of classic searching and sorting algorithms are examined for inefficiencies in their memory behavior, and simple memory optimizations are applied to them. The performance results demonstrate that these memory optimizations significantly reduce cache misses and improve overall performance. Reductions in cache misses range from 40% to 90%, and although these reductions come with an increase in instruction count, they translate into execution time speedups of up to a factor of two. Since cache-conscious algorithm design is uncommon, it is not surprising that there is a lack of analytical tools to help algorithm designers understand the memory behavior of algorithms. This thesis also investigates techniques for analyzing the cache performance of algorithms. To explore the feasibility of a purely analytical technique, this thesis introduces collective analysis, a framework within which cache performance can be predicted as a function of both cache and algorithm configuration.