Program execution speed critically depends on increasing cache hits, as cache hits are orders of magnitude faster than misses. To increase cache hits, we focus on the problem of cache replacement: choosing which cache line to evict upon inserting a new line. This is challenging because it requires planning far ahead and currently there is no known practical solution. As a result, current replacement policies typically resort to heuristics designed for specific common access patterns, which fail on more diverse and complex access patterns. In contrast, we propose an imitation learning approach to automatically learn cache access patterns by leveraging Belady's, an oracle policy that computes the optimal eviction decision given the future cache accesses. While directly applying Belady's is infeasible since the future is unknown, we train a policy conditioned only on past accesses that accurately approximates Belady's even on diverse and complex access patterns, and call this approach Parrot. When evaluated on 13 of the most memory-intensive SPEC applications, Parrot increases cache miss rates by 20% over the current state of the art. In addition, on a large-scale web search benchmark, Parrot increases cache hit rates by 61% over a conventional LRU policy. We release a Gym environment to facilitate research in this area, as data is plentiful, and further advancements can have significant real-world impact.

中文翻译：

---

一种缓存替换的模仿学习方法

程序执行速度主要取决于缓存命中的增加，因为缓存命中比未命中快几个数量级。为了增加缓存命中，我们关注缓存替换问题：在插入新行时选择要驱逐的缓存行。这是具有挑战性的，因为它需要提前规划，目前还没有已知的实用解决方案。因此，当前的替换策略通常采用为特定常见访问模式设计的启发式方法，但在更多样化和更复杂的访问模式上失败。相比之下，我们提出了一种模仿学习方法，通过利用 Belady's 自动学习缓存访问模式，Belady 是一种预言机策略，可根据未来的缓存访问计算最佳驱逐决策。虽然直接应用 Belady's 是不可行的，因为未来是未知的，我们训练一个仅以过去的访问为条件的策略，即使在多样化和复杂的访问模式下也能准确地近似 Belady，并将这种方法称为 Parrot。在对 13 个内存最密集的 SPEC 应用程序进行评估时，Parrot 将缓存未命中率比当前的技术水平提高了 20%。此外，在大规模 Web 搜索基准测试中，Parrot 将缓存命中率比传统 LRU 策略提高了 61%。我们发布了一个 Gym 环境以促进该领域的研究，因为数据丰富，进一步的进步会对现实世界产生重大影响。Parrot 将缓存未命中率提高了 20%。此外，在大规模 Web 搜索基准测试中，Parrot 将缓存命中率比传统 LRU 策略提高了 61%。我们发布了一个 Gym 环境以促进该领域的研究，因为数据丰富，进一步的进步会对现实世界产生重大影响。Parrot 将缓存未命中率提高了 20%。此外，在大规模 Web 搜索基准测试中，Parrot 将缓存命中率比传统 LRU 策略提高了 61%。我们发布了一个 Gym 环境以促进该领域的研究，因为数据丰富，进一步的进步会对现实世界产生重大影响。