With the development of high-performance computing and big data applications, the scale of data transmitted, stored, and processed by high-performance computing cluster systems is increasing explosively. Efficient compression of large-scale data and reducing the space required for data storage and transmission is one of the keys to improving the performance of high-performance computing cluster systems. In this paper, we present SW-LZMA, a parallel design and optimization of LZMA based on the Sunway 26010 heterogeneous many-core processor. Combined with the characteristics of SW26010 processors, we analyse the storage space requirements, memory access characteristics, and hotspot functions of the LZMA algorithm and implement the thread-level parallelism of the LZMA algorithm based on Athread interface. Furthermore, we make a fine-grained layout of LDM address space to achieve DMA double buffer cyclic sliding window algorithm, which optimizes the performance of SW-LZMA. The experimental results show that compared with the serial baseline implementation of LZMA, the parallel LZMA algorithm obtains a maximum speedup ratio of 4.1 times using the Silesia corpus benchmark, while on the large-scale data set, speedup is 5.3 times.

中文翻译：

---

SW-LZMA：基于SW26010众核处理器的LZMA并行实现

随着高性能计算和大数据应用的发展，高性能计算集群系统传输、存储和处理的数据规模呈爆炸式增长。对大规模数据进行高效压缩，减少数据存储和传输所需的空间，是提高高性能计算集群系统性能的关键之一。在本文中，我们介绍了 SW-LZMA，这是一种基于 Sunway 26010 异构众核处理器的 LZMA 的并行设计和优化。结合SW26010处理器的特点，分析了LZMA算法的存储空间需求、内存访问特性、热点功能，并基于Athread接口实现了LZMA算法的线程级并行。此外，我们对 LDM 地址空间进行了细粒度布局，实现了 DMA 双缓冲区循环滑动窗口算法，优化了 SW-LZMA 的性能。实验结果表明，与 LZMA 的串行基线实现相比，并行 LZMA 算法使用 Silesia 语料库基准获得最大加速比为 4.1 倍，而在大规模数据集上，加速比为 5.3 倍。