当前位置: X-MOL 学术Future Gener. Comput. Syst. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Facilitating external sorting on SMR-based large-scale storage systems
Future Generation Computer Systems ( IF 6.2 ) Pub Date : 2020-11-07 , DOI: 10.1016/j.future.2020.10.032
Chih-Hsuan Chen , Shuo-Han Chen , Yu-Pei Liang , Tseng-Yi Chen , Tsan-sheng Hsu , Hsin-Wen Wei , Wei-Kuan Shih

In the big data era, retaining the capability to process and store the sheer amount of data has become a necessity for data-intensive computing. To meet the requirement of big data processing, the storage-centric computing concept of processing data within storage devices has gained its popularity over the years, because the latency and energy consumed by moving data between host systems and storage devices gradually exceed that of processing data. To process data for data-intensive computing, one of the fundamental data processing technique is external sorting, which is widely used in database management systems (DBMS) and Hadoop framework. On the other hand, to store the ever-increasing volumes of data, shingled magnetic recording (SMR) drives have been proposed to increase the areal density of conventional hard disk drives (HDDs) via overlapping adjacent tracks. The SMR drive is widely regarded as a promising technology for the big data application because SMR drives can boost the capacity of HDDs without significant technology changes. Nevertheless, the overlapped track layout of SMR drive imposes the sequential write constraint on incoming write traffic, thus worsening the efficiency of performing external sorting on SMR drives. Such an observation motivates us to propose an SMR-based External Merge Sort (SMR-EMS) strategy for SMR-based large-scale storage systems with the goals of alleviating the negative impacts of sequential write constraint and enhancing the performance of external sorting on SMR drives via utilizing the concept of storage-centric computing. Experiments were conducted to demonstrate the capability of the proposed strategy on improving the efficiency of external merge sorting on SMR drives.



中文翻译:

促进基于SMR的大型存储系统的外部排序

在大数据时代,保持处理和存储大量数据的能力已成为数据密集型计算的必要条件。为了满足大数据处理的需求,多年来在存储设备内处理数据的以存储为中心的计算概念已逐渐普及,因为在主机系统和存储设备之间移动数据的等待时间和能耗逐渐超过了处理数据的延迟和能耗。 。为了处理用于数据密集型计算的数据,一种基本的数据处理技术是外部排序,该方法广泛用于数据库管理系统(DBMS)和Hadoop框架中。另一方面,要存储不断增长的数据量,提出了一种带盖磁记录(SMR)驱动器,以通过重叠相邻磁道来提高传统硬盘驱动器(HDD)的面密度。SMR驱动器被广泛认为是大数据应用中的一项有前途的技术,因为SMR驱动器可以在不进行重大技术更改的情况下提高HDD的容量。但是,SMR驱动器的重叠磁道布局对传入的写流量施加了顺序写约束,从而降低了对SMR驱动器执行外部排序的效率。这样的观察促使我们提出一个建议。SMR驱动器的重叠磁道布局对传入的写流量施加了顺序写约束,从而降低了对SMR驱动器执行外部排序的效率。这样的观察促使我们提出一个建议。SMR驱动器的重叠磁道布局对传入的写流量施加了顺序写约束,从而降低了对SMR驱动器执行外部排序的效率。这样的观察促使我们提出一个建议。SMR为基础的è xternal中号二哥小号ORT(SMR-EMS)战略与减轻连续写入约束的负面影响,并通过利用概念增强对SMR驱动器外部排序的性能目标,基于SMR-大规模存储系统以存储为中心的计算。进行实验以证明所提出的策略在提高SMR驱动器上的外部合并排序效率方面的能力。

更新日期:2020-11-18
down
wechat
bug