Skip to main content

Advertisement

SpringerLink
Log in

A Survey of Non-Volatile Main Memory Technologies: State-of-the-Arts, Practices, and Future Directions

  • Survey
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Non-Volatile Main Memories (NVMMs) have recently emerged as a promising technology for future memory systems. Generally, NVMMs have many desirable properties such as high density, byte-addressability, non-volatility, low cost, and energy efficiency, at the expense of high write latency, high write power consumption, and limited write endurance. NVMMs have become a competitive alternative of Dynamic Random Access Memory (DRAM), and will fundamentally change the landscape of memory systems. They bring many research opportunities as well as challenges on system architectural designs, memory management in operating systems (OSes), and programming models for hybrid memory systems. In this article, we first revisit the landscape of emerging NVMM technologies, and then survey the state-of-the-art studies of NVMM technologies. We classify those studies with a taxonomy according to different dimensions such as memory architectures, data persistence, performance improvement, energy saving, and wear leveling. Second, to demonstrate the best practices in building NVMM systems, we introduce our recent work of hybrid memory system designs from the dimensions of architectures, systems, and applications. At last, we present our vision of future research directions of NVMMs and shed some light on design challenges and opportunities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ousterhout J, Gopalan A, Gupta A, Kejriwal A, Lee C, Montazeri B, Ongaro D, Park S J, Qin H, Rosenblum M, Rumble S, Stutsman R, Yang S. The RAMCloud storage system. ACM Transactions on Computer Systems, 2015, 33(3): Article No. 7. https://doi.org/10.1145/2806887.

  2. Zhang H, Chen G, Ooi B C, Tan K L, Zhang M H. In-memory big data management and processing: A survey. IEEE Transactions on Knowledge and Data Engineering, 2015, 27(7): 1920-1948. https://doi.org/10.1109/TKDE.2015.2427795.

    Article  Google Scholar 

  3. Malladi K T, Shaeffer I, Gopalakrishnan L, Lo D, Lee B C, Horowitz M. Rethinking DRAM power modes for energy proportionality. In Proc. the 45th Annual IEEE/ACM International Symposium on Microarchitecture, December 2012, pp.131-142. https://doi.org/10.1109/MICRO.2012.21.

  4. Mutlu O. Memory scaling: A systems architecture perspective. In Proc. the 5th IEEE International Memory Workshop, May 2013, pp.21-25. https://doi.org/10.1109/IMW.2013.6582088.

  5. Nair P J, Kim D H, Qureshi M K. ArchShield: Architectural framework for assisting DRAM scaling by tolerating high error rates. In Proc. the 40th Annual International Symposium on Computer Architecture, June 2013, pp.72-83. https://doi.org/10.1145/2485922.2485929.

  6. Wu X J, Reddy A L N. SCMFS: A file system for storage class memory. In Proc. the International Conference for High Performance Computing, Networking, Storage and Analysis, Nov. 2011, Article No. 39. https://doi.org/10.1145/2063384.2063436.

  7. Ahn J, Yoo S, Choi K. DASCA: Dead write prediction assisted STT-RAM cache architecture. In Proc. the 20th International Symposium on High Performance Computer Architecture, Feb. 2014, pp.25-36. https://doi.org/10.1109/HPCA.2014.6835944.

  8. Wu D, He B, Tang X, Xu J, Guo M. RAMZzz: Rank-aware DRAM power management with dynamic migrations and demotions. In Proc. the 2012 International Conference for High Performance Computing, Networking, Storage and Analysis, Nov. 2012, Article No. 32. https://doi.org/10.1109/SC.2012.99.

  9. Lu Y, Wu D, He B, Tang X, Xu J, Guo M. Rank-aware dynamic migrations and adaptive demotions for DRAM power management. IEEE Transactions on Computers, 2015, 65(1): 187-202. https://doi.org/10.1109/TC.2015.2409847.

    Article  MathSciNet  Google Scholar 

  10. Lu Y, He B, Tang X, Guo M. Synergy of dynamic frequency scaling and demotion on DRAM power management: Models and optimizations. IEEE Transactions on Computers, 2015, 64(8): 2367-2381. https://doi.org/10.1109/TC.2014.2360534.

    Article  MathSciNet  MATH  Google Scholar 

  11. Foong A, Hady F. Storage as fast as rest of the system. In Proc. the 8th IEEE International Memory Workshop, May 2016. https://doi.org/10.1109/IMW.2016.7495289.

  12. Volos H, Tack A J, Swift M M. Mnemosyne: Lightweight persistent memory. In Proc. the 16th International Conference on Architectural Support for Programming Languages and Operating Systems, March 2011, pp.91-104. https://doi.org/10.1145/2248487.1950379.

  13. Venkataraman S, Tolia N, Ranganathan P, Campbell R H. Consistent and durable data structures for non-volatile byte-addressable memory. In Proc. the 9th USENIX Conference on File and Storage Technologies, February 2011, pp.61-75.

  14. Ramos L E, Gorbatov E, Bianchini R. Page placement in hybrid memory systems. In Proc. the 25th International Conference on Supercomputing, May 31–June 4, 2011, pp.85-95. https://doi.org/10.1145/1995896.1995911.

  15. Dhiman G, Ayoub R, Rosing T. PDRAM: A hybrid PRAM and DRAM main memory system. In Proc. the 46th ACM/IEEE Design Automation Conference, July 2009, pp.664-669. https://doi.org/10.1145/1629911.1630086.

  16. Qureshi M K, Srinivasan V, Rivers J A. Scalable high performance main memory system using phase-change memory technology. In Proc. the 36th Annual International Symposium on Computer Architecture, June 2009, pp.24-33. https://doi.org/10.1145/1555754.1555760.

  17. Yoon H, Meza J, Ausavarungnirun R, Harding R A, Mutlu O. Row buffer locality aware caching policies for hybrid memories. In Proc. the 30th IEEE International Conference on Computer Design, Sept. 30–Oct. 3, 2012, pp.337-344. https://doi.org/10.1109/ICCD.2012.6378661.

  18. Liu H, Chen Y, Liao X, Jin H, He B, Zheng L, Guo R. Hardware/software cooperative caching for hybrid DRAM/NVM memory architectures. In Proc. the International Conference on Supercomputing, June 2017, Article No. 26. https://doi.org/10.1145/3079079.3079089.

  19. Condit J, Nightingale E B, Frost C, Ipek E, Lee B, Burger D, Coetzee D. Better I/O through byte-addressable, persistent memory. In Proc. the 22nd ACM SIGOPS Symposium on Operating Systems Principles, October 2009, pp.133-146. https://doi.org/10.1145/1629575.1629589.

  20. Dulloor S R, Kumar S, Keshavamurthy A, Lantz P, Reddy D, Sankaran R, Jackson J. System software for persistent memory. In Proc. the 9th European Conference on Computer Systems, April 2014, Article No. 15. https://doi.org/10.1145/2592798.2592814.

  21. Agarwal N, Wenisch T F. Thermostat: Application-transparent page management for two-tiered main memory. In Proc. the 22nd International Conference on Architectural Support for Programming Languages and Operating Systems, April 2017, pp.631-644. https://doi.org/10.1145/3093315.3037706.

  22. Coburn J, Caulfield A M, Akel A, Grupp L M, Gupta R K, Jhala R, Swanson S. NV-heaps: Making persistent objects fast and safe with next-generation, non-volatile memories. ACM SIGARCH Computer Architecture News, 2011, 39(1): 105-118. https://doi.org/10.1145/1961295.1950380.

    Article  Google Scholar 

  23. Volos H, Tack A J, Swift M M. Mnemosyne: Lightweight persistent memory. ACM SIGPLAN Not., 2011, 47(4): 91-104. https://doi.org/10.1145/1961296.1950379.

    Article  Google Scholar 

  24. Chakrabarti D R, Boehm H J, Bhandari K. Atlas: Leveraging locks for non-volatile memory consistency. ACM SIGPLAN Notices, 2014, 49(10): 433-452. https://doi.org/10.1145/2714064.2660224.

    Article  Google Scholar 

  25. Yang J, Kim J, Hoseinzadeh M, Izraelevitz J, Swanson S. An empirical guide to the behavior and use of scalable persistent memory. In Proc. the 18th USENIX Conference on File and Storage Technologies, February 2020, pp.169-182.

  26. Poremba M, Zhang T, Xie Y. NVMain 2.0: A user-friendly memory simulator to model (non-)volatile memory systems. IEEE Computer Architecture Letters, 2015, 14(2): 140-143. https://doi.org/10.1109/LCA.2015.2402435.

    Article  Google Scholar 

  27. Sánchez D, Kozyrakis C. ZSim: Fast and accurate microarchitectural simulation of thousand-core systems. In Proc. the 40th Annual International Symposium on Computer Architecture, June 2013, pp.475-486. https://doi.org/10.1145/2485922.2485963.

  28. Duan Z, Liu H, Liao X, Jin H. HME: A lightweight emulator for hybrid memory. In Proc. the Design, Automation Test in Europe Conference and Exhibition, March 2018, pp.1375-1380. https://doi.org/10.23919/DATE.2018.8342227.

  29. Volos H, Magalhaes G, Cherkasova L, Li J. Quartz: A lightweight performance emulator for persistent memory software. In Proc. the 16th Annual Middleware Conference, November 2015, pp.37-49. https://doi.org/10.1145/2814576.2814806.

  30. Zhu G, Lu K, Wang X, Zhou X, Shi Z. Building emulation framework for non-volatile memory. IEEE Access, 2017, 5: 21574-21584. https://doi.org/10.1109/ACCESS.2017.2715346.

    Article  Google Scholar 

  31. Zhang W, Li T. Exploring phase change memory and 3D die-stacking for power/thermal friendly, fast and durable memory architectures. In Proc. the 18th International Conference on Parallel Architectures and Compilation Techniques, Sept. 2009, pp.101-112. https://doi.org/10.1109/PACT.2009.30.

  32. Park H, Yoo S, Lee S. Power management of hybrid DRAM/PRAM-based main memory. In Proc. the 48th ACM/EDAC/IEEE Design Automation Conference, June 2011, pp.59-64. https://doi.org/10.1145/2024724.2024738.

  33. Lee S, Bahn H, Noh S H. CLOCK-DWF: A write-history-aware page replacement algorithm for hybrid PCM and DRAM memory architectures. IEEE Transactions on Computers, 2014, 63(9): 2187-2200. https://doi.org/10.1109/TC.2013.98.

    Article  MathSciNet  MATH  Google Scholar 

  34. Salkhordeh R, Asadi H. An operating system level data migration scheme in hybrid DRAM-NVM memory architecture. In Proc. the Design, Automation & Test in Europe Conference & Exhibition, March 2016, pp.936-941. https://doi.org/10.3850/9783981537079_0605.

  35. Khouzani H A, Hosseini F S, Yang C. Segment and conflict aware page allocation and migration in DRAM-PCM hybrid main memory. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2016, 36(9): 1458-1470. https://doi.org/10.1109/TCAD.2016.2615845.

    Article  Google Scholar 

  36. Chen D, Jin H, Liao X, Liu H, Guo R, Liu D. MALRU: Miss-penalty aware LRU-based cache replacement for hybrid memory systems. In Proc. the Design, Automation & Test in Europe Conference & Exhibition, March 2017, pp.1086-1091. https://doi.org/10.23919/DATE.2017.7927151.

  37. Jin H, Chen D, Liu H, Liao X, Guo R, Zhang Y. Miss penalty aware cache replacement for hybrid memory systems. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020, 39(12): 4669-4682. https://doi.org/10.1109/TCAD.2020.2966482.

    Article  Google Scholar 

  38. Mladenov R. An efficient non-volatile main memory using phase change memory. In Proc. the 13th International Conference on Computer Systems and Technologies, June 2012, pp.45-51. https://doi.org/10.1145/2383276.2383284.

  39. Loh G H, Hill M D. Efficiently enabling conventional block sizes for very large die-stacked DRAM caches. In Proc. the 44th Annual IEEE/ACM International Symposium on Microarchitecture, December 2011, pp.454-464. https://doi.org/10.1145/2155620.2155673.

  40. Meza J, Chang J, Yoon H, Mutlu O, Ranganathan P. Enabling efficient and scalable hybrid memories using fine-granularity DRAM cache management. IEEE Computer Architecture Letters, 2012, 11(2): 61-64. https://doi.org/10.1109/LCA.2012.2.

    Article  Google Scholar 

  41. Wang X, Liu H, Liao X, Chen J, Jin H, Zhang Y, Zheng L, He B, Jiang S. Supporting super-pages and lightweight page migration in hybrid memory systems. ACM Transactions on Architecture and Code Optimization, 2019, 16(2): 1-26. https://doi.org/10.1145/3310133.

    Article  Google Scholar 

  42. Wang X, Liu H, Liao X, Jin H, Zhang Y. TLB coalescing for multi-grained page migration in hybrid memory systems. IEEE Access, 2020, 8: 66304-66314. https://doi.org/10.1109/ACCESS.2020.2983065.

    Article  Google Scholar 

  43. Park Y, Park S K, Park K H. Linux kernel support to exploit phase change memory. In Proc. the Linux Symposium, July 2010, pp.217-224.

  44. Wei W, Jiang D, McKee S A, Xiong J, Chen M. Exploiting program semantics to place data in hybrid memory. In Proc. the International Conference on Parallel Architecture and Compilation, October 2015, pp.163-173. https://doi.org/10.1109/PACT.2015.10.

  45. Sha E H M, Chen X Z, Zhuge Q, Shi L, Jiang W. Designing an efficient persistent in-memory file system. In Proc. the IEEE Non-Volatile Memory System and Applications Symposium, Aug. 2015. https://doi.org/10.1109/NVMSA.2015.7304365.

  46. Chen X, Sha E H M, Wang X, Yang C, Jiang W, Zhuge Q. Contour: A process variation aware wear-leveling mechanism for inodes of persistent memory file systems. IEEE Transactions on Computers. https://doi.org/10.1109/TC.2020.3002537.

  47. Xue D, Huang L, Li C, Wu C. Dapper: An adaptive manager for large-capacity persistent memory. IEEE Transactions on Computers, 2019, 68(7): 1019-1034. https://doi.org/10.1109/MC.2020.2964916.

    Article  MathSciNet  MATH  Google Scholar 

  48. Xu J, Swanson S. NOVA: A log-structured file system for hybrid volatile/non-volatile main memories. In Proc. the 14th U SENIX Conference on File and Storage Technologies, February 2016, pp.323-338.

  49. Yang J, Izraelevitz J, Swanson S. Orion: A distributed file system for non-volatile main memories and RDMA-capable networks. In Proc. the 17th USENIX Conference on File and Storage Technologies, February 2019, pp.221-234.

  50. Dong M, Bu H, Yi J, Dong B, Chen H. Performance and protection in the ZoFS user-space NVM file system. In Proc. the 27th ACM Symposium on Operating Systems Principles, October 2019, pp.478-493. https://doi.org/10.1145/3341301.3359637.

  51. Coburn J, Caulfield A M, Akel A, Grupp L M, Gupta R K, Jhala R, Swanson S. NV-heaps: Making persistent objects fast and safe with next-generation, nonvolatile memories. In Proc. the 16th International Conference on Architectural Support for Programming Languages and Operating Systems, March 2011, pp.105-118. https://doi.org/10.1145/1950365.1950380.

  52. Liu R S, Shen D Y, Yang C L, Yu S C, Wang C Y M. NVM Duet: Unified working memory and persistent store architecture. In Proc. the 19th International Conference on Architectural Support for Programming Languages and Operating Systems, March 2014, pp.455-470. https://doi.org/10.1145/2541940.2541957.

  53. Denny J E, Lee S, Vetter J S. NVL-C: Static analysis techniques for efficient, correct programming of nonvolatile main memory systems. In Proc. the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing, May 2016, pp.125-136. https://doi.org/10.1145/2907294.2907303.

  54. Zhang L, Swanson S. Pangolin: A fault-tolerant persistent memory programming library. In Proc. the USENIX Annual Technical Conference, July 2019, pp.897-912.

  55. Krishnan R M, Kim J, Mathew A, Fu X, Demeri A, Min C, Kannan S. Durable transactional memory can scale with timestone. In Proc. the 25th International Conference on Architectural Support for Programming Languages and Operating Systems, March 2020, pp.335-349. https://doi.org/10.1145/3373376.3378483.

  56. Gu J, Yu Q, Wang X, Wang Z, Zang B, Guan H, Chen H. Pisces: A scalable and efficient persistent transactional memory. In Proc. the USENIX Annual Technical Conference, July 2019, pp.913-928.

  57. Wu M, Zhao Z, Li H, Li H, Chen H, Zang B, Guan H. Espresso: Brewing Java for more non-volatility with nonvolatile memory. In Proc. the 33rd International Conference on Architectural Support for Programming Languages and Operating Systems, March 2018, pp.70-83. https://doi.org/10.1145/3173162.3173201.

  58. Seok H, Park Y, Park K W, Park K H. Efficient page caching algorithm with prediction and migration for a hybrid main memory. ACM SIGAPP Applied Computing Review, 2011, 11(4): 38-48. https://doi.org/10.1145/2107756.2107760.

    Article  Google Scholar 

  59. Hirofuchi T, Takano R. RAMinate: Hypervisor-based virtualization for hybrid main memory systems. In Proc. the 7th ACM Symposium on Cloud Computing, October 2016, pp.112-125. https://doi.org/10.1145/2987550.2987570.

  60. Duan Z, Liu H, Liao X, Jin H, Jiang W, Zhang Y. HiNUMA: NUMA-aware data placement and migration in hybrid memory systems. In Proc. the 37th IEEE International Conference on Computer Design, November 2019, pp.367-375. https://doi.org/10.1109/ICCD46524.2019.00058.

  61. Yang D, Liu H, Jin H, Zhang Y. HMvisor: Dynamic hybrid memory management for virtual machines. SCIENCE CHINA Information Sciences. https://doi.org/10.1007/s11432-019-2729-5. (to be published)

  62. Wang Z, Shan S, Cao T, Gu J, Xu Y, Mu S, Xie Y, Jiménez D A. WADE: Writeback-aware dynamic cache management for NVM-based main memory system. ACM Transactions on Architecture and Code Optimization, 2013, 10(4): Article No. 51. https://doi.org/10.1145/2555289.2555307.

  63. Hu J, Xue C J, Zhuge Q, Tseng W, Sha E H M. Towards energy efficient hybrid on-chip scratch pad memory with non-volatile memory. In Proc. the Design, Automation & Test in Europe Conference & Exhibition, March 2011, pp.746-751. https://doi.org/10.1109/DATE.2011.5763127.

  64. Cho S, Lee H. Flip-N-Write: A simple deterministic technique to improve PRAM write performance, energy and endurance. In Proc. the 42nd Annual IEEE/ACM International Symposium on Microarchitecture, December 2009, pp.347-357. https://doi.org/10.1145/1669112.1669157.

  65. Li Y, Li X, Ju L, Jia Z. A three-stage-write scheme with flip-bit for PCM main memory. In Proc. the 20th Asia and South Pacific Design Automation Conference, January 2015, pp.328-333. https://doi.org/10.1109/ASPDAC.2015.7059026.

  66. Li Z, Wang F, Feng D, Hua Y, Tong W, Liu J, Liu X. Tetris write: Exploring more write parallelism considering PCM asymmetries. In Proc. the 45th International Conference on Parallel Processing, August 2016, pp.159-168. https://doi.org/10.1109/ICPP.2016.25.

  67. Palangappa P M, Mohanram K. CompEx: Compression-Expansion coding for energy, latency, and lifetime improvements in MLC/TLC NVM. In Proc. the 2016 IEEE International Symposium on High Performance Computer Architecture, March 2016, pp.90-101. https://doi.org/10.1109/HPCA.2016.7446056.

  68. Chen Y T, Cong J, Huang H, Liu B, Liu C, Potkonjak M, Reinman G. Dynamically reconfigurable hybrid cache: An energy-efficient last-level cache design. In Proc. the Design, Automation & Test in Europe Conference & Exhibition, March 2012, pp.45-50. https://doi.org/10.1109/DATE.2012.6176431.

  69. Liu J, Jaiyen B, Veras R, Mutlu O. RAIDR: Retention-aware intelligent DRAM refresh. In Proc. the 39th Annual International Symposium on Computer Architecture, June 2012, pp.1-12. https://doi.org/10.1109/ISCA.2012.6237001.

  70. Lee D, Kim Y, Seshadri V, Liu J, Subramanian L, Mutlu O. Tiered-latency DRAM: A low latency and low cost DRAM architecture. In Proc. the 19th IEEE International Symposium on High Performance Computer Architecture, Feb. 2013, pp.615-626. https://doi.org/10.1109/HPCA.2013.6522354.

  71. David H, Fallin C, Gorbatov E, Hanebutte U R, Mutlu O. Memory power management via dynamic voltage/frequency scaling. In Proc. the 8th ACM International Conference on Autonomic Computing, June 2011, pp.31-40. https://doi.org/10.1145/1998582.1998590.

  72. Pourshirazi B, Zhu Z. Refree: A refresh-free hybrid DRAM/PCM main memory system. In Proc. the IEEE International Parallel and Distributed Processing Symposium, May 2016, pp.566-575. https://doi.org/10.1109/IPDPS.2016.58.

  73. Hay A, Strauss K, Sherwood T, Loh G H, Burger D. Preventing PCM banks from seizing too much power. In Proc. the 44th Annual IEEE/ACM International Symposium on Microarchitecture, December 2011, pp.186-195.

  74. Zhao J, Li S, Yoon D H, Xie Y, Jouppi N P. Kiln: Closing the performance gap between systems with and without persistence support. In Proc. the 46th Annual IEEE/ACM International Symposium on Microarchitecture, December 2013, pp.421-432. https://doi.org/10.1145/2540708.2540744.

  75. Awad A, Blagodurov S, Solihin Y. Write-aware management of NVM-based memory extensions. In Proc. the International Conference on Supercomputing, June 2016, Article No. 9. https://doi.org/10.1145/2925426.2926284.

  76. Zhang L, Neely B, Franklin D, Strukov D, Xie Y, Chong F T. Mellow writes: Extending lifetime in resistive memories through selective slow write backs. In Proc. the 43rd ACM/IEEE Annual International Symposium on Computer Architecture, June 2016, pp.519-531. https://doi.org/10.1109/ISCA.2016.52.

  77. Qureshi M K, Karidis J, Franceschini M, Srinivasan V, Lastras L, Abali B. Enhancing lifetime and security of PCM-based main memory with start-gap wear leveling. In Proc. the 42nd Annual IEEE/ACM International Symposium on Microarchitecture, Dec. 2009, pp.14-23. https://doi.org/10.1145/1669112.1669117.

  78. Azevedo R, Davis J D, Strauss K, Gopalan P, Manasse M, Yekhanin S. Zombie memory: Extending memory lifetime by reviving dead blocks. In Proc. the 40th Annual International Symposium on Computer Architecture, June 2013, pp.452-463. https://doi.org/10.1145/2485922.2485961.

  79. Ipek E, Condit J, Nightingale E B, Burger D, Moscibroda T. Dynamically replicated memory: Building reliable systems from nanoscale resistive memories. ACM S IGPLAN Notices, 2010, 45(3): 3-14. https://doi.org/10.1145/1735970.1736023.

    Article  Google Scholar 

  80. Jin H, Li Z, Liu H, Liao X, Zhang Y. Hotspot-aware hybrid memory management for in-memory key-value stores. IEEE Transactions on Parallel and Distributed Systems, 2020, 31(4): 779-792. https://doi.org/10.1109/TPDS.2019.2945315.

    Article  Google Scholar 

  81. Zhou J, Shen Y, Li S, Huang L. NVHT: An efficient key-value storage library for non-volatile memory. In Proc. the 3rd IEEE/ACM International Conference on Big Data Computing, Applications and Technologies, December 2016, pp.227-236. https://doi.org/10.1145/3006299.3006318.

  82. Li W, Jiang D, Xiong J, Bao Y. HiLSM: An LSM-based key-value store for hybrid NVM-SSD storage systems. In Proc. the 17th ACM International Conference on Computing Frontiers, May 2020, pp.208-216. https://doi.org/10.1145/3387902.3392621.

  83. Eisenman A, Gardner D, AbdelRahman I, Axboe J, Dong S, Hazelwood K, Petersen C, Cidon A, Katti S. Reducing DRAM footprint with NVM in Facebook. In Proc. the 13th EuroSys Conference, April 2018, Article No. 42. https://doi.org/10.1145/3190508.3190524.

  84. Kim J, Lee S, Vetter J S. PapyrusKV: A high-performance parallel key-value store for distributed NVM architectures. In Proc. the International Conference for High Performance Computing, Networking, Storage and Analysis, November 2017, Article No. 57. https://doi.org/10.1145/3126908.3126943.

  85. Arulraj J. Data management on non-volatile memory. In Proc. the 2019 International Conference on Management of Data, June 2019, p.1114. https://doi.org/10.1145/3299869.3328523.

  86. Lersch L, Hao X, Oukid I, Wang T, Willhalm T. Evaluating persistent memory range indexes. Proc. VLDB Endowment, 2019, 13(4): 574-587. https://doi.org/10.14778/3372716.3372728.

  87. Lu B, Hao X, Wang T, Lo E. Dash: Scalable hashing on persistent memory. Proc. VLDB Endow., 2020, 13(10): 1147-1161. https://doi.org/10.14778/3389133.3389134.

  88. Mahapatra P, Hill M D, Swift M M. Don’t persist all: Efficient persistent data structures. arXiv:1905.13011, 2019. https://arxiv.org/abs/1905.13011, Oct. 2020.

  89. Ni Y, Chen S, Lu Q, Litz H, Zhu P, Miller E L, Wu J. Closing the performance gap between DRAM and PM for in-memory index structures. Technical Report, University of California, 2020. https://www.ssrc.ucsc.edu/media/pubs/329b041d657e2c2225aa68fb33e72ecca157e6df.pdf, Oct. 2020.

  90. Chen Y, Lu Y, Yang F, Wang Q, Wang Y, Shu J. FlatStore: An efficient log-structured key-value storage engine for persistent memory. In Proc. the 25th International Conference on Architectural Support for Programming Languages and Operating Systems, March 2020, pp.1077-1091. https://doi.org/10.1145/3373376.3378515.

  91. Liu W, Liu H, Liao X, Jin H, Zhang Y. NGraph: Parallel graph processing in hybrid memory systems. IEEE Access, 2019, 7: 103517-103529. https://doi.org/10.1109/ACCESS.2019.2931058.

    Article  Google Scholar 

  92. Malicevic J, Dulloor S, Sundaram N, Satish N, Jackson J, Zwaenepoel W. Exploiting NVM in large-scale graph analytics. In Proc. the 3rd Workshop on Interactions of NVM/FLASH with Operating Systems and Workloads, October 2015, Article No. 2. https://doi.org/10.1145/2819001.2819005.

  93. Gill G, Dathathri R, Hoang L, Peri R, Pingali K. Single machine graph analytics on massive datasets using Intel Optane DC persistent memory. Proc. VLDB Endow., 2020, 13(10): 1304-1318. https://doi.org/10.14778/3389133.3389145.

  94. Long Y, She X, Mukhopadhyay S. Design of reliable DNN accelerator with un-reliable ReRAM. In Proc. the Design, Automation & Test in Europe Conference & Exhibition, March 2019, pp.1769-1774. https://doi.org/10.23919/DATE.2019.8715178.

  95. Li S, Xu C, Zou Q, Zhao J, Lu Y, Xie Y. Pinatubo: A processing-in-memory architecture for bulk bit-wise operations in emerging non-volatile memories. In Proc. the 53rd Annual Design Automation Conference, June 2016. https://doi.org/10.1145/2897937.2898064.

  96. Han L, Shen Z, Liu D, Shao Z, Huang H H, Li T. A novel ReRAM-based processing-in-memory architecture for graph traversal. ACM Transactions on Storage, 2018, 14(1): Article No. 9. https://doi.org/10.1145/3177916.

  97. Zilberberg O, Weiss S, Toledo S. Phase-change memory: An architectural perspective. ACM Computing Surveys, 2013, 45(3): Article No. 29. https://doi.org/10.1145/2480741.2480746.

  98. Wu C, Zhang G, Li K. Rethinking computer architectures and software systems for phase-change memory. ACM Journal on Emerging Technologies in Computing Systems, 2016, 12(4): Article No. 33. https://doi.org/10.1145/2893186.

  99. Boukhobza J, Rubini S, Chen R, Shao Z. Emerging NVM: A survey on architectural integration and research challenges. ACM Transactions on Design Automation of Electronic Systems, 2017, 23(2): Article No. 14. https://doi.org/10.1145/3131848.

  100. Mittal S, Vetter J S. A survey of software techniques for using non-volatile memories for storage and main memory systems. IEEE Transactions on Parallel and Distributed Systems, 2016, 27(5): 1537-1550. https://doi.org/10.1109/TPDS.2015.2442980.

    Article  Google Scholar 

  101. Li Y, Ghose S, Choi J, Sun J, Wang H, Mutlu O. Utility-based hybrid memory management. In Proc. the IEEE International Conference on Cluster Computing, Sept. 2017, pp.152-165. https://doi.org/10.1109/CLUSTER.2017.130.

  102. Gandhi J, Basu A, Hill M D, Swift M M. BadgerTrap: A tool to instrument x86-64 TLB misses. ACM SIGARCH Computer Architecture News, 2014, 42(2): 20-23. https://doi.org/10.1145/2669594.2669599.

    Article  Google Scholar 

  103. Chen C, Hsiu P, Kuo T, Yang C, Wang C. Age-based PCM wear leveling with nearly zero search cost. In Proc. the 49th Annual Design Automation Conference, June 2012, pp.453-458. https://doi.org/10.1145/2228360.2228439.

  104. Gao S, He B, Xu J. Real-time in-memory checkpointing for future hybrid memory systems. In Proc. the 29th ACM on International Conference on Supercomputing, June 2015, pp.263-272. https://doi.org/10.1145/2751205.2751212.

  105. Islam N S, Rahman M, Lu X, Panda D K. High performance design for HDFS with byte-addressability of NVM and RDMA. In Proc. the International Conference on Supercomputing, June 2016, Article No. 8. https://doi.org/10.1145/2925426.2926290.

  106. Knyaginin D, Gaydadjiev G N, Stenström P. Crystal: A design-time resource partitioning method for hybrid main memory. In Proc. the 43rd International Conference on Parallel Processing, September 2014, pp.90-100. https://doi.org/10.1109/ICPP.2014.18.

  107. Yang J, Wei Q, Wang C, Chen C, Yong K, He B. NV-tree: A consistent and workload-adaptive tree structure for nonvolatile memory. IEEE Transactions on Computers, 2016, 65(7): 2169-2183. https://doi.org/10.1109/TC.2015.2479621.

    Article  MathSciNet  MATH  Google Scholar 

  108. Arulraj J, Levandoski J, Minhas U, Larson P. Bztree: A high-performance latch-free range index for non-volatile memory. Proc. VLDB Endow., 2018, 11(5): 553-565. https://doi.org/10.1145/3164135.3164147.

    Article  Google Scholar 

  109. Ma K, Zheng Y, Li S, Swaminathan K, Li X, Liu Y, Sampson J, Xie Y, Narayanan V. Architecture exploration for ambient energy harvesting nonvolatile processors. In Proc. the 21st IEEE International Symposium on High Performance Computer Architecture, Feb. 2015, pp.526-537. https://doi.org/10.1109/HPCA.2015.7056060.

  110. Zhu T, Hu H, Qian W, Zhou H, Zhou A. Fault-tolerant precise data access on distributed log-structured merge-tree. Frontiers of Computer Science, 2019, 13(4): 760-777. https://doi.org/10.1007/s11704-018-7198-6.

    Article  Google Scholar 

  111. Doshi K, Giles E, Varman P. Atomic persistence for SCM with a non-intrusive backend controller. In Proc. the IEEE International Symposium on High Performance Computer Architecture, March 2016, pp.77-89. https://doi.org/10.1109/HPCA.2016.7446055.

  112. Olson M A, Bostic K, Seltzer M I. Berkeley DB. In Proc. the USENIX Annual Technical Conference, June 1999, pp.183-191.

  113. Sears R, Brewer E. Stasis: Flexible transactional storage. In Proc. the 7th Symposium on Operating Systems Design and Implementation, November 2006, pp.29-44.

  114. Izraelevitz J, Yang J, Zhang L et al. Basic performance measurements of the Intel Optane DC persistent memory module. arXiv:1903.05714, 2019. https://arxiv.org/abs/1903.05714, Oct. 2020.

  115. Patil O, Ionkov L, Lee J, Mueller F, Lang M. Performance characterization of a DRAM-NVM hybrid memory architecture for HPC applications using Intel Optane DC persistent memory modules. In Proc. the International Symposium on Memory Systems, September 2019, pp.288-303. https://doi.org/10.1145/3357526.3357541.

  116. Weiland M, Brunst H, Quintino T et al. An early evaluation of Intel’s Optane DC persistent memory module and its impact on high-performance scientific applications. In Proc. the International Conference for High Performance Computing, Networking, Storage and Analysis, November 2019, Article No. 76. https://doi.org/10.1145/3295500.3356159.

  117. Peng I B, Gokhale M B, Green E W. System evaluation of the Intel Optane byte-addressable NVM. In Proc. the International Symposium on Memory Systems, September 2019, pp.304-315. https://doi.org/10.1145/3357526.3357568.

  118. Zhou Y, Philbin J, Li K. The multi-queue replacement algorithm for second level buffer caches. In Proc. the 2001 USENIX Annual Technical Conference, June 2001, pp.91-104.

  119. Liu H, Ye Y, Liao X, Jin H, Zhang Y, Jiang W, He B. Space-oblivious compression and wear leveling for non-volatile main memories. In Proc. the 36th International Conference on Massive Storage Systems and Technology, October 2020.

  120. Liu H, Liu R, Liao X, Jin H, He B, Zhang Y. Object-level memory allocation and migration in hybrid memory systems. IEEE Transactions on Computers, 2020, 69(9): 1401-1413. https://doi.org/10.1109/TC.2020.2973134.

    Article  MATH  Google Scholar 

  121. Hassan A, Vandierendonck H, Nikolopoulos D S. Software-managed energy-efficient hybrid DRAM/NVM main memory. In Proc. the 12th ACM International Conference on Computing Frontiers, May 2015, Article No. 23. https://doi.org/10.1145/2742854.2742886.

  122. Hu X, Wang X, Li Y, Zhou L, Luo Y, Ding C, Jiang S, Wang Z. LAMA: Optimized locality-aware memory allocation for key-value cache. In Proc. the USENIX Annual Technical Conference, July 2015, pp.57-69.

  123. Shun J, Blelloch G E. Ligra: A lightweight graph processing framework for shared memory. ACM SIGPLAN Not., 2013, 48(8): 135-146. https://doi.org/10.1145/2442516.2442530.

    Article  Google Scholar 

  124. Yu S, Chen P. Emerging memory technologies: Recent trends and prospects. IEEE Solid-State Circuits Magazine, 2016, 8(2): 43-56. https://doi.org/10.1109/MSSC.2016.2546199.

    Article  Google Scholar 

  125. Lu Y, Shu J, Chen Y, Li T. Octopus: An RDMA-enabled distributed persistent memory file system. In Proc. the USENIX Annual Technical Conference, July 2017, pp.773-785.

  126. Shan Y, Tsai S, Zhang Y. Distributed shared persistent memory. In Proc. the 2017 Symposium on Cloud Computing, September 2017, pp.323-337. https://doi.org/10.1145/3127479.3128610.

  127. Chen T, Liu H, Liao X, Ji H. Resource abstraction and data placement for distributed hybrid memory pool. Frontiers of Computer Science, 2020. (accepted)

  128. Barbalace A, Iliopoulos A, Rauchfuss H, Brasche G. It’s time to think about an operating system for near data processing architectures. In Proc. the 16th Workshop on Hot Topics in Operating Systems, May 2017, pp.56-61. https://doi.org/10.1145/3102980.3102990.

  129. Gu B, Yoon A S, Bae D et al. Biscuit: A framework for near-data processing of big data workloads. In Proc. the 43rd International Symposium on Computer Architecture, June 2016, pp.153-165. https://doi.org/10.1109/ISCA.2016.23.

  130. Yan H, Ahn E C, Duan L. Enabling NVM-based deep learning acceleration using nonuniform data quantization: Work-in-progress. In Proc. the 2017 International Conference on Compilers, Architectures and Synthesis for Embedded Systems Companion, October 2017, Article No. 20. https://doi.org/10.1145/3125501.3125516.

  131. Chen A. Utilizing the variability of resistive random access memory to implement reconfigurable physical unclonable functions. IEEE Electron Device Letters, 2015, 36(2): 138-140. https://doi.org/10.1109/LED.2014.2385870.

    Article  Google Scholar 

  132. Torres L, Brum R, Cargnini L, Sassatelli G. Trends on the application of emerging non-volatile memory to processors and programmable devices. In Proc. the IEEE International Symposium on Circuits and Systems, May 2013, pp.101-104. https://doi.org/10.1109/ISCAS.2013.6571792.

  133. Liauw Y, Zhang Z, Kim W, Gamal A E, Wong S S. Nonvolatile 3D-FPGA with monolithically stacked RRAM-based configuration memory. In Proc. the 2012 IEEE International Solid-State Circuits Conference, Feb. 2012, pp.406-408. https://doi.org/10.1109/ISSCC.2012.6177067.

  134. Zhao W, Belhaire E, Chappert C, Mazoyer P. Spin transfer torque (STT)-MRAM-based runtime reconfiguration FPGA circuit. ACM Trans. Embed. Comput. Syst., 2009, 9(2): Article No. 14. https://doi.org/10.1145/1596543.1596548.

  135. Chen Y, Wong W, Li H, Koh C, Zhang Y, Wen W. On-chip caches built on multilevel spin-transfer torque RAM cells and its optimizations. ACM Journal on Emerging Technologies in Computing Systems, 2013, 9(2): Article No. 16. https://doi.org/10.1145/2463585.2463592.

Download references

Author information

Authors and Affiliations

  1. National Engineering Research Center for Big Data Technology and System, Huazhong University of Science and Technology, Wuhan, 430074, China

    Hai-Kun Liu, Di Chen, Hai Jin, Xiao-Fei Liao, Kan Hu & Yu Zhang

  2. Services Computing Technology and System Laboratory, Huazhong University of Science and Technology, Wuhan, 430074, China

    Hai-Kun Liu, Di Chen, Hai Jin, Xiao-Fei Liao, Kan Hu & Yu Zhang

  3. Cluster and Grid Computing Laboratory, Huazhong University of Science and Technology, Wuhan, 430074, China

    Hai-Kun Liu, Di Chen, Hai Jin, Xiao-Fei Liao, Kan Hu & Yu Zhang

  4. School of Computing Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Hai-Kun Liu, Di Chen, Hai Jin, Xiao-Fei Liao, Kan Hu & Yu Zhang

  5. School of Computing, National University of Singapore, Singapore, 117418, Singapore

    Binsheng He

Authors
  1. Hai-Kun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Di Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hai Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao-Fei Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Binsheng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kan Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hai-Kun Liu.

Supplementary Information

ESM 1

(PDF 128 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, HK., Chen, D., Jin, H. et al. A Survey of Non-Volatile Main Memory Technologies: State-of-the-Arts, Practices, and Future Directions. J. Comput. Sci. Technol. 36, 4–32 (2021). https://doi.org/10.1007/s11390-020-0780-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-020-0780-z

Keywords

Search

Navigation