In anticipation of native DBMS architectures based on non-volatile main memory
https://doi.org/10.15514/ISPRAS-2020-32(1)-9
Abstract
Many experts in the field of data management believe that the emergence of non-volatile byte-addressable main memory (NVM) available for practical use will lead to the development of a new type of ultra-high-speed database management systems (DBMS) with single-level data storage (native in-NVM DBMS). However, the number of researchers who are actively engaged in research of architectures of native in-NVM DBMS has not increased in recent years. The most active researchers are PhD students that are not afraid of the risks, which, of course, exist in this new area. The second section of the article discusses the state of the art in NVM hardware. The analysis shows that NVM in the DIMM form factor has already become a reality, and that in the near future we can expect the appearance on the market NVM-DIMMs with the speed of conventional DRAM and endurance close to that of hard drives. The third section is devoted to the review of related works, among which the works of young researchers are the most advanced. In the fourth section, we state and justify that the work performed so far in the field of in-NVM DBMS, did not lead to the emergence of a native architecture. This is hampered by the set of limiting factors analyzed by us. In this regard, in the fifth section, we present a sketch of the native architecture of an in-NVM DBMS, the choice of which is influenced only by the goals of simplicity and efficiency. In conclusion, we summarize the article and argues the need for additional research into many aspects of the native architecture of an in-NVM DBMS.
About the Author
Sergey Dmitrievich Kuznetsov
Ivannikov Institute for System Programming of the Russian Academy of Sciences,
Lomonosov Moscow State University,
Moscow Institute of Physics and Technology (State University),
National Research University, Higher School of Economics,
Plekhanov Russian University of Economics
Russian Federation
Doctor of Technical Sciences, Professor, Chief Researcher at ISP RAS, Professor at the Departments of System Programming of MSU, MIPT, and HSE, Senior Researcher at REU
Russian Federation
Doctor of Technical Sciences, Professor, Chief Researcher at ISP RAS, Professor at the Departments of System Programming of MSU, MIPT, and HSE, Senior Researcher at REU
References
1. Chris Mellor. Escaping the DRAM price trap: Storage Class Memory, what it is and why it matters. Blocks & Files, 2019. Available at: https://blocksandfiles.com/2018/11/28/2019-the-year-of-storage-class-memory/, accessed 02-06-2019
2. Anton Shilov. Samsung Ships First Commercial Embedded MRAM (eMRAM) Product. AnandTech, March 6, 2019. Available at: https://www.anandtech.com/show/14056/samsung-ships-first-commercial-emram-product, accessed 02-06-2019.
3. Lucian Armasu. Intel STT-MRAM Technology Is Ready for Mass Production. Tom's Hardware, February 21, 2019. Available at: https://www.tomshardware.com/news/intel-stt-mram-mass-production,38665.html, accessed 02-06-2019.
4. Joy Arulraj. The Design and Implementation of a Non-Volatile Memory Database Management System. PhD Thesis, Computer Science Department, School of Computer Science, Carnegie Mellon University, 2018, 51 p.
5. Joy Arulraj, Andrew Pavlo. How to Build a Non-Volatile Memory Database Management System. In Proc. of the 2017 ACM International Conference on Management of Data, 2017, pp. 1753-1758.
6. Joy Arulraj, Andrew Pavlo. Non-Volatile Memory Database Management Systems. Synthesis Lectures on Data Management. Morgan & Claypool Publishers, 2019, 173 p.
7. Joy Arulraj. Data Management on Non-Volatile Memory. Award Talk. In Proc. of the 2019 International Conference on Management of Data, 2019, p. 1114.
8. Ismail Oukid. Architectural principles for database systems on storage-class memory. PhD.thesis, Technische Universität Dresden, 2017, 189 p.
9. I. Oukid, D. Booss, A. Lespinasse, W. Lehner, T. Willhalm, and G. Gomes. Memory management techniques for large-scale persistent-main-memory systems. Proceedings of the VLDB Endowment, Vol. 10, Issue 11, 2017, pp. 1166-1177.
10. I. Oukid and W. Lehner. Data structure engineering for byte-addressable non-volatile memory. In Proc. of the 2017 ACM International Conference on Management of Data, 2017, pp. 1759-1764.
11. С.Д. Кузнецов. Новые устройства хранения данных и их влияние на технологию баз данных. Программная инженерия, no. 4. 2018, стр. 147–155 / Sergey D. Kuznetsov. New Storage Devices and the Future of Database Management. Baltic Journal of Modern Computing, Vol. 6, No. 1. 2018, pp. 1-12. DOI: 10.22364/bjmc.2018.6.1.01.
12. Байтоадресуемая энергонезависимая память и СУБД. Презентация Андрея Николаенко на Tarantool Conference 2018 / Byte-addressable non-volatile memory and DBMS. Presentation by Andrey Nikolayenko at the Tarantool Conference 2018. Available at: https://ibs.ru/media/media/baytoadresuemaya-energonezavisimaya-pamyat-i-subd/, accessed 09.06.2019 (in Russian).
13. Sparsh Mittal, and Jeffrey S. Vetter A Survey of Software Techniques for Using Non-Volatile Memories for Storage and Main Memory Systems. IEEE Transactions on Parallel and Distributed Systems, vol. 27, no. 5, 2016, pp. 1537-1550.
14. Micron Technology. Available at: https://www.micron.com/, accessed 18-06-2019.
15. Panasonic and United Microelectronics Corporation Agreed to Develop Mass Production Process for Next Generation ReRAM, Feb 01, 2017. Available at: https://news.panasonic.com/global/press/data/2017/02/en170201-3/en170201-3.html, accessed 18-06-2019.
16. Crossbar. Available at: https://www.crossbar-inc.com/company/about-crossbar/, accessed 18-06-2019.
17. Everspin Technologies Available at: https://www.everspin.com/, accessed 18-06-2019.
18. Crocus Technology. Available at: https://crocus-technology.com, accessed 22-06-2019.
19. Crocus Nano Electronics. Available at: http://crocusnano.com, acccessed 22-06-2019.
20. L. Wei, J. G. Alzate, U. Arslan, J. Brockman, N. Das, K. Fischer, T. Ghani,O. Golonzka, P. Hentges, R. Jahan, P. Jain, B. Lin, M. Meterelliyoz, J. O’Donnell,C. Puls, P. Quintero, T. Sahu, M. Sekhar, A. Vangapaty, C. Wiegand, F. Hamzaoglu. 7Mb STT-MRAM in 22FFL FinFET Technology with 4ns Read Sensing Time at 0.9V Using Write-Verify-Write Scheme and Offset-Cancellation Sensing Technique. In Proc. of the IEEE International Solid-State Circuits Conference (ISSCC), 2019, pp. 214-216.
21. P. Jain, U. Arslan, M. Sekhar, B. C. Lin, L. Wei, T. Sahu, J. Alzate-vinasco,A. Vangapaty, M. Meterelliyoz, N. Strutt, A. B. Chen, P. Hentges, P. A. Quintero,C. Connor, O. Golonzka, K. Fischer, F. Hamzaogl. 3.6Mb 10.1Mb/mm2Embedded Non-Volatile ReRAM Macro in 22nm FinFET Technology with Adaptive Forming/Set/Reset Schemes Yielding Down to 0.5V with Sensing Time of 5ns at 0.7. In Proc. of the IEEE International Solid-State Circuits Conference (ISSCC), 2019, pp. 212-214.
22. Stratis D. Viglas. Data Management in Non-Volatile Memory. In Proc. of the 2015 ACM SIGMOD International Conference on Management of Data, 2015, pp. 1707-1711
23. Naveed Ul Mustafa, Adrià Armejach, Ozcan Ozturk, Adrián Cristal, Osman S. Unsal. Implications of non-volatile memory as primary storage for database management systems. In Proc. of the 2016 International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation (SAMOS), 2016, pp. 164-171
24. Mihnea Andrei, Christian Lemke, Günter Radestock, Robert Schulze, Carsten Thiel, Rolando Blanco, Akanksha Meghlan, Muhammad Sharique, Sebastian Seifert, Surendra Vishnoi, Daniel Booss, Thomas Peh, Ivan Schreter, Werner Thesing, Mehul Wagle, Thomas Willhalm. SAP HANA Adoption of Non-Volatile Memory. Proceedings of the VLDB Endowment, vol. 10, no. 12, 2017, pp. 1754-1765.
25. Alexander van Renen, Viktor Leis, Alfons Kemper, Thomas Neumann, Takushi Hashida, Kazuichi Oe, Yoshiyasu Doi, Lilian Harada, Mitsuru Sato. Managing non-volatile memory in database systems. In Proc. of the 2018 ACM SIGMOD International Conference on Management of Data, 2018, pp. 1541-1555.
26. Joy Arulraj, Andrew Pavlo, Subramanya R. Dulloor. Let’s Talk About Storage & Recovery Methods for Non-Volatile Memory Database Systems. In Proc. of the 2015 ACM SIGMOD International Conference on Management of Data, pp. 707-722
27. Joy Arulraj, Matthew Perron, Andrew Pavlo. Write-Behind Logging. Proceedings of the VLDB Endowment, vol. 10, no. 4, pp. 337-348, 2017.
28. J. Arulraj, J. Levandoski, U. F. Minhas, and P.-A. Larson. BzTree: A high-performance latch-free range index for non-volatile memory. Proceedings of the VLDB Endowment, vol. 11, no. 5, pp. 553-565, 2018.
29. Joy Arulraj, Andrew Pavlo, Krishna Teja Malladi. Multi-Tier Buffer Management and Storage System Design for Non-Volatile Memory. arXiv:1901.10938, 2019.
30. J. Gray, P. McJones, M. Blasgen, B. Lindsay, R. Lorie, T. Price, F. Putzolu, and I. Traiger. The recovery manager of the system R database manager. ACM Computing Surveys, vol. 13, no. 2, 1981, pp. 223-242.
31. Philip A. Bernstein, Vassos Hadzilacos, Nathan Goodman. Concurrency control and recovery in database systems. Addison-Wesley, 1987, 370 p.
32. Ismail Oukid, Daniel Booss, Wolfgang Lehner, Peter Bumbulis, Thomas Willhalm. SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery. In Proc. of the Tenth International Workshop on Data Management on New Hardware (DaMoN'14), 2014.
33. Ismail Oukid, Wolfgang Lehner, Thomas Kissinger, Thomas Willhalm, Peter Bumbulis. Instant Recovery for Main-Memory Databases. In Proc. of the 7th Biennial Conference on Innovative Data Systems Research (CIDR’15), 2015.
34. Ismail Oukid, Johan Lasperas, Anisoara Nica, Thomas Willhalm, Wolfgang Lehner. FPTree: A Hybrid SCM-DRAM Persistent and Concurrent B-Tree for Storage Class Memory. In Proc. of the 2016 ACM SIGMOD International Conference on Management of Data, 2016, pp. 371-386.
35. Ismail Oukid, Wolfgang Lehner. Towards a Single-Level Database Architecture on NVM. Presentation abstract. In Proc. of the 8th Annual Non-Volatile Memories Workshop, 2017.
36. Ismail Oukid. Architectural Principles for Database Systems on Storage-Class Memory. This paper is a summary of the author’s PhD thesis of the same title. Lecture Notes in Informatics (LNI), Gesellschaft für Informatik, Bonn, 2019.
37. Michael Stonebraker, Ugur Cetintemel. «One Size Fits All»: An Idea Whose Time Has Come and Gone. In Proc. of the 21st International Conference on Data Engineering, 2005, pp. 2-11.
38. Douglas Comer. Ubiquitous B-Tree. ACM Computing Surveys, vol, 11, issue 2, 1979, pp. 121-137.
39. Tobin J. Lehman, Michael J. Carey. A Study of Index Structures for Main Memory Database Management Systems. In Proc. of the 1986 ACM SIGMOD international conference on Management of data, 1986, pp. 239-250.
40. Rudolf Bayer. R. Bayer, E. McCreight. Organization and Maintenance of Large Ordered Indexes. Acta Informatica, vol. 1, issue 3, 1972, pp. 173-189.
41. Yingjun Wu, Joy Arulraj, Jiexi Lin, Ran Xian, Andrew Pavlo. An empirical evaluation of in-memory multi-version concurrency control. Proceedings of the VLDB Endowment, vol. 10, issue 7, 2017, pp. 781-792.
42. Kenneth C. Knowlton. A fast storage allocator. Communications of the ACM, vol. 8, issue 10, 1965, pp. 623-624.
43. Xiangyao Yu, George Bezerra, Srinivas Devadas, Michael Stonebraker, Andrew Pavlo. Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores. Proceedings of the VLDB Endowment, vol. 8, issue 3, 2014, pp. 209-220.
44. Кузнецов С.Д. Транзакционные параллельные СУБД: новая волна. Труды ИСП РАН, том 20, 2011, стр. 189-251 / Sergey D. Kuznetsov. Transactional Massive-Parallel DBMSs: A New Wave. Trudy ISP RAS/Proc. ISP RAS, vol. 20, 2011, pp. 189-251 (in Russian).
45. Rachael Harding, Dana Van Aken, Andrew Pavlo, Michael Stonebraker. An Evaluation of Distributed Concurrency Control. Proceedings of the VLDB Endowment, vol. 10, issue 5, 2017, pp. 553-564.
46. Witold Litwin. Linear Hashing: A New Tool for File and Table Addressing. In Proc. of the 6th International Conference on Very Large Data Bases, 1980, pp. 212-223.
47. S. Pelley, T. F. Wenisch, and K. LeFevre. Do query optimizers need to be SSD-aware? In Proc. of the 2nd International Workshop on Accelerating Data Management Systems using Modern Processor and Storage Architecture, 2011.
48. F. Faerber, A. Kemper, P. Å Larson, J. Levandoski, T. Neumann, and A. Pavlo. Main Memory Database Systems. Foundations and Trends in Databases, vol. 8, No. 1-2, 2016, pp. 1–130.
Review
For citations:
Kuznetsov S.D. In anticipation of native DBMS architectures based on non-volatile main memory. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2020;32(1):153-180. (In Russ.) https://doi.org/10.15514/ISPRAS-2020-32(1)-9
Email this article 