DESIGN CHALLENGES ON ENTERPRISE-SCALE STORAGE SYSTEMS EMPLOYING HARD DRIVES AND NAND FLASH BASED SOLID-STATE DRIVES
Open Access
- Author:
- Kim, Youngjae
- Graduate Program:
- Computer Science and Engineering
- Degree:
- Doctor of Philosophy
- Document Type:
- Dissertation
- Date of Defense:
- August 17, 2009
- Committee Members:
- Dr Anand Sivasubramaniam And Dr Bhuvan Urgaonkar, Dissertation Advisor/Co-Advisor
Anand Sivasubramaniam, Committee Chair/Co-Chair
Bhuvan Urgaonkar, Committee Chair/Co-Chair
Prasenjit Mitra, Committee Member
Qian Wang, Committee Member
Raj Acharya, Committee Member - Keywords:
- Storage Systems
Flash Memory - Abstract:
- Flash memory overcomes some key shortcomings of hard disk drives (HDDs), including faster access to non-sequential data and lower power consumption. Economic forces, driven by the desire to introduce flash into the enterprise market without changing existing software based, have resulted in the emergence of solid state drives (SSDs), flash packaged in HDD from factors and capable of working with device drivers and I/O buses designed for HDDs. Unlike the use of DRAM for caching or buffering, however, certain idiosyncrasies of SSDs make their integration into HDD-based systems non-trivial. Flash memory suffers from limits on its reliability, in an order of magnitude more expensive than the disk, and can be sometimes even slower than the HDD (due to excessive GC induced by high intensity of random writes). Given the complementary properties of HDDs and SDDs in terms of cost, performance, and lifetime, the current consensus among several storage experts is to view SSDs not as a replacement for HDD but rather as a complementary device within the storage hierarchy. In my dissertation, I designed and evaluated such a hybrid system called HybridStore to provide (a) improved capacity planning techniques to administrators with the overall goal of operating within cost-budgets and (b) improved performance/lifetime guarantees during episodes of deviations from expected workloads through several novel mechanisms such as fragmentation busting and write regulation. As an illustrative example of HybridStore’s efficacy, a combination of 1 SSD and 6 low-speed, cheaper and higher capacity HDDs is recommended the most cost-effective storage configuration in HybridStore for a predominantly random-write dominant I/O trace from an OLTP application running at a large financial institution. Also, HybridStore employing HDD with small SSD is able to reduce the average response time for Financial trace by about 71% as compared to a HDD-based system. In addition to HybridStore project, I developed a novel design technique of the Flash Translation Layer (FTL) in the SSD. It provides improved performance, reduced garbage collection overhead, and better overloaded behavior compared to state-of- the art FTL schemes. For example, the Financial trace shows a 78% improvement in average response time (due to a 3-fold reduction in operations of the garbage collector), compared to a state-of-the-art FTL scheme. Finally, I also developed and validated flash simulation framework call FlashSim. While a number of well-regarded simulation environments exist for HDDs, the same is not yet true for SSDs. This is due to SSDs having been in the storage market for relatively less time as well as the lack of information (hardware configuration and software methods) about state-of-the-art SSDs that is publicly available. FlashSim aimed at filling this void in performance evaluation of emerging storage systems that employ SSDs.