Flash storage disaggregation
Ana Klimovic,Christos Kozyrakis,Eno Thereska,Binu John,Sanjeev Kumar +4 more
- 18 Apr 2016
- pp 29
TL;DR: It is shown that Flash disaggregation allows scaling CPU and Flash resources independently in a cost effective manner through resource-efficient scale-out and is used to draw conclusions about data and control plane issues in remote storage.
read more
Abstract: PCIe-based Flash is commonly deployed to provide datacenter applications with high IO rates. However, its capacity and bandwidth are often underutilized as it is difficult to design servers with the right balance of CPU, memory and Flash resources over time and for multiple applications. This work examines Flash disaggregation as a way to deal with Flash overprovisioning. We tune remote access to Flash over commodity networks and analyze its impact on workloads sampled from real datacenter applications. We show that, while remote Flash access introduces a 20% throughput drop at the application level, disaggregation allows us to make up for these overheads through resource-efficient scale-out. Hence, we show that Flash disaggregation allows scaling CPU and Flash resources independently in a cost effective manner. We use our analysis to draw conclusions about data and control plane issues in remote storage.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
LegoOS: a disseminated, distributed OS for hardware resource disaggregation
Yizhou Shan,Yutong Huang,Yilun Chen,Yiying Zhang +3 more
- 08 Oct 2018
TL;DR: LegionOS as discussed by the authors is a new OS designed for hardware resource disaggregation, which appears to users as a set of distributed servers and can span multiple processor, memory, and storage hardware components.
Swift: Delay is Simple and Effective for Congestion Control in the Datacenter
Gautam Kumar,Nandita Dukkipati,Keon Jang,Hassan M. G. Wassel,Xian Wu,Behnam Montazeri,Yaogong Wang,Kevin Springborn,Christopher Alfeld,Michael C. Ryan,David Wetherall,Amin Vahdat +11 more
- 30 Jul 2020
TL;DR: In large-scale testbed experiments, Swift delivers a tail latency of <50μs for short RPCs, with near-zero packet drops, while sustaining ~100Gbps throughput per server, while providing high throughput for long RPCs.
Cirrus: a Serverless Framework for End-to-end ML Workflows
Joao Carreira,Pedro Fonseca,Alexey Tumanov,Andrew Zhang,Randy H. Katz +4 more
- 20 Nov 2019
TL;DR: This work proposes Cirrus---an ML framework that automates the end-to-end management of datacenter resources for ML workflows by efficiently taking advantage of serverless infrastructures and shows that Cirrus outperforms frameworks specialized along a single dimension.
221
KVell: the design and implementation of a fast persistent key-value store
Baptiste Lepers,Oana Balmau,Karan Gupta,Willy Zwaenepoel +3 more
- 27 Oct 2019
TL;DR: KVell, the first persistent KV able to utilize modern NVMe SSDs at maximum bandwidth, is implemented and compared against available state-of-the-art LSM and B tree KVs, both with synthetic benchmarks and production workloads.
145
ReFlex: Remote Flash ≈ Local Flash
Ana Klimovic,Heiner Litz,Christos Kozyrakis +2 more
- 04 Apr 2017
TL;DR: ReFlex is presented, a software-based system for remote Flash access, that provides nearly identical performance to accessing local Flash and uses a dataplane kernel to closely integrate networking and storage processing to achieve low latency and high throughput at low resource requirements.
140
References
The Google file system
Sanjay Ghemawat,Howard Gobioff,Shun-Tak Albert Leung +2 more
- 19 Oct 2003
TL;DR: This paper presents file system interface extensions designed to support distributed applications, discusses many aspects of the design, and reports measurements from both micro-benchmarks and real world use.
The Hadoop Distributed File System
Konstantin Shvachko,Hairong Kuang,Sanjay Radia,Robert J. Chansler +3 more
- 03 May 2010
TL;DR: The architecture of HDFS is described and experience using HDFS to manage 25 petabytes of enterprise data at Yahoo! is reported on.
•Book
The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines
Luiz Andre Barroso,Urs Hoelzle +1 more
- 01 Jan 2008
TL;DR: The architecture of WSCs is described, the main factors influencing their design, operation, and cost structure, and the characteristics of their software base are described.
Mesos: a platform for fine-grained resource sharing in the data center
Benjamin Hindman,Andy Konwinski,Matei Zaharia,Ali Ghodsi,Anthony D. Joseph,Randy H. Katz,Scott Shenker,Ion Stoica +7 more
- 30 Mar 2011
TL;DR: The results show that Mesos can achieve near-optimal data locality when sharing the cluster among diverse frameworks, can scale to 50,000 (emulated) nodes, and is resilient to failures.
The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines
Luiz Andre Barroso,U. Hölzle +1 more
TL;DR: The architecture of WSCs is described, the main factors influencing their design, operation, and cost structure, and the characteristics of their software base are described.
1.4K
Related Papers (5)
Ana Klimovic,Heiner Litz,Christos Kozyrakis +2 more
- 04 Apr 2017
Aleksandar Dragojevic,Dushyanth Narayanan,Orion Hodson,Miguel Castro +3 more
- 02 Apr 2014