Building a large-scale share flash infrastructure

Building a large-scale share flash infrastructure 1

RSI, conceptually, blows server components – CPU, GPU, storage, memory, network – across separate racks, enabling each to be sized and upgraded in full rack boxes, connected by high-capacity/low latency, PCIe links. Layer on top of that virtual server software, so virtual servers can be constructed from pieces of the rack-scale components, and you have a game-changing config that makes enterprise infrastructure competitive with cloud services.

But – there’s always a but – using non-volatile memory and storage over a network typically means unacceptable latency. That’s where a Stanford’s team ReFlex – a software storage server – comes in.


There are several tradeoffs in making remote flash access acceptable. The biggest is low latency, but there’s also high throughput – saturating an NVMe device with as few CPU cores as possible.

Managing multi-tenancy in a shared flash pool requires isolation, so applications aren’t stepping on each others toes. And it’s desirable to be highly flexible in sharing the flash, as well as other deployment issues, such as scale and network protocols.

In testing, the Stanford team found that ReFlex achieved remote flash performance equivalent to local flash accesses over 10Gb Ethernet using TCP/IP. They explain

ReFlex achieves high performance with limited compute requirements using a novel dataplane kernel that tightly integrates networking and storage. The dataplane design avoids the overhead of interrupts and data copying, optimizes for locality, and strikes a balance between high throughput (IOPS) and low tail latency.


The big problem with multi-tenancy on flash devices is the huge difference between read and write performance. Writes can take many milliseconds, while reads are sub-millisecond affairs. This means an app that does a lot of writing, say metadata updates or streaming video, uses a lot of an NVMe device’s resources.

The Stanford team implemented a QoS scheduler with global visibility into the entire workload across all tenants. The maximum IOPS depends on the read/write ratio of all requests. The scheduler looks at each workload’s service level objective (SLO), prioritizes latency critical apps over best effort apps, and ensures that SLOs for apps are met.

As a result of this and other optimizations, ReFlex is capable of serving up to 850K IOPS per core while only adding 21µs of latency over direct access to local flash. That’s remarkably good.

The Storage Bits take

Intel’s visionaries have been promoting the RSI concept for years, but it looks like 2018 will be the year that all the necessary pieces – PCIe v4 in particular – will come together to make it technically and economically feasible. With the crash in flash prices, racks full of flash are more affordable than they’ve ever been, even at multi-hundred TB scale.

This is very good news for our data-intensive future. It will be interesting to see if any of the enterprise storage vendors productize something like ReFlex.

Courteous comments welcome, of course. The paper ReFlex: Remote Flash ≈ Local Flash won a Most Memorable paper award at NVMW 19.

About the author

E-Crypto News was developed to assist all cryptocurrency investors in developing profitable cryptocurrency portfolios through the provision of timely and much-needed information. Investments in cryptocurrency require a level of detail, sensitivity, and accuracy that isn’t required in any other market and as such, we’ve developed our databases to help fill in information gaps.

Related Posts

E-Crypto News Executive Interviews

Crypto Scams

Beanstalk Farms Loses $80M In A Massive DeFi Governance Flash-Loan Hack
Beanstalk Farms Loses $80M In A Massive DeFi Governance Flash-Loan Hack
April 23, 2022
Joon Pak Head of Crypto at Prove talks to Us about Crypto Fraud And More
April 11, 2022
Mintable CEO Zach Burks Talks to Us about the Opensea Stolen NFTs and Their Recovery
March 21, 2022
Crypto Crime
Crypto Crime Surges To Record Highs As Thieves Follow Market Buzz – Chainalysis 2022 Report
February 24, 2022
Bots Circumvent 2FA Login At Coinbase And Other Crypto Exchanges In 2022
Bots Have Circumvented 2FA Logins At Coinbase And Other Crypto Exchanges In 2022
February 17, 2022

Automated trading with HaasBot Crypto Trading Bots

Blockchain/Cryptocurrency Questions and Answers

Roundtable Interview-What is the Effect of The Russia-Ukraine War on Cryptocurrency Prices?
March 4, 2022
How Does Bitcoin Casino Work + 2021 Beginner’s Guide
November 8, 2021
How to Buy and Sell Cryptocurrency
November 8, 2021
What Are Bitcoin Futures And How Will They Work In 2022?
November 4, 2021
The Unconventional Guide to Ethereum
October 28, 2021

CryptoCurrencyUSDChange 1hChange 24hChange 7d
Bitcoin28,864 0.24 % 1.98 % 5.00 %
Ethereum1,766.3 0.02 % 5.56 % 12.72 %
Tether1.001 0.14 % 0.02 % 0.02 %
USD Coin0.9989 0.16 % 0.08 % 0.04 %
BNB302.75 0.24 % 3.39 % 1.59 %
XRP0.3889 0.35 % 1.83 % 7.77 %
Binance USD1.001 0.01 % 0.23 % 0.22 %
Cardano0.9566 0.22 % 0.68 % 6.96 %
Solana42.15 0.09 % 7.02 % 19.46 %
Dogecoin0.08371 7.52 % 5.01 % 3.87 %

Bitcoin (BTC) $ 28,881.00
Ethereum (ETH) $ 1,770.43
Tether (USDT) $ 1.00
USD Coin (USDC) $ 1.00
BNB (BNB) $ 302.95
XRP (XRP) $ 0.390019
Binance USD (BUSD) $ 1.00
Cardano (ADA) $ 0.461806
Solana (SOL) $ 42.38
Dogecoin (DOGE) $ 0.084133