(Small image shows, on a logarithmic scale, the access time and cost per gigabyte for DRAM, Flash and a 15K disk. See the ACM article for details.)

In a recent ACM article," Can flash memory become the foundation for a new tier in the storage hierarchy?", Adam Leventhal, one of the three authors of DTrace, discusses the challenge in finding places to use flash memory with the right balance between cost and performance.

Adam says, "Flash should be viewed not as a replacement for existing storage, but rather as a means to enhance it." He then reviews the properties of Flash and Today's Storage Hierarchy. An interesting comparison he makes is:

• 10Tb on 7,200 RPM disks cost about $3k and dissipates about 112 watts
• 10Tb on 15,000 RPM disks cost about $22k and dissipates about 473 watts

A strategy that allows a given performance level to be achieved with smaller, slower, cheaper and cooler disks versus faster, hotter, expensive disks is very appealing on many levels.

It has been suggested that Flash can be used as a Log Device. "To that end you need a device that mimics the important properties of NVRAM (fast, persistent writes), while avoiding the downsides (cost, size, battery power)", says Adam. To overcome the write performance of Flash, he continues, "it's possible to build a flash-based device that can service write operations very quickly by inserting a DRAM write cache and then treating that write cache as nonvolatile by adding a supercapacitor to provide the necessary power to flush outstanding data in the DRAM to flash in the case of power loss."

Many apps, such as databases can easily use and benefit from such log devices. ZFS can also benefit from log devices as a separate intent-log device (a slog in ZFS jargon).

Flash can also be used as a Cache to address read latency. Adam: "Storage systems typically keep a DRAM cache of data the system determines a consumer is likely to access so that it can service read requests from that cache rather than waiting for the disk. In ZFS, this subsystem is called the adaptive replacement cache (ARC)."

Flash is suitable to be a second level ARC - L2ARC in ZFS terms. A small server with room for 128Gb of DRAMs can easily accommodate 768GB or more using flash SSDs in its internal drive bays. The L2ARC can help take full advantage of the flash SSDs.

"Under normal operation," Adam writes, "the L2ARC treats flash as cheap and vast storage. As it writes blocks of data to populate the cache devices, however, the L2ARC includes a directory so that after a power loss, the contents of the cache can be identified, thus pre-warming the cache."

Sun blogger milek, inspired by Adam's article suggests that you could "... buy additional entry-level array like Sun's 2540 and you put 12x143GB SSD drives for L2ARC (and at least one for SLOG if required). This gives you about 1.5TB of cache! You can cache relatively large database here."

In addition he continues, "I like the way L2ARC works - if you unmount ZFS pool and mount it again old L2ARC content will be re-used (thanks to ZFS checksums possible stale data will be detected, skipped and read from disks). What it means is that if you connect your 2540 (or whatever) full of SSD drives to a cluster and you failover your database along with your ZFS pool to another node your 1.5TB of cache will be still warm. So the impact of failover on your database performance can be greatly reduced."

Another insight from Adam, "generally, this new flash tier can be thought of as a radical form of hierarchical storage management (HSM) without the need for explicit management."

More Information

ZFS L2ARC One Pager [...read more...]

		fullsource
Print Email Current Profile Archive Blog Generate newsletter