AMD Launches NVMe RAID Support For Ryzen Threadripper Platformby Billy Tallis on October 2, 2017 9:00 AM EST
A week later than originally planned, AMD is launching NVMe RAID support for their Ryzen Threadripper platform. Grab the Windows 10 drivers and release notes here. This updated RAID capability should work with any X399/Threadripper motherboard and any NVMe SSDs, enabling RAID modes 0, 1 and 10 for up to 10 NVMe SSD connected to the CPU's PCIe lanes, but a motherboard firmware update may be required first.
AMD's Ryzen Threadripper launched with support for RAID modes 0, 1 and 10 for arrays of SATA drives connected through the X399 chipset's eight SATA ports. The X399 chipset doesn't offer RAID capability for NVMe SSDs, but given its limited PCIe lane counts, this isn't much of an inconvenience. The Ryzen Threadripper processors themselves have plenty of PCIe lanes to go around. In a typical configuration, the motherboard will have up to three M.2 slots providing PCIe x4 connections to the Threadripper CPU (leaving 48 PCIe lanes for GPUs and four lanes for the connection to the X399 chipset). These PCIe x4 ports can also operate as SATA ports to support M.2 SATA SSDs. However, since these ports are provided by the CPU and not the chipset, they're not covered by the RAID capabilities of the chipset.
To allow for RAID arrays of NVMe SSDs, the Ryzen Threadripper platform needed another RAID implementation. Nobody has manufactured a true NVMe hardware RAID controller—one that completely abstracts the individual drives in the array and presents a single NVMe device to the host. Like all previous NVMe RAID implementations and almost all motherboard RAID implementations for SATA drives, AMD's NVMe RAID for Ryzen Threadripper is at heart a software RAID system. That means it needs new drivers for the OS, and matching support in the motherboard's UEFI to enable booting from a RAID array.
Chipset RAID implementations for SATA typically rely on switching the SATA controller out of AHCI mode and into a proprietary RAID mode. This ensures that the operating system's standard storage drivers won't be able to access the drives, so there's no danger of the array being accidentally corrupted by an OS that isn't aware the drive is part of an array. When Intel added NVMe RAID to their consumer platform with the Sunrise Point 100-series chipsets accompanying Skylake processors, they included a similar mechanism that hides NVMe devices that are connected through the chipset and instead makes them accessible through non-standard interfaces on the SATA controller.
For Intel's X299 HEDT platform for Skylake-X, the same chipset RAID capabilities are available, but the more interesting feature is Intel's Virtual RAID on CPU (VROC) for NVMe SSDs attached to the CPU's PCIe lanes. VROC is a software RAID system that builds atop the Intel Volume Management Device (VMD) feature of Skylake-X and Skylake-SP processors. VMD allows some of the CPU's PCIe ports to be partitioned off into a separate PCI domain behind a virtual PCI bridge. VMD has largely been described by Intel as a feature intended to simplify support for hotplug and enclosure management (status LEDs) of NVMe SSDs, but it apparently also helps build a robust software RAID stack. Intel has not shared much technical information about how VROC works, but they have announced that its features will be segmented by requiring the purchase of a hardware key to unlock RAID capability, and a premium key to unlock RAID-5.
AMD has not described any features of their Zen architecture that sound particularly similar to Intel's VMD or the NVMe remapping feature of Intel's consumer platform chipsets, so it's not clear what hardware features (if any) are helping to enable their NVMe RAID implementation. From a user perspective, setting up NVMe RAID is very similar to setting up chipset SATA RAID: configure the array through the motherboard firmware and load an extra driver when installing Windows. Unlike Intel's plans for VROC, AMD isn't trying for any extra product segmentation and isn't charging extra for any of their NVMe RAID functionality.
One important warning: the new RAID system is incompatible with the previous SATA RAID system for X399. Users with an existing SATA RAID array will not be able to access it using the new RAID drivers. The only migration path AMD currently offers is to re-create the array using the new drivers and firmware, and restore the data from a back-up. For users booting off a RAID array, this means re-installing Windows.
Post Your CommentPlease log in or sign up to comment.
View All Comments
Duncan Macdonald - Monday, October 2, 2017 - linkLow volume market. Threadripper, EPYC and high end Xeon CPUs are the only ones with enough PCIe lanes to make good use of such a card.
bill.rookard - Monday, October 2, 2017 - linkNot necessarily. I have a older Asus P7F7-E WS motherboard which has full x8 slots x 4. With 3 add in cards for M.2 (keep one for the GPU, or just go with IGP and run all 4 slots) - they could run at x2 each, x4 drives, at 2TB M.2 drives I could run 32TB of all SSD NVMe storage. A RAID5 setup would leave me 30TB usable.
Billy Tallis - Tuesday, October 3, 2017 - linkThat board only has 16 PCIe 2.0 lanes coming off the CPU. Back then, PCIe switch chips like the NF200 were cheap enough to be used in enthusiast products, but PCIe 3 switches have never been so cheap. And since Threadripper has so many lanes coming directly off the CPU instead of being fanned out by the PCH or a separate switch chip, it has more than six times the I/O bandwidth available compared to your Lynnfield system.
colonelclaw - Monday, October 2, 2017 - link"Nobody has manufactured a true NVMe hardware RAID controller"
Are you sure? https://www.broadcom.com/products/storage/raid-con...
Billy Tallis - Monday, October 2, 2017 - linkYeah, I'm sure. Go read up on the trouble they've had getting their Linux driver for those chips accepted into the kernel. It's an ugly mess, but amounts to more of a NVMe passthrough capability than hardware RAID.
msroadkill612 - Wednesday, October 4, 2017 - linkWhat is your opinion of the highpoint 4x nvme raid card please?
Billy Tallis - Wednesday, October 4, 2017 - linkI haven't tested it yet, but I plan to. It's another software RAID solution, but one with both Windows and Linux support. Last I heard, they felt pretty good about the sequential I/O performance but were still working to improve the random I/O performance.
msroadkill612 - Monday, October 9, 2017 - linkfyi
i think this may be a bench of 2x highpoint 4xnvme cards and 6x panasonic sm961 (~same as evo specs afaik)
It also scales well.
Expensive at $400, vs asus ~$220 list - which makes getting a tr mobo w/ as many native nvme ports included as possible a bargain.
lilmoe - Monday, October 2, 2017 - linkU.2?
BrokenCrayons - Monday, October 2, 2017 - linkYeah, they had a few hits people liked. I don't see the appeal, but my musical tastes are different.