AnandTech Storage Bench 2013

When Anand built the AnandTech Heavy and Light Storage Bench suites in 2011 he did so because we didn't have any good tools at the time that would begin to stress a drive's garbage collection routines. Once all blocks have a sufficient number of used pages, all further writes will inevitably trigger some sort of garbage collection/block recycling algorithm. Our Heavy 2011 test in particular was designed to do just this. By hitting the test SSD with a large enough and write intensive enough workload, we could ensure that some amount of GC would happen.

There were a couple of issues with our 2011 tests that we've been wanting to rectify however. First off, all of our 2011 tests were built using Windows 7 x64 pre-SP1, which meant there were potentially some 4K alignment issues that wouldn't exist had we built the trace on a system with SP1. This didn't really impact most SSDs but it proved to be a problem with some hard drives. Secondly, and more recently, we've shifted focus from simply triggering GC routines to really looking at worst case scenario performance after prolonged random IO.

For years we'd felt the negative impacts of inconsistent IO performance with all SSDs, but until the S3700 showed up we didn't think to actually measure and visualize IO consistency. The problem with our IO consistency tests is that they are very focused on 4KB random writes at high queue depths and full LBA spans--not exactly a real world client usage model. The aspects of SSD architecture that those tests stress however are very important, and none of our existing tests were doing a good job of quantifying that.

We needed an updated heavy test, one that dealt with an even larger set of data and one that somehow incorporated IO consistency into its metrics. We think we have that test. The new benchmark doesn't even have a name, we've just been calling it The Destroyer (although AnandTech Storage Bench 2013 is likely a better fit for PR reasons).

Everything about this new test is bigger and better. The test platform moves to Windows 8 Pro x64. The workload is far more realistic. Just as before, this is an application trace based test--we record all IO requests made to a test system, then play them back on the drive we're measuring and run statistical analysis on the drive's responses.

Imitating most modern benchmarks Anand crafted the Destroyer out of a series of scenarios. For this benchmark we focused heavily on Photo editing, Gaming, Virtualization, General Productivity, Video Playback and Application Development. Rough descriptions of the various scenarios are in the table below:

AnandTech Storage Bench 2013 Preview - The Destroyer
Workload Description Applications Used
Photo Sync/Editing Import images, edit, export Adobe Photoshop CS6, Adobe Lightroom 4, Dropbox
Gaming Download/install games, play games Steam, Deus Ex, Skyrim, Starcraft 2, BioShock Infinite
Virtualization Run/manage VM, use general apps inside VM VirtualBox
General Productivity Browse the web, manage local email, copy files, encrypt/decrypt files, backup system, download content, virus/malware scan Chrome, IE10, Outlook, Windows 8, AxCrypt, uTorrent, AdAware
Video Playback Copy and watch movies Windows 8
Application Development Compile projects, check out code, download code samples Visual Studio 2012

While some tasks remained independent, many were stitched together (e.g. system backups would take place while other scenarios were taking place). The overall stats give some justification to what we've been calling this test internally:

AnandTech Storage Bench 2013 Preview - The Destroyer, Specs
  The Destroyer (2013) Heavy 2011
Reads 38.83 million 2.17 million
Writes 10.98 million 1.78 million
Total IO Operations 49.8 million 3.99 million
Total GB Read 1583.02 GB 48.63 GB
Total GB Written 875.62 GB 106.32 GB
Average Queue Depth ~5.5 ~4.6
Focus Worst case multitasking, IO consistency Peak IO, basic GC routines

SSDs have grown in their performance abilities over the years, so we wanted a new test that could really push high queue depths at times. The average queue depth is still realistic for a client workload, but the Destroyer has some very demanding peaks. When we first introduced the Heavy 2011 test, some drives would take multiple hours to complete it; today most high performance SSDs can finish the test in under 90 minutes. The Destroyer? So far the fastest we've seen it go is 10 hours. Most high performance SSDs we've tested seem to need around 12 - 13 hours per run, with mainstream drives taking closer to 24 hours. The read/write balance is also a lot more realistic than in the Heavy 2011 test. Back in 2011 we just needed something that had a ton of writes so we could start separating the good from the bad. Now that the drives have matured, we felt a test that was a bit more balanced would be a better idea.

Despite the balance recalibration, there's just a ton of data moving around in this test. Ultimately the sheer volume of data here and the fact that there's a good amount of random IO courtesy of all of the multitasking (e.g. background VM work, background photo exports/syncs, etc...) makes the Destroyer do a far better job of giving credit for performance consistency than the old Heavy 2011 test. Both tests are valid; they just stress/showcase different things. As the days of begging for better random IO performance and basic GC intelligence are over, we wanted a test that would give us a bit more of what we're interested in these days. As Anand mentioned in the S3700 review, having good worst case IO performance and consistency matters just as much to client users as it does to enterprise users.

We're reporting two primary metrics with the Destroyer: average data rate in MB/s and average service time in microseconds. The former gives you an idea of the throughput of the drive during the time that it was running the Destroyer workload. This can be a very good indication of overall performance. What average data rate doesn't do a good job of is taking into account response time of very bursty (read: high queue depth) IO. By reporting average service time we heavily weigh latency for queued IOs. You'll note that this is a metric we've been reporting in our enterprise benchmarks for a while now. With the client tests maturing, the time was right for a little convergence.

AnandTech Storage Bench 2013 - The Destroyer

While IO consistency was lacking, the Hawk does surprisingly well in our new Storage Bench 2013. It's definitely no challenger to SanDisk Extreme II or Seagate 600, but the excellent sequential performance makes up for the lack of random write performance and consistency. A look at the average service time shows that the IO consistency is what is holding the Hawk back. 

AnandTech Storage Bench 2013 - The Destroyer

Performance Consistency Random & Sequential Performance
Comments Locked


View All Comments

  • karasaj - Tuesday, June 25, 2013 - link

    Every time I kept reading that, I kept thinking about how no name suppliers would be bad... hard to remember it's from Toshiba, haha. Good luck to them. Nice review :) Power consumption seems obnoxiously low at load.
  • mcveigh - Tuesday, June 25, 2013 - link

    Strontium Hawk is the name of my spirit animal!
  • jmke - Tuesday, June 25, 2013 - link

    not the best product name I must admit... might not do well in Benelux ;)
  • Pessimism - Tuesday, June 25, 2013 - link

    Why poo-poo the manufacturer for taking a stand to end the 1000^3 garbage? Everyone should follow suit and label with formatted capacity.
  • hedleyroos - Tuesday, June 25, 2013 - link

    Your use of the word "poo-poo" is ironically funny to people who speak Afrikaans (and probably Dutch and Flemish) since "stront" means "poo". They have zero chance of succeeding in those markets.
  • Kristian Vättö - Tuesday, June 25, 2013 - link

    Like I said, it's pretty much useless for a small OEM like Strontium to try and change the industry.
  • piroroadkill - Tuesday, June 25, 2013 - link

    It should then say 240GiB...
  • dealcorn - Tuesday, June 25, 2013 - link

    OK. Laptops outsell desktops and you were unable to test with HIPM and DIPM enabled and no mention was made of DEVSLP support which is a big deal for any Haswell mobile device. By impairing the relevance of the review to typical use cases, someone properly earns a demerit.

    You need a Haswell mobile device to test with if you want to maintain relevance to the mainstream market. I would complain to my bosses that you need better hardware support to perform at the level they and readers expect of you.
  • Kristian Vättö - Tuesday, June 25, 2013 - link

    I'll be brutally honest here. Just because we are AnandTech doesn't mean that we have piles of laptops laying around, especially ones that are based on a chip that was released less than a month ago. All reviewers want Haswell-based devices at the moment, the supply is extremely tight. Usually laptops only have a review time of two weeks or so because the same system may be sent to others once it's been returned. Obviously reviewers who are actually going to review the system are the first priority, so it'd be really hard for me to get one because 1) I would only be using it for one test 2) I couldn't send it back anytime soon. In other words, the manufacturer wouldn't get much bang for their marketing $ because they wouldn't get much visibility, which is the reason review samples exist in the first place.

    I know there's the option of buying one but again, I'll be honest here: It would be around $1000 for just one test. I would definitely take one if Anand paid for it but as far as I've understood, Anand isn't into spending thousands on test equipment (keep in mind that the financial situation isn't all that good for us since it's usually the marketing budgets that get cut when bad times hit, so it's harder for us to get advertisers). There's a ton of stuff I'd love to have as they would really take our SSD tests (especially power related) to a next level but I'm not the one making decisions.

    We have talked with Intel and ASUS and asked if there's anyway HIPM/DIPM could be enabled on a desktop system (even via custom firmware) but as far as I know, they are not up for that.

    Trust me, I would take a Haswell laptop on a heartbeat if someone gave me one, but I hope you also understand that we don't get whatever we want from manufacturers.
  • dealcorn - Tuesday, June 25, 2013 - link

    Well handled and responsive to my concern. So, BK is telling his team we have a winning hand but we need to release products faster. Anandtech is arguably the premier web site consumers turn to for help understanding how new technology benefits them. Intel, however, makes no effort to ensure you have access to the hardware necessary to explain why consumers should value the new stuff. If Intel is trying to increase the cadence, they should step up their game. If Intel does not understand that, it lessens BK's credibility.

Log in

Don't have an account? Sign up now