One of the key aspects of the most recent Intel processor launch, its Core 9th Generation processors, is that the new design affords some hardware-based protection for a couple of the Spectre and Meltdown family of security vulnerabilities. When these vulnerabilities were first discovered, they were patched using a combination of software and firmware, which unfortunately led to some performance regressions over an unpatched processor. The ultimate goal is for a hardware patch, which is always enabled, that loses zero performance – we’re testing out what the new patches have for us today.

A Recap: Spectre and Meltdown

At the start of 2018, it was announced that two new architecture-based attack vectors had been discovered in modern computer processors. These vectors are based in what are called side-channel attacks: the ability to probe or infer what something else on the processor is doing without actually seeing the workload or its data. But more fundamentally than that, the reason these attacks can occur is down to how the processor is designed. Side-channel attacks on processor speculation, for example, can expose data that wasn’t even intended to be accessed by the program in the first place. And other side-channel attacks can break through memory permission barriers between software like a Buick through a barn.

Because security is the number one factor in all modern day computing (and unfortunately in some cases, an afterthought), this became a serious potential problem quite quickly. The specific vulnerabilities have so far mostly fallen under two broad headings which are now synonymous with these attacks: Spectre and Meltdown. For completeness, there are similar attacks that don’t fall under these headings, but side-channel security is still at the heart of what is being broken.

All vulnerable processor makers were told about these attack vectors around six months before the public, in what is known in the industry as responsible disclosure – find a vulnerability, inform the vulnerable parties, and give them sufficient time to fix the issue before going public, in order to prevent any nefarious individuals exploiting it (and creating what is called a 0-day). In that time, the known attacks were analysed and software-related patches were built. For example, over the course of 2018, Intel started with its newest and high-priority platforms first, and slowly worked its way back through its product catalogue. Other companies had executed similar plans, however Intel has by far been the most high profile of them all.

One big issue surrounding these attacks is unsecured embedded systems running high-performance processors – those that are deployed in the field and can be easily compromised. This is beyond the scope of this article, but it’s important to understand how many devices might still be vulnerable in ten years.

How To Fix Spectre and Meltdown: Security

Since the disclosure of these new attack vectors, new vulnerabilities similar to them have been discovered but follow in the same vein. In order to protect against these issues, one extreme answer is a full top-to-bottom redesign of how modern computer processors are designed. This is a massive undertaking, so in the end we have a sliding scale of security patches.

Perhaps the easiest patch to implement is the operating system software patch. For home and office users, this is going to be the one that they experience first, given the prevalence of Windows and Mac OS to provide regular updates to the security of their platforms.

However, not everything can be solved by a traditional software patch, especially when the offending areas of the CPU design cannot be adjusted in at a high level software. More fundamental aspects of the CPU might require a microcode update, which involves uploading very small and low-level code to the CPU to make important changes to the CPU – and particularly to patch around errata.

The ideal way to distribute microcode updates is as part of a larger system firmware (e.g. BIOS) update, this way the updated microcode is available to the CPU immediately at boot. However it's also possible to distribute microcode updates at the OS level – to be loaded alongside the OS – and this is what has been happening given the serverity of the issue and the fact that OEMs tend to give up on firmware updates rather quickly. As a result, all the major OS vendors have been able to distribute the necessary microcode to affected CPUs through mainstream OS update mechanisms, allowing most PCs to be hardened against these attacks by combining the microcode fixes with OS changes to leverage the fixes.

But even as great as microcode is, it's still a software workaround for a hardware issue, incurring some performance penalties along the way. Ultimately there needs to be a hardware fix in place to ensure that the processor is secure at all times. This cannot happen on any current products, and requires new products over the next few years to have fixes in place.

Ultimately there are two types of hardware fix: a hardware patch or a hardware redesign. The hardware patch does the same as the software or microcode patches: stops the security issue. A hardware resign fundamentally changes the way the hardware operates, removing the issue entirely.

As stated, security is always a number one priority here. However priority one-point-five is performance. This is the reason for this review.

How To Fix Spectre and Meltdown: Performance

The reason why some of these new attack vectors have been so surprising is because they have arisen out of techniques designed to increase the performance of a modern day microprocessor. A key innovation to performance in the modern era is speculation – the ability for the processor to ‘predict’ (for lack of a better word) what data will be needed by the software at a future date, saving long trips out to main memory and hiding those delays. One of the slowest things you can do on a modern processor is main memory latency, so a lot of work has gone in to optimizing the hardware that performs speculation and pre-fetching data. As a result, speculative execution is one of the cornerstones of modern, high-performance processors.

The security problem arises when data is pre-fetched but not needed. If another program can access that data or infer about what the data contains, which may have been within a secure area in main memory, it’s a security nightmare. Or even another program can access that speculative memory even during the speculation itself, it’s a nasty thing to track. In an ideal world, everyone keeps to their own lane and everything runs fast and smooth. However, in order to keep that part of the memory secure, a fix needs to occur: either a patch, or a fundamental redesign of the hardware.

The patches for security issues such as speculation, therefore, have the potential to be dire for performance. If all of a sudden one of the techniques for increasing single core performance was disabled, that’s a lot of performance out of the window. Depending on whom you ask, the performance results of the initial software and firmware patches varied.

Speaking with Intel and their own internal performance numbers on server workloads, there were three main trends to speak of: most of the performance regression happened with memory and IO-related workloads (due to what is called kernel switching, one of the fixes applied), it has more of an effect with faster storage such as NVMe-based SSD (because HDDs already take a longer time to be ready), and it affects the newest processors the least. Going through the numbers on their tests for cloud servers, an IO/compute mix workload on modern hardware, such as HHVM Wordpress, would experience a 9-10% decrease in performance. An older platform such as Broadwell-EP or Haswell-EP performs similarly in all tests before and after patch, but may experience a 30-60% decrease an synthetic IO bound workloads such as FIO (worst case scenario) – Skylake-SP is less affected.

However, it was published in other media that some video game servers saw their game servers move from being 20% active to 50% active when the patches were applied. This increases costs and an inability to scale, so it has been suggested that some performance-limited deployments may not be using these fixes because of this. Others, such as compute-limited HPC, have seen no decrease in performance. Notably, it wasn’t even a topic at Supercomputing this year as to performance related issues.

The ultimate goal is to have a patch that minimizes the performance deficit, but keeps 100% of security. This has been erroneously reported in some press as the holy grail ‘hardware fix’, but those publications fail to differentiate between a hardware patch and a hardware redesign. A ‘simple’ hardware patch would still have many of the same performance issues as a software patch, just with the guarantee that said fixes are always there. But ultimately a hardware patch that minimizes performance loss is the goal.

Intel’s March to Hardware Fixes

If you have been following our coverage this year of Intel’s product portfolio, back in September it was announced that Intel’s next generation of server processors, Cascade Lake-SP, would implement a number of fixes at the hardware level. Lisa Spelman, VP of Intel’s Data Center Group, confirmed this in our interview, saying that the processor design hit the intercept for when the fixes were ready. Intel also stated that its Whiskey Lake 15W processors, built in the 14++ process, would have similar hardware fixes. And most recently, the new 9th generation Coffee Lake Refresh process would also have the same fixes. As a result, we get a table like this of what is fixed and what is not:

Spectre and Meltdown on Intel
Families with Any Hardware Mitigations
Spectre Variant 1 Bounds Check Bypass OS/VMM OS/VMM OS/VMM OS/VMM
Spectre Variant 2 Branch Target Injection SW + FW SW + HW SW + FW
Meltdown Variant 3 Rogue Data Cache Load Hardware Hardware Hardware Firmware
Meltdown Variant 3a Rogue System Register Read Firmware
  Variant 4 Speculative Store Bypass SW + FW
  Variant 5 L1 Terminal Fault Hardware Hardware Hardware Firmware

So for the newest Core i9 Coffee Lake Refresh processors, CFL-R Intel has two hardware fixes, for Meltdown (aka Variant 3) and the closely related L1TF (aka Variant 5). All others will be covered by a mix of firmware and software patches.

As mentioned, the hardware fix is a minimum requirement for security, but says nothing about performance. The question then becomes if this fix is purely a patch, or something more. Out of all the vulnerabilities, Spectre Variant 2 has been quoted as the biggest source for performance regression.

Luckily, Intel has made this testing easy for us. The new 9th generation Coffee Lake Refresh processors are microarchitecturally identical to the 8th generation Coffee Lake family. As a result, we can take one processor from each, fix them to identical frequencies/core counts/memory, run them in the same motherboard with the same BIOS, and compare the benchmark results. This is what we did. If the hardware fix is more than just a patch, we should see the 9th generation outperform the 8th generation.

Test Setup

For our testing we took the newest Core i9-9900K and the special edition Core i7-8086K and initiated a face off. Each processor was set to four cores each with no hyperthreading, with the frequency set to 3.0 GHz for all workloads at all times. The motherboard of choice was our ASRock Z390 Pro Gaming Core i7, using BIOS P3.20, and we used 4x8GB of Crucial Ballistix DDR4-2666.

We ran through our whole benchmark suite, both our professional/office tests and our gaming tests using an MSI GTX 1080 at four different resolution/quality combinations. The results may shock you.


Rather than several pages of graphs with two values in each, here’s a summary of the results split up by section.

In all cases, performance was within the margin of error between both processors. The biggest single CPU test gain was 4.0% (in LuxMark C++) and the worst was -2.7% (in Agisoft). The usual culprits for this sort of test, DigiCortex and WinRAR, were both within the margin of error. As a result, this hardware fix appears to essentially be a hardware implementation of the fixes already rolled out via microcode for the current Coffee Lake processors.

Cascade Lake-SP has similar fixes, but also has a hardware patch for Spectre Variant 2. In our interview with Lisa Spelman, she stated that Cascade overall will offer higher performance than Skylake-SP.

The long and short of matters then is that based on the testing we've done thus far, it doesn't look like Coffee Lake Refresh recovers any of the performance the original Coffee Lake loses from the Meltdown and Spectre fixes. Coffee Lake was always less impacted than older architectures, but whatever performance hit it took remains in the Refresh CPU design.

But at least the security for the two variants that Coffee Lake has hardware patches for is baked in, no matter where that processor goes. And Cascade will improve that further. It will be interesting to see where the next set of hardware fixes will land. If we can get some numbers, we’ll let you know.

Comments Locked


View All Comments

  • FunBunny2 - Monday, December 3, 2018 - link

    "So basically disabling all major benefits from speculative memory accesses to begin with"

    ??? last time I looked, that's been part of multiprogammed machines since the 60s.

    "The STRETCH was not a complete success. It guessed which path would be taken from a branch instruction and obeyed instructions along that path. If it had taken the wrong branch, it had to undo the effects of those instructions executed which really compromised its overall performance. Much time was spent at Aldermaston writing compilers that would allow the STRETCH to guess correctly more often."
  • Ian Cutress - Monday, December 3, 2018 - link

    Speculation occurs whether HT is enabled or not, it's HT agnostic. If anything, HT is used to hide memory latency when sufficient resources allow. See my comment above
  • Urthor - Monday, December 3, 2018 - link

    Interesting you report as "within the margin of error"

    Will Anand ever report on its P-values???
  • Ratman6161 - Monday, December 3, 2018 - link

    I know your site relies on advertising. But you need to find a way to keep yourselves from being a vehicle for frauds and scams. For example on this page "thousands of people from the us cancel Netflix..."

    This links to a fraud site that true to get people to turn over their credit card to a fake streaming service.
  • Ryan Smith - Monday, December 3, 2018 - link

    Hey Ratman, I haven't seen that one. Could you please send me an email with screenshots and other info?
  • LJM - Tuesday, December 4, 2018 - link

    I keep getting:
    "Millionaire Mom from Helsinki Exposes 3-step Secret to Work from Home"
    "The Rich in Finland Are Trying to Ban This Bitcoin Video"
    "Finland Company Fires Manager After Making Thousands Rich By Mistake"

    It's ridiculous.
  • LJM - Tuesday, December 4, 2018 - link

    Pretty much all ads in the "Ads by Revcontent
    From The Web" section look like a scam every time.
  • Hixbot - Tuesday, December 4, 2018 - link

    Ryan, is you can successfully convince your web overlords to ditch revcontent, you will be doing a major service to your readers. The "ads" are clickbait that often use NSFW images of women that I'd categorize as misogynistic, many of the "ads" lead to scams and malware.

    This is tech site, I suggest find an ad provider that advertises tech.
  • TheJian - Tuesday, December 4, 2018 - link

    please retest with 8700k and earlier. If I'm losing 10% I want to know when the class action suit starts as I want some money back! I didn't pay for a chip that only next 90% of it's perf or less. My 4790k hasn't had any fixes yet due to this. Again, they owe ALL OF US MONEY, and more on older stuff clearly according to intel themselves.

    I hope you ask AMD what is wrong (or not) with their 7nm chips upcoming. I'm going to buy one probably anyway assuming they are NOT as bad as Intel (weren't already, so...why would next gen be worse). I wouldn't even patch HOME pc's for anything you have to sit at my PC to exploit (doesn't happen at home). But I'd still want some rebates or something to shut up.

    It's broke, slowing me down is still broke even if mitigated. And testing without HT is useless as others noted. Whether it masks stuff or not, who buys an Intel chip and turns off HT? NOBODY. If that's Intel's fix, I want a LOT of my money back...LOL.
    OpenBSD disabled it, can't get to the article right now on vpn (toms hates my vpn..ROFL) too lazy to turn it off.
    "Red Hat strongly recommends to mitigate the problem, you manually enable Linux specific kernel parameters or potentially disabling features like Intel hyperthreading, after the available updates have been applied."

    Yeah, not an option IMHO. Oracle in that article says disabling HT doesn't fix it all anyway, and just slows you down of course.
    "The real fix to all these problems, Intel admits, is by replacing today's processors. "
    Ahh, so, did you do this on purpose? Whenever you have a major slowdown in PC sales (eh, last 5 years declining), you release the bugs (much like fbi/doj leaking right and left on fake news) and claim you need to fix them? Er, Uh, just buy our crap again, we promise we'll get it right this time, but you ALL have to replace every chip we've ever made...LOL. I want my freaking money back. :(

    I'll buy AMD next due to 7nm anyway, but this is the last time I buy Intel even if they win 2020-2021. Color me pissed for some time. I have at least two chips to buy at 7nm, and I'm thinking AMD wins my vote for the next one or two after that by default. Intel now has to EARN my purchase and that will take a LONG while.
  • The_Assimilator - Tuesday, December 4, 2018 - link

    Good luck winning a lawsuit with the complaint "Intel owe me a refund because they didn't forecast a major security flaw being discovered in their products", you fucking dumbass.

Log in

Don't have an account? Sign up now