Launching the #CPUOverload Project: Testing Every x86 Desktop Processor since 2010by Dr. Ian Cutress on July 20, 2020 1:30 PM EST
We want to have every desktop CPU since 2010 tested on our new benchmarks.
Updating our testing suite is all well and good, but in order for users to find the data relevant, it has to span as many processors as possible. Using tools such as Intel's ARK, Wikipedia, CPU-World and others, I have compiled a list of over 800+ x86 processors (actually 900+ when this article goes live) which qualify. At the highest level, I am splitting these into four categories:
- Intel Consumer (Core i-series, HEDT)
- Intel Enterprise (Xeon, Xeon-W, 1P, 2P)
- AMD Consumer (Ryzen, FX, A-Series)
- AMD Enterprise (EPYC, Opteron)
Within both AMD and Intel, the consumer and enterprise arms of each company are discretely different business units, with product teams, and rarely is there any cross-over. The separation of the departments is easy to follow, in that ‘Consumer’ basically stands for mainstream processors that are aimed at machines a user or an OEM could build with off-the-shelf parts, and typically do not support ECC memory. ‘Enterprise’ is going to refer to processors that might end up in workstations, servers or data centers, that have professional grade features, and most of these parts do support ECC memory.
Three boxes of CPUs. Less than half
The next level of separation for the processors, for our purposes, is going to be under the heading ‘family’. Family is a term that typically groups the processors by the microarchitecture, but could also have separation based on socket or features. For CPU Overload, choosing one high-level category breaks down like this:
- AMD Consumer (360+ processors), inc Pro
- Ryzen 3000 (Threadripper, Ryzen 9/7/5/3; 7nm Zen2)
- Ryzen 2000 (Threadripper, Ryzen 7/5/3; 12+ Zen+)
- Ryzen 1000 (Threadripper, Ryzen 7/5/3; 14nm Zen)
- Bristol Ridge (A12 to Athlon X4, 28nm AM4-based Excavator v2 APUs)
- Carrizo (Athlon X4, 28nm FM2+ Excavator)
- Kaveri Refresh (A10 to Athlon X2, FM2+)
- Kaveri (A10 to Sempron X2, FM2+)
- Richland (A10 to Sempron X2, FM2+ Piledriver)
- Trinity (A10 to Sempron X2, FM2/2+ Steamroller)
- Llano (A8 to Sempron X2, 32nm FM1 K10)
- Kabini (FM1, 28nm Jaguar)
- Vishera (FX-9590 to FX-4300, 32nm AM3 Piledriver)
- Zambezi (FX-8100 to FX-4100, 32nm AM3 Bulldozer)
- AM3 Phenom II X6 to X4 (K10, Thuban/Zosma/Deneb)
- (Optional) Other AM3 (K10, Zosma, Deneb, Propus, Heka, etc)
- (Optional) Other AM2 (Agena, Toliman, Kuma)
Neither AMD nor Intel provides complete lists of the processors it launched within a certain family. Intel does it best, under its ark.intel.com (known as ARK) platform, however sometimes there are obscure CPUs that do not make the official list due to partnerships or being geo-specific. The only way we end up knowing these obscure CPUs exist is because someone has ended up with the processor in their system and run diagnostic tests. Intel calls these processors ‘off-roadmap’ CPUs, and while it only provides information on them through ARK if you know the exact processor number already. Scouring the various resources available online to draw up that picture proved one thing: no list is complete. I doubt the one I have is complete either.
For example, most users believe that the last AMD FX processor that was made by AMD was the massive 220W FX-9590 for the AM3 platform. This is not the case.
AMD released two FX CPUs on FM2+, and these were only sold in HP pre-built systems. The two CPUs are the FX-770K and the FX-670K. Typically FX processors are known for being only on the AM3+ platform on the desktop, however AMD and HP struck a deal to give the premium FX name to these other CPUs and they were never launched at retail – in order to get these we found a seller on eBay that had pulled them out of old systems.
In some lists we found online, it was very easy to get mixed up because some companies have not kept their naming consistent. Take the strange case of the Athlon X4 750. Despite the name not having a suffix, is classified as a newer family to the Athlon X4 750K. The X4 750 being Piledriver based and the X4 750K which is Steamroller based.
Then there are region specific CPUs, like the FX-8330, which was only released in China.
This is the current standing of the 'Intel Consumer' Core i7 and Core i5 processors up to Coffee Lake. Ones marked in yellow are ones that I have immediately available, ready for testing, and ones marked in red are still to-be-obtained. The common thread is that Intel has supplied all the Core i7-K processors for most generations, and the i5-K processors for some of them. The big blocks of yellow for Kaby Lake and Skylake were also sourced from Intel. The other singular dotted processors are either ones that I have purchased personally from my own stash, or ones that we have obtained via partners, such as the S/T processors when we covered some low-power hardware a few years ago. Needless to say, there are plenty of gaps, especially on the latest (and unannounced processors), but also going further back, before I was in charge of the CPU reviews.
Some of the feedback we have had with this project is that the database technically does not need every CPU that ever existed to be relevant, and even then, for some CPUs if we reduce the frequency multiplier, it will perform the same as a processor we do not have. While for some CPUs that is true, it has to be as long as the testing does not fall foul of the power states, the turbo states, the points at which turbo frequencies are enabled and the appropriate frequency/voltage curve binning (and if the cache sizes line up). While this can be the case, it is often on a case-by-case basis. However for the scope of this project, and for this project to be authentic in the data it is going to give, one of the rules I am imposing is that the data has to come from testing with the CPU on hand - no synthetic numbers from 'simulated' processors.
Rule: No Synthetic Numbers from Simulated Processors
I mentioned that sourcing is one of the most difficult parts of this project, and the obvious answer to get hardware is to go direct to the manufacturer and request it: both manufacturers end up being big parts of this project regardless of their active participation, but the best scenario is that they should be the hardware source.
As the four areas above (AMD/Intel, Consumer/Enterprise) are for lack of a better description different companies, the press contact points for the consumer and enterprise sides of each company are different. As a result, we have different relationships with each of the four, and one of the interesting barriers to sampling is rebuilding relations when long-term contacts leave. Sometimes this happens for the better (sampling improves), or for the worse (a severe reluctance to offer anything).
Unfortunately sometimes there a wall - business unit policies for sampling often mean only one CPU here or there can be offered due to what’s available for media distribution, or if the company, the press contact, or the product manager does not see any value to the business in sampling a component (such as an Intel Pentium or an AMD A9), then it is unlikely we would get that sample from them. Part of that relationship with these companies is demonstrating the value of this data.
Another aspect is not actually having any samples - these are PR teams, not infinitely packed stock rooms. So if the team we are in contact with does not have access to certain parts that we request, such as consumer-grade parts that were built specifically for certain OEMs that are not under the ‘consumer’ PR team’s remit, or even some of the low priority parts in a stack, they can't loan them to us. It sounds somewhat odd that a big company like Intel or AMD wouldn't have access to a part that I'm looking for, but take those HP-only FX CPUs I mentioned earlier – despite it being a consumer grade CPU, because it ended up being a B2B transaction to supply these parts, it would have never passed the hands of the PR team, and any deal with the OEM may have put reviews of the hardware solely at the discretion of the OEM. Or the region-only parts, then only the PR team in that region will have access to them. (I eventually picked up those parts on my own dime on eBay, but this isn’t always possible.)
Nonetheless, we have approached as many people internally at both companies, as well as some OEMs and resellers, with our CPU Overload project idea. Both of the consumer arms of Intel and AMD have already provided a good first-round bounty of the latest hardware, and in most cases complete stacks of the newest generations. The enterprise hardware is a little tricky to get hold of. But many thanks to our Intel and AMD contacts that are already on-board with CPU Overload, as we try to work closer with the other units.
One thing to mention is that the newer the processor, the easier it is to get direct from the manufacturer. Typically these parts are already within their sampling quotas. However, if I go and ask for a Sandy Bridge Core i3-2125 from 2011, a sample to share is unlikely to be at their fingertips - there might be one in the drawer in a lab somewhere, but that is never a guarantee. This is where the project will have to look to private sales, forums, and online auction sites to play a role as we move further into the past. Depending on how the project goes, we may reach out to our readers (either in a project update, or on my twitter @IanCutress) for certain parts to complete the stacks. This has already worked for at least three hard-to-find CPUs, such as the HP FX CPUs (the FX-770K and FX-670K), and the Athlon X4 750 (not 750K), which we picked up from eBay and China respectively.
For the initial few months of the project, we have around 200 CPUs to begin. This breaks down into the following:
|CPU Overload Project Status|
|CPUs on Hand||Key Notes|
|Intel Consumer||138 / 406||29 / 46 : HEDT
72 / 241 : Core
37 / 119 : Pentium/Celeron
|AMD Consumer||137 / 366
|42 / 105 : AM4 and TR
43 / 108 : FM2+/FM1/AM1
52 / 153 : AM3/3+
Xeon E and
|27 / 155||75% of E3-1200 v5
75% of E3-1200 v4
36% of E3-1200 v3
|AMD EPYC||11 / 40||Known Socketed EPYC
Lots of unknown in Cloud
|Others||Opteron 6000: 2 / 121
|Total (Phase 1)||313 / 967|
For the first phase, we are almost at a good level, having 33.7% of the processors needed. However, the models we do have a fairly localized in the Skylake/Kaby Lake-S sets, Intel's HEDT range, and some of AMD's stack. There is still a good number of interesting segments missing in what we have to hand.
The K10, LGA1366 Xeons and the older Opterons are part of the secondary scope of this project. Some of them are easy to obtain with bottomless pockets and a trip to eBay, and others require more research. There is a potential 'Phase 1.5', if we were to go after all of the Xeon E5-1600 and E5-2600 processors as well. Then it becomes an issue of tackling single socket vs dual socket systems, as well as suitable NUMA software.
So out of the two main issues with a project of this size, sourcing and benchmark longevity, we're trying to tackle both with one go - retest everything with a new benchmark suite on with the latest stable OS. The only factor left is time - retesting all these CPUs doesn't happen overnight. The key numbers to begin with testing however will be the headline Intel processors back to Sandy Bridge, and the AMD parts back to FX.
Post Your CommentPlease log in or sign up to comment.
View All Comments
vasily - Monday, July 20, 2020 - linkYou might want to check out Phoronix Test Suite and openbenchmarking.org.
colinisation - Monday, July 20, 2020 - linkwould love to see the following processors added
5775C (overclocked to 4Ghz) - just purely to see what impact the eDRAM has on workloads
Phenom II X4
Highest Bulldozer core
VIA's highest performance x86 core
faizoff - Monday, July 20, 2020 - linkWhat a gargantuan project this is going to be. And I cannot wait, oddly enough I've been using the bench tool the past few weeks to get a sense of how much difference an upgrade for me would make.
I am probably one of the many (or few) people that have still held on to their i5 2500k and this is one of the places I can select that CPU and compare the benchmarks with newer releases.
This project looks to be an amazing read once all done and will be especially looking forward to those segments "how well does x CPU run today?"
Alim345 - Monday, July 20, 2020 - linkAre you going to make benchmark scripts available? They should be useful for individual comparisons, since many users might have overclocked CPUs which were more common in 2010-2015.
brantron - Monday, July 20, 2020 - linkJust to fill out the starting set:
7700K needs a common AMD counterpart, i.e. Ryzen 2600
Sandy or Ivy Bridge i7
That would also make for a good article, as it should be possible to overclock any of those to ~4.5 GHz for a more apples to apples comparison.
StormyParis - Monday, July 20, 2020 - linkThank you for that. My main question is not "what should I buy" because that's always very well covered, and on a fixed budget there's never much choice anyway, but "should I upgrade *now* which is only worth it when last time's amount of money gets you at least 2x performance. I'ive got a 7yo Core i5... I'll look into it !
eastcoast_pete - Monday, July 20, 2020 - linkIan, thanks for this!
One aspect I've wondered about for a while is whether you could include performance/Watt in your tests and comparisons going forward? I know that's usually done for server CPUs, but I also find it of interest for desktop and laptop CPUs.
thebigteam - Monday, July 20, 2020 - linkI think I have the below list of Intel CPUs available if needed, likely with working mobos too. Would be very happy to clean out the closet and get these to you guys :) Likely some 2009/2010 Athlons as well
inighthawki - Monday, July 20, 2020 - linkThank you so much for changing your gaming benchmark methodology. I tend to play my games at 1440p on lowest settings for maximum framerates, which is far more often than not CPU bound. It was always so annoying seeing the benchmarks be GPU bound when I'm trying to see how much a new CPU helps.
Smell This - Monday, July 20, 2020 - linkChicken
With AM3, AM2+ and AM2 processors, AM3+ processors broke backwards-compatibility.
A mobo like the MSI 790FX K9A2 Platinum transitioned nearly 250 processors from S754-939, to AM2-AM3, beginning with the single-core Athlon 64 3000+ 'Orleans' up to the PhII x6 DDR3 Thubans.
These were the progeny of the K8 or 'Hammer' projects. A Real Man would never leave them behind ...