Four years ago we reviewed Facebook's self-designed and open source server, Freedom. Coinciding with releasing their datacenter design, they founded the Open Compute Foundation, creating a home for the design documents, licenses and bringing vendors together under one roof.

Today, the Open Compute Foundation is doing well. Many high profile companies such as Yandex, IBM and Intel are members, and in 2014 Microsoft joined the initiative as a second major open hardware partner; releasing its Open Cloud Server designs to a pool of open hardware spanning everything from servers and switches to datacenters. The international community has grown substantially, with Summits in all major continents attracting a larger crowd each time. More so, the number of companies adopting hardware from OCP and contributing back is rising as well, and with good reason: Mark Zuckerberg indicated at the 2015 Summit in San Diego that Facebook achieved a $2 billion (USD) cost reduction (!), achieved in part by using open purpose-built hardware instead of regular proprietary gear.

To Follow and Like, at Scale

When Facebook talks about the incredible amounts of money it has saved by using Open Compute, keep in mind that even though the vanity-free hardware is designed to be cost-effective, it's the actual software that enables hardware efficiency. The company has always sought to use commonly available software components to build its services, and when it picked one, plenty of time and money are spent on performance engineering to optimize the software. Performance engineering that enables Facebook to handle 6 billion likes, 930 million photos and 12 billion messages every day, and those numbers only represent activity its social network product generates; Instagram and WhatsApp are not exactly toy workloads either.

Facebook has carried out plenty of improvements in open-source projects, or started new implementations, most of them contributed back to the community, good karma indeed. Notable performance-related projects are HHVM -- short for HipHop VM, a JIT'ing PHP virtual machine with its own PHP language dialect called Hack, adding static typing in the mix. HHVM is advertised as 'a more predictable PHP', and if Wikipedia's migration serves as any indication it makes existing PHP powered sites quite a bit faster as well. Another initiative called rocksdb, implements a properly scaleable, persistent key-value store for fast storage written in C++ and presto, a distributed SQL engine for Big Data stored in common storage systems like Cassandra, JDBC DBs and HDFS. Presto provides functionality similar to Hive, and is currently in use at Dropbox and Airbnb.

Scale Your Devops

A (dev)ops engineer at Facebook can request any kind of hardware configuration he likes, as long it is one of the following five:

Increasing the number of parts used in your systems exponentially increases the amount of money and time that will be spent in procuring, validation and maintenance. And keep in mind that in order to avoid vendor lock-in, each configuration must be available from at least two suppliers, so it's easy to imagine why FB prefers to keep it simple when it comes to servers: 5 SKUs using the same base platform, each targeted at a certain kind of application.

The SKUs mentioned are variations on Facebook's latest Xeon E5 server platform, Leopard, reviewed in detail in this article. The web tier is concerned with gathering every piece of information from the entire stack and rendering it to HTML/JSON, for which it needs a decent CPU, but not much else. Object storage, like photos is quite the opposite, and requires just a simple CPU (Atom C2000) to serve object from a large storage backend. At the other end of the spectrum, you have the data crunching units that require a decent chunk of processing power, memory capacity and I/O.

Server Generations
Comments Locked

26 Comments

View All Comments

  • SuperVeloce - Wednesday, April 29, 2015 - link

    From Mass storage: "Compared to hard disks optical media touts greater reliability, with Blu-ray discs having a life expectancy of 50 years and some discs could even be able to live on for a century."

    Yeah sure. Like my expensive gold color cd's from different vendors, baked on different high quality writers, now mostly not working anymore after some 15-20 years. Despite being held in almost perfect environment all these years
  • Uplink10 - Wednesday, April 29, 2015 - link

    Someday they are going to figure out that:
    -SAS HDDs are costlier but if you are using RAID it does not matter, they should use consumer drives and not overpriced enterprise drives
    -I calculated sometimes back if Bluray cold storage is cheaper than HDDs but it is not and more so you cannot change the data once you write it, it is better to go with HDDs
  • toyotabedzrock - Wednesday, April 29, 2015 - link

    You have to wonder what these networking chip vendors are hiding in the firmware that makes them so resistant to open sourcing the code.
  • Casper42 - Monday, May 4, 2015 - link

    Johan, some of the HP info at the end was interesting, but incomplete.
    If you (or anyone reading this) plan to talk to HP, they will also talk about their relatively new CloudLine "CL" type machines as well.
    They come in standard 1RU/2RU designs as well as OpenRack designs coming soon.
    And the SL line is all being morphed over to Project Apollo which uses the XL prefix.
    Apollo 2500 is now live, 4X00 will replace SL4500, 6000 has already replaced S6500, and the 8000 was a net-new add for Gen9 focused on big HPC farms.
    So anything SL is, or soon will be, a dead platform. (The SLs you mention could be an exception since they are not widely commercially available)
  • Netpower - Tuesday, June 2, 2015 - link

    One general problem with this design is how to take care of power line disturbances entering the power shelves via the 277V AC lines. The 48V DC is filtered via the 48V battery but you must add a filter/power line conditioner somewhere to make sure that transients and sags doesn't kill your power shelves. The 380V DC approach by (http://www.emergealliance.org) is much more reliable and still have all the advantages with higher efficiency, lower cable losses etc.
  • Astana - Wednesday, September 16, 2020 - link

    It is believed that the nicotine addiction of a smoker prevents you from quitting. But modern research has shown that the quitting process is complicated by psychological addiction. That's why many people can't cope with the habit of smoking. But there are results with https://heroindetoxeurope.com/ drvorobjev hospital they're all successful.
    The problem of smoking is not only the problem of men, but also increasingly - women. Many women are wondering: "How can you quit smoking and not get better? Psychologists have proven that when a woman quits smoking, she subconsciously tries to replace, to compensate for the process of smoking with food. This feeling is also familiar to men who quit smoking, but they do not attach much importance to it. As a result, very soon an extra 3-4 kg or more will appear.

Log in

Don't have an account? Sign up now