Facebook’s latest homemade hardware is a 128-port modular switch

Facebook has been building its own servers and storage gear for years, and last June announced its first-ever networking gear in the form of a top-of-rack switch called “Wedge.” On Wednesday, the company furthered its networking story with a new switch platform called “6-pack,” which is essentially a bunch of Wedge switches crammed together inside a single box.

The purpose of 6-pack was to build a modular platform that can handle the increase in network traffic that Facebook’s recently deployed “Fabric” data center architecture enables. The Facebook blog post announcing 6-pack goes into many more details of the design, but here is the gist:

“It is a full mesh non-blocking two-stage switch that includes 12 independent switching elements. Each independent element can switch 1.28Tbps. We have two configurations: One configuration exposes 16x40GE ports to the front and 640G (16x40GE) to the back, and the other is used for aggregation and exposes all 1.28T to the back. Each element runs its own operating system on the local server and is completely independent, from the switching aspects to the low-level board control and cooling system. This means we can modify any part of the system with no system-level impact, software or hardware. We created a unique dual backplane solution that enabled us to create a non-blocking topology.”

In an interview about 6-pack, lead engineer Yuval Bachar described its place in the network fabric as the level above the top-of-rack Wedge switches. Facebook might have hundreds of 6-pack appliances within a given data center managing traffic coming from its untold thousands of server racks.

A 6-pack line card.

A 6-pack line card.

“We just add those Lego blocks, as many as we need, to build this,” he said.

Matt Corddry, Facebook’s director of engineering and hardware team lead, said all the focus on building networking gear is because Facebook user growth keeps growing as more of the world comes online, and the stuff they’re sharing is becoming so much richer, in the form of videos, high-resolution photos and the like.

That might be the broader goal, but Facebook also has a business-level goal that’s behind its decision to build its own gear in the first place, and to launch the open source Open Compute Project. Essentially, Facebook wants to push hardware vendors to deliver the types of technology it needs. If it can’t get them to build custom gear, it and dozens of other large-scale Open Compute partners can with immense buying power can at least push the Dells and HPs and Ciscos of the world in the right direction.

Corddry said there’s nothing to report yet about Wedge or 6-pack being used anywhere outside Facebook but, he noted, “Our plan is to release the full package of Wedge in the near future to Open Compute.”

6-packdiff2

If you’re interested in hearing more about Facebook’s data center fabric, check out our recent Structure Show podcast interview with Facebook’s director of network engineering, Najam Ahmad.

[soundcloud url=”https://api.soundcloud.com/tracks/178628647″ params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

Download This Episode

Subscribe in iTunes

The Structure Show RSS Feed

IBM to build two supercomputers for the U.S. Department of Energy

IBM said today that it will develop two new supercomputers for the U.S. Department of Energy that are based on IBM’s new Power servers and will contain NVIDIA GPU accelerators and Mellanox networking technology. The new supercomputers, to be named Summit and Sierra, will be ready to roll in 2017; IBM will end up scoring a cool $325 million in government contracts.

Why applications are still Microsoft’s biggest asset in the cloud

Microsoft is talking a lot about the scale of its cloud computing platform lately, but scale alone won’t help it steal revenue from Amazon Web Services or Google. Microsoft’s advantage is in commercial software, which brings more profi and acts as a gateway to other services.

The enterprise CIO needs a comprehensive strategic plan and quick

There are many who profess to know what goes on within the mind of the CIO and across the IT organization as a whole. The challenge is: If you have not been responsible for the role, it is increasingly difficult to truly understand the complicated world that encompasses enterprise IT organizations. Could they be simplified? In a word, yes. But that is easier said than done. One needs an appreciation for the demands coming from not just technology, but also from other organizations within the company and the IT organization itself. But even that statement does not provide the full depth of the complexity facing today’s CIO.

The CIO balancing act

Today’s CIO is facing a balancing act between legacy solutions, methodologies and the modern-day buzzword bingo. Whether from cloud computing, big data analytics, data center complications, new architectures, new programming languages or just simply (relatively) the changes in the business direction, the complication is far and wide. And even if a CIO agrees and wants to move to a new solution like cloud, there may be other limiting factors to consider.

IT as a strategic weapon

Strategy is not a new or foreign concept to the IT organization. The vast majority of CIOs and IT organizations have a well-defined strategy that outlines how the IT organization supports the company as a whole. At times however, strategy becomes a victim to the interrupt-driven nature of IT requests. Always being one to want to please, the latest request becomes the newest focus for the team.

One opportunity missed by many organizations is how to transition from being the “hero” to being the sought-after strategic weapon for a company. There is a big difference between the two and it resonates greatly on IT’s intrinsic value to the company. The modern-day CIO is shifting from problem solving to providing business leverage. That is not to say that the IT organization gives up the problem solving. It remains, but is table stakes in today’s IT requirements.

Spanning the industries

The shift in thinking is not relegated to a specific region or industry. Silicon Valley, including its wide geography from San Francisco to San Jose, is not alone in the opportunity. Neither are new upstarts in the web scale category. Every single industry and region has the same challenge. Recall that companies operate in a global economy and need to respond accordingly. Eat or be eaten. Even the incumbent is not immune to the changes sitting at the front door.

Cloud implementation v2.0

One way IT organizations are changing the conversation between IT and Line of Business (LoB) teams is in the introduction of cloud computing. Beyond the common use-cases (CRM, HRIS, Email, etc), the implementations vary greatly. One trend coming up is a move to ‘cloud implementation v2.0’. Organizations were quick to try cloud-based services with very mixed results. In many cases, the attempt was fairly haphazard. IT organizations are now stepping back and rethinking their approach to cloud in a more holistic fashion. Where does it apply, how, why and when? But it goes much broader than that.

Shifting gears to focus on data

In order to understand where to apply cloud, understanding the larger objective is critical. This is where data-centric conversations come into play. In the end, it is not just about the application and data, but also about the value to the company. Add in conversations like Big Data, Analytics, Internet of Things (IoT), Industrial Internet and one can see how the complexity just grew exponentially.

The clock is ticking…

The growing complexity for the CIO and IT organization does not translate to more available time. Quite the contrary. The demands that companies are placing on their IT organization are increasing exponentially. This is where a new strategic vision is needed. In order to respond in a timely manner, CIOs will need to rethink their organization, processes, focuses and technology in a holistic manner. It will take time to evolve to the new model. But timing is of the essence. The demand is here today and is only increasing.