Encountering the Hamilton Manifesto

More than five years ago James Hamilton of Amazon fame, posted on his personal blog a presentation he gave about networking called Datacenter Networks Are in My Way.   Here is a link to his post and my last check showed that the slides to the presentation are still available. I copied four of the slides in the thumbnail to the left to save the reader from a click out.

Hamilton’s post was interesting to the networking industry because it came from someone who was building at web-scale and at a time that the early seeds of SDN had been planted, but not yet harvested. This is not going to be a post about SDN, it is not about white boxes versus traditional chassis switches, network fragility or how web scale companies build networks versus enterprise companies. This is going to be post about what I call the Hamilton Manifesto and the first encounters with the manifesto in the market.

I recently wrote a post and described the evolution we have seen in the market around the collapsing of IT silos. About this time last year, I started speaking to the various vendors and end-users in the converged infrastructure, scale-out storage and scale-out application markets. As the vendors competing in these market verticals began to increase their deployment sizes, something interesting happened, the conversation turned to the network being in the way of the deployment. To be clear, I am not referring to treating switches as servers, I am talking about the design of the network and how it operates from an engineering perspective. If you are not familiar with what I am referring to I will write a follow-on post which will be an adaptation of this post , which I wrote on the Plexxi blog describing the Modern Era of networking versus the legacy Protocol Era that we have been living in far too long.

The network as an obstacle has been typically framed to me in these generalized vignettes:

[Storage/CI Vendor Perspective] – I know why people buy my solution; they buy it because they want my storage, but as we try to sell them more racks, the networking part becomes more complicated. Once we go beyond the ToR, we have to go sell the networking team in the account and they really don’t care that we are selling storage and compute. My sales teams do not know how to sell to the network guy and all this step does is slow down sales and everyone gets frustrated.

[Enterprise Cloud Deployment Team] – We have a traditional network that we upgrade every few years and we have been using the same network vendor for 10+ years. We tried a software defined data center (SDDC) a year ago and it worked, but it was not enough. We now want to try to attack the hardware part of the network. We want a network that we can tailor to our application needs, but the design we have been using for years is static and brittle.

[Webscale SAAS End-User] – We looked at SDN in 2012 and decided it was too immature and we stayed with tried and true L3 Fat Tree / CLOS network model. Four years later we are spending too much time scripting stuff to program switches, we have all these complex DC network designs. There has to be a way to build a network that allows us to scale out in the same way we add compute and storage and just works.

[CI Vendor] – We have had a few years of great success, but the deployment clusters were small to just a few racks. Now we are trying to deployments of tens of racks and rows and the all the network engineering required just to build at scale puts a damper on the larger deals. Small deals of rack or two are no problem – 20 racks of CI is major undertaking.

The big Red Herring in networking over the past ten years is most things labeled “innovative” when the word that should be used is “iterative.” What most users want is a better network design a Modern Network – not more tools for the legacy Protocol Era network design.  That will be the subject of the next blog post.

/wrk