Scale Out Networking and Solving Big Problems Matter

Another week has passed without a post, but it has not been because I was not thinking about posting.  It really has been a struggle to find time to write.  I want to continue to expand my thoughts about the datacenter specific to the network.  All the prior stuff I have written about building the network vertically and Moore’s Law Exhaustion (MLE) are well threaded throughout this presentation by James Hamilton.  See slide 6 for specifics to the following.

When I listen to storage and compute people tell me about the how much information exists and how fast we can process information, it is all noise to me unless the network can scale to meet the demands of compute and storage.  If you want to read a recent study on these trends, this paper in Science Magazine is a good start.  Big Data as the Wall Street analysts like to refer to it is meaningless if the data is predominantly static or difficult to acquire and process.  What the Science Magazine paper misses is the significant change that is going to occur in the network in the next five years.  The change I am referring to I have mentioned before, but I am going to attempt to put some numbers around the event.

The transition from 1G to 10G server connections in the datacenter is going to be a significant event and I would submit that it will end the era in which (1) networks are built vertically with oversubscription and (2) end the era of Moore’s Law for networking.  I have posted in the past that I was not a believer in the whole exabyte flood thesis and the doom and gloom warnings about video clogging the internet.  This was all just silly, lazy analysis.  Networks are built with throttles and congestion blocking points all over the place; they are called oversubscription ratios and routers.  Try building out datacenter network models and scaling the bandwidth from the rack server to another rack server in the datacenter and then out of the datacenter to another datacenter.  I spent a good week doing this exercise which further convinced me that networking is going to change.  I have some ideas on that, but the point of this post was to attempt to put some numbers around the 1G to 10G transition in the datacenter and not detail my biases on network design in the future.

Below is model I have been working on using (i) 1G and 10G server port projections and (ii) MPU shipments from Intel and AMD.  There are many ways to change the model based on assumptions of networking ports, future sales, server life-cycles, etc.  I tried to keep the model at least feeling real without making any outlandish predications.  I did not calculate for high rates of VMs in the model.  This model does not even attempt to predict traffic levels.  The calculations this model is making reflect the installed network capacity of servers in datacenters for the next four years.  The conclusions do not mean that future capacity will be used 100%.  I did not even try to add in CPU utilization rates.  All I am calculating in this model is how much network capacity in Petabytes (PBs) will be installed in datacenters in the next four years.  I used two forecasts to get to the network capacity projections.

The first set of numbers I used was MPU shipments from INTC and AMD since 2003 and projections to 2015.  The second set of numbers was the 1G and 10G server port shipment forecasts.  The models diverge, but both show a significant ramp in server network port capacity.  At present we are probably around ~55 PBs of installed network capacity.  Over the next four years, this capacity is expected to grow from 55 PBs to ~750 PBs or more; maybe an exabyte of network capacity add in four years.  To put that into perspective, I would say that if a high rate of utilization of this capacity is achieved, parts of the network break and if the network cannot scale to support this capacity, then it is not worth funding the upgrade.  I do not believe the later, I believe some smart people will find a way to put this amount of capacity into the network and the practices of the past will not be the practices of the future.  If you like solving big problems, I just described your next career endeavor.  I would also state that this type of network capacity growth changes the fundamentals of how networks are built – hence it could usher in a new set of leaders for networking.

I call this scale out networking.  I have called the last ten years the Lost Decade for Venture Capital and I have written that large scale venture capital backed innovation might dead.  I followed up the lost decade post with some more thoughts here, but I am now thinking that the large scale VC backed innovation may not be dead.  As usual, I could be wrong.

/wrk

* It is all about the network stupid, because it is all about compute. *

** Comments are always welcome in the comments section or in private.  Just hover over the Gravatar image and click for email. **