The Evolving Role of the Network Engineer

A few weeks ago I posted a blog on what I have experienced over the past four years at Plexxi.  That post led the Packet Pusher team of Ethan and Greg to reach out and we recorded a podcast about the changing role of the network engineer and IT silos. In preparation for the podcast, my colleagues Mat Mathews, Mike Welts and I collaborated on the following that I edited a final time after the podcast. This post started as a dialog about what we are seeing in the market, what our customers and prospects want to engage about, how we position Plexxi to the network engineer and where we see this all going now that market clarity has begun to emerge.

We see network engineers on a parallel course to what Sys Admins went through or are going through with the DevOps process.  SysAdmins used to focus on the gear such as patching servers, installing operating systems and software.  DevOps is focused on outcomes – quickly and systematically get applications deployed. This has led the SysAdmin to break down the process of application deployment into a set of workflows versus individual hardware and software components; or said differently IT silos.

Rather than focusing on individual boxes and configurations, the role of the network engineer will evolve to become focused on outcomes. Rather than the mechanics of the tasks, the outcome of the tasks becomes the measurable component of success or failure. For example, ensuring that applications get what they need from the network is a measurable outcome and is vastly more than just scripting it is really about design and system properties. Focusing on the design goal of the system forces the network engineer to break down application workflows into events that can be systematically categorized and used to automate network behavior.

A far more interesting day job is solving for the where, how and when problem for network resources which includes but is not limited to how bandwidth is allocated. In the Packet Pusher podcast, Greg references the absence of computer science degrees. Until he said that during the recording, I had not thought about that point. In the role we see for the network engineer, the daily task becomes more about architecture and outcomes, than command line proficiently and this role does draw on traditional computer science skill sets.

During the lead in to the podcast, Ethan referred to several network design constraints that I had posted from Google. My point was not a technical discussion on centralized versus distributed state. The point was the team at Google built a system to deliver an outcome and the network team was tasked to find solutions to several traditional networking limitations. The network team was not an isolated silo, shunted off to a windowless office to tweak parameters via the CLI.  The network team was important part of the combined IT team delivering on the design outcome of the system and the measureable design outcome metric was application performance and experience.

Network Engineers need to grab the bull by the horns and start to really understand what drives their business. They need to learn to decompose what have been traditional IT silos into services that the network can provide to make better for the business.  We think the network engineer has an opportunity to lead and push on their technology vendors to give them the capabilities and tools to make their IT systems better.

Network Engineers have an opportunity to evolve beyond being the smartest product practitioner and into a champion for the agile business.  For the past four years I have heard many IT managers/CxOs in companies bemoan the networking team for the time it takes them to do what should be simple and easy tasks. Why does it take a trouble ticket to change a VLAN when an application needs a change?

One way to solve the agility gap is to begin to script all of the network engineering tasks so they can be executed with less typing – but the other way is to break down what the business needs into events, and build a network and infrastructure that actually understands those events. This is what the webscale companies have done and most people miss the forest for the trees and focus on white boxes and other red herrings when they talk about webscale. This leads to the next question of what really excites network engineers that we speak to? It is definitely not spanning tree and CLI commands, but rather the construction of the logic and policy management of dealing with business/application level events. Translated into business relevancy, a network engineer who can perform at a high level in the role of mapping business logic to IT system outcomes is extremely valuable to their employer.

What we have learned in talking to our customers and prospects is what they really want is an automated infrastructure that serves the needs of their applications, which are the customers of infrastructure.  The new cloud architectures will let the applications dynamically drive compute and storage resources.  Historically we have designed for peak, fixed bandwidth between all nodes, which forced network engineers to over buy and grow into the design or under buy and have a frequent upgrade events.   Neither approach was desirable.  The cloud model approach lets a network engineer build a network that follows the consumption model dynamically and also lets you incrementally add resources.  Examples here are Google and Amazon and I am not suggesting that enterprises think of building on that scale, but I am suggesting that the value that the web giants showed the world is that by automating the infrastructure as a whole, they could build at scale and have a much more responsive infrastructure and this would not have been possible without the network team.  The cloud giants did not sit around and build these huge infrastructure systems without the network engineer.  The network engineer was an integral part of the team and I think the future role of the network engineer in the enterprise will be focused on breaking down application workflows into events that can be systematically categorized and used to automate network behavior, which I think is a far more interesting day job than being a CLI jockey, but this cannot be done well without the core knowledge that comes with being a network engineer.

Dealing with Reality

The half-life of traditional applications is long, but they are certainly in decay. Every time a legacy application is re-architected, it moves to a converged model whereby the application components expect to be dynamically matched to resources as needed.

The decision to buy or rent will always continually be re-evaluated based on economics and switching costs. Switching costs will start to come down as application architectures start to standardize on a common set of portable technologies, but that will take time. There is a parallel to car: you rent a car for specific times/needs and you buy a car for specific times/needs, your IT needs will vary and enterprises need to not trade high fixed cost asset burden for a high usage cost

Network engineers need to look back into the business. If the business is being driven by rapid development of new applications/services, they have likely started to develop around some of the existing network limitations, which is not always ideal (e.g. only vMotion/DRS within the rack, don’t do backups during the day, etc.). The new process is to start to develop classes of events that create network impacts and look at how a new software-driven network approach could provide services to respond to those events and give the business back some degrees of freedom they did not think was available.

/wrk