SYS-CON MEDIA Authors: Xenia von Wedel, Peter Silva, Glenn Rossman, Ava Smith, Elizabeth White

Blog Feed Post

Automation through workflow state

The benefits of automation are well understood: more agile service provisioning, faster time to insight when there are issues, and a reduction in human error as manual interaction is reduced. Much of the premise behind long-term SDN architectural advantages is steeped in the hope that SDN will help enable and ultimately promote automation. But while centralizing control has significant operational advantages, by itself, it doesn’t actually address the most important requirement for automation.

If automation is going to be more than just reducing keystrokes, there will have to be a rise of workflow state.

Referential space

Successfully managing a network is an exercise in constant iteration through network state. Whenever something needs to be done, the architect or operator examines her current frame of reference to figure out the starting point. That frame of reference usually starts with some implicit understanding of how the network is designed. From there, she takes some action. Maybe she pings an endpoint, checks the state of a BGP neighbor, or examines some interface statistics. Whatever the first step, the point is that she knows when she starts that there is work after the first step.

The information gleaned from the first step yields additional understanding. Her frame of reference changes as she now knows more than before. With her new position in referential space, she takes the next step. And the next, and the next after that. Each step yields a different piece of information, and the process of iterating through a constantly changing referential space ultimately yields some outcome or resolution.

Byproducts of iterative workflows

There are two major byproducts of this iterative approach to workflow. The first is that the starting point is rarely based on an absolute understanding of fact. Rather it is an interpretation that the individual operator or architect creates based on a number of somewhat soft conditions – knowledge, experience, intuition, whatever. This means that for each task, the workflow is somewhat unique, depending on the operator and the environment.

The impact here is important. If workflows are unique based on the operator and the conditions (i.e., the referential space or frame of reference), then the outcomes driven by those workflows are difficult to repeat. Part of why networking is so hard is that so much of it borders on arcane dark art. Science demands repeatability, but the very nature of workflow management in networking makes that challenging.

The second byproduct of networking’s iterative nature is that workflows frequently depend on a set of chained tasks, each of which has a dependency on the preceding task. To make things worse, that dependency is actually rarely known at the start of a a workflow. It’s not that tasks cannot be predictably chained – first, you look at the physical layer, and then you move up stack perhaps. But each subsequent task is executed based on not just the previous task but also the output of the previous task. This creates a complex set of if/then statements in most workflows.

Part of the challenge in automation is providing the logic to navigate the conditional nature of networking workflows.

“Network engineers need to think like programmers”

With the rise of movements like DevOps, “network engineers thinking like programmers” has become a popular phrase. This is a very important change in how we handle network architecture and operations. But there are subtleties here that get lost in the cliche.

First, when people toss the phrase around, they often mean that network engineers need to pick up a scripting language (Python, Ruby, even Perl). Thinking like a software developer has very little to do with programming languages. Languages are a way of expressing intent, but it’s entirely possible to know Python and think nothing like a developer.

Second, when people refer to programming in the context of DevOps, they generally mean that network operators need to think about configuration less as a collection of commands and more like code. Once you make that shift, then you can think about things like source code management, automated testing, and rapid deployment.

But networking needs to do more than just treat configuration as code.  DevOps has more to do with deploying and validating changes. It doesn’t fundamentally change how workflows are executed, and it barely touches more operational tasks like troubleshooting network conditions.

Before anyone picks a religious battle over DevOps here, my point is not that DevOps is bad. It’s just that DevOps by itself is not sufficient. And there are things that ought to be done that are separate from DevOps.

Tiny feedback loops

So if thinking like a programmer isn’t about learning a programming language and it’s more than treating configuration as code, what is it?

Software development is really about creating something out of lots of tiny feedback loops. When you write functions, you don’t just execute some task. You generally execute that task and then return a value. The value provides some immediate feedback about the outcome. In some cases, the function returns the value of a computation; in other cases, it simply returns an indication that the function succeeded or failed.

These values are obviously then used by other functions, which allows us to string together small building blocks into complex chains. The important part? These chains can then be repeatably executed in a deterministic way.

Networking workflows shouldn’t be that different. Each individual activity yields some value (sometimes a specific value as when looking at some counter, other times a success or failure as with a ping). The problem is that while networking commands frequently return information, it is up to the operator themselves to parse this information, analyze what it means, and then take the next action.

Workflow state

What we need if we really want to make automation happen in ways that extend beyond just scripting keystrokes is a means of creating deterministic networking workflows. For this to happen, we need people who construct workflows to think more like developers. Each activity within a workflow needs to be a tiny feedback loop with explicit workflow state that is programmatically passed between workflow elements.

We actually instinctively do this at times. XML, NETCONF, and the like have been used to encapsulate networking inputs and outputs for awhile with the intent of making things parseable and thus more automatable.

But we stopped short. We made the outputs more automation-friendly without ever really creating workflows. So while we can programmatically act on values, it only works if someone has automated a particular workflow. As an industry, we haven’t gotten to actually addressing the workflow problem.

Maybe it’s the highly conditional nature of networking combined with the uniqueness of individual networks. Or maybe it’s that outside of a few automation savants, our industry doesn’t generally think about workflows the way a software developer would.

The bottom line

Networking workflows rely way too heavily on an iterative pass through referential space. The reason change is so scary and troubleshooting so hard is that very little in networking is actually deterministic. But if we really want to improve the overall user experience en route to making workflows both repeatable and reliable, we do need to start thinking a bit more like developers. It all starts with a more explicit understanding of the workflows we rely on, and the expression of feedback via some form of workflow state.

And for everyone betting on abstractions, just know that abstracting a poorly-defined workflow results in an equally poor abstraction. We need to be starting elsewhere.

[Today’s fun fact: Only male fireflies can fly. Take that, females!]

The post Automation through workflow state appeared first on Plexxi.

Read the original blog entry...

More Stories By Michael Bushong

The best marketing efforts leverage deep technology understanding with a highly-approachable means of communicating. Plexxi's Vice President of Marketing Michael Bushong has acquired these skills having spent 12 years at Juniper Networks where he led product management, product strategy and product marketing organizations for Juniper's flagship operating system, Junos. Michael spent the last several years at Juniper leading their SDN efforts across both service provider and enterprise markets. Prior to Juniper, Michael spent time at database supplier Sybase, and ASIC design tool companies Synopsis and Magma Design Automation. Michael's undergraduate work at the University of California Berkeley in advanced fluid mechanics and heat transfer lend new meaning to the marketing phrase "This isn't rocket science."

Latest Stories
SYS-CON Events announced today that Windstream, a leading provider of advanced network and cloud communications, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Windstream (Nasdaq: WIN), a FORTUNE 500 and S&P 500 company, is a leading provider of advanced network communications, including cloud computing and managed services, to businesses nationwide. The company also offers broadband, p...
The 4th International DevOps Summit, co-located with16th International Cloud Expo – being held June 9-11, 2015, at the Javits Center in New York City, NY – announces that its Call for Papers is now open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's large...
"There is a natural synchronization between the business models, the IoT is there to support ,” explained Brendan O'Brien, Co-founder and Chief Architect of Aria Systems, in this SYS-CON.tv interview at the 15th International Cloud Expo®, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Verizon Enterprise Solutions is simplifying the cloud-purchasing experience for its clients, with the launch of Verizon Cloud Marketplace, a key foundational component of the company's robust ecosystem of enterprise-class technologies. The online storefront will initially feature pre-built cloud-based services from AppDynamics, Hitachi Data Systems, Juniper Networks, PfSense and Tervela. Available globally to enterprises using Verizon Cloud, Verizon Cloud Marketplace provides a one-stop shop fo...
Leysin American School is an exclusive, private boarding school located in Leysin, Switzerland. Leysin selected an OpenStack-powered, private cloud as a service to manage multiple applications and provide development environments for students across the institution. Seeking to meet rigid data sovereignty and data integrity requirements while offering flexible, on-demand cloud resources to users, Leysin identified OpenStack as the clear choice to round out the school's cloud strategy. Additional...
The major cloud platforms defy a simple, side-by-side analysis. Each of the major IaaS public-cloud platforms offers their own unique strengths and functionality. Options for on-site private cloud are diverse as well, and must be designed and deployed while taking existing legacy architecture and infrastructure into account. Then the reality is that most enterprises are embarking on a hybrid cloud strategy and programs. In this Power Panel at 15th Cloud Expo (http://www.CloudComputingExpo.com...
We are all here because we are sold on the transformative promise of The Cloud. But what good is all of this ephemeral, on-demand infrastructure if your usage doesn't actually improve the agility and speed of your business? How must Operations adapt in order to avoid stifling your Cloud initiative? In his session at DevOps Summit, Damon Edwards, co-founder and managing partner of the DTO Solutions, will highlight the successful organizational, process, and tooling patterns of high-performing c...
The definition of IoT is not new, in fact it’s been around for over a decade. What has changed is the public's awareness that the technology we use on a daily basis has caught up on the vision of an always on, always connected world. If you look into the details of what comprises the IoT, you’ll see that it includes everything from cloud computing, Big Data analytics, “Things,” Web communication, applications, network, storage, etc. It is essentially including everything connected online from ha...
Software-driven innovation is becoming a primary approach to how businesses create and deliver new value to customers. A survey of 400 business and IT executives by the IBM Institute for Business Value showed businesses that are more effective at software delivery are also more profitable than their peers nearly 70 percent of the time (1). DevOps provides a way for businesses to remain competitive, applying lean and agile principles to software development to speed the delivery of software that ...
Docker offers a new, lightweight approach to application portability. Applications are shipped using a common container format and managed with a high-level API. Their processes run within isolated namespaces that abstract the operating environment independently of the distribution, versions, network setup, and other details of this environment. This "containerization" has often been nicknamed "the new virtualization." But containers are more than lightweight virtual machines. Beyond their small...
The move in recent years to cloud computing services and architectures has added significant pace to the application development and deployment environment. When enterprise IT can spin up large computing instances in just minutes, developers can also design and deploy in small time frames that were unimaginable a few years ago. The consequent move toward lean, agile, and fast development leads to the need for the development and operations sides to work very closely together. Thus, DevOps become...
Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.

ARMONK, N.Y., Nov. 20, 2014 /PRNewswire/ --  IBM (NYSE: IBM) today announced that it is bringing a greater level of control, security and flexibility to cloud-based application development and delivery with a single-tenant version of Bluemix, IBM's

An entirely new security model is needed for the Internet of Things, or is it? Can we save some old and tested controls for this new and different environment? In his session at @ThingsExpo, New York's at the Javits Center, Davi Ottenheimer, EMC Senior Director of Trust, reviewed hands-on lessons with IoT devices and reveal a new risk balance you might not expect. Davi Ottenheimer, EMC Senior Director of Trust, has more than nineteen years' experience managing global security operations and asse...
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at Internet of @ThingsExpo, James Kirkland, Chief Ar...