What DevOps Means for Static Analysis
Editorial Note: I originally wrote this post for the NDepend blog. You can check out the original here, at their site. While you’re there, have a look at NDepend’s static analysis offering.
For most of my career, software development has, in a very specific way, resembled mailing a letter. You write the thing, and then you go through the standard mail piece rigmarole. This involves putting it into an envelope, addressing the envelope, putting a stamp on, it and then walking it over to the mailbox. From there, you stuff it into the mailbox.
At this point, you might as well have dropped the thing into some kind of rip in space-time for all you understand what comes next. Off it goes into the ether, and you hope that it arrives at its destination through some kind of logistical magic. So it has generally gone with software.
We design it, architect, and lovingly write it. We package it up, test it, correct defects in it, and then we call it done. From there, we fire it into the mailbox-black-hole of the software world: operations. They take it and deploy it, or whatever, and then, by some magic we don’t concern ourselves about, it runs in the real world. Or so it has generally gone.
Problems with the Traditional Approach
With the benefit of hindsight, you can probably guess the main problem with this state of affairs. So rather than enumerate it dryly in a series of bullet points, let me offer it up in story format.
You work as an application developer in some very large enterprise. There, you build web apps. And you take pride in your work. You write clean code, you maintain the unit test suite, you collaborate dutifully with QA, and you generally do your best.
In fact, this effort even extends beyond your own dev environment and into as many environment as you can see. You run load, smoke, and integration tests in QA and sandbox environment. And, as a whole unit, your team does everything it can to ensure the integrity of the work. But beyond the pre-prod environment, the fate of your application becomes an utter mystery. Some group of folks located in a different timezone take it from there. You wish it well as it heads to production.
And then, one day, six months later, you get some incident report. Apparently, some guy in Hungary or somewhere was doing something when somehow he get a null reference exception. But don’t worry, here’s a brief description of what he said and a few thousand lines of some random log file. Good luck with your repro!
Enter DevOps
No doubt this sounds at least passingly familiar, especially if you’ve been at the software game for a while. Personally, I cannot tell you how many defects I’ve closed in my life with “closed, could not reproduce.” I did this not out of a lack of diligence, but because I found myself over-matched. As a developer, I was so far removed from production issues that reproducing them evoked images of needles and haystacks.
The DevOps movement, among other things, seeks to remedy this problem. It eliminates the historically massive bureaucratic divide between building and running software — between the development and operations organizations. The developers pair up with operations folks and concern themselves with the particulars of running and deploying software.
DevOps carries enormous significant for the business of software and for release cadence and customer support. But on this blog, we talk about static analysis. What impact does the DevOps movement have there?
Faster Releases
You can play the chicken and the egg game with DevOps and faster releases. I’d personally put my money on the faster release cadence occurring first, since agile practitioners have stumped for that since before even the agile manifesto. Tighten the feedback loop, they advocated.
As this happened and shippable increments of software dwindled from years to mere weeks, the mechanisms for packaging, deploying, and releasing software had to race to keep up. This led to more automation in these areas, which naturally gives rise to DevOps, informally, and then as a movement.
This deployment pace acts as a bottleneck. But removing this bottleneck also removes some slack time for checking, double checking, and peer reviewing code. Generally speaking, the faster pace from “dev done” to “in the wild” puts added emphasis on automated checks and review of code. Static analysis becomes critical to simultaneously sustaining pace and quality.
Automation Prioritized with Developers Spread Thinner
If removing the deployment bottleneck spread developers a bit thin, then DevOps itself spread them even thinner. In addition to their normal responsibilities writing the software itself , they now had other duties. They had to concern themselves with automating operational concerns. This might involve simply pairing with the operations folks, but it also might mean learning new tools, writing scripts, or even building applications.
So you can imagine the result. Take slack time away from developers. Then take actual development time away from them. You could always hire more software developers, but budgets frequently don’t allow that. Instead, you need to economize for developer time in any way that you can.
Operational Issue Reduction Presents New Potential Bottlenecks
I’ll add a third consideration to the mix. And this one provides yet a third reinforcement of the importance of static analysis in the age of DevOps. Historically, deployment activities happened infrequently, and presented huge, ungainly messes. I can myself recall observing nightmarish, multi-week deployments that exhausted all parties involved. By the time the dust settled, the application crashing a few times a day seemed like a pleasant respite by comparison.
As deployments became more frequent, smoother, and relative non-issues, they stopped drowning out all other stars in the sky with the sheer magnitude of their brightness. Code quality issues, technical debt, and escaped defects started to look uglier and uglier by comparison. In general, user expectations from software have gone up.
This renders static analysis more important than ever for shining a light on potential issues and for helping you catch code quality problems in the early stages and before they become runaway technical debt.
The Verdict
I think it’s fair to say that the evolution of software techniques in general, and, specifically, the advent of DevOps have made automated code analysis more important than ever. This holds true both for dynamic analysis (APM and the like) and static analysis.
In fact, I might even argue that you could consider static analysis an integral part of a DevOps approach. Don’t worry, I won’t muddy the waters further around what DevOps is and is not by weakening the definition. Enough people are doing that already. But I will go so far as to say that having automated testing and code analysis is table stakes for any sophisticated automatic deployment setup.
A bit tangential to your point about static analysis, but I just want to gripe: I’m currently working for one of these organizations that define DevOps as “Devs do the Dev *and* the Ops”, but I think it’s a big metaphorical Penguins on Icebergs fail. I suspect that organizations falsely view DevOps as a pure dichotomy between Devs Doing All The Ops, versus Devs Kicking Their Code Over The Wall Into The Ops Silo, whereas in my experience the best arrangements have been somewhere in between, such as with an Ops person embedded in each team, and with the Devs… Read more »
FWIW, I empathize. I certainly recognize the philosophical business value of the right, cross-disciplined approach, but provisioning servers and automating deployments doesn’t really interest me either.