Stories about Software


Agile: False Hope and Real Promise

I went shopping for jeans last weekend, which was, as always, somewhat of an aggravating experience. It’s not just jeans, but, really, any wearable thing: shoes, work shirts, belts, even socks. The source of my irritation is the fact that I can’t simply replace items I have that are worn out because the thing you bought last time is always decommissioned. Strangely, major things like cars have better staying power than clothes. There’s a very simple use case missing: As a shoe wearer, given that I love a pair of shoes and thus wear them out in 2 years, I want to go back to the same store and buy the exact same pair of friggin’ shoes so that my life is not interrupted. And the software of life fails to perform.

The reason for this is the nebulous concept of fashion. Naturally, wearable manufacturers and retailers don’t want to be ‘stale’ so they churn things at a rate that prevents me from being pragmatic with my shopping and turns my trips into maddening treasure hunts instead of simple errands. So I contemplate fashion a bit with a natural bias toward annoyance, and this has informed a pretty cynical take on the subject. I have an ongoing hypothesis that the thing that drives fashion isn’t some guy sitting in Paris, saying “I’m going to make those idiots wear really tight jeans this year and then laugh at them.” Instead, its the shifting tectonic plates of poor people trying to look like rich people and rich people trying not to look like poor people.

By way of narrative, guy in Paris releases skinny jeans and charges $700 for them. They’re new and expensive, so rich people buy them so that they can be seen wearing new and expensive things — signaling their wealth and cosmopolitan tastes. Bulk clothing manufacturers start knocking off Paris designer and selling it to the masses for $39 a pair and they sell out like crazy as everyone wants to ape the A list celebrities and influencers wearing these things. Rich people get annoyed that they’re no longer distinguishable on sight and this demand drives Paris guy to dream up some new, weird thing that they can buy to look rich. Rinse, repeat.


This narrative isn’t limited to the arena of rich and poor people with clothes, however. It also extends to things like industry thought leadership and buzzwords. Crowd of cool kids starts doing something and succeeding/making money. They give it a name and everyone rushes to mimic their success, adopting this thing. Imitators adopt it en masse, bring mediocrity and misunderstanding to it, and the cool kids lament that everyone is getting it wrong and that it’s time for another industry shake-up. Queue the “Is {Insert Tech} Dead” link bait and search for the next great thing.

“Agile” is probably the poster child for this sort of thing in the development world. Back around the turn of the millennium, various industry thinkers were experimenting with ways to move software development away from software development practices that mimicked construction and physical engineering since the results produced by this traditional, “waterfall” approach were spotty at best. A group of representatives of some of these experiments came together to find common ground and the result was called, “The Agile Manifesto.” In the interceding 13 years, the industry has come to accept that the Agile approach is “the right thing” while agreeing that, by and large, no one does it right.

Off the top, I’ve heard the following expressed recently (paraphrased from my recollection):

  • Agile is dead.
  • Scrum is a scam that makes grunts stand while they deliver their status reports.
  • If XP doesn’t work it’s because you’re doing it wrong.
  • Agile is stupid because you really can’t work sanely without comprehensive documentation and specs.
  • Agile is a marketing gimmick to create phony demand for meaningless certifications.

And yet, you’d be hard-pressed to find any non-agile shop that didn’t look down shamefacedly and mumble “yeah, we do waterfall.” ┬áSo, the reasonable conclusions from this are:

  • Waterfall (or pseudo-waterfall things like RUP) are really out of style.
  • Agile is mainstream, but the real fashionistas are so over it because everyone’s messing it up and ruining it.
  • Cool kids are looking for what’s next: a post-Agile world.

Agile, Distilled

I’m not certified in Scrum or XP or anything else. I have no project management credentials and no letters after my name. I certainly have experience in these methodologies and understand the ceremonies and the roles, but I’d hardly count myself an encyclopedia of how everything should operate. It’s always bemused me that arguments emerge over the finer points of the exact approaches to these methodologies and that people can actually be certified in exact adherence when the focus of the Agile Manifesto seems to me to be best summarized by the idea of bringing pragmatism and empirical experimentation to software development. But, opportunities for snark notwithstanding, it’s easy to see why people become disillusioned. Anytime there is some sort of process and any failures are blamed on a lack of true, absolute adherence, you have a recipe for quackery.

But at it’s core, I think Agile methodologies can be distilled to a single compound principle: tighten the feedback loop and improve the feedback. Think about the different things that are done in various flavors of agile: automated testing/TDD, short sprints, pair programming, retrospectives, big/visible charts, etc. All of it is about delivering feedback faster, delivering feedback more obviously, and acting quickly on the feedback. This is really, really non-controversial. The sooner you get feedback on what you’re working on, the sooner you can improve. So, where does everything go off the rails?

Is it that things don’t go well because people aren’t following the processes right? Is it that the processes themselves are broken? Is it that the processes are good, but a bunch of hucksters are peddling fools’ gold? Is it that people are just too entrenched in their ways to be compatible with these principles? Personally, I don’t think it’s any of this. I think that the fundamental problem is one of expectations (though some of the hucksters do nothing to mitigate this). Specifically, I think the main source of woe is the assumption that “going Agile” will cure what ails you because the only thing standing between the broken, over-budget pile of crap you’re about to ship and software nirvana is having a different set of meetings and adopting a different desk arrangement. The Agile promise isn’t realized by tweaking process; it’s realized by getting a scheme in place for receiving meaningful feedback and using that feedback, with a lot of practice, to get better at making software.

10,000 Hours

Wait, what? Practice? That doesn’t have anything to do with Kanban boards, swim lanes, stand-ups, and all of the other Agile stuff that gets so much focus. In the words of the esteemed Alan Iverson, “What are we talking about? Practice? We’re talking about practice, man.” The wide world is talking about the magic dust you can sprinkle on a team to make it crank out better software, faster, and I’m in here talking about practice.

During my semi-weekly drive to Detroit, I’ve been listening to audio books, including, most recently, Malcom Gladwell’s “Outliers.” Without offering up too much of a spoiler, one of the things that he talks about is that people like Bill Gates don’t reach the incredible levels of success that they do by being dominant in a vacuum, but rather they combine incredible fortune and opportunity with cleverness, ingenuity and a whole lot of really hard work. Bill Gates was born at a time where him being a teenager coincided with the very first machines that allowed for desktop based computing, rather than punch cards. He had the incredible good fortune of gaining access to such a computer in the late 60’s, as a teenager, when almost no adults or professionals on Earth had such access. Addicted to the fun of programming, he worked nights and weekends during his teenage years, programming, at a time when few others in the world had that opportunity. By the time he was an adult, ready to start Microsoft, he was one of the few people on Earth with 10,000 hours of experience programming this way.

The hypothesis for which the book is best known is that 10,000 hours of deliberate practice is the general average amount of time put in before a person acquires mastery of a field. As I understand it, there has been some corroboration of this hypothesis as well as some counter-evidence, but the broad point remains that achieving mastery requires a whole lot of deliberate work and concerted, sustained effort to improve. You know how you get really good at delivering software? Do it for years and years, constantly learning from your mistakes. You know how you don’t get really good at delivering software? By having slightly different meetings, shipping crap more frequently, and hoping for different results.

The way I see it, big A Agile, at its core, is not about fixing you or your software. It’s about making your practice more meaningful and efficient. To return to the basketball metaphor, Agile isn’t a weekend retreat that turns you into Michael Jordan. It’s a weekend retreat that shows you how to do a bunch of layup and dribbling drills that, after 10,000 hours of practice, will make you pretty good, as long as you keep challenging yourself. The great shame of Agile or any other kind of movement is that of false hope in easy fixes. Done well, it will expose your pain points more quickly, provide you with tools for removing obstacles, illuminate a path to improvement, and show you how to self-evaluate regularly. But it won’t make you awesome. Only practice and continuous learning will do that.

newest oldest most voted
Notify of
Andy Bailey
Being in the software development business myself I find it both amusing and frustrating to see so called “professionals” take Agile and try to fix the processes in stone thus achieving the very opposite of agility. I was in a meeting this very week where I was told that my team don’t follow Agile practices properly because not everyone has a role, that we don’t do this, that and the other according to so called Agile standard practices. My rejoinder that one of the pillars of Agile is flexibility in approach fell on deaf ears and was poo-pooed with “You… Read more »
Erik Dietrich
“You cannot be agile without following the standard…” ouch. The irony. That’s definitely an anti-pattern (in my opinion) that I’ve observed in various places and one of the big symptoms you see is absurd arguments over “who is more agile,” which actually remind me of the clunky old “Capability Maturity Model” concept that we’re supposed to be moving away from. We humans do seem to love our rank culture. On the other side of the spectrum are teams that change the name of their status meetings to “standup” and say they’re doing Scrum. I think this is where a lot… Read more »
Andy Bailey

Feedback in the development cycle is as essential as any other part of it. Without it the team cannot react and incorporate necessary changes to a project.
The “other team” have no QM process, no feedback from Product Management and little from their public testers. There is also no process by which this feedback is incorporated into their development process except on an ad hoc basis.
We have all of this as well as a fairly relaxed and lightweight Scrum process.

Jeff S.

“mindless process following”

This is what resonates with me. This is what really gets under my skin. Not just with software development, but anything that people blindly do or think. Legions of wanna-be’s ignorantly regurgitating whatever in order to impress others.

Following a recipe doesn’t make you a chef.

Erik Dietrich

I hadn’t heard it before, but I like that quote. And absolutely true.

Geoff Mazeroff
This post is apropos given I just watched Leon’s talk about keeping software weird (http://vimeo.com/54042336 — about 30 mins long). He talks about agile as a subculture and now it’s gone mainstream, so it’s strayed from its roots a bit. I also enjoyed his comment about how many of us look to the bright spots (heroes, processes, etc.) and think “If I only eat what they eat and write with the same pen and live in the same city… I can be successful too!” Thanks for thoughtful commentary; it certainly made me reflect on my own successes and failures (just… Read more »
Erik Dietrich

Will have to give the talk a watch tonight on the treadmill (hotel wifi permitting). Your last sentence is a nice summary of the attitude that originally inspired this post. There’s no magic bounty of low hanging fruit that you can pick to get good at software — it’s a skill that requires practice. Agile gives you better ways to practice and improve, not better software.

Amitai Schlair
I care about being wrong faster so I can be right sooner (example: How to efficiently learn a programming language). Therefore I care about being willing to be wrong, being able to tell when I’m wrong, and figuring out who can help me determine whether I’m wrong. That’s what I want from — and provide for — the people I work with. If we’re always finding ways to be wrong faster, together, then we’re almost certainly doing our work effectively. If we’re not working together to be wrong faster, then we’re not — and it’s that feeling of effectiveness I’ll… Read more »
Erik Dietrich

Fail fast. Most definitely. I think that having a culture where it’s okay and encouraged to be wrong (at first) is critical. How many points are we going to deliver sprint 1? Who knows — let’s guess. Our guess will be wrong and we’ll know more next week.

Andrew Hopkinson

Better even still, do some work up front and try and reduce failure. A lot of times you don’t have the luxury of “fixing it later” or a dedicated test team. I worked in a team with an Agile devotee, who’s mantra was “do it now and fix it later”. That came back to bite us in the butt time and time again.

Erik Dietrich

I could certainly get on board with “failure prevention > fail fast > fail slow,” provided the failure prevention culture didn’t become one that discouraged experimentation and reaching.