DaedTech

Stories about Software

Jul
17

By

Easy Deployment: the Alpha and the Omega

Category: Language Agnostic Tags: , |

A bit of housekeeping…you may have noticed that the social media buttons look a bit different if you’re not accessing through RSS. The old plugin that I was using seems not to be supported anymore, and the Facebook button vanished for a bit. I tried out a replacement and liked it, so I kept it. My thanks to Active Bits for the Social Sharing Toolkit.

Wrong But Fast

There’s a pretty good chance that your deployment process is both too painful and not painful enough. But before I return to that cryptic statement, let me talk a bit about something I’ve observed in developers — especially ones that are newer to the industry. Here’s an example of a series of exchanges that has become pretty familiar to me:

User: It would be nice if the profile screen had a way I could change my password.

Young Buck: That’ll take like, literally, two seconds. I’ll be right back!

(Fifteen minutes later)

Young Buck: Okay, I pushed it out to the server, so you can change your password now.

User: Wow! It’s live already? That’s really cool! Thank you! Let me try it out. Let’s see… oh, hmmm. When I try to log in it looks like it crashes.

Young Buck: That doesn’t seem right. I mean, the only thing I changed was…. oh! I know exactly what happened. Give me three minutes, and I’ll be back.

(30 minutes later)

Young Buck: Alright, should be good.

User: Well, I can get in again, and there’s the change password button, but when I click, nothing happens.

Young Buck: That’s not possible. You must have forgotten to clear your cache. Unless…wait, I think I know what’s going on here. Give me 10 minutes.

User: Uh, tell ya what — just don’t worry about it. Maybe I’ll try it again next week.

Young Buck: (Crestfallen) But it’ll only take 10 minutes and I know exactly what the problem is.

User: (With pity) I’m sure you do, but I’ve got a lot of things to get done today.

This is not professional behavior. Imagine if you took your car in for repairs somewhere and things went this way. You’d probably have the Better Business Bureau on the phone shortly or at least be headed to another mechanic. The young buck is sloppy because he’s brash and arrogant. Right?

Or does it just come off that way a little because of how sure he seems when really he’s just eager to please the user and prove himself? Personally, I find that in the overwhelming majority of cases this is really what’s going on. People often get into programming because they like solving problems. And many programmers were some of the smartest kids in their classes growing up — the ones waving their hands frantically, demanding that the teacher give them a chance to show that they know the answer.

FranticStudent

As entry level programmers, the school game has been all that they know. It’s a balance between rushing to get the right answer (teacher calling on students, timed exams, cramming in homework, etc.) and getting answers right, with the former often winning. In college, most programming assignments are evaluated by programs that allow you to submit your code as often as you want and get immediate feedback. There are also office hours, so students who visit professors and TAs the most frequently and submit the most work tend to get the best grades and the most feedback. Computer Science students are used to a paradigm where ideas are valued over execution.

Welcome to the Real World, Grasshopper

In the professional world, however, execution reigns supreme. Ideas are cheap. You may be able to rattle off the quick-sort algorithm in pseudo-code faster than anyone around you, but that’s not going to win you any startup capital. With software, even the intellectual property system (USPTO, anyway) is a joke seemingly designed to let Apple, Microsoft and Google endlessly sue each other and occasionally to swat little guys like bugs. Having the idea first and/or quickly is not as important as getting the idea right in the end.

It takes people some time to learn this upon entering the work force, and exchanges like the one I mention above are common. Users more or less say, “Go away and come back to me when you have something that makes my life better and not a second before. I don’t care if you thought of it in five seconds or if it took you five minutes or if it would have taken you five minutes except that the database was actually not normalized to BCNF and blah, blah, blah. Whatever.” Some people figure this out quickly, and some never figure it out. But whatever the speed may be and however much your group may or may not have come to terms with this, there needs to be structure in place to stop the madness.

In other words, a lot of the developers in your group are going to be eager to please. This is especially true if they regularly interact with their users. There is going to be pressure on them to say things like, “sure, I’ll have that for you in 10 minutes.” But they need not to say things like that. If they can say things like that and they can successfully (attempt to) make them happen, your deployment process is not painful enough.

Hurts so Good

Deployment is not to be taken lightly, especially if there is a release and the users are going to be seeing the result of the work. If you can deploy effortlessly in minutes, there’s a very good chance your process is not painful enough. The situation I’ve been describing above suffers from this very problem — it’s too easy for eager crowd-pleasers to deploy and thus it’s too easy for them to depend on users to be their fast feedback mechanism.

Developers are smart and often opportunistic. Getting fast feedback is extremely important to them, and they’ll naturally seek out ways to procure it. If you let them, they’ll use end users as their feedback mechanism (and, in a tone-deaf sense that ignores end-user perception, this is actually optimal), but you can’t permit this. Rather than following that path of least resistance, or at least familiarity from school/hobbyist days, you need to choke off that path and force them to carve a new riverbed. You need to make deployment more painful.

Now there’s “antiseptic on a cut” painful and “shark gnawing on your leg” painful. You want to gun for the former. A lot of deployment processes that enable developers to SPAM end users with non-functional updates are the product of amateur hour: xcopying files to the server, putting an executable on a share drive, zipping things up and emailing them, etc. These things tend to be both easy to do and easy to botch, so simply setting policies in place that prevent developers from doing them is antiseptic on the cut.

Deployments and especially releases to end users need to follow some process, and coding is simply one stop along the way — not the entirety of it. Ideally, there should be the coding and then developer testing, but from there, automated unit and integration tests, code reviews, static analysis, exploratory/manual testing by QA and observation by a UX group can all be part of the mix. These good practices serve to improve quality in and of themselves, but they also serve to prevent spurious, sloppy releases. If you know you can make a change in five minutes and have it in front of a user in six minutes, you’re going to do that. But if you know that you can make the change in five minutes and it’ll be days of going through the entire release process before you hear back from the end user, you’ll start finding other ways to get fast feedback (such as running unit tests, asking other developers to take a look and working closely with QA). By making deployment more painful you ensure that a lot more care goes into it.

But Hurts Too Much

If you’ve been reading along and grinding your teeth in objection to my premise that your deployment process needs to hurt more, I understand. Things shouldn’t be painful, and deliberately hurting yourself is a form of madness. If your deployment process hurts, it’s too painful, even though I just told you it’s not painful enough. But you have to prevent pain in the right way. Xcopy deployment is like being a boxer addicted to morphine — your process is horrible but pain free. Now imagine that you realize that being addicted to drugs is a problem, so you cut back and start feeling the pain each time a professional puncher unloads on you. In one sense, you’re feeling too much pain because it hurts to get punched in the face, but at the same time you’re not feeling enough pain because the amount you’re feeling isn’t causing you to consider another vocation.

The analogy here may not line up exactly, but the idea is similar. A lot of development and deployment processes are problematically painful in that they’re error-prone and difficult, but not painful enough in that they don’t prevent over-eager deploys and bad decisions. The solution? Get off the junk and stop letting yourself get pummeled by human wrecking balls. Or, in software terms, have a process that makes bad deployments hard and good ones very easy.

Now, getting to this release nirvana is not, itself, simple, but life is simple once you get there. The path to it generally involves a lot of automated testing and good planning. It involves a predictable release cadence, such as a sprint, and a commitment to following the process, not making exceptions nor cramming things in at the 11th hour or pushing back release dates. It involves continuous integration rather than periodic, nightmarish feature branch integrations. It involves a resistance to patching frantically when you make mistakes (you’ll learn a valuable lesson for next time). It involves a simple, fully-automated, easily repeatable build process. It involves a single click/button push deployment process. Summed up, it means that every time someone on your team checks in code, a series of automated tests and checks ensure that the code would be a good candidate for production or it is rejected from checkin until it would be. It means that your code could be shipped with a reasonably high degree of confidence on every checkin and that whether or not to actually ship is a business decision — not a technical one.

I encourage you to take a look at your build process and deployment process. Is it easy because Jim in accounting could do it? If so, it’s too easy and it’s definitely causing you problems. Is it hard enough that you do a lot of checking beforehand because you won’t want to do it again if things go wrong? That’s an improvement because it forces your hand for early testing and vetting, but it’s still a time-wasting problem. First, think about putting obstacles in place to guard against careless deployment, then think about refining those obstacles into process-helping practices, and finally, think about smoothing over the obstacles in the form of complete automation, leaving nothing but good, easy process.

Jun
12

By

It’s a Work in Progress

Category: The Life of a Programmer Tags: |

I’ll have that for you next week. Oh, you want it to do that other thing that we talked about earlier where it hides those controls depending on what the user selects? I’ll have it for you next month. Oh, and you also want the new skinning and the performance improvements too? I’ll get started and we’ll set about six months out as the target date. Unless you want to migrate over to using Postgres. Then make it a year.

Have you ever had any conversations that went like this? Ever been the person making this promise or at least the one that would ultimately be responsible for delivering it? If so, I bet you felt like, “man, I’m totally just making things up, but I guess we’ll worry about that later.” Have you ever been on the hearing end of this kind of thing? I bet you felt like, “man, he’s totally just making things up, and none of this will ever actually happen.”

As you’ll know if you’ve been checking in at this blog for a while, my opinions about the “waterfall methodology” are no secret. It isn’t my intention to harp on that here but rather to point out that the so-called waterfall approach to software development is simply part of what I consider to be a larger fail: namely, putting a lot of distance between promises and deliveries.

I get the desire to do this–I really do. It’s the same reason that we root for underdogs in sports or we want the nice guy not to finish last. Having success is good, but sudden, unexpected success is awesome. It creates a narrative that is far more compelling than “guy who always delivers satisfactory results does it yet again.” Instead, it says, “I just pulled this thing called an iPod out of nowhere and now I’m going to be the subject of posthumous books and cheesy project manager motivational calendars in a decade.”

Companies have done things like this for a long, long time–particularly companies that sell tangible goods. It’s been the story of a consumer-based world where you innovate, patent, and then sell temporarily monopolized widgets, making a fortune by giving the world something new and bold. You probably wind up on the cover of various magazines with feel-good stories about the guy behind the revolutionary thing no one saw coming, except for one visionary (or a team of them) toiling away in anonymity and secrecy, ready to pull back the curtain and dazzle us with The Prestige.

Magician

But here’s the thing. The world is getting increasingly service oriented. You don’t really buy software anymore because now you subscribe and download it from “the cloud” or something. Increasingly, you rent tools and other things that perhaps you would previously have purchased. Even hardware goods and electronics have become relatively disposable, and there is the expectation of constant re-selling. Even the much-ballyhooed innovator, Apple, hasn’t really done anything interesting for a while. We’re in a world where value is delivered constantly and on a slight incline, rather than in sudden, massive eruptions and subsequent periods of resting on laurels.

What does the shifting global economic model have to do with the “waterfall” method of getting things done? Well, it pokes a hole in the pipe-dream of “we’ll go off and bury ourselves in a bunker for six months and then emerge with the software that does everything you want and also makes cold fusion a reality.” You won’t actually do that because no one really does anymore, and even back when people did, they failed a lot more often than they succeeded in this approach.

I love to hear people say, “it’s a work in progress.” That means that work is getting done and those doing it aren’t satisfied, and those two components are essential to the service model. These words mean that the speaker is constantly adding value. But more than that, it means that the speaker is setting small, attainable goals and that those listening can expect improvements soon and with a high degree of confidence. The person doing the work and the target audience are not going to get it perfect this week, next week, next month, or maybe ever. But they will get it better and better at each of those intervals, making everyone involved more and more satisfied. And having satisfaction steadily improve sure beats having no satisfaction at all for a long time and then a very large chance of anger and lawsuits. Make your work a work in progress, and make people’s stock in your labor a blue chipper rather than a junk bond.

Apr
15

By

What Drives Waterfall Projects?

Category: Anti Patterns, The Life of a Programmer Tags: , |

To start off the week, I have a satirical post about projects developed using the waterfall ‘methodology.’ (To understand the quotes, please see my post on why I don’t think waterfall is actually a methodology at all). I figured that since groups that use agile approaches and industry best practices have a whole set of xDD acronyms, such as TDD, BDD, and DDD, waterfall deserved a few of its own. So keep in mind that while this post is intended to be funny, I think there is a bit of relevant commentary to it.

Steinbeck-Driven Development (SDD)

For those of you who’ve never had the pleasure to read John Steinbeck’s “Of Mice and Men,” any SDD practitioner will tell you that it’s a heartwarming tale of two friends who overcome all odds during the Great Depression, making it cross-country to California to start a rabbit petting zoo. And it’s that outlook on life that they bring to the team when it comes to setting deadlines, tracking milestones, and general planning.

scan0003

Relentlessly optimistic, the SDD project manager reacts to a missed milestone by reporting to his superiors that everything is a-OK because the team will just make it up by the time they hit the next one. When the next milestone is missed by an even wider margin, same logic applies. Like a shopping addict or degenerate gambler blithely saying, “you gotta spend money to make money,” this project manager will continue to assume on-time delivery right up until the final deadline passes with no end in site. When that happens, it’s no big deal–they just need a week to tie up a few loose ends. When that week is up, it’ll just be one more week to tie up a few loose ends. When that week expires, they face reality. No, just kidding. It’ll just be one more week to tie up a few loose ends. After enough time goes by, members of the team humor him with indulgent baby talk when he says this: “sure it will, man, sure it will. In a week, everything will be great, this will all be behind us, and we’ll celebrate with steaks and lobster at the finest restaurant in town.”

Spoiler alert. At the end of Steinbeck’s novel, the idyllic rabbit farm exists only in the mind of one of the friends, shortly before he’s shot in the back of the head by the other, in an act that is part merciful euthanasia and part self-preservation. The corporate equivalent of this is what eventually happens to our project manager. Every week he insists that everything will be fine and that they’re pretty close to the promised land until someone puts the project out of its misery.

Shooting-Star-Driven Development (SSDD)

Steinbeck-Driven Development is not for everyone. It requires a healthy ability to live in deluded fantasy land (or, in the case of the novel, to be a half-wit). SSDD project managers are not the relentless optimists that their SDD counterparts are. In fact, they’re often pretty maudlin, having arrived at a PM post on a project that everyone knows is headed for failure and basically running out the clock until company bankruptcy or retirement or termination or something. These are the equivalents of gamblers that have exhausted their money and credit and are playing at the penny tables in the hopes that their last few bucks will take them on an unprecedented win streak. Or, perhaps more aptly, they’re like a lonely old toy-maker, sitting in his woodshop, hoping for a toy to come to life and keep them company.

This PM and his project are doomed to failure, so he rarely bothers with status meetings, creates a bare minimum of power points, and rarely ever talks about milestones. Even his Gantt charts have a maximum of three nested dependencies. It’s clear to all that he’s phoning it in. He knows it’s unlikely, but he pins his slim hope to a shooting star: maybe one of his developers will turn out to be the mythical 100x developer that single-handedly writes the customer information portal in the amount of time that someone, while struggling to keep a straight face, estimated it would take to do.

As projects go along and fall further and further behind schedule and the odds of a shooting star developer become more and more remote, the SSDD project manager increasingly withdraws. Eventually, he just kind of fades away. If Geppetto were a real life guy, carving puppets and asking stars to make them real children, he’d likely have punched out in an 19th century sanitarium. There are no happy endings on SSDD projects–just lifeless, wooden developers and missed deadlines.

Fear-Driven Development (FDD)

There is no great mystery to FDD projects. The fate of the business is in your hands, developers. Sorry if that’s a lot of pressure, but really, it’s in your hands.

The most important part of a FDD project is to make it clear that there will be consequences–dire consequences–to the business if the software isn’t delivered by such and such date. And, of course, dire consequences for the business are pretty darned likely to affect the software group. So, now that everyone knows what’s at stake, it’s time to go out and get that complex, multi-tiered, poorly-defined application built in the next month. Or else.

Unlike most waterfall projects, FDD enters the death march phase pretty much right from the start of coding. (Other waterfall projects typically only start the death march phase once the testing phase is cancelled and the inevitability of missing the deadline is clear.) The developers immediately enter a primal state of working fourteen hours per day because their very livelihoods hang in the balance. And, of course, fear definitely has the effect of getting them to work faster and harder than they otherwise would, but it also has the side effect of making the quality lower. Depending on the nature of the FDD project and the tolerance level of the customers for shoddy or non-functional software, this may be acceptable. But if it isn’t, time for more fear. Consequences become more dire, days become longer, and weekends are dedicated to the cause.

The weak have nervous breakdowns and quit, so only the strong survive to quit after the project ends.

Passive-Aggressive-Driven Development (PADD)

One of the most fun parts of waterfall development is the the estimation from ignorance that takes place during either the requirements or design days. This is where someone looks at a series of Visio diagrams and says, “I think this project will take 17,388.12 man-hours in the low risk scenario and 18,221.48 in the high-risk scenario.” The reason I describe this as fun is because it’s sort of like that game you play where everyone guesses the number of gumballs in a giant jar of gumballs and whoever is closest without going over wins a prize. For anything that’s liable to take longer than a week, estimation in a waterfall context is a ludicrous activity that basically amounts to making things up and trying to keep a straight face as you convince yourself and others that you did something besides picking a random number.

Well, I broke this task up into 3,422 tasks and estimated each of those, so if they each take four hours, and everything goes smoothly when we try to put them all together with an estimate for integration of… ha! Just kidding! My guess is 10,528 hours–ten because I was thinking that it’d have to be five digits, the fve because it’s been that many days that we’ve been looking at these Gantt charts and sequence diagrams, and twenty-eight because that was my number in junior high football. And you can’t bid one hour over me because I’m last to guess!

But PADD PMs suck all of the fun out of this style of estimation by pressuring the hours guessers (software developers) into retracting and claiming less time. But they don’t do it by showing anger–the aggression is indirect. When the developer says that task 1,024, writing the batch file import routine, will take approximately five hours, the PADD PM says, “Oh, wow. Must be pretty complicated. Jeez, I just assumed that a senior level developer could bang that out in no more than two. My bad.” Shamed, the developer retracts: “No, no–you’re right. I figured the EDI would be more complicated than it was, so I just realized that my estimate is actually two hours.”

Repeated in aggregate, the PADD PM is some kind of spectacular black belt/level 20/guru/whatever metric is used to measure PM productivity, because he just reduced the time to market by 60% before a single line of code was ever written. Amazing! Of course, talk at the beginning of the project is cheap. The real measure of waterfall project success is figuring out who to blame and getting others to absorb the cost when the project gets way behind schedule. And this is where the PADD master really shines.

To his bosses, he says, “man, I guess I just had too much faith in our guys–I mean, I know you hire the best.” To the developers, he says, “boy, your estimates seemed pretty reasonable to me, so I would have assumed that everything would be going on time if you were just putting in the hours and elbow grease… weird.” To the end-users/stakeholders, he says, “it’s strange, all of our other stakeholders who get us all of the requirements clearly and on time get their software on time–I wonder what happened here.”

There’s plenty of blame to go around, and PADD PMs make sure everyone partakes equally and is equally dissatisfied with the project.

Nov
14

By

The Developer Incentive Snakepit

Category: The Life of a Programmer Tags: , , | 4 Comments

Getting Incentives Wrong

A while ago I was talking to a friend of mine, and he told me that developers in his shop get monetary bonuses for absorbing additional scope in a “waterfall” project. (I use quotes because I don’t think that “waterfall” is actually a thing — I think it’s just collective procrastination followed by iterative development) . That sounds reasonable and harmless at first. Developers get paid more money if they are able to adapt successfully to late-breaking changes without pushing back the delivery date. Fair enough…. or is it?

Image credit to Renaud d’Avout d’Auerstaedt via wikimedia commons.

Let’s digress for a moment.  In colonial India, the ruling British got tired of all of the cobras hanging around and being angry and poisonous and whatnot, so they concocted a scheme to address this problem. Reasoning that more dead cobras meant fewer living cobras, they began to offer a reward/bounty for any cobra corpses brought in to them by locals. As some dead cobras started coming in, circumstances initially improved, but after a while the number of dead cobras being brought in exploded even though the number of living cobras actually seemed to increase a little as well.  How is this possible?  The British discovered that Indian locals were (cleverly) breeding cobras that they could kill and thus cash in on the reward. Incensed, the British immediately discontinued the program. The cobra breeders, who had no more use for the things, promptly discarded them, resulting in an enormous increase in the local, wild cobra population. This is an iconic example of what has come to be called “The Law of Unintended Consequences“, which is a catch-all for describing situations where an incentive-based plan has an effect other than the desired outcome, be it oblique or directly counter. The Cobra Effect is an example of the latter.

Going back to the “money for changes absorbed” incentive plan, let’s consider if there might be the potential for “cobra breeders” to start gaming the system or perhaps less cynical, subconscious games that might go on. Here are some that I can think of:

  1. Change requests arise when the actual requirements differ from those defined initially, so developers have a vested interest in initial requirements being wrong or incomplete.
  2. Change requests will happen if the marketing/UX/PM/etc stakeholders don’t communicate requirements clearly to developers, so developers have a vested interest in avoiding these stakeholders.
  3. Things reported as bugs/defects/issues require changes to the software, which developers have to fix for free, but if those same items were, for some reason, re-classified as change requests, developers would make more money.
  4. The concept of “change request” doesn’t exist in agile development methodologies since any requirement is considered a change request, and thus developers would lose money if a different development methodology were adopted.

So now, putting our more cynical hats on, let’s consider what kind of outcomes these rational interests will probably lead to:

  1. Developers are actively obstructionist during the “requirements phase”.
  2. Developers avoid or are outright hostile toward their stakeholders.
  3. Developers battle endlessly with QA and project management, refusing to accept any responsibility for problems and saying things like “there was nothing in the functional spec that said it shouldn’t crash when you click that button!”
  4. Developers actively resist changes that would benefit the business, favoring rigidity over flexibility.

Think of the irony of item (4) in this scenario.  The ostensible goal of this whole process is to reward the development group for being more flexible when it comes to meeting the business demands.  And yet when incentives are created with the intention of promoting that flexibility, they actually have the effect of reducing it.  The managers of this department are the colonial British and the cash for absorbed change requests is the cash for dead cobras.  By rewarding the devs for dead cobras instead of fewer cobras, you wind up with more cobras; the developers’ goal isn’t more flexibility but more satisfied change requests, and the best way to achieve this goal is to generate software that needs a lot of changes as far as the business is concerned.  It’s an example of the kind of gerrymandering and sandbagging that I alluded to in an earlier post and it makes me wonder how many apparently absurd developer behaviors might be caused by nonsensical or misguided incentive structures triggering developers to game the system.

So what would be a better approach?  I mean with these Cobra Effect situations, hindsight is going to be 20/20.  It’s easy for us to say that it’s stupid to pay people for dead cobras, and it was only after ruminating on the idea for a while that the relationship between certain difficulties of the group in question and the incentive structure occurred to me.  These ideas seem generally reasonable on their faces and people who second guess them might be accused of arm-chair quarterbacking.

Get Rid of Cobras with Better Incentives

I would say that the most important thing is to align incentives as closely as possible with goals and to avoid rewarding process adherence.  For example, with this change request absorbed incentive, what is the actual goal, and what is being rewarded?  The actual reward is easy, particularly if you consider it obtusely (and you should): developers are rewarded when actual requirements are satisfied on time in spite of not matching the original requirements.  There’s nothing in there about flexibility or business needs — just two simple criteria: bad original requirements and on-time shipping.  I think there’s value in stating the incentive simply (obtusely) because it tends to strip out our natural, mile-a-minute deductions.  Try it on the British Empire with its cobra problem:  more cobra corpses means more money.  There’s nothing that says the cobra corpses have to be local or that the number of cobras in the area has to decrease.  That’s the incentive provider being cute and clever and reasoning that one is the logical outcome of the other.

Flexibility image credit to “RDECOM” via wikimedia commons.

Going back to the bad software incentives, the important thing is to pull back and consider larger goals.  Why would a company want change requests absorbed?  Probably because absorbed change requests (while still meeting a schedule) are a symptom (not a logical outcome, mind you) of flexibility.  Aha, so flexibility is the goal, right?  Well, no, I would argue.  Flexibility is a means rather than an end.  I mean, outside of a circus contortionist exhibit, nobody gets paid simply to be flexible.  Flexibility is a symptom of another trend, and I suspect that’s where we’ll find our answer.

Why would a company want to be flexible when it comes to putting out software?  Maybe they have a requirement from the marketing department stating that they want to cut a sort of hip, latest-and-greatest feel, so the GUI has to constantly be updated with whatever latest user experience/design trends are coming from Apple or whoever is currently the Georgio Armani of application design.  Maybe they’re starting on a release now and they know their competition is coming out with something in 3 months that they’re going to want to mimic in the next release, but they don’t yet know what that thing is.  Maybe they just haven’t had any resources available to flesh out more than a few wireframes for a few screens but don’t want all progress halted, “waterfall” style, until every last detail of every last screen is hammered out in IRS Tax Code-like detail.

Aha!  Now we’re talking about actual business goals rather than slices of process that someone thinks will help the goals be achieved.  Why not reward the developers (and QA, marketing, UX, etc as well) for those business goals being met rather than for adherence to some smaller process?  Sadly, I think the answer is a command and control style of hierarchical management that often seeks to justify positions with fiefdoms of responsibility and opacity.  In other words, many organizations will have a CEO state these business goals and then hand down a mandate of “software should be flexible” to some VP, who in turn hands down a mandate of “bonuses for change requests absorbed” to the manager of the software group or some such thing.  It is vital to resist that structure as much as possible since providing an incentive structure divorced from broader goals practically ensures that the prospective recipients of the structure care nothing whatsoever for anything other than gaming the system in their own favor.  And in some cases, such as our case, this leads to open and predictable conflicts with other groups that have (possibly directly contradicting) incentive structures of their own.  As an extreme example, imagine a tech group where the QA team gets money for each bug they find and the developers lose money for the same.  A good way to ensure quality… or a good way to ensure fistfights in your halls?

Take-Away

Why keep things so simple and not fan out the deductive thinking about incentives, particularly in the software world?  Well, software people by nature are knowledge workers that are paid to use well-developed, creative problem-solving skills.  Incentives that work on assembly line workers such as “more money for more holes drilled per hour” are more likely to backfire when applied to people who make their living inventing ways to automate, enhance and optimize processes.  Software people will become quickly adept at gaming your system because it’s what you pay them to do.  If you reward them for process, they’ll automate to maximize the reward, whether or not it’s good for the business.  But if you reward them for goals met, they will apply their acute problem solving skills to devising the best process for solving the problem — quite possibly better than the one you advise them to follow.

If you’re in a position to introduce incentives, think carefully about the incentives that you introduce and how closely they mirror your actual goals.  Resist the impulse to reward people for following some process that you assume is the best one.  Resist the impulse to get clever and think “If A, then B, then C, then D, so I’ll reward people for D in order to get A.”  That’s the sort of thinking that leads to people on your team dropping cobras in your lunch bag.  Metaphorical cobras.  And sometimes real cobras.  If you work with murderers.

Sep
19

By

There is No Such Thing as Waterfall

Category: The Life of a Programmer Tags: , , | 6 Comments

Don’t try to follow the waterfall model. That’s impossible. Instead, only try to realize the truth: there is no waterfall.

Waterfall In Practice Today

I often see people discuss, argue and debate the best approach or type of approach to software development. Most commonly, this involves a discussion of the merits of iterative and/or agile development, versus the “more traditional waterfall approach.” What you don’t see nearly as commonly, but do see every now and then is how the whole “waterfall” approach is based on a pretty fundamental misunderstanding, wherein the man (Royce) who coined the term and created the iconic diagram of the model was holding it up as a strawman to say (paraphrased) “this is how to fail at writing software — what you should do instead is iterate.” Any number of agile proponents may point out things like this, and it isn’t too hard to make the case that the waterfall development methodology is flawed and can be problematic. But, I want to make the case that it doesn’t even actually exist.

I saw a fantastic tweet earlier from Energized Work that said “Waterfall is typically 6 months of ‘fun up front’, followed by ‘steady as she goes’ eventually ramping up to ‘ramming speed’.” This is perfect because it indicates the fundamental boom and bust, masochistic cycle of this approach. There is a “requirements phase” and a “design phase”, which both amount basically to “do nothing for a while.” This is actually pretty relaxing (although frustrating for ambitious and conscientious developers, as this is usually unstructured-unstructured time). After a few weeks or months or whatever of thumb-twiddling, development starts, and life is normal for a few weeks or months while the “chuck it over the wall” deadline is too far off for there to be any sense of how late or bad the software will turn out to be. Eventually, though, everyone starts to figure out that the answers to those questions are “very” and “very”, respectively, and the project kicks into a death march state and “rams” through the deadline over budget, under-featured, and behind schedule, eventually wheezing to some completion point that miraculously staves off lawyers and lawsuits for the time being.

This is so psychically exhausting to the team that the only possible option is 3 months of doing nothing, er, excuse me, requirements and design phase for the next project, to rest. After working 60 hour weeks and weekends for a few weeks or months, the developers on the team look forward to these “phases” where they come in at 10 AM, leave at 4 PM, and sit around writing “shall” a lot, drawing on whiteboards, and googling to see if UML has reached code generation Shangri La while they were imprisoned in their cubicles for the last few months. Only after this semi-vacation are they ready to start the whole chilling saga again (at least the ones that haven’t moved on to greener pastures).

Diving into the Waterfall

So, what actually happens during these phases, in a more detailed sense, and what right have I to be so dismissive of requirements and design phases as non-work? Well, I have experienced what I’m describing firsthand on any number of occasions, and found that most of my time is spent waiting and trying to invent useful things to do (if not supporting the previous release), but I realize that anecdotal evidence is not universally compelling. What I do consider compelling is that after these weeks or months of “work” you have exactly nothing that will ever be delivered to your end-users. Oh, you’ve spent several months planning, but when was the last time planning something was worked at anywhere near as much as actually doing something? When you were a high school or college kid and given class time to make “idea webs” and “outlines” for essays, how often was that time spent diligently working, and how often was that time spent planning what to do next weekend? It wasn’t until actual essay writing time that you stopped screwing around. And, while we like to think that we’ve grown up a lot, there is a very natural tendency to regress developmentally when confronted with weeks of time after which no real deliverable is expected. For more on this, see that ambitious side project you’ve really been meaning to get back into.

But the interesting part of this isn’t that people will tend to relax instead of “plan for months” but what happens when development actually starts. Development starts when the team “exits” the “design phase” on that magical day when the system is declared “fully designed” and coding can begin. In a way, it’s like Christmas. And the way it’s like Christmas is that the effect is completely ruined in the first minute that the children tear into the presents and start making a mess. The beautiful design and requirements become obsolete the second the first developer’s first finger touches the first key to add the first character to the first line of code. It’s inevitable.

During the “coding phase”, the developers constantly go to the architect/project manager/lead and say “what about when X happens — there’s nothing in here about that.” They are then given an answer and, if anyone has time, the various SDLC documents are dutifully updated accordingly. So, developers write code and expose issues, at which time requirements and design are revisited. These small cycles, iterations, if you will, continue throughout the development phase, and on into the testing phase, at which time they become more expensive, but still happen routinely. Now, those are just small things that were omitted in spite of months of designing under the assumption of prescience — for the big ones, something called a “change request” is required. This is the same thing, but with more emails and word documents and anger because it’s a bigger alteration and thus iteration. But, in either case, things happen, requirements change, design is revisited, code is altered.

Whoah. Let’s think about that. Once the coding starts, the requirements and design artifacts are routinely changed and updated, the code changes to reflect that, and then incremental testing is (hopefully) done. That doesn’t sound like a “waterfall” at all. That sounds pretty iterative. The only thing missing is involving the stakeholders. So, when you get right down to it, “Waterfall” is just dysfunctional iterative development where the first two months (or whatever) are spent screwing around before getting to work, where iterations are undertaken and approved internally without stakeholder feedback, and where delivery is generally late and over budget, probably by an amount in the neighborhood of the amount of time spent screwing around in the beginning.

The Take-Away

My point here isn’t to try to persuade anyone to alter their software development approach, but rather to try to clarify the discussion somewhat. What we call “waterfall” is really just a specifically awkward and inefficient iterative approach (the closest thing to an exception I can think of is big government projects where it actually is possible to freeze requirements for months or years, but then again, these fail at an absolutely incredible rate and will still be subject to the “oh yeah, we never considered that situation” changes, if not the external change requests). So there isn’t “iterative approach” and “waterfall approach”, but rather “iterative approach” and “iterative approach where you procrastinate and scramble at the end.” And, I don’t know about you, but I was on the wrong side of that fence enough times as a college kid that I have no taste left for it.

« Older Posts
Newer Posts »