DaedTech

Stories about Software

By

Don’t Take My Advice

A Brief History of My Teeth

When it comes to teeth, I hit the genetic jackpot. I’m pretty sure that, instead of enamel, my teeth are coated in some sort of thin layer of stone. I’ve never had a cavity and my teeth in general seem impervious to the variety of maladies that afflict other people’s teeth. So this morning, when I was at the dentist, the visit proceeded unremarkably. The hygienist scraped, prodded, and fluoridized my teeth with a quiet efficiency, leaving me free to do little but listen to the sounds of the dentist’s office.

HygienistThose sounds are about as far from interesting as sounds get: the occasional hum of the HVAC, people a few rooms over exchanging obligatory inanities about the weather, coughs, etc. But for a brief period of time, the hygienist in the next station over took to scolding her patient like a child (he wasn’t). She ominously assured him that his lack of flossing would catch up to him someday, though I quietly wondered which day that would be, since he was pushing retirement age. I empathized with the poor man because I too had been seen to by that hygienist during previous visits.

The first time I encountered her a couple of years ago, she asked me if I drank much soda, to which I replied that I obviously did since my employer stocked the fridge with it for free. I don’t drink coffee, so if I want to partake in the world’s most socially acceptable upper to combat grogginess, it’s Diet Pepsi or Mountain Dew for me. She tsked at me, told me a story of some teenager whose teeth fell out because he had a soda fountain in his basement, and then handed me a brochure with pictures of people’s mouths who were apparently ravaged by soda to such a state where I imagine that the only possible remedy was a complete head transplant. My 31 year steak of complete imperviousness to cavities in spite of drinking soda was now suddenly going to end as my teeth fell inexorably from my head. That is, unless I repented my evil ways and started drinking water and occasionally tea.

I agreed to heed her advice in the way that one generally does when confronted with some sort of pushy nutjob on a mission–I pretended to humor her so that she would leave me alone. When I got home, I threw out the Pamphlet of Dental Doom and didn’t think about her for the next six months until I came back to the dentist. The first thing that she asked when I came in was whether I was still drinking a lot of soda or not. I was so dumbfounded that I didn’t have the presence of mind to lie my way out of this confrontation. I couldn’t decide whether it’d be creepier if she remembered me from six months ago and remembered that I drank a lot of soda or if she had made herself some kind of note in my file. Either way, I was in trouble. The previous time she had been looking to save the sinful, but now she was just pissed. Like the man I silently commiserated with this morning, she proceeded to spin spiteful tales of the world of pain coming my way.

Freud might have some interesting ideas about a person that likes to stick her fingers in other people’s mouths while trying to scare them, and I don’t doubt that there’s a whole host of ground for her to cover on the proverbial therapist couch, but I’ll speak to the low hanging fruit. She didn’t like that I ignored her advice. She is an expert and I am an amateur. She used her expertise to make a recommendation to me, and I promptly blew it off, which is a subtle slap in the face should an expert choose to view it that way.

It Isn’t Personal

It’s pretty natural to feel slighted when you offer expert advice and the recipient ignores it. This is especially true when the advice was solicited. I recall going along once with my girlfriend to help her pick out a computer and feeling irrationally irritated when we got to the store and she saw one that she liked immediately, losing complete interest in my interpretation of the finer points of processor architectures and motherboard wiring schemes. I can also recall it happening at times professionally when people solicit advice about programming, architecture, testing, tooling, etc. I’m happy–thrilled–to provide this advice. How dare anyone not take it.

But while it’s often hard to reason your way out of your feelings, it’s a matter of good discipline to reason past irrational reactions. As such, I strive not to take it personally when my advice, even when solicited, goes unheeded. It’s not personal, and I would argue it’s probably a good sign that the members of your team may be nurturing budding self-sufficiency.

Let’s consider three possible cases of what happens when a tech lead or person with more experience offers advice. In the first case, the recipient attempts to heed the advice but fails due to incompetence. In the second, the recipient takes the advice and succeeds with it. In the third and most interesting case, the recipient rejects the advice and does something else instead. Notice that I don’t subdivide this case into success and failure. I honestly don’t think it matters in the long term.

In the first case, you’re either dealing with someone temporarily out of their depth or generally incompetent, who might be considered an outlier on the lower end of the spectrum. The broad middle is populated with the second case people who take marching orders well enough and are content to do just that. The third group also consists of outliers, but often high-achieving ones. Why do I say that? Well, because this group is seeking data points rather than instructions. They want to know how an expert would handle the situation, not so that they can copy the expert necessarily, but to get an idea. Members of this group generally want to blaze their own trails, though they may at times behave like the second group for expediency.

But this third group consists of tomorrow’s experts. It doesn’t matter if they succeed or fail in the moment because, hey, success is success, but you can be very sure they’ll learn from any failures and won’t repeat their mistakes. They’re learning lessons by fire and experimentation that the middle-of-the-roaders learn as cargo cult practice. And they’re not dismissing your advice to offend you–they earnestly want to understand and assimilate your expertise–but rather to learn from you.

So when this happens to you as a senior team member/architect/lead/etc., try to fight the urge to be miffed or offended and exercise some patience. Give them some rope and see what they do–they can always be reigned in later if they aren’t doing well, but it’s hard to let the rope out if you’ve extinguished their experimental and creative spirit. The last thing you want to be is some woman in a dentist’s office, getting irrationally angry that adults aren’t properly scared of the Cavity Creeps.

By

The Council of Elders Anti-Pattern

Convene the High Council

Today I’d like to talk about something that I’ve observed on some occasions throughout my career. It sort of builds on the concepts that I’ve touched on in previous writings “How to Keep your Best Programmers” and the Expert Beginner posts, including Bruce Webster’s “Dead Sea Effect.” The phenomenon is an anti-pattern that is unique to shops with fairly mature Dead Sea/Expert Beginner presences, and I’ll call it “Council of Elders.”

To understand what I mean by this, let’s consider what happens in an organization with a more vital software group full of engaged, competent, and improving engineers and not bogged down in the politics and mediocrity of seniority-based advancement. In such a group, natural divisions of labor emerge based on relative expertise. For example, you might have an effective team with the “database guy” and the “architecture gal” and “markup guru” and the “domain knowledge expert” or some such thing. Each resident expert handles decision making within his or her sphere of influence and trusts other team members to likewise do the right thing in their own areas of strength. There might be occasional debate about things that blur the lines or occur on the boundary between two experts, but that’s just part of a healthy group dynamic.

And truly, this is a healthy group. There is no dead weight, and all of the group members have autonomy within a subset of the work that is done, meaning that people are happy. And that happiness breeds mutual trust and productivity. I’m oversimplifying and prettying up the counterexample a bit, but it really is something of a virtuous cycle with team members happily playing to their own strengths and areas of interest.

TheCouncilBut what happens in particularly salty Dead Sea shops where authority comes neither from merit nor from expertise, but rather from tenure with the company? What happens when the roost is ruled by Expert Beginners? Well, for one thing, the lords and ladies here tend to guard their fiefdoms more jealously since expertise on the part of others is more threat than benefit. Perhaps more importantly and broadly, however, knowledge and expertise are devalued in favor of personal politics and influence with debates being won on the basis of “who are you and how loud/charming/angry/glib/etc. are you” rather than on the merit of ideas. The currency of Dead Sea departments is everything but ideas–in benevolent ones, it may be “how long have you been here” or “how nice a person are you,” and, in “high pressure culture” ones, it might simply be psychopathy or other cutthroat forms of “might makes right.” And with evaluation of ideas out the window, every council member is freed to hold forth as an expert on every topic, regardless of how much or little he knows about that topic. Nobody is going to dispute anything he says–non-members are cowed into submission and fellow members recognize the importance of putting on a unified public front since they want to be be able to do the same without being questioned.

If you put this political yeast in the oven and let it rise, some of the results are fairly predictable: idea stagnation, increasingly bone-headed solutions to problems, difficulty keeping talent, etc. But an interesting consequence isn’t necessarily intuitive–namely that you’ll wind up with a kind of cabal of long-tenured people that collectively make all decisions, however big or small. I call this the “Council of Elders,” and it’s like one of those Magic Eye paintings that’s hidden but couldn’t be more obvious once you see it.

The Council of Elders is sort of like the Supreme Court of the department, and it’s actually surprisingly democratic as opposed to the more expected ladder system which ranks people by years and months with the company (or if you’re a fan of The Simpsons, the system in the Stonecutters episode where all members of the club are assigned a numeric rank in the order of joining, which determines their authority). The reason that it’s democratic is that actually assigning rank based on years/months of tenure would unceremoniously poke a hole in any illusion of meritocracy. So the council generally makes entrance to the club a matter of tenure, but status within the club a shifting matter of alliances and status games once past the velvet rope.

The Council is fundamentally incapable of delegation or prioritization of decision making. Since entrance is simply a matter of “paying your dues” (i.e. waiting your turn) and largely earned by Expert Beginners, it’s really impossible to divide up decision making based on expertise. The members tend to be very good at (or at least used to) company politics and procedures but not much else. They mostly have the same ‘skill’ set. The lack of prioritization comes from the main currency in the group being status. If a decision, however small, is made by someone not on the Council, it threatens to undermine the very Council itself, so a policy of prevention is adopted and any attempts at circumvention are met with swift and terrible justice (in the form of whatever corporate demerits are in place).

Recognizing Your Elders

What does this council look like in the wild, and how do you know if one is pulling your strings? Here’s a set of symptoms that you’re being governed by a Council of Elders:

  • In any meeting convened to make a decision, the same group of people with minor variants is always present.
  • Members of the software group are frequently given vague instructions to “talk to so and so before you do anything because his cousin’s dog’s trainer once did something like that or something.”
  • There is a roster, emailed out or tacked up, of people who can “clear” you to do something (e.g. they give code reviews, approve time spent, etc.)
  • Sparks fly when something wasn’t “cleared” with someone, even when a few others “approved” it and it has apparently nothing to do with him. “Someone should have told me.” More than one or two people like this and you have a Council.
  • People in the group often feel caught between a rock and a hard place when involved in the political posturing of the Council (e.g. Yoda tells the intern Padiwan to implement the new hours tracking app using SQL Server, but then Mace Windu screams at him that this is a MySQL shop–the Council settles this matter later with no apologies)
  • There is a junta of team members that seem to do nothing but shuffle from meeting to meeting all day like mid-level managers.
  • There are regular meetings viewed by newbies and junior developers as rites of passage.

These are just easy ways to detect Councils in their natural habitat: “where there’s smoke, there’s fire” situations. But lest you think that I’m trying to paint every software department with senior developers that go to meetings with this brush, here is a list of specific criteria — “minimum Council Requirements,” if you will:

  • Lack of clearly defined roles on the group/team or else lack of clear definition for the assigned roles (e.g. there is no ‘architect’ or if there is, no one really knows what that means).
  • A “dues paying” culture and promotions, power and influence largely determined by length of tenure.
  • Lack of objective evaluation criteria of performance, skill and decision-making acumen (i.e. answers to all questions are considered subjective and matters of opinion).
  • Proposed changes from above are met with claims of technical infeasibility and proposed changes from juniors/newbies or other groups are met with vague refusals or simply aren’t acknowledged at all.
  • Actions of any significance are guarded with gatekeeper policies (official code reviews, document approval, sign-off on phases, etc).
  • Line manager tacitly approves of, endorses or is the product of the Council.
  • A moderate to large degree of institutional bureaucracy is in place.
  • New technologies, techniques, approaches, etc are met systematically with culturally entrenched derision and skepticism.

Not a Harmless Curiosity

At this point, you might think to yourself, “So what? It might not be ideal, but rewarding people with a little gratifying power in exchange for company loyalty is common and mostly harmless.” Or perhaps you’re thinking that I’m overly cynical and that the Council generally has good advice to dispense–wisdom won in the trenches. Of course all situations are different, but I would argue that a Council of Elders has a relatively noticeable and negative effect on morale, productivity, and general functioning of a group.

  • The Council is a legitimate bottleneck in its micromanagement of other team members, severely hampering their productivity even with the best assumption of its judiciousness
  • A group of people that spends all its time debating one another over how to rule on every matter doesn’t actually get much work done. The SCOTUS, for example, hasn’t represented a lot of clients lately, but that’s because their job is to be a ruling council–yours probably has titles like, “Senior Software Engineer.”
  • People lacking expertise but put in positions of authority tend to overcompensate by forming strong opinions and sticking to them stubbornly. A room full of people meeting this criterion is going to resemble the US House of Representatives with its gridlock more than a software team.
  • Major problems are likely to languish without solutions because the committee doesn’t do much prioritizing and is likely to be sidetracked for a few days by the contentious issue of what color to make the X for closing the window.
  • Important decisions are made based on interpersonal dynamics among the council rather than merit. Jones might have the best idea, but Jones shorted the check at lunch that day, so the other Council Members freeze him out.
  • Councils make it hard to give meaningful roles or titles to anyone and thus give rise to preposterous situations in which departments have eight ‘architects’ and five developers, or where a project manager decides what database to use while a DBA writes requirements. If everyone on the Council has authority and is an expert at everything, project roles are meaningless anyway.
  • Even under the best circumstances, democratic voting on software design trends toward mediocrity: see Design by Committee Anti-Pattern
  • People toiling away under the rule of a Council, if they don’t leave, will tend to wind up checked out and indifferent. Either that, or they’ll be assimilated into the Council culture.
  • The Council is a natural preservation agent of the Dead Sea problem. Being micromanaged by a team of people whose best qualification is being around for a while and negotiating power will separate the wheat from the chaff, but not in the good way.
  • The only thing that unites the Council is outside threats from above or below. If managers or newer members want change, the Council will lock together like a Praetorian Legion to preserve the status quo, however dysfunctional that may be.
  • Suggestions for improvement are construed with near universality as threats.

If you’re in a position to do so, I’d suggest breaking up the cartel. Figure out what people are good at and give them meaningful roles. Keep responsibilities divided, and not only will autonomy become more egalitarian, but also people tenured and new alike will develop complementary skills and areas of expertise. With people working together rather than forming shifting alliances, you’ll find prioritizing tasks, dividing up work, and getting things done becomes easier. The Council’s reptile brain will take over and it will fight you every step of the way when you try to disband it, but once you do, it will be good for everyone.

By

A Metaphor to Help You Suck at Writing Software

“No plan survives contact with the enemy” –Helmuth von Moltke the Elder

Bureaucracy 101

Let’s set the scene for a moment. You’re a workaday developer in a workman kind of shop. A “waterfall” shop. (For back story on why I put quotes around waterfall, see this post). There is a great show of force when it comes to building software. Grand plans are constructed. Requirements gathering happens in a sliding sort of way where there is one document for vague requirements, another document for more specific requirements, a third document for even more specific requirements than that, and repeat for a few more documents. Then, there is the spec, the functional spec, the design spec, and the design document. In fact, there are probably several design documents.

There aren’t just the typical “waterfall” phases of requirements->design->code->test->toss over the wall, but sub-phases and, when the organism grows large enough, sub-sub-phases. There are project managers and business managers and many other kinds of managers. There are things called change requests and those have their own phases and documents. Requirements gathering is different from requirements elaboration. Design sub-phases include high-level, mid-level and low-level. If you carefully follow the process, most likely published somewhere as a mural-sized state machine or possibly a Gantt chart unsurpassed in its perfect hierarchical beauty, you will achieve the BUFD nirvana of having the actual writing of the code require absolutely no brain power. Everything will be so perfectly planned that a trained ape could write your software. That trained ape is you, workaday developer. Brilliant business stakeholder minds are hard at work perfecting the process of planning software in such fine grained detail that you need not trouble yourself with much thinking or problem solving.

Dude, wait a minute. Wat?!? That doesn’t sound desirable at all! You wake up in the middle of the night one night, sit bolt upright and are suddenly fundamentally unsure that this is really the best approach to building a thing with software. Concerned, you approach some kind of senior business project program manager and ask him about the meaning of developer life in your organization. He nods knowingly, understandingly and puts one arm on your shoulders, extending the other out in broad, professorial arc to help you share his vision. “You see my friend,” he says, “writing software is like building a skyscraper…” And the ‘wisdom’ starts to flow. Well, something starts to flow, at any rate.

Let’s Build a Software Skyscraper

Like a skyscraper, you can’t just start building software without planning and a lot of upfront legwork. A skyscraper can’t simply be assembled by building floors, rooms, walls, etc independently and then slapping them altogether, perhaps interchangeably. Everything is necessarily interdependent and tightly coupled. Just like your software. In the skyscraper, you simply can’t build the 20th floor before the 19th floor is built and you certainly can’t interchange those ‘parts’ much like in your software you can’t have a GUI without a database and you can’t just go swapping persistence models once you have a GUI. In both cases every decision at every point ripples throughout the project and necessarily affects every future decision. Rooms and floors are set in stone in both location and order of construction just as your classes and modules in a software project have to be built in a certain order and can never be swapped out from then on.Jenga

But the similarities don’t end with the fact that both endeavors involve an inseparable web of complete interdependence. It extends to holistic approaches and cost as well. Since software, like a skyscraper, is so lumbering in nature and so permanent once built, the concept of prototyping it is prima facie absurd. Furthermore, in software and skyscrapers, you can’t have a stripped-down but fully functional version to start with — it’s all or nothing, baby. Because of this it’s important to make all decisions up-front and immediately even when you might later have more information that would lead to a better-informed decision. There’s no deferral of decisions that can be made — you need to lock your architecture up right from the get-go and live with the consequences forever, whatever and however horrible they might turn out to be.

And once your software is constructed, your customers better be happy with it because boy-oh-boy is it expensive, cumbersome and painful to change anything about it. Like replacing the fortieth floor on a skyscraper, refactoring your software requires months of business stoppage and a Herculean effort to get the new stuff in place. It soars over the budget set forth and slams through and past the target date, showering passerby with falling debris all the while.

To put it succinctly in list form:

  1. There is only one sequence in which to build software and very little opportunity for deviation and working in parallel.
  2. Software is not supposed to be modular or swappable — a place for everything and everything in its place
  3. The concept of prototyping is nonsensical — you get one shot and one shot only.
  4. It is impossible to defer important decisions until more information is available. Pick things like database or markup language early and live with them forever.
  5. Changing anything after construction is exorbitantly expensive and quite possibly dangerous

Or, to condense even further, this metaphor helps you build software that is brittle and utterly cross-coupled beyond repair. This metaphor is the perfect guide for anyone who wants to write crappy software.

Let’s Build an Agile Building

Once you take the building construction metaphor to its logical conclusion, it seems fairly silly (as a lot of metaphors will if you lean too heavily on them in their weak spots). What’s the source of the disconnect here? To clarify a bit, let’s work backward into the building metaphor starting with good software instead of using it to build bad software.

AgileBuildingA year or so ago, I went to a talk given by “Uncle” Bob Martin on software professionalism. If I could find a link to the text of what he said, I would offer it (and please comment if you have one) but lacking that, I’ll paraphrase. Bob invited the audience to consider a proposition where they were contracting to have a house built and maintained with a particular contractor. The way this worked was you would give the contractor $100 and he would build you anything you wanted in a day. So, you could say “I want a two bedroom ranch house with a deck and a hot-tub and 1.5 bathrooms,” plop down your $100 and come back tomorrow to find the house built to your specification. If it turned out that you didn’t like something about it or your needs changed, same deal applied. Want another wing? Want to turn the half bath into a full bath? Want a patio instead of a deck? Make your checklist, call the contractor, give him $100 and the next day your wish would be your house.

From there, Bob invited audience members to weigh two different approaches to house-planning: try-it-and-see versus waterfall’s “big design up front.” In this world, would you hire expert architects to form plans and carpenters to flesh them out? Would you spend weeks or months in a “planning phase”? Or would you plop down $100 and say, “well, screw it — I’ll just try it and change it if I don’t like it?” This was a rather dramatic moment in the talk as the listener realized just before Bob brought it home that given a choice between agile, “try it and see” and waterfall “design everything up front” nobody sane would choose the latter. The “waterfall” approach to houses (and skyscrapers) is used because a better approach isn’t possible and not because it’s a good approach when there are alternatives.

Wither the Software-Construction Canard?

Given the push toward Agile software development in recent years and the questionable parallels of the metaphor in the first place, why does it persist? There is no shortage of people who think this metaphor is absurd, or at least misguided:

  1. Jason Haley, “It’s not like Building a House”
  2. Terence Parr, “Why writing software is not like engineereing”
  3. James Shore, “That Damned Construction Analogy”
  4. A whole series of people on stackoverlow
  5. Nathaniel T. Schutta, Why Software Development IS Like Building a House (Don’t let the title fool you – give this one a detailed read)
  6. Thomas Guest, “Why Software Development isn’t Like Construction”

If you google things like “software construction analogy” you will find literally dozens of posts like these.

So why the persistence? Well, if you read the last article, by Thomas Guest, you’ll notice a reference to Steve McConnell’s iconic book “Code Complete.” This book has an early chapter that explores a variety of metaphors for software development and offers this one up. In my first daedtech post I endorsed the metaphor but thought we could do better. I stand by that endorsement not because it’s a good metaphor for how software should be developed but because it’s a good metaphor for how it is developed. As in our hypothetical shop from the first section of the post, many places do use this approach to write (often bad) software. But the presence of the metaphor in McConnell’s book and for years and years before that highlights one of the main reasons for persistence: interia. It’s been around a long time.

But I think there’s another, more subtle reason it sticks around. Hard as it was to find pro posts about the software-construction pairing, the ones I did find share an interesting trait. Take a look at this post, for instance. As “PikeWake” gets down to explaining the metaphor, the first thing that he does is talk about project managers and architects (well, the first thing is the software itself, but right after that come the movers and shakers). Somewhere below that the low-skill grunts who actually write the software get a nod as well. Think about that for a moment. In this analogy, the most important people to the software process are the ones with corner offices, direct reports and spreadsheets, and the people who actually write the software are fungible drones paid to perform repetitive action, rather than work. Is it any wonder that ‘supervisors’ and other vestiges of the pre-Agile, command and control era love this metaphor? It might not make for good software, but it sure makes for good justification of roles. It’s comfortable in a world where companies like github are canning the traditional, hierarchical model, valuing the producers over the supervisors, and succeeding.

Perhaps that’s a bit cynical, but I definitely think there’s more than a little truth there. If you stripped out all of the word documents, Gantt charts, status meetings and other typical corporate overhead and embraced a world where developers could self-organize, prioritize and adapt, what would people with a lot of tenure but not a lot of desire or skill at programming do? If there were no actual need for supervision, what would happen? These can be unsettling, game changing questions, so it’s easier to cast developers as low-skill drones that would be adrift without clever supervisors planning everything for them than to dispense with the illusion and realize that developers are highly skilled, generally very intelligent knowledge workers quite capable of optimizing processes in which they participate.

In the end, it’s simple. If you want comfort food for the mid-level management set and mediocrity, then invite someone in to sweep his arm professorially and spin a feel-good tale about how building software is like building houses and managing people is like a father nurturing his children. If you want to be effective, leave the construction metaphor in the 1980s where it belongs.

By

The Developer Incentive Snakepit

Getting Incentives Wrong

A while ago I was talking to a friend of mine, and he told me that developers in his shop get monetary bonuses for absorbing additional scope in a “waterfall” project. (I use quotes because I don’t think that “waterfall” is actually a thing — I think it’s just collective procrastination followed by iterative development) . That sounds reasonable and harmless at first. Developers get paid more money if they are able to adapt successfully to late-breaking changes without pushing back the delivery date. Fair enough…. or is it?

Image credit to Renaud d’Avout d’Auerstaedt via wikimedia commons.

Let’s digress for a moment.  In colonial India, the ruling British got tired of all of the cobras hanging around and being angry and poisonous and whatnot, so they concocted a scheme to address this problem. Reasoning that more dead cobras meant fewer living cobras, they began to offer a reward/bounty for any cobra corpses brought in to them by locals. As some dead cobras started coming in, circumstances initially improved, but after a while the number of dead cobras being brought in exploded even though the number of living cobras actually seemed to increase a little as well.  How is this possible?  The British discovered that Indian locals were (cleverly) breeding cobras that they could kill and thus cash in on the reward. Incensed, the British immediately discontinued the program. The cobra breeders, who had no more use for the things, promptly discarded them, resulting in an enormous increase in the local, wild cobra population. This is an iconic example of what has come to be called “The Law of Unintended Consequences“, which is a catch-all for describing situations where an incentive-based plan has an effect other than the desired outcome, be it oblique or directly counter. The Cobra Effect is an example of the latter.

Going back to the “money for changes absorbed” incentive plan, let’s consider if there might be the potential for “cobra breeders” to start gaming the system or perhaps less cynical, subconscious games that might go on. Here are some that I can think of:

  1. Change requests arise when the actual requirements differ from those defined initially, so developers have a vested interest in initial requirements being wrong or incomplete.
  2. Change requests will happen if the marketing/UX/PM/etc stakeholders don’t communicate requirements clearly to developers, so developers have a vested interest in avoiding these stakeholders.
  3. Things reported as bugs/defects/issues require changes to the software, which developers have to fix for free, but if those same items were, for some reason, re-classified as change requests, developers would make more money.
  4. The concept of “change request” doesn’t exist in agile development methodologies since any requirement is considered a change request, and thus developers would lose money if a different development methodology were adopted.

So now, putting our more cynical hats on, let’s consider what kind of outcomes these rational interests will probably lead to:

  1. Developers are actively obstructionist during the “requirements phase”.
  2. Developers avoid or are outright hostile toward their stakeholders.
  3. Developers battle endlessly with QA and project management, refusing to accept any responsibility for problems and saying things like “there was nothing in the functional spec that said it shouldn’t crash when you click that button!”
  4. Developers actively resist changes that would benefit the business, favoring rigidity over flexibility.

Think of the irony of item (4) in this scenario.  The ostensible goal of this whole process is to reward the development group for being more flexible when it comes to meeting the business demands.  And yet when incentives are created with the intention of promoting that flexibility, they actually have the effect of reducing it.  The managers of this department are the colonial British and the cash for absorbed change requests is the cash for dead cobras.  By rewarding the devs for dead cobras instead of fewer cobras, you wind up with more cobras; the developers’ goal isn’t more flexibility but more satisfied change requests, and the best way to achieve this goal is to generate software that needs a lot of changes as far as the business is concerned.  It’s an example of the kind of gerrymandering and sandbagging that I alluded to in an earlier post and it makes me wonder how many apparently absurd developer behaviors might be caused by nonsensical or misguided incentive structures triggering developers to game the system.

So what would be a better approach?  I mean with these Cobra Effect situations, hindsight is going to be 20/20.  It’s easy for us to say that it’s stupid to pay people for dead cobras, and it was only after ruminating on the idea for a while that the relationship between certain difficulties of the group in question and the incentive structure occurred to me.  These ideas seem generally reasonable on their faces and people who second guess them might be accused of arm-chair quarterbacking.

Get Rid of Cobras with Better Incentives

I would say that the most important thing is to align incentives as closely as possible with goals and to avoid rewarding process adherence.  For example, with this change request absorbed incentive, what is the actual goal, and what is being rewarded?  The actual reward is easy, particularly if you consider it obtusely (and you should): developers are rewarded when actual requirements are satisfied on time in spite of not matching the original requirements.  There’s nothing in there about flexibility or business needs — just two simple criteria: bad original requirements and on-time shipping.  I think there’s value in stating the incentive simply (obtusely) because it tends to strip out our natural, mile-a-minute deductions.  Try it on the British Empire with its cobra problem:  more cobra corpses means more money.  There’s nothing that says the cobra corpses have to be local or that the number of cobras in the area has to decrease.  That’s the incentive provider being cute and clever and reasoning that one is the logical outcome of the other.

Flexibility image credit to “RDECOM” via wikimedia commons.

Going back to the bad software incentives, the important thing is to pull back and consider larger goals.  Why would a company want change requests absorbed?  Probably because absorbed change requests (while still meeting a schedule) are a symptom (not a logical outcome, mind you) of flexibility.  Aha, so flexibility is the goal, right?  Well, no, I would argue.  Flexibility is a means rather than an end.  I mean, outside of a circus contortionist exhibit, nobody gets paid simply to be flexible.  Flexibility is a symptom of another trend, and I suspect that’s where we’ll find our answer.

Why would a company want to be flexible when it comes to putting out software?  Maybe they have a requirement from the marketing department stating that they want to cut a sort of hip, latest-and-greatest feel, so the GUI has to constantly be updated with whatever latest user experience/design trends are coming from Apple or whoever is currently the Georgio Armani of application design.  Maybe they’re starting on a release now and they know their competition is coming out with something in 3 months that they’re going to want to mimic in the next release, but they don’t yet know what that thing is.  Maybe they just haven’t had any resources available to flesh out more than a few wireframes for a few screens but don’t want all progress halted, “waterfall” style, until every last detail of every last screen is hammered out in IRS Tax Code-like detail.

Aha!  Now we’re talking about actual business goals rather than slices of process that someone thinks will help the goals be achieved.  Why not reward the developers (and QA, marketing, UX, etc as well) for those business goals being met rather than for adherence to some smaller process?  Sadly, I think the answer is a command and control style of hierarchical management that often seeks to justify positions with fiefdoms of responsibility and opacity.  In other words, many organizations will have a CEO state these business goals and then hand down a mandate of “software should be flexible” to some VP, who in turn hands down a mandate of “bonuses for change requests absorbed” to the manager of the software group or some such thing.  It is vital to resist that structure as much as possible since providing an incentive structure divorced from broader goals practically ensures that the prospective recipients of the structure care nothing whatsoever for anything other than gaming the system in their own favor.  And in some cases, such as our case, this leads to open and predictable conflicts with other groups that have (possibly directly contradicting) incentive structures of their own.  As an extreme example, imagine a tech group where the QA team gets money for each bug they find and the developers lose money for the same.  A good way to ensure quality… or a good way to ensure fistfights in your halls?

Take-Away

Why keep things so simple and not fan out the deductive thinking about incentives, particularly in the software world?  Well, software people by nature are knowledge workers that are paid to use well-developed, creative problem-solving skills.  Incentives that work on assembly line workers such as “more money for more holes drilled per hour” are more likely to backfire when applied to people who make their living inventing ways to automate, enhance and optimize processes.  Software people will become quickly adept at gaming your system because it’s what you pay them to do.  If you reward them for process, they’ll automate to maximize the reward, whether or not it’s good for the business.  But if you reward them for goals met, they will apply their acute problem solving skills to devising the best process for solving the problem — quite possibly better than the one you advise them to follow.

If you’re in a position to introduce incentives, think carefully about the incentives that you introduce and how closely they mirror your actual goals.  Resist the impulse to reward people for following some process that you assume is the best one.  Resist the impulse to get clever and think “If A, then B, then C, then D, so I’ll reward people for D in order to get A.”  That’s the sort of thinking that leads to people on your team dropping cobras in your lunch bag.  Metaphorical cobras.  And sometimes real cobras.  If you work with murderers.

By

Because I Said So Costs You Respect

Do you remember being a kid, old enough to think somewhat deductively and logically, but not old enough really to understand how the world works? Do you remember putting together some kind of eloquent argument about why you should be able to sleep at a friend’s house or stay out later than normal, perfecting your reasoning, presentation and polish only to be rebuffed? Do you remember then having a long back and forth, asking why to everything, only to be smacked in the face with the ultimate in trump cards: “because I’m your parent and I say so?” Yeah… me too. Sucked, didn’t it?

There are few things in life more frustrating than pouring a good amount of effort into something only to have it shot down without any kind of satisfying explanation of the rationale. For children this tends to be unfortunate for self interested reasons: “I want to do X in the immediate future and I can’t.”  But as you get older and are motivated by more complex and nuanced concerns, these rejections get more befuddling and harder to understand.  A child making a case for why he should own a Red Rider BB Gun will understand on some level the parental objection “it’s dangerous” and that the parental objection arises out of concern for his welfare. So when the “enough — I’m your parent and I say so” comes up it has the backing of some kind of implied reasoning and concern.  An adult making a case that adopting accounting software instead of doing things by hand would be a money saving investment will have a harder time understanding  an answer of “no, we aren’t going to do that because I’m your boss and I say so.”

This is hard for adults to understand because we are sophisticated and interpersonally developed enough to understand when our goals align with those of others or of organizations. In other words, we reasonably expect an answer of “no” to the question “can I go on vacation for 8 months and get paid?” because this personal goal is completely out of whack with others and with an organization’s. So even if the explanation or reasoning aren’t satisfying, we get it. The “because I said so” is actually unnecessary since the answer of “that makes no sense for anyone but you” is perfectly reasonable. But when goals align and it is the means, rather than the ends, that differ, we start to have more difficulty accepting when people pull rank.

I remember some time back being asked to participate in generating extra documentation for an already documentation-heavy process. The developers on the project were taking requirements documents as drawn up by analysts and creating “requirements analysis” documents, and the proposal was to add more flavors of requirements documents to this. So instead of the analysts and developers each creating their own single word document filled with paragraph narratives of requirements, they were now being asked to collaborate on several each with each subsequent version adding more detail to the document. So as a developer, I might write my first requirements document in very vague prose, to be followed by a series of additional documents, each more detailed than the last.

I objected to this proposal (and really even to the original proposal). What I wanted to do was capture the requirements as data items in a database rather than a series of documents filled with prose. And I didn’t like the idea of having several documents with each one being a “more fleshed out” version of the last document — there’s a solution for that and it’s called “version control”. But when I raised these objections to the decision makers on the project, I was rebuffed. If you’re curious as to what the rationale for favoring the approach I’ve described over the one I suggest, I am as well, even to this day. You see, I never received any explanation other than a vague “well, it might not be the greatest, but we’re just going to go with it.” This explanation neither explained the benefit of the proposed approach nor any downside to my approach. Instead, I was a kid again, hearing “because I’m your parent and I say so.” But I wasn’t a little kid asking to stay out late — I was an adult with a different idea for how to achieve the same productivity and effectiveness goals as everyone else.

In the end, I generated the documents I was required to generate. It wasn’t the end of the world, though I never did see it as anything other than a waste of time. As people collaborating toward a larger goal, it will often be the case that we have to do things we don’t like, agree with or approve of, and it might even be the case that we’re offered no explanation at all for these things. That is to be expected in adult life, but I would argue that it should be minimized because it chips away at morale and satisfaction with one’s situation.

Once, twice, or every now and then, most people will let a “just do it because I say so” roll off their back. Some people might let many more than that slip by, while others might get upset earlier in the process. But pretty much everyone is going to have a limit with this kind of thing. Pretty much everyone will have a threshold of “because I say so” responses beyond which they will check out, tune out, leave, blow up, or generally lose it with that in some form. So I strongly recommend avoiding the tempting practice to “pull rank” and justify decisions with “because it’s my project” or “because I’ve decided that.” It’s sometimes nice not to have to justify your decisions — especially to someone with less experience than you and when you’re in a hurry — but the practice of defending your rationale keeps you sharp and on your toes, and it earns the respect of others. “Because I say so” is a well that you can only go to so many times before it dries up and leaves a desert of respect and loyalty. Don’t treat your coworkers like children when they want to know “why” and when they have legitimate questions — they deserve better than “because I’m in charge and I say so.”