How to Prioritize Bugs on Your To-Do List
Editorial Note: I originally wrote this post for the NDepend blog. You can check out the original here, at their site. While you’re there, take a look around at NDepend and explore its new tech debt measurement features.
People frequently ask me questions about code quality. People also frequently ask me questions about efficiency and productivity. But it seems we rarely wind up talking about the two together. How can you most efficiently improve quality via the fixing of bugs. Or, more specifically, how should you prioritize the list of bugs that you have?
Let me be clear about something up front. I’m not going to offer you some kind of grand unified scheme of bug prioritization. If I tried, the attempt would come off as utterly quixotic. Because software shops, roles, and offerings vary so widely, I cannot address every possible situation.
Instead, I will offer a few different philosophies of prioritization, leaving the execution mechanics up to you. These should cover most common scenarios that software developers and project managers will encounter.
Maximizing Self Interest
I’ll lead with the scenario probably most common to software developers. Stop me if this sounds familiar. The first interaction point you have with a bug is receiving an email from Jira or TFS or Rally or whatever. From there, you log in, read the details, and check the pre-assigned priority. You check that because of the bug-fixing algorithm imposed on you by management: fix any priority 1 bugs, or, if you have none of those, any priority 2 bugs, and so on down the line.
In this world, bug fixing becomes a matter of looking after your own self interest. Prioritizing your own to do list, consequently, becomes simple. Management and the business have made the important, strategic decisions already and will evaluate you on the basis of quantity of defects fixed. Thus you should prioritize the easiest to fix first, so that you fix as many as possible.
This may sound cynical to you, but I’m fine with that. I have a fundamental distaste for the specialization obsession we have that separates fixing and prioritization. Organizations that freeze technical people out of the strategic discussion of priority reap what they sow. Robbed of the ability to act in the organization’s best interests, developers should act in their own. Of course, I would prefer developers participate in the “how do we act in the company’s best interests” discussions.
Minimizing Reputation Issues
Once we move beyond the simple prioritization scheme of “pre-assigned order,” things get interesting. For the rest of the post, I’ll talk about prioritization approaches that involve nuance and strategic thinking. In other words, these assume that you have control over the prioritization in the first place.
Perhaps the simplest of these approaches involves reasoning about the “embarrassment factor” for the team with the user base. If you have a bug where users see a yellow screen of death half of the time they log in, you look like amateurs. Thus you want to prioritize bugs in such a way as to minimize this effect.
With this scheme, you must think beyond immediate user experience, however. Security holes or back end data processing may not have quite the same immediate “you’ve got to be kidding” factor, but they present serious long term threats to reputation. With the reputation preservation approach to prioritizing, you weigh the short and long term to see what provides the largest threat. Then you go fix that.
Show Me the Money
You may have noticed a potential flaw in the reputation focus. Specifically, it carries an implicit assumption that reputation equates to value (or money). In other words, we avoid things that damage the company’s reputation because we assume that this will be bad for the company, fiscally. But can we safely assume that?
Sure, correlation probably exists in many cases. But remember that an embarrassing defect may not actually affect sales or the bottom line in any way at all. People might see some goofy formatting on your landing page, laugh at you, and then buy your product anyway.
Instead, you can prioritize by cost of the defect to your business. In this fashion, reputation presents just one possible vector for costing you money. Others can include outages, refunds, and anything that damages top line revenue. Likewise, bugs in your code can affect the bottom line as well, particularly if they hamper automation of processes.
Tackle bugs in the order of the amount of money that you think they cost you. And, don’t worry. I don’t mean to suggest that you should know exactly how much money any given bug will cost. You can’t read the future. Rather, just approximate and use your valuations and use them to guide you.
So far, the approaches have concerned prioritizing defects in a conceptual queue. In other words, look at all of the defects, assign them individual “scores” of some kind, and force rank them. Let’s consider something a bit different for this last philosophy.
Mistakes are inevitable in any process, but released production defects do not need to be. With advances in how we think of techniques like continuous testing, continuous deployment, and smart automated test suites, we can catch a staggering amount of defects before they escape. Sure, perfection is hard, but you can navigate to a place where a defect surprises and dismays, rather than a place where giant lists of defects seem par for the course.
Against this backdrop, we regard defect prioritization differently. Rather than force ranking them according to money or catastrophic impact, we find the very existence of the bug problematic. It becomes less important to fix the bug itself and more important to fix the thing that allowed a bug in the first place.
To concretely illustrate my point, imagine a car assembly line in which all cars coming out had a broken axle. Would you force rank the cars from most to least expensive and fix their axles one at a time? Or would you address whatever in the assembly line caused the cars to have broken axles?
With a holistic approach, you fix the defects themselves, particularly when doing so carries tactical importance. But you place even more emphasis on fixing the development process generating the defects.