DaedTech

Stories about Software

By

Is This Problem Worth Solving?

I’ve done a little bit of work lately on a utility that reads from log files that are generated over the course of a month. Probably about 98% of the time, users would only be interested in this month’s file and last month’s file. How should I handle this?

At different points in my career, I’d have answered this question differently. Early on, my answer would have been to ignore the sentence about “98% of the time” and just implement a solution wherein the user had to pick which file or files he wanted. For a lot of my career, I would have written the utility to read this month and last month’s files by default and to take an optional command line parameter to specify a different file to read (“sensible defaults”). These days, my inclination is more toward just writing it to read this month’s and last month’s file, shipping it, and seeing if anyone complains — seeing if that 2% is really 2% or if, maybe, it’s actually 0%.

Part of this evolution in me was the evolution of the software industry itself. When I started out doing a lot of C and C++ in the Linux/Unix worlds, gigantic documents and users’ manuals were the norm. If you were a programmer, it was your responsibility to write massive, sprawling APIs and if you used someone’s code, it was your responsibility to use those massive APIs and read those gigantic manuals. So, who cares what percentage of time a user might do what? You just write everything that could ever possibly happen into your code and then tell everyone to go read the manual.

Then things started to get a little more “agile” and YAGNI started to prevail. On top of that, the user became more of a first class citizen, so “let’s default to the most common path” became the attractive thing to do and “RTFM” went the way of the dodo. The iconic example would be a mid 2000’s Windows or web application that would give you a sensible experience and then offer a dizzying array of features in some “Advanced Settings” window.

The next step was born with the release of the iPhone when it started to become acceptable and then normal to write dead simple things that didn’t purport to do everything. Apple’s lead here was, “this is the app, this is what it does, take it or leave it.” The “advanced settings” window was replaced by “we’ll tell you what settings you want,” which requires no window.

This shifting environment over the last 15 years informed my perspective but wasn’t entirely responsible for it. I’d say what was responsible for the shift were two realizations. First, I realized, “a ‘business spec’ isn’t nearly as important as understanding your users, their goals, and how they will use what you give them.” It became important to understand that one particular use case was so dominant that making it a pleasant experience at the cost of making another experience less pleasant was clearly worthwhile. The second realization came years later, when I learned that your users do, frequently, want you to tell them how to use your stuff.

Some of this opinion arose from spending good bits of time in a consulting capacity, where pointing out problems and offering a handful of solutions typically results in, “well, you’re the expert — which one should I do?” You hear that enough and you start saying instead, “here’s a problem I’ve noticed and here’s how I’ve had success in the past fixing this problem.” It makes sense when you think about it. Imagine having someone out to fix your HVAC system and he offers you 4 possible solutions. You might ask a bit about cost/benefit and pros/cons, but what you’ll probably wind up saying is, “so…. what would you do and/or what should I do?”

TerrifiedOfFurnace

There’s an elegance to coding up what a user will do 98% of the time and just shipping that, crude as it sounds. As I mentioned, it will confirm whether your “98%” estimate was actually accurate. But, more importantly, you’ll get a solution for 98% of the problem to market pretty quickly, letting the customer realize the overwhelming majority of ROI right away. On top of that, you’ll also not spend a bunch of money chasing the 2% case up front before knowing for sure what the impact of not having it will be. And finally, you add the possibility for a money-saving work-around. If the utility always reads this month’s and last month’s files, and we need to read one from a year ago… rather than writing a bunch of code for that… why not just rename the file from a year ago and run it? That’ll cost your client 10 seconds per occurrence (and these occurrences are rare, remember) rather than hundreds or thousands of dollars in billable work as you handle all sorts of edge cases around date/time validation, command line args, etc.

I always wince a little when I offer anecdotes of the form, “when I was younger, I had position X but I’ve come to have position Y” because it introduces a subtle fallacy of “when you get wiser like me, you’ll see that you’re wrong and I’m right.” But in this case, the point wasn’t to discredit my old ways of thinking, per se, but rather to explain how past experiences have guided the change in my approach. I’ve actually stumbled a fair bit into this, rather than arrived here with a lot of careful deliberation. You see, there were a lot of times that I completely whiffed on considering the 2% case at all and just shipped, realizing to my horror only after the fact that I’d forgotten something. Bracing for getting reamed as soon as someone figured out that I’d overlooked the 2% case, I battened down the hatches and prepared to face the fire only to face… absolutely nothing. No one noticed or cared, and the time spent on the 2% would have been a total waste.

So next time you find yourself thinking about how to handle some bizarre edge case or unlikely scenario, pull back and ask yourself whether handling that is worth delaying the normal cases or not. Ask yourself if your user can live with a gap in the edge case or work-around it. And really, ask yourself what the ROI for implementation looks like. This last exercise, more so than learning any particular framework, language or library, is what distinguishes a problem solver from a code slinger.

newest oldest most voted
Notify of
Matthew Whetton
Guest

I think this is very true and to some degree is very similar to the concept of a minimum viable product, albeit from a non-commercial point of view (which I think applies equally)

Erik Dietrich
Guest

I read the Lean Startup recently, and am inclined to agree with this assessment (though I hadn’t made the connection until you mentioned it).

Andrei Dragotoniu
Guest

I quite like the view on whether you need to implement an edge case or not. What matters to me is the potential to lose income. if that is bound to happen I think the edge case needs to be dealt with rather than wait for someone to find it and possibly exploit it.

There are situations however when it’s so unlikely to happen that it’s not worth spending time fixing it.

Erik Dietrich
Guest

Exactly. I’m not trying to argue that edge cases aren’t worth implementing across the board, but rather that the conversation about their worth should happen before any implementation is done.

Geoff Mazeroff
Guest
Geoff Mazeroff

You may wince at that anecdote of “when I was younger”, yet that shows that best practices evolve. I like to think that there’s an intent for any approach (large manuals, sensible defaults, etc.); those intents change as technology improves. I imagine 10 years from now we’ll look back and say, “Remember in 2015 when it was the norm to do X? How was that even tolerated?”

Andrei Dragotoniu
Guest

yeah that already happened to me. It gets even better when you look at some old code and think to yourself …. I can’t believe I wrote that, what on earth was I thinking 🙂

Erik Dietrich
Guest

Sure, I wouldn’t dispute that. I wince at it being used as a form of persuasion in an argument. For instance, consider a political argument where someone says, “I was a Democrat too when I was younger, but I then I matured and became a Republican.” It’s a subtle way to make what essentially amounts to an obtuse argument, “I’m older than you so I’m right.”

That’s why I say I wince when I frame things this way — the idea that it might be perceived I’m trying to do that.

Eric
Guest
Eric

Lots of good points. I’ve had the benefit of working with a variety of codebases over the years. I’ve come to gain an appreciation for the efforts that others have put in. Its natural to look at legacy code and think “I could have done better”. It may very well be true. A solid routine written in C 20 years ago will probably still look good today. But on the flip side the realisation that comes to me is that somebody knuckled down and produced version 1.0. The fact that I have been hired to upgrade to version 5.0 is… Read more »

Jacob Appleton
Guest
Jacob Appleton

I’ve been reading “The Lean Startup” lately, and as Matthew mentions below, this blog post seems to dovetail nicely with the concepts discussed in that book. I can’t help but see MVP, which for mine is what you’re describing here, as a product of the technology available to us today as developers. We live in a world where software can be constantly verified (I use .NET and NCrunch, and constantly know (to a reasonable degree) that my software is working as I expect it to), easily deployed with the click of a button (or without even that if you are… Read more »

Erik Dietrich
Guest

That’s a very good point. It’s not just that the philosophy around software has changed a lot over the last couple of decades. The underlying technology pipeline and feedback loop has also changed a lot and, in fact, probably *driven* the philosophy changes.

Cheval
Guest
Cheval

I’d like to agree with you, but when something goes really bad (eg. leap second), you always need that 2% of functionality to be easy as well.

Erik Dietrich
Guest

I think that’s a second discussion. Whether to do it up front or not is what I’m talking about here. Keeping your software in a state where it’s easy to implement the 2% (or any arbitrary functionality) is a matter of what, these days, is called “software craftsmanship,” and is something I fully support. Just because we decide not to incur the cost of doing the 2% doesn’t mean we shouldn’t know how it would be accomplished if we needed to do it.

Kerry Patrick
Guest
Kerry Patrick

Great article. The most important thing you wrote there is “understanding your users, their goals, and how they will use what you give them”, because the flipside of that decision is judging something to be a 2% edge case when in reality it’s a 10-15% edge case. I’ve worked at companies where my end users were very close by, and I’ve seen the “learned helplessness” response where users had come to expect that each line of business app would require a hinky workaround on a monthly or quarterly basis because that edge case hadn’t been implemented and the developer had… Read more »

Erik Dietrich
Guest

I’ve seen the anti-pattern you’re describing as well. 🙂 Not a pretty thing.

It’s good that you mention the flip-side too, because that can happen and has happened to me. You take your best guess and economize and then prepare to adapt. “I’m not doing this, Joe End User, because I suspect that it will only happen about 2% of the time. If I’m wrong, and it turns out that it happens a lot more, here’s how we’ll handle that.”

Timothy Boyce
Guest
Timothy Boyce

It is not just that the single 2% case might be bigger than expected. Let’s say you guessed right every time. The problem is that all of the 2% cases combined that were not implemented can add up to a far more significant number.

MickiSon2
Guest
MickiSon2

I completely agree with your assessment. As I’ve become more experienced I’ve definitely learned to identify edge cases and work with my clients to ask ‘do you REALLY need this edge case addressed because it’s going to cost $$$ ?’

Erik Dietrich
Guest

Exactly. That’s the tradeoff that we, as professionals, need to help stakeholders understand.

Eric
Guest
Eric

“First, I realized, “a ‘business spec’ isn’t nearly as important as understanding your users, their goals, and how they will use what you give them.” It became important to understand that one particular use case was so dominant that making it a pleasant experience at the cost of making another experience less pleasant was clearly worthwhile.” – I think this is a crucial point. Pretty much everything you’ve said resonates with me. Sadly we’re not always at liberty to employ this kind of thinking. In certain projects there is a significant divide between between developers and end users.

Erik Dietrich
Guest

I’ve worked in situations like that, with a lot of indirection between the users and developers. I’ve now spent a significant portion of my career actively looking to cut out such middle men. It’s rarely easy.