DaedTech

Stories about Software

By

What TDD Is and Is Not

In my travels, I’ve heard a lot of people embrace test driven development (TDD) and I’ve heard a lot of people say that it isn’t for them. For the most part, it’s been my experience that people who say it isn’t for them have either never really, actually done it or haven’t done it enough to be good at it.

Lest you think I’m just being judgmental, here’s a post of mine from about 3 years ago, before I had truly taken the humbling plunge of forcing myself to follow the red-green-refactor discipline to the letter. My language here is typical of someone who buys into unit testing and even to TDD, but who hasn’t become facile enough with the process to avoid feeling the need to tinker with it and qualify the work. If you cut to the heart of what’s going on in my post there (and I say this hat in hand) it’s less “here’s my own special way of doing TDD that I think is actually superior” and more “I haven’t yet gotten good enough at TDD to use it everywhere and that’s rather frustrating.”

Stoplight

Not long after I wrote that post, I forced myself, on a project, to follow the discipline, to the letter, and I’ve never looked back after the initial pain of floundering during what I used to describe as the “prototyping stage” of my coding. I solved the ‘problem’ of TDD not being compatible with that prototyping by thinking my designs through more carefully up-front, solving only problems that actually exist, and essentially not needing it anymore. TDD wasn’t the problem for me when I wrote that post; the problem was that I wasn’t being efficient in my approach.

It’s been my experience that most of the criticism of TDD comes from people like me, years ago. These are people who either have never actually tried TDD or who have tried it for an amount of time too short to become proficient with it. I can rarely recall someone who became proficient with the practice one day saying, “you know what, enough of this — I don’t see the benefit.” For me this statement is hard to imagine because TDD shortens the feedback loop between implementing something and know if it works, and developers innately crave tight feedback loops.

And since the majority of criticism seems to come from those least familiar with the discipline, there are a lot of misconceptions and straw man arguments, as one might expect. So today, I’d like to offer some clarity by way of defining what TDD is and what it it isn’t.

What TDD Isn’t

I think it’s important that I first discuss what TDD is not. There are a lot of misconceptions that surround the test driven development approach to writing software, and there’s a pretty good chance that you’ve at least been exposed to them, if not slightly misled by them. Not only does this make it hard to understand how the practice works, but it may even lead you or your team to reject it for reasons that aren’t actually valid.

  1. First, TDD is not a replacement for user acceptance testing. Someone who practices it does not believe that it’s a valid substitute for running the application and making sure that it does what the requirements state that it needs to do.
  2. Classic TDD is also not comprehensive automated testing. The by-product of it is mainly unit tests, which are tests for finely grained pieces of code, such as classes and methods. Other kinds of automated tests, such as integration tests and end to end system tests involving databases or other external constructs will be a separate prong of your overall testing strategy.
  3. One thing that I frequently hear as an indictment of TDD is that it doesn’t encourage you to think through your design because you initially do the simplest thing to make tests pass. This is not accurate at all. It simply distributes the planning over the course of your development as you go rather than forcing it all to be done up front.
  4. Test driven development is not and does not claim to be any sort of load testing, concurrency testing, or anything else that you might put under the category of “smoke” or “stress” testing. The tests generated by TDD are not meant to test the behavior of your system under adverse conditions.
  5. Another common misconception is that TDD means that you write all tests for the system before you start writing your code. Frankly, doing so would be wildly impractical, and detractors who cite this impracticality as a reason not to do TDD are arguing against a straw man.
  6. TDD is not a quality assurance strategy. When your software department is contemplating new initiatives and dividing them up according to who will own then, TDD does not belong to the testing group.
  7. Detractors of TDD often point out that it doesn’t address corner cases in application or even class and method logic. That is true, but TDD doesn’t aim to address these. They belong with the other automated tests that will be written later.
  8. And, believe it or not, TDD is not primarily a testing activity. This is probably the hardest for people to wrap their heads around when learning the practice. But if you think about the acronym – test driven development, it is primarily development. The tests driving it are simply a characteristic of the development.

What TDD Is

Having gone over a series of things that TDD is not, hopefully I’ve cleared up some misconceptions and narrowed the field a bit. So let’s take a look at what TDD actually is. I should note here that the flavor of TDD that I’m addressing is “Classic,” triangulation-oriented TDD rather than the behavior driven “London” school of TDD.

  1. As mentioned in the last section, TDD is a development approach. It’s a development approach that happens to produce unit tests as you go, which you can then save for later. I suppose you could discard them and retain some of the benefit of the approach, but that would certainly be a waste since you’re going to want automated tests for your system anyway.
  2. Another facet of the test driven development approach is that you avoid “paralysis by analysis,” a situation in which you are so overwhelmed by the complexity of the problem that you simply stare at the screen or otherwise procrastinate, unsure how to proceed. TDD ensures that you’re constantly solving manageable problems.
  3. TDD produces test cases that cover and address every line of code that you write. With this 100% test coverage, you can change your code fearlessly in response to altered requirements, architectural needs, or other unforeseen circumstances. You’ll never have to look at the system nervously, wondering if you’re breaking things. Your tests will tell you.
  4. Besides allowing you to change code easily, test driven development also guides you toward a flexible design. The reason for this is that TDD forces you to assemble your code with testing seams in it, which are entry points, such as constructor and method arguments, that allow you to take advantage of polymorphism for easier testing. A side effect of this is that your code that’s testable is also easier to configure and mix and match in production.
  5. TDD is also a way to ensure that you’re writing as little code as necessary. Since every change to production code requires a failing test, you have to think through exactly what the system needs before you ever touch it. This prevents speculative coding and throwing in things that you assume you’ll need later, such as property setters you never actually use.

So how is all of this accomplished? Well, TDD is a discipline that follows a specific process and relatively simple process.

  1. As I mentioned briefly, the first step is that you expose some kind of deficiency in the system that you’d like to address. This could be a bug, a missing feature, a new requirement – anything that your codebase does not currently do that you want it to. With the deficiency picked out, you write a test that fails because of the deficiency.
  2. With the failing test in place, you then implement the simplest possible solution in your code to get the failing test to pass, ensuring that all of your other tests also still pass. This makes the system completely functional.
  3. Once the system is completely functional, you look at your quick and dirty fix and see if it needs to be refactored toward better design. If so, you perform the refactoring, ensuring that the tests still pass.

So there you have it: a brief overview of what TDD really is. If you’re interested in more on this subject and you have a Pluralsight subscription, check out my course on continuous testing TDD using a tool called NCrunch, which is all about speeding up your feedback loop during development. Most of this post is from the transcript of that course. If you don’t have a Pluralsight subscription and are interested in a trial, drop me a line and I’ll give you a free week of Pluralsight subscription.

By the way, if you liked this post and you're new here, check out this page as a good place to start for more content that you might enjoy.

By

There’s No Excuse

If you have a long, error-prone process, automate it. This is the essential underpinning of what we all do for a living and informs our day to day routines. Whether we’re making innovative new consumer applications, games, line of business applications, or anything else, we look at other people’s lives and say, “we can make that automatic.” “Don’t use a pen and paper for your todo list when there’s GTasks.” “Don’t play risk with a board and dice when you can do it online (or play an Elder Scrolls game instead).” “Don’t key all that nonsense in by hand.”

And yet, far too often we fail to automate our own lives and work. I wonder if there’s some kind of cognitive blindspot we have. When other people mindlessly perform laborious tasks, we jump in and point out how dumb what they’re doing is and how we can sell them something that will rock their world. But when we mindlessly perform laborious tasks (data entry, copy and paste programming, etc) we don’t think twice or, if we do, we assure ourselves that it’s too complicated or not worth automating or something.

I was going through some old documentation the other day to look up what was needed to add another instance of a feature to a legacy application. What it boiled to was something along the lines of “we have an application that sells computers and we want to add a new kind of PCI card to the configurable options.” Remarkably, this involved hand-adding things to all sorts of different meta data tables in a database, along with updating some stored procedures. The application, generally speaking, was smart enough to do the rest without code changes, even including GUI updates, so that was a win. But hand-adding things to tables was… annoying.

If only there were something that would let people add things to the database without using the query explorer tool and doing it by hand… something where you could write instructions in some kind of “language,” if you will, and translate these instructions into something visual and easy to understand so that people could add meta-data more simply. Hmmm.

Wait, I’ve got it! How about instead of a document explaining how to add a bunch of records to the database, you write an administrative function into your GUI that automates it? What a victory! You get the meta-data added and your new PCI option, and you also remove the possibility that you’ll mangle or forget one of the entries and leave the system in a poor state. This is really the only option. It’s not an either-or kind of situation. Hand-adding things to your database is facepalm.

What’s the moral of this story? To me, it’s this: if you have a giant document detailing manual steps for programmers to follow to get something done, what you really have is a spec/user story for your next development cycle. Automate all the things, and then burn those documents at a cathartic, gleeful camp fire. You can turn your onerous processes into roasted marshmallows.

By

Cleaning Up Your Build

Today, I’d like to make a post that’s relatively short and to the point. And it’s relatively short and to the point because what I’m going to talk about is so basic and fundamental that it won’t take long to say it. It is absolutely critical that you have nothing standing between the stuff checked into your project’s source control and a completely successful build and deployment of your software.

Here’s the first thing that should absolutely be true. Someone brand new to the team should be able to take a computer with nothing special installed on it, point it at the latest good version of your project in source control, get the latest code, build it in the IDE, and be able to run your software when the build is done. If when this person runs the software, weird things happen or it crashes, you’ve failed. If this person can’t even successfully build the software, you’ve failed badly. If this person can’t even compile the software, things are really ugly. If this person can’t get the software out of source control, you’re probably using Rational Clear Case, and that poor person coming to the team has no idea what’s coming for the next months and years. Not being able to get things out of source control without jumping through hoops is total fail.

It is absolutely critical that right away, on the first day, someone can get your software from source control, build it and run it without issues. If this isn’t possible right now, make a project plan/user story/whatever for making it be possible in the future and give this a high priority. Think of yourself as a restaurant that has severe health code violations. Sure you could ignore them and continue cranking out your signature dish, “Spaghetti with Botulism,” but it’s not advisable. You need to have a clean and sanitary work environment, free from nasty cruft and residue that’s worked its way into being a normal part of your process.

Once you’ve got a “source control to runtime” path that’s pristine, you need to make sure this is also the case for deployment. You should be able to fire up a clean target machine or VM, deploy your deliverables to it in some automated fashion, and have it work. What you shouldn’t need to do is install MS Word on there or remember to copy over those six license files and that .trx thing in that one directory. Oh yeah, and a registry setting or something.

As soon as you’re doing stuff like that, you have a polluted build because you have a point of failure. You have your “automated” process and then you have the thing that you just have to remember to do every time or things go badly. If any of your process is manual, you WILL mess it up and cause problems. We’re humans and its inevitable. This is especially true if you aren’t agile and deployments happen only rarely. If you can, eliminate it as a step, but if you can’t, then automate it. Deploying should be dead simple.

And something else to bear in mind is that past sins aren’t forgiven. In other words, if you have a deployment process now that’s simple and one click and works every time with something like XCopy, that doesn’t mean you’re out of the woods. The “on a clean machine” requirement is critical. If you’re XCopying over existing files to deploy, you might have some weird one-off thing that you did to the server 2 years ago and have forgotten all about. You need to make sure you can nuke your whole deployment, redo it, and have it work.

If it sounds as though I’m being a hardliner or extremist, perhaps that’s the case, but I think it’s justifiable on this subject. You can’t negotiate with cargo cult build processes. They have to be eliminated because there is absolutely no upside and absolutely pure downside. Think about your own source control, build and deployment processes and ask yourself if there are things that need to be weeded out. And you know what? Just take a crack at it. I’ve done this sort of thing myself on a lot of different projects and I’ve always found it’s never as hard to fix the problems as you think it’s going to be. Usually it’s just that no one thinks to try.

By

Interface Segregation Principle: A Practical Example

I’ve had this partially completed post in my drafts folder for a while, and, thanks to a sort of half-hearted New Years resolution to either finish or discard really old drafts, I’m going to finish this one. I changed the title and re-did the focus a bit, since this one was a year and a half old, and the code in question here is hazier and hazier in my mind. But it’s a tale of Singletons, needless coupling, poor cohesion, and woe.

I’ve been interviewing some candidates of late and one of the things I typically ask about is familiarity with the SOLID principles. An encouraging amount of people say things like, “oh sure, classes should have only one purpose” or “oh, yeah, we use IoC containers!” It seems as though S, O, and D get the most love, while L and I (Liskov Substitution Principle and Interface Segregation Principle, respectively) are rarely mentioned. Today, I’ll talk about I with an example that may hit home with you.

I worked on a project once with a fairly unique ‘architecture.’ It was a GUI-heavy desktop application with a reasonably straightforward set of domain objects that were read from persistence and stored in memory. This was accomplished using what really kind of amounted to an old school, C-style memory map of structs. There was a root object, and everything was hierarchically possessed by this root object. So, let’s say that you were modeling a parking lot full of cars — you’d navigate to the car parts via calls like Root.Cars[12].Engine.Battery or Root.Cars[5].Body.Exterior.Paint. And naturally, Root was implemented as a Singleton so that you could access this memory map from anywhere at any time to read or write. Yep. That was a long project.

This Singleton probably started out modestly enough, but by the time I had worked on the project, its sizable gravitational pull had roped smaller satellites, code-moons and free-falling bits of flotsam into it, creating a runaway juggernaut of anti-pattern. This thing was, if memory serves, pushing 10K lines of code and perhaps even spreading out into some partial classes. I think by the time I moved onto another project, it was showing the first signs of having an event horizon.

LargePlanet

There were several different skins on this desktop app, and for some reason, each of them had their own mini-versions of this Singleton (a much more modest 1K to 2K LOC, each), I guess to contain the skin-specific sprawl that would have created a circular reference situation with the behemoth itself. It was one of these mini-versions that inspired this post, along with a story a friend of mine told me after I’d left the project.

A few of us had, while working in this code base, endeavored to extract at least tiny pockets of code from the massive gravitational field of the Singletons so that we could write some unit tests and avoid the bug whack-a-mole game that ensued whenever you changed something in the global state. I was probably the most successful in fighting this battle and, my friend, having less success after I’d left, complained to me one day over lunch. He said that he’d had a perfectly testable class, but in a code review someone in a position of relative authority had pointed out that something he was doing was already done in one of the satellite Singletons and he should use that code. His request to hide the Singleton behind a wrapper or even an interface implementation was denied. A large portion of his class became thoroughly untestable because naturally, the first call to his stuff triggered the first lazy call to the satellite, which triggered the first lazy call to the black hole Singleton, which fired up every threading model, logger, aspect, and bit of file I/O in the history of creation, crushing the test runner like an insignificant gnat.

And this, to me, is perhaps the best argument for the Interface Segregation Principle that I’ve ever heard. As a quick recap, the ISP says that “clients should not be forced to depend upon interfaces that they don’t use.” In my friend’s case, depending on the utility method that he was being forced to use, in turn, made him depend on an entire, 1K+ LOC Singleton being initialized and, indirectly, a whole host of other application functionality. This was about the most egregious violation that I’d ever encountered.

What’s the alternative? Well, years later, it’s hard to list specifics, but how about pulling that utility out somewhere? If it relied on no global state, this is a no-brainer — just put it into its own class somewhere. Static, instance — it doesn’t matter — just not in that Singleton. If it relied on global state, then create a small interface with one method, have the satellite Singleton implement it, and hand that interface to the class in question. In either case, my friend’s class depends on a single method for that functionality and not the entire application domain and the code responsible for loading it (as well as loggers and other ancillary nonsense).

The Interface Segregation Principle asks you to keep your dependencies to a minimum. This will help you unit test, but it will also help your sanity at maintenance time. After all, would you rather debug a class that you knew depended on an external method or two, or one whose behavior could be explained by any one of tens of thousands of lines of code? These things will matter to you or whoever maintains the application. A lot. So think of this example and think of the ISP as you’re writing your code.

By

Intro to Unit Testing 10: The Business Value of Unit Tests

Backstory

I worked for a consulting firm for a while. We didn’t make anything particularly exciting — line of business applications and the like was about the extent of it. The billing model for clients was dead simple and resembled the way that lawyers charge; consultants had an hourly rate and we kept diligent track of our time, to the nearest quarter hour. There was a certain feel-good element to this oversimplification of knowledge work in the same way that it’s pleasant to lean back, watch Superman defeat Lex Luthor and delight in a PG world where Good v Evil grudge matches always end with Good coming out the victor.

It’s pleasant to think that writing software has the predictable, low-thought cadence of an activity like chopping wood where each 15 minutes spent produces a fairly constant amount of value to the recipient of the labor. (Cue background song, Lou Reed, “A Perfect Day“) Chop for 15 minutes, collect $3, hand over X chopped logs. Chop for 1 hour, collect $12, hand over 4X chopped logs. Write software for 15 minutes, produce a working, 15 LOC application for $25. Write software for 1 hour, produce a working, 60 LOC application for $100. Oh, such a perfect day.

When I started at the company, I asked some people if they wrote unit tests. The answer was generally ‘no’ and the justification for this was that you’d have to run it by the client and the client most likely wouldn’t want to pay for you to write unit tests. What they meant by this was that since we billed in quarter hour increments and supplied invoices with detailed logs of all activity, it’d be sort of hard to sneak in 15 minutes of writing automated test code. Presumably, the fear was that the client would say, “what’s this ‘unit testing’ stuff and why did you do it when you didn’t say anything about it.” I say “presumably” because this wasn’t the reason people didn’t unit test at this company just like whatever excuse they have at your company isn’t the real reason for not unit testing there. The real reason is usually not knowing how to do it.

Why did I start out with this anecdote and its centerpiece of the quarter hour billing and development cadence? Well, simply because software development is a creative exercise and far too spastic to flow along smoothly in a low viscosity stream of lines of code per minute. You may sit and stare blankly at a computer screen while contemplating design for half an hour, code for 4 minutes, stare blankly again for an hour, code for 20 minutes, and then finish the product. So, 24 minutes — is that a billable half hour or 15 minutes? Closer to 30, I suppose. Do you count the blank staring? On the one hand, this was the real work — the knowledge work — in a way that the typing certainly wasn’t. So should you bill 1.5 hours instead and just count the typing as a brainless exercise? Or should you bill 2 hours because the work is a gestalt? I personally think that the answer is obvious and the gestalt billing model cuts right to the notion that software development is a holistic exercise that involves delivering a working product, and the breakdown may include typing, thinking, white-boarding, searching Stack Overflow, debugging, squinting at a GUI, talking to another developer, going for a 5 minute walk for perspective, running a static analysis tool, tracking down a compiler warning, copying 422 files to a target directory, and yes, my friend, unit testing.

Those are all things that you do as part of writing good software. And, in a consulting paradigm, you wouldn’t cut one of them out and say, “the client wouldn’t want to pay for that,” because the client doesn’t know what its talking about when you’re under the software-writing hood — that’s why they’re paying you. They wouldn’t want to pay for “searching Stack Overflow” or “Squinting at the GUI” either, but you don’t refuse to do those things when you’re writing software. And so refusing to unit test for this same reason is a cop-out. When a younger developer at that firm asked me why I wrote unit tests and how I accounted for them in my billing, this was essentially the argument I gave — I asked him how he accounted for the time he spent compiling, debugging, and running the application and, bright guy that he is, he understood what I was saying immediately.

Core Business Value

This may seem like a roundabout and long introduction to this chapter, but it really cuts to the core of the business value proposition for unit tests. During development, why do developers compile, run, and debug? Well, they do it to see if their code is doing what they think it should do. Write some code, then make sure it’s doing what you expect. So why write unit tests? To make sure your code is doing what you expect, and to make sure it keeps doing what you expect via automation. The core business value of unit tests is that they serve as progress markers, sign-posts, and guard rails on the road to an application that does what you expect.

Unit tested applications are more predictable and better documented than their non-unit tested counter parts (assuming the same amount of API documentation and commenting are done), and there is an enormous amount of business value in predictability and clarity of intent. With a good unit test suite, you’ll know in minutes if you’ve introduced a regression bug. Without that unit test suite, when will you know? When you run the application GUI? When QA runs it a week later? When the customer runs it a month later? When something randomly goes haywire a year later? Each one of these delays becomes exponentially more expensive.

That’s not the only value-add from a business perspective (and I’ll list some other ones next), but it’s the main one, as far as I’m concerned. It also explains why the notion that you need to carve out some extra time for unit tests and figure out whether the customer wants them or not is preposterous. Do you think the customer is going to get angry if you explain that part of your development process is to execute the code you just wrote to make sure it doesn’t crash? If the answer to that is “no, of course not, that’s ridiculous” then you also have the answer to whether or not a customer would care if you happened to automate that process.

Of course, one thing to bear in mind is that a customer may not want to pay for you to learn on the job to unit test, and that’s a fair point. But if the customer (or your company, internally, if you aren’t a consultant) doesn’t want to foot the bill for this, then you should strongly consider picking it up on your own and then switching customer/company if they don’t buy in to something as fundamental as automating predictability. Unit tests are the software equivalent of accountants practicing double entry bookkeeping, doctors washing their hands, electricians turning power back on before leaving and plumbers doing the same with the water. Imagine if your plumber sweat welded a joint for your new shower, sized it up and then said, “meh, I’m sure it’s fine” and left without ever running the water. That’s what tens of thousands of us do every day when we just assume some piece of code works because it worked a month ago and you don’t remember touching it since then. Ship it? Meh, sure, whatever — it’s probably fine. The business value of unit tests is a stronger assurance that we know what’s going on than “meh, sure, whatever.”

Ancillary Business Value

Here are some other ways in which unit tests add value to the business beyond confirming that the application behaves as expected.

First, unit tests tend to serve nicely as documentation. This may sound strange at first; you’re probably thinking, “how is a bunch of code documentation when we have a whole activity associated with documenting our code?” Well, fact of the matter is that documentation in the form of writeups, code comments, instruction manuals, etc tends to get out of date as the product ages. Unit tests, however, are never out of date because if they were, the build would fail (or at least you’d see red when you ran them) and you’d be forced to go back and “fix the documentation.” If you keep your unit test methods clean and give them good names as described in earlier chapters of this series, they’ll also read more like a book than like code, and they’ll document purpose and intended behavior of the system.

Unit tests also guard against regression. When you write the tests, you’re confirming that the software does what you expect it to at that moment. But what about later? Maybe later you forget what you intended in that moment or decide that you intended something different and you change the code. Will it still work? In a lot of legacy code bases, the answer to that question is, “yikes — who knows?” With a thoughtfully unit tested code base, you can rig it so that a test goes red if a design assumption that you made is no longer true. For instance, say you write some method with the intention that it never return null, and say that eventually you and your teammates build on this method and its assumed post-condition, grabbing values returned by the method and never checking for null before dereferencing them. If someone later modifies that method and adds a condition in which it returns null, the only thing standing between them and introducing a regression bug is a unit test that fails if that method returns null.

The practice of automated unit testing has after the fact benefits such as documentation and guards against regression bugs, but it also helps during the course of development by having a positive impact on your design. I’ve long been a fan of and have previously linked to this excellent talk by Michael Feathers called “The Deep Synergy Between Testability and Good Design.” The general idea is that writing code with the knowledge that you’re going to be writing tests for it (or practicing TDD) leads you to write small, factored classes and methods that are loosely coupled and that this practice, in turn, creates flexible and maintainable code. Or, consider the converse and think of how hard it is to write unit tests for giant, procedural methods and classes. Unit tests make it harder to do things that make your code awful.

Lastly, I’ll throw in a benefit that summarizes my take on this entire subject and really drives things home. It’s a huge bit of editorializing, but I feel somewhat entitled to do so in my own conclusion. I believe that a serious piece of value added by unit testing is that it lends you or your group legitimacy and credibility. In this day and age, the question, “should you unit test your code” is basically considered to be settled case law in the industry. So the question, to a large extent, boils down to whether you write tests or whether you have excuses, legitimate or otherwise (and there are legitimate ones, such as “I don’t know how, yet.”) Don’t be in the camp that has excuses.

Forget justifying what you or your organization has done up to this point, and imagine yourself as a customer of software development. You’ve got a budget, and you’re looking to have some software written that you don’t have time, yourself, to write. All other things being equal, which group do you hire? Do you hire a group that responds to “do you unit test” with “no, we don’t think our customers would want that?” How about a group that responds with “well, there’s this database and this GUI and sometimes there’s hardware, so we really can’t?” Or do you hire a group that responds with, “we sure do, would you like to see some samples?” I bet it’s the last one, if you’re honest with yourself.

So be that last group. Add value to your users and your business. Write good software and consider your design carefully, and, just as importantly, automate the process of ensuring your software does what you think it does. Your credibility and the credibility of your software is at stake.