DaedTech

Stories about Software

Jun
19

By

Introduction to Unit Testing Part 3: Unit Testing Sucks

Category: Language Agnostic Tags: , | 18 Comments

I don’t know about you, but I remember desperately wanting to be able to drive right up until I was fifteen years old and I got my learner’s permit. I thought about it a lot–how fun it would be, how much freedom I would have, how my trusty old bike would probably get rusty from disuse. About a month after getting my permit, I desperately wanted my license and to drive on my own without supervision. But I’m omitting a month there, during which an unexpected thing happened. I realized that driving was stupid and awful and it sucked and I hated it and I’d never do it, so just forget it!

It was in that month that the abstraction of operating a car and having freedom became the reality of hitting the gas when I meant to hit the brake pulling out of my driveway or not knowing when I was supposed to go after stopping at a stop sign. It was a weird mix of frustration, anger, and fear that tends to accompany new activities–even ones that you know will benefit you. And that’s why the title of this post isn’t simple link bait. I did that not to satirize a position, but to empathize. Like many things when you’re new to them, starting to unit testing quite frankly sucks. It’s frustrating, foreign, and hard to get right. Accordingly, it’s easy to abandon it when you have deadlines to meet.

Accident

This post is about minimizing frustration and barriers to adoption by staying focused and setting reasonable expectations. I would argue that if you’re new to writing tests, writing a few and enjoying localized success without high coverage is a lot more important than suddenly becoming a TDD (Test-Driven Development) expert with 100% test coverage right out of the gate (or at least trying to become one). Incremental progress is good.

Don’t Try TDD Just Yet

I’m a little torn as I write this, but the first thing that I’ll suggest is that you not try TDD if you have no experience unit testing. Some might disagree with this suggestion, but I think that you’re going to be trying to learn too many new things all at once and will be a lot more likely to get frustrated. Unit tests are simply pieces of code that you write, as covered in more detail in the last post in the series. It’s a new kind of code to be writing, but you’re just learning about new methods to call and attributes (or annotations, in Java) to use. You’ll get there.

But TDD is an entirely new way of writing code. It’s a discipline in which you do not write any production code until you have written a unit test that fails. Then you get that test and all other tests to pass and refactor the code as needed. Does that sound crazy (if you discount the fact that a number of developers you respect probably do it)? Exactly. Probably not for you right now. It’s a bridge too far, and you’re more likely to throw up your arms in disgust and quit if you try to learn both things right now. I speak from experience, as, years ago, I was introduced to unit testing and TDD at the same time. I was overwhelmed until I just went back to figuring out the whole unit testing thing alone first. Maybe that wouldn’t happen to you, but I’d caution you to be wary of learning these two things simultaneously.

So let’s stick to learning what unit tests are and how to write them.

Test New Classes Only

In my pluralsight course, I use the example of a method that identifies numbers as prime or not, and in a series of posts I did last fall on TDD, I use the example of something that calculates a bowling score. I’ve also done other code katas and exercises like these in the past to show people both the mechanics of unit testing and TDD.

When I do this, one of the things people frequently say is something along the lines of “pff…sure, when you’re writing something stupid and easy like a prime number finder, but there’s no way that would work on our code base.” I then surprise these people by agreeing with them. I’m sure it wouldn’t work on your code base. Why? Well, because unit tests don’t just magically spring up like mushrooms after a few days of rain. They’re more like roses–you have to plan for them from the start and carefully cultivate an environment in which they can thrive.

Some years back, I saw an excellent talk on “The Deep Synergy Between Testability and Good Design,” by Michael Feathers. I highly suggest watching this talk if you haven’t seen it, but to summarize, he states (and I agree) that well-designed and factored code goes hand in hand with testability. You’re much more likely to find that code written to be testable is good code and, conversely, code written without unit tests in mind is not the greatest. And so if you’re deeply invested in a code base that has never been covered by unit tests, it doesn’t surprise me to hear that you don’t think unit testing would work on your code. I imagine it wouldn’t.

But don’t throw out unit testing because it looks like it wouldn’t work in your code base. Just resolve to do it on new classes that you create. As you go along and get better at unit testing, you’ll start to understand how to write testable classes. It will thus get easier and easier to test all new additions to the code, and you’ll start to get the hang of it with relatively minimal impact on your existing code, your process, or your time. Starting to unit test doesn’t mean that you’re suddenly responsible for testing every line of code in history, nor does it mean you must test every single new line. Just start out by writing a few that you think will help.

Test Existing Code by Extracting Little Classes

Once you get the feel for adding unit tests for new classes/code that you add to the code base, it’s a good time to start taking baby steps toward getting tests in place for your legacy (non-tested) code. Now, some procedural, monolithic mass of code that wasn’t testable a month ago when you started out isn’t magically testable now because you have some practice. It’s still a problem.

You’re going to have to chip away at it. And you’re going to have to do this by developing a new skill: identifying pieces of functionality that you can pull out into new classes and test. Go look through methods and classes and find things that don’t have a lot of dependencies on class fields or (yuck) global/static variables. Excellent candidates for this are methods with pure in-memory operations and ones that deal largely with primitives. Do you have some gigantic method that has a whole region buried in it that does nothing but cobble together a string to be used later in the method? Pull that out into a new class, and write unit tests that make assertions about the string it returns.

As you practice this, you’ll get a better and better feel for what you can pull out with a minimum of friction. You’ll find yourself not only getting more of your codebase under test, but also that you’re improving its design and modularity.

Know When to Fold ‘Em

This is another one that’s hard to type, but you really have to learn to look at code and just say, “nope, not happening.” There are classes and methods that you simply are not going to be able to test unless you come back with a green belt in unit testing–or pair with someone who has hers. And, even then, the prognosis may be that you need to rewrite the legacy class/method altogether to make it testable. Here is a quick list of things that, early in your unit-testing career, you should consider to be deal-breakers and simply move on from to avoid frustration. As a beginner, avoid testing code (class methods and properties) that:

  1. Calls static methods. At best, a static method is functional and returns something that depends only on its inputs. If this is the case (such at functions like Math.Pow() or Math.Abs()), the code is still testable, but a far more common case, especially if the static methods are ones in your own code base, is that they manipulate some kind of global state. Global state is testability kryptonite. I’ll explain more later, but for now, please take my word for it.
  2. Invokes singletons. The singleton design pattern is used almost universally as a politically correct way to hide your global variables in plain sight. For what this means to testability, see the last bullet. If it calls singletons, forget it, move on.
  3. Dispatches background workers or manages threading. When unit tests are run, the unit test runner is responsible for managing threading and it will run your tests in parallel. If you’re trying to make sure your threads and thread management are in one state for production and another for testing, you are about to ruin your day and probably your week. It’s not worth it–don’t try.
  4. Accesses files, connects to databases, calls web services, etc. I mentioned this in the first post in the series, but that was in the context of saying that these things aren’t considered unit tests. Well, another issue here is that they’re also relatively brittle and long running. If you write tests that do these things, they’re going to fail at weird times and in unpredictable ways. You’ll be used to all of your tests passing and suddenly one fails and then passes again, and it turns out it’s because Bill from accounting bumped into the database server and its Nic card is a little “tricky.” If you have unit tests that fail for borderline-inconceivable reasons beyond your control, you will become discouraged.
  5. Code that triggers any of the above anywhere in the call stack. You don’t escape the problems of threading, global state, or externalities by not using them directly. If you trigger them, it’s the same difference.
  6. Classes that require crazy amounts of instantiation. If you want to test a method, but it has forty-five parameters, most of which are classes that are difficult or complex to create, forget it. That code sorely needs reworking, and creating massive, brittle tests for it this early in your career will be a world of pain. Chip away at making the design better before you tackle it.

Don’t worry–I’m not suggesting that you give up on a long timeline, and I’ll continue on with this series and discuss strategy for addressing these things later. But for now, just consider them signals that this code is out of bounds for testing. If you don’t, there’s a high likelihood that you’ll spin your wheels and get angry, frustrated, and irritable, making it more likely that you’ll give up. I can’t eliminate the frustration of being new at something like driving, but I can at least steer you away from six-way traffic lights and three-lane roundabouts.

Jun
14

By

Introduction to Unit Testing Part 2: Let’s Write a Test

Category: .NET Tags: , , , | 5 Comments

In the last post in this series, I covered the essentials of unit testing without diving into the topic in any real detail. The idea was to offer sort of a guerrilla guide to what unit testing is for those who don’t know but feel they should. Continuing on that path and generating material for my upcoming presentation, I’m going to continue with the introduction.

In the previous post, I talked about what unit testing is and isn’t, what its actual purpose is, and what some best practices are. This is like explaining the Grand Canyon to someone that isn’t familiar with it. You now know enough to say that it’s a big hole in the earth that provides some spectacular views and that you can hike down into it, seeing incredible shifts in flora and fauna as you go. You can probably convince someone you’ve been there in a casual conversation, even without having seen it. But, you won’t really get it until you’re standing there, speechless. With unit testing, you won’t really get it until you do it and get some benefit out of it.

So, Let’s Write that Test

Let’s say that we have a class called PrimeFinder. (Anyone who has watched my Pluralsight course will probably recognize that I’m recycling the example class from there.) This class’s job is to determine whether or not numbers are prime, and it looks like this:

public class PrimeFinder
{
    public bool IsPrime(int possiblePrime)
    {
        return possiblePrime != 1 && !Enumerable.Range(2, (int)Math.Sqrt(possiblePrime) - 1).Any(i => possiblePrime % i == 0);
    }       
}

Wow, that’s pretty dense looking code. If we take the method at face value, it should tell us whether a number is prime or not. Do we believe the method? Do we have any way of knowing that it works reliably, apart from running an entire application, finding the part that uses it, and poking at it to see if anything blows up? Probably not, if this is your code and you needed my last post to understand what a unit test was. But this is a pretty simple method in a pretty simple class. Doesn’t it seem like there ought to be a way to make sure it works?

I know what you’re thinking. You have a scratchpad and you copy and paste code into it when you want to experiment and see how things work. Fine and good, but that’s a throw-away effort that means nothing. It might even mislead when your production code starts changing. And checking might not be possible if you have a lot of dependencies that come along for the ride.

But never fear. We can write a unit test. Now, you aren’t going to write a unit test just anywhere. In Visual Studio, what you want to do is create a unit test project and have it refer to your code. So if the PrimeFinder class is in a project called Daedtech.Production, you would create a new unit test project called DaedTech.Production.Test and add a project reference to Daedtech.Production. (In Java, the convention isn’t quite as cut and dry, but I’m going to stick with .NET since that’s my audience for my talk). You want to keep your tests out of your production code so that you can deploy without also deploying a bunch of unit test code.

Once the test class is in place, you write something like this, keeping in mind the “Arrange, Act, Assert” paradigm described in my previous post:

[TestMethod]
public void Returns_False_For_One()
{
    var primeFinder = new PrimeFinder(); //Arrange

    bool result = primeFinder.IsPrime(1); //Act

    Assert.IsFalse(result); //Assert
}

The TestMethod attribute is something that I described in my last post. This tells the test runner that the method should be executed as a unit test. The rest of the method is pretty straightforward. The arranging is just declaring an instance of the class under test (commonly abbreviated CUT). Sometimes this will be multiple statements if your CUTs are more complex and require state manipulation prior to what you’re testing. The acting is where we test to see what the method returns when we pass it a value of 1. The asserting is the Assert.IsFalse() line where we instruct the unit test runner that a value of false for result means the test should pass, but true means that it should fail since 1 is not prime.

Now, we can run this unit test and see what happens. If it passes, that means that it’s working correctly, at least for the case of 1. Maybe once we’re convinced of that, we can write a series of unit tests for a smattering of other cases in order to convince ourselves that this code works. And here’s the best part: when you’re done exploring the code with your unit tests to see what it does and convince yourself that it works (or perhaps you find a bug during your testing and fix the code), you can check the unit tests into source control and run them whenever you want to make sure the class is still working.

Why would you do that? Well, might be that you or someone else later starts playing around with the implementation of IsPrime(). Maybe you want to make it faster. Maybe you realize it doesn’t handle negative numbers properly and aim to correct it. Maybe you realize that method is written in a way that’s clear as mud and you want to refactor toward readability. Whatever the case may be, you now have a safety net. No matter what happens, 1 will never be prime, so the unit test above will be good for as long as your production code is around–and longer. With this test, you’ve not only verified that your production code works now; you’ve also set the stage for making sure it works later.

Resist the Urge to Write Kitchen Sink Tests

kitchensink

When I talked about a smattering of tests, I bet you had an idea. I bet your idea was this:

[TestMethod]
public void Test_A_Bunch_Of_Primes()
{
    var primes = new List() { 2, 3, 5, 7, 11, 13, 17 };
    var primeFinder = new PrimeFinder();


    foreach(var prime in primes)
        Assert.IsTrue(primeFinder.IsPrime(prime));
}

After all, it’s wasteful and inefficient to write a method for each case that you want to test when you could write a loop and iterate more succinctly. It’ll run faster, and it’s more concise from a coding perspective. It has every advantage that you’ve learned about in your programming career. This must be good. Right?

Well, not so much, counterintuitive as that may seem. In the first place, when you’re running a bunch of unit tests, you’re generally going to see their result in a unit test runner grid that looks something like a spreadsheet. Or perhaps you’ll see it in a report. If when you’re looking at that, you see a failure next to “IsPrime_Returns_False_For_12” then you immediately know, at a glance, that something went wrong for the case of 12. If, instead, you see a failure for “Test_A_Bunch_Of_Primes”, you have no idea what happened without further investigation. Another problem with the looping approach is that you’re masking potential failures. In the method above, what information do you get if the method is wrong for both 2 and 17? Well, you just know that it failed for something. So you step through in the debugger, see that it failed for 2, fix that, and move on. But then you wind up right back there because there were actually two failures, though only one was being reported.

Unit test code is different from regular code in that you’re valuing clarity and the capture of intent and requirements over brevity and compactness. As you get comfortable with unit tests, you’ll start to give them titles that describe correct functionality of the system and you’ll use them as kind of a checklist for getting code right. It’s like your to-do list. If every box is checked, you feel pretty good. And you can put checkboxes next to statements like “Throws_Exception_When_Passed_A_Null_Value” but not next to “Test_Null”.

There are some very common things that new unit testers tend to do for a while before things click. Naming test methods things like “Test_01” and having dozens of asserts in them is very high on the list. This comes from heavily procedural thinking. You’ll need to break out of that to realize the benefit of unit testing, which is inherently modular because it requires a complete breakdown into components to work. If nothing else, remember that it’s, “Arrange, Act, Assert,” not, “Arrange, Act, Assert, Act, Assert, Assert, Assert, Act, Act, Assert, Assert, Act, Assert, etc.”

Wrap-Up

The gist of this installment is that unit tests can be used to explore a system, understand what it does, and then guard to make sure the system continues to work as expected even when you are others are in it making changes later. This helps prevent unexpected side effects during later modification (i.e. regressions). We’ve also covered that unit tests are generally small, simple and focused, following the “Arrange, Act, Assert” pattern. No one unit test is expected to cover much ground–that’s why you build a large suite of them.

Jun
10

By

Introduction to Unit Testing (Don’t Worry, Your Secret is Safe with Me)

Category: You Asked For It Tags: , | 8 Comments

Have you ever been introduced to someone and promptly forgotten their name? Or have you ever worked with or seen someone enough socially that you ought to know their name? They know yours, and you’re no longer in a position that it’s socially acceptable for you to ask them theirs. You’re stuck, so now you’re sentenced to an indefinite period of either avoiding calling them by name, avoiding them altogether, or awkwardly trying to get someone else to say their name. Through little to no fault of your own, you’re going to be punished socially for an indefinite period of time.

I think the feeling described strongly parallels the feeling that a developer has when the subject of unit testing comes up. Maybe it’s broached in an interview by the interviewer or interviewee. Perhaps it comes up when a new team member arrives or when you arrive as the new team member. Maybe it’s at a conference over drinks or at dinner. Someone asks what you do for unit testing, and you scramble to hide your dirty little secret–you don’t have the faintest idea how it works. I say that it’s a dirty little secret because, these days, it’s generally industry-settled case law that unit testing is what the big boys do, so you need either to be doing it or have a reason that you’re not. Unless you come up with an awesome lie.

The reasons for not doing it vary. “We’re trying to get into it.” “I used to do it, but it’s been a while.” “Oh, well, sometimes I do, but sometimes I don’t. It’s hard when you’re in the {insert industry here} industry.” “We have our own way of testing.” “We’re going to do that starting next month.” It’s kind of like when dentists ask if you’re flossing. But here’s the thing–like flossing, (almost) nobody really debates the merits of the practice. They just make excuses for not doing it.

I’m not here to be your dentist and scold you for not flossing. Instead, I’m here to be your buddy: the buddy with you at the party that knows you don’t know that guy’s name and is going to help you figure it out. I’m going to provide you with a survival guide to getting started with unit testing–the bare essentials. It is my hope that you’ll take this and get started following an excellent practice. But if you don’t, this will at least help you fake it a lot better at interviews, conferences, and other discussions.

(If you’re a grizzled unit-testing veteran this will be review for you, though you can feel free to read through and critique the finer points)

Let’s Get our Terminology Straight

One common pitfall for the unit-test-savvy faker is misuse of the term. Nothing is a faster giveaway that you’re faking it than saying that you unit test and proceeding to describe something you do that isn’t unit testing. This is akin to claiming to know the mystery person’s name and then getting it wrong. Those present may correct you or they may simply let you embarrass yourself thoroughly.

Unit testing is testing of units in your code base–specifically, the most granular units or the building blocks of your application. In theory, if you write massive, procedural constructs, the minimum unit of code could be a module. But in reality, the unit in question is a class or method. You can argue semantics about the nature of a unit, but this is what people who know what they’re talking about in the industry mean by unit test. It’s a test for a class and most likely a specific method on that class.

Here are some examples of things that are not unit tests and some good ways to spot fakers very quickly:

  1. Developer Testing. Compiling and running the application to see if it does what you want it to do is not unit testing. A lot of people, for some reason, refer to this as unit testing. You can now join the crowd of people laughing inwardly as they stridently get this wrong.
  2. Smoke/Quality Testing. Running an automated test that chugs through a laundry list of procedures involving large sections of your code base is not a unit test. This is the way the QA department does testing. Let them do their job and you stick to yours.
  3. Things that involve databases, files, web services, device drivers, or other things that are external to your code. These are called integration tests and they test a lot more than just your code. The fact that something is automated does not make it a unit test.

So what is a unit test? What actually happens? It’s so simple that you’ll think I’m kidding. And then you’ll think I’m an idiot. Really. It’s a mental leap to see the value of such an activity. Here it is:

[TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
public void Returns_False_For_1()
{
    var finder = new PrimeFinder();
    Assert.IsFalse(finder.IsPrime(1)); 
}

I instantiate a class that finds primes, and then I check to make sure that IsPrime(1) returns false, since 1 is not a prime number. That’s it. I don’t connect to any databases or write any files. There are no debug logs or anything like that. I don’t even iterate through all of the numbers, 1 through 100, asserting the correct thing for each of them. Two lines of code and one simple check. I am testing the smallest unit of code that I can find–a method on the class yields X output for Y input. This is a unit test.

I told you that you might think it’s obtuse. I’ll get to why it’s obtuse like a fox later. For now, let’s just understand what it is so that at our dinner party we’re at least not blurting out the wrong name, unprovoked.

What is the Purpose of Unit Testing?

Unit testing is testing, and testing a system is an activity to ensure quality. Right? Well, not so fast. Testing is, at its core, experimentation. We’re just so used to the hypothesis being that our code will work and the test confirming that to be the case (or proving that it isn’t, resulting in changes until it does) that we equate testing with its outcome–quality assurance. And we generally think of this occurring at the module or system level.

Testing at the unit level is about running experiments on classes and methods and capturing those experiments and their results via code. In this fashion, a sort of “state of the class” is constructed via the unit test suite for each unit-tested class. The test suite documents the behavior of the system at a very granular level and, in doing so, provides valuable feedback as to whether or not the class’s behavior is changing when changes are made to the system.

So unit testing is part of quality assurance, but it isn’t itself quality assurance, per se. Rather, it’s a way of documenting and locking in the behavior of the finest-grained units of your code–classes–in isolation. By understanding exactly how all of the smallest units of your code behave, it is straightforward to assemble them predictably into larger and larger components and thus construct a well designed system.

Some Basic Best Practices and Definitions

So now that you understand what unit testing is and why people do it, let’s look a little bit at some basic definitions and generally accepted practices surrounding unit tests. These are the sort of things you’d be expected to know if you claimed unit-testing experience.

  • Unit tests are just methods that are decorated with annotations (Java) or attributes (C#) to indicate to the unit test runner that they are unit tests. Each method is generally a test.
  • A unit test runner is simply a program that executes compiled test code and provides feedback on the results.
  • Tests will generally either pass or fail, but they might also be inconclusive or timeout, depending on the tooling that you use.
  • Unit tests (usually) have “Assert” statements in them. These statements are the backbone of unit tests and are the mechanism by which results are rendered. For instance, if there is some method in your unit test library Assert.IsTrue(bool x) and you pass it a true variable, the test will pass. A false variable will cause the test to fail.
  • The general anatomy of a unit test can be described using the mnemonic AAA, which stands for “Arrange, Act Assert.” What this means is that you start off the test by setting up the class instance in the state you want it (usually be instantiating it and then whatever else you want), executing your test, and then verifying that what happened is what you expected to happen.
  • There should be one assertion per test method the vast majority of the time, not multiple assertions. And there should (almost) always be an assertion.
  • Unit tests that do not assert will be considered passing, but they are also spurious since they don’t test anything. (There are exceptions to this, but that is an intermediate topic and in the early stages you’d be best served to pretend that there aren’t.)
  • Unit tests are simple and localized. They should not involve multi-threading, file I/O, database connections, web services, etc. If you’re writing these things, you’re not writing unit tests but rather integration tests.
  • Unit tests are fast. Your entire unit test suite should execute in seconds or quicker.
  • Unit tests run in isolation, and the order in which they’re run should not matter at all. If test one has to run before test two, you’re not writing unit tests.
  • Unit tests should not involve setting or depending on global application state such as public, static, stateful methods, singeltons, etc.

Lesson One Wrap-Up

I’m going to stop here and probably continue this as a series of posts. I’m going to be giving an “introduction to unit tests” talk, probably this week, so I’m doing double duty with blog posts and prepping for that talk. When finished, I’ll probably post the PowerPoint slides for anyone interested. Hopefully if you’ve been faking it all along with unit tests you can now at least fake it a little bit better by actually knowing what they are, how they work, why people write them, and how people use them. Perhaps together we can get you started adding some value by writing them, but at least for now this is a start.

Apr
17

By

Language Basics from Unit Tests

Category: Language Agnostic Tags: , |

Let’s say that in a green field code base someone puts together a type that conceptually is a collection of non-integer values. For the sake of discussion, let’s call it a graph. A graph object might store a series of two-element tuples or perhaps a series of some value type like “point.” The graph might then perform operations on this data, such as IncreaseX() or IncreaseY() or Invert() or Divide()–operations that iterate through the points and do things to them. The actual mechanics of this don’t matter a whole lot. It’s the concept that’s important.

Now let’s say that in the graph the internal representation of the points is a floating point data type such as, well, float. I’m going to save the nuance of floating point arithmetic for a future practical math post, but suffice it say that floats can exhibit some weird-seeming behavior when it comes to comparisons, truncation/rounding, certain kinds of casting and type representations, etc. 

[TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
public void Mind_Equals_Blown()
{
    float x = 0.2f;
    float y = 0.1f;
    float z = x + y;

    Assert.IsTrue(z == x + y);  //What the - why does this fail?!?
}

And let’s also say that the person responsible for authoring this graph class hasn’t read a practical math post about floating point arithmetic and is completely oblivious to these potential pitfalls.

And, finally, let’s say that this graph class becomes a mainstay of the business logic in a particular application. It’s modified, extended, and relied heavily upon without a whole lot of attention paid to its internal workings. At least until stuff mysteriously doesn’t work. But when that happens, the culprit isn’t immediately obvious, so strange work-arounds and cargo-cult, oddball solutions spring up when symptoms occur. Extension methods are written, and sometimes entirely different modules are added to the code base because the existing one is “tricky” or “not to be trusted.”

At the application level, this causes maintenance issues, a lot of heated and fruitless arguments, and voodoo approaches to code. From a user interface perspective, this causes quirky behavior. Occasionally a linear graph is completely displaced out of the graph and rendered on some menu somewhere, or the screen goes blank for a few seconds and then the display is restored. Defects and defect reports are created and developers dispatched to track down the issue, but after a few days of fruitless efforts, some project manager quietly sets the defect’s priority from “critical” to “cosmetic” and the software is shipped. It’s embarrassing, but whatcha gonna do. Ya know, computers have a mind of their own sometimes!

MessedUpGraph

Catching it Early

What if, instead of doing things the old-fashioned but all-too-common way, the authors of this code had been writing unit tests and/or practicing TDD? Well, there’s a very good chance that the issue stemming from the graph library is caught immediately as its API methods are being fleshed out from a functionality perspective. There’s a good chance that someone is writing a test and gets to the point that we were at in the code sample above, where they are utterly dumbfounded as to why 1+1 does not equal 2 in float land.

And then, good things happen. The developer in question takes to google or stack overflow, or perhaps he talks to other, more experienced developers on his team. He then gets an explanation, learns something about the language, and leaves the code in a correct state. Contrast this with the non-tested approach of “code it up, build a bad house on the bad foundation, and then ship the result because it’s too late.”

And what if the TDD/unit tests don’t expose this issue? Well, what they’ll do in either case is decouple the code base. So when the issue eventually does crop up via weird GUI behavior, it will be much easier to isolate. When it’s isolated, it will be much easier for the unit-test-savvy developers to write a test that exposes the defect to learn the lesson and fix the issue. It’s still a win.

The point about unit tests helping catch errors and leading to a more decoupled design is hardly controversial. But the benefits go beyond that. Unit tests provide a fast feedback loop for all points in the code base, which lends itself very well to poking and prodding things and experimenting. And that, in turn, leads to better understanding of not only the code, but also the language. If you can execute and get feedback on code extremely quickly, you’re much more likely to ask questions like, “I wonder what happens if I do x…” and then to do it and see. And that sort of experimentation, much like immersion in natural language, leads much more quickly to fluency.

Mar
11

By

Characterization Tests

Category: Language Agnostic Tags: , | 1 Comment

The Origins of Legacy Code

I’ve been reading the Michael Feathers book, Working Effectively with Legacy Code and enjoying it immensely. It’s pushing ten years old, but it stands the test of time quite well–probably much better than some of the systems it uses as examples. And there is a lot of wisdom to take from it.

When Michael describes “legacy code,” he isn’t using the definition as you’re probably accustomed to seeing. I’d hazard a guess that your definition would be something along the lines of “code written by departed developers” or maybe just “old, bad code.” But Michael defines legacy code as any code that isn’t covered by automated regression tests (read: unit tests). So it’s entirely possible and common for developers to be writing code that’s legacy code as soon as it’s checked in.

HouseOfCardsI like this definition a lot, and not, as some might suspect, out of any purism. I’m not equating “legacy” with “bad,” embracing the definition as a backhanded way to say that people who don’t develop the way that I do write bad code. The reason I like the “test-less” definition of “legacy code” is that it brings to the fore the largest association that I have with legacy code, which is fear of changing it.

Think about what runs through your head when you’re tasked with making changes to some densely-packed, crusty old system. It’s probably a sense of honest to goodness unease or demotivation as you realize the odds of getting things right are low and the odds of headaches and blame are high. The code is rat’s nest of dependencies and weird work-arounds, and you know that touching it will be painful.

Now consider another situation that’s different but with similar results. You have some assignment that you’ve worked on for weeks or months. It’s complicated, the customer isn’t sure what he wants, there have been lots of hiccups and setbacks, and there’s budget and deadline pressure. At the bitter end, after a few all-nighters, a bit of scope reduction, and some concessions, you somehow finally get all of the key features working for the most part. You check in the code for shipping, thinking, “I have no idea how this is working, but thank God it is, and I am never touching that again!” You’ve written code that was legacy code from the get-go.

Legacy code isn’t just the bad code that the team before you wrote, or some crusty old stuff from three language versions ago, or some internal homegrown VBA and Excel written by Steve, who’s actually an accountant. Legacy code is any code that you don’t want to touch because it’s fragile.

Getting Things Under Control

In his book, Michael Feathers lays out a lot of excellent strategies for taming out-of-control legacy code. I highly recommend giving it a read. But he coins a term and technique that I’d like to mention today. It’s something that I think programmers should be aware of because it helps lower the barriers to getting started with unit testing. And that term is “characterization tests.”

Characterization tests are the “I’m Okay, You’re Okay,” Rorschach approach to documenting code. There are no wrong answers–just documenting the way things exist. So if you have a method called AddTwoNumbers(int, int) and it returns 12 when you feed it 1 and 1, you don’t say “that’s wrong.” Instead you write a test that documents that return value and you move on, seeing and documenting what it does with other inputs.

Sound crazy? Well, it’s really not. It’s not crazy because things like this actually happen in real life. When code goes live, people work their processes around it, however much its behavior may be goofy or unintended. Once code is in the wild, “right” and “wrong” cease to matter, and the requirements as they existed some time in the past are forgotten. There only is “what is.” Sounds very zen, and perhaps it is.

One of the most common objections when it comes to unit testing is from developers that work on legacy systems where code is hard to test and no tests exist. They’ll say that they’d do things differently if starting from scratch (which usually turns out not to be true), but that there’s just no tackling it now. And this is a valid objection–it can be very hard to get anything under test. But characterization tests at least remove one barrier to testing, which is having extensive experience writing proper unit tests.

With characterization tests, it’s really easy. Just write a unit test that gets in the vicinity of what you want to document, finagle it until it doesn’t throw runtime exceptions, assert something–anything–and watch the test fail. When it fails, make note of what the expected and actual were, and just change the expected to the actual. The test will now pass, and you can move on. Change some method parameters or variables in other classes or even globals–whatever you have access to and can change without collapsing the system.

Through this poking, prodding, and documenting, you’ll start getting a rudimentary picture of what the system does. You’ll also start getting the hang of the characterization test approach (and perhaps unit testing for real as an added bonus). But most importantly, you’ll finally have the beginnings of an automated safety net. There’s no right and wrong per se, but you will start to be able to see when your changes to the system are making it behave differently in ways you didn’t expect. In legacy, different is dangerous, so it’s invaluable to have this notification system in place.

Characterization tests aren’t going to save the day, and they probably aren’t going to be especially easy to write. At times (global state, external dependencies, etc.) they may even be impossible. But if you can get some in place here and there, you can start taking the fear out of interacting with legacy code.

By the way, if you liked this post and you're new here, check out this page as a good place to start for more content that you might enjoy.
« Older Posts
Newer Posts »