DaedTech

Stories about Software

Jun
19

By

Introduction to Unit Testing Part 3: Unit Testing Sucks

Category: Language Agnostic Tags: , | 18 Comments

I don’t know about you, but I remember desperately wanting to be able to drive right up until I was fifteen years old and I got my learner’s permit. I thought about it a lot–how fun it would be, how much freedom I would have, how my trusty old bike would probably get rusty from disuse. About a month after getting my permit, I desperately wanted my license and to drive on my own without supervision. But I’m omitting a month there, during which an unexpected thing happened. I realized that driving was stupid and awful and it sucked and I hated it and I’d never do it, so just forget it!

It was in that month that the abstraction of operating a car and having freedom became the reality of hitting the gas when I meant to hit the brake pulling out of my driveway or not knowing when I was supposed to go after stopping at a stop sign. It was a weird mix of frustration, anger, and fear that tends to accompany new activities–even ones that you know will benefit you. And that’s why the title of this post isn’t simple link bait. I did that not to satirize a position, but to empathize. Like many things when you’re new to them, starting to unit testing quite frankly sucks. It’s frustrating, foreign, and hard to get right. Accordingly, it’s easy to abandon it when you have deadlines to meet.

Accident

This post is about minimizing frustration and barriers to adoption by staying focused and setting reasonable expectations. I would argue that if you’re new to writing tests, writing a few and enjoying localized success without high coverage is a lot more important than suddenly becoming a TDD (Test-Driven Development) expert with 100% test coverage right out of the gate (or at least trying to become one). Incremental progress is good.

Don’t Try TDD Just Yet

I’m a little torn as I write this, but the first thing that I’ll suggest is that you not try TDD if you have no experience unit testing. Some might disagree with this suggestion, but I think that you’re going to be trying to learn too many new things all at once and will be a lot more likely to get frustrated. Unit tests are simply pieces of code that you write, as covered in more detail in the last post in the series. It’s a new kind of code to be writing, but you’re just learning about new methods to call and attributes (or annotations, in Java) to use. You’ll get there.

But TDD is an entirely new way of writing code. It’s a discipline in which you do not write any production code until you have written a unit test that fails. Then you get that test and all other tests to pass and refactor the code as needed. Does that sound crazy (if you discount the fact that a number of developers you respect probably do it)? Exactly. Probably not for you right now. It’s a bridge too far, and you’re more likely to throw up your arms in disgust and quit if you try to learn both things right now. I speak from experience, as, years ago, I was introduced to unit testing and TDD at the same time. I was overwhelmed until I just went back to figuring out the whole unit testing thing alone first. Maybe that wouldn’t happen to you, but I’d caution you to be wary of learning these two things simultaneously.

So let’s stick to learning what unit tests are and how to write them.

Test New Classes Only

In my pluralsight course, I use the example of a method that identifies numbers as prime or not, and in a series of posts I did last fall on TDD, I use the example of something that calculates a bowling score. I’ve also done other code katas and exercises like these in the past to show people both the mechanics of unit testing and TDD.

When I do this, one of the things people frequently say is something along the lines of “pff…sure, when you’re writing something stupid and easy like a prime number finder, but there’s no way that would work on our code base.” I then surprise these people by agreeing with them. I’m sure it wouldn’t work on your code base. Why? Well, because unit tests don’t just magically spring up like mushrooms after a few days of rain. They’re more like roses–you have to plan for them from the start and carefully cultivate an environment in which they can thrive.

Some years back, I saw an excellent talk on “The Deep Synergy Between Testability and Good Design,” by Michael Feathers. I highly suggest watching this talk if you haven’t seen it, but to summarize, he states (and I agree) that well-designed and factored code goes hand in hand with testability. You’re much more likely to find that code written to be testable is good code and, conversely, code written without unit tests in mind is not the greatest. And so if you’re deeply invested in a code base that has never been covered by unit tests, it doesn’t surprise me to hear that you don’t think unit testing would work on your code. I imagine it wouldn’t.

But don’t throw out unit testing because it looks like it wouldn’t work in your code base. Just resolve to do it on new classes that you create. As you go along and get better at unit testing, you’ll start to understand how to write testable classes. It will thus get easier and easier to test all new additions to the code, and you’ll start to get the hang of it with relatively minimal impact on your existing code, your process, or your time. Starting to unit test doesn’t mean that you’re suddenly responsible for testing every line of code in history, nor does it mean you must test every single new line. Just start out by writing a few that you think will help.

Test Existing Code by Extracting Little Classes

Once you get the feel for adding unit tests for new classes/code that you add to the code base, it’s a good time to start taking baby steps toward getting tests in place for your legacy (non-tested) code. Now, some procedural, monolithic mass of code that wasn’t testable a month ago when you started out isn’t magically testable now because you have some practice. It’s still a problem.

You’re going to have to chip away at it. And you’re going to have to do this by developing a new skill: identifying pieces of functionality that you can pull out into new classes and test. Go look through methods and classes and find things that don’t have a lot of dependencies on class fields or (yuck) global/static variables. Excellent candidates for this are methods with pure in-memory operations and ones that deal largely with primitives. Do you have some gigantic method that has a whole region buried in it that does nothing but cobble together a string to be used later in the method? Pull that out into a new class, and write unit tests that make assertions about the string it returns.

As you practice this, you’ll get a better and better feel for what you can pull out with a minimum of friction. You’ll find yourself not only getting more of your codebase under test, but also that you’re improving its design and modularity.

Know When to Fold ‘Em

This is another one that’s hard to type, but you really have to learn to look at code and just say, “nope, not happening.” There are classes and methods that you simply are not going to be able to test unless you come back with a green belt in unit testing–or pair with someone who has hers. And, even then, the prognosis may be that you need to rewrite the legacy class/method altogether to make it testable. Here is a quick list of things that, early in your unit-testing career, you should consider to be deal-breakers and simply move on from to avoid frustration. As a beginner, avoid testing code (class methods and properties) that:

  1. Calls static methods. At best, a static method is functional and returns something that depends only on its inputs. If this is the case (such at functions like Math.Pow() or Math.Abs()), the code is still testable, but a far more common case, especially if the static methods are ones in your own code base, is that they manipulate some kind of global state. Global state is testability kryptonite. I’ll explain more later, but for now, please take my word for it.
  2. Invokes singletons. The singleton design pattern is used almost universally as a politically correct way to hide your global variables in plain sight. For what this means to testability, see the last bullet. If it calls singletons, forget it, move on.
  3. Dispatches background workers or manages threading. When unit tests are run, the unit test runner is responsible for managing threading and it will run your tests in parallel. If you’re trying to make sure your threads and thread management are in one state for production and another for testing, you are about to ruin your day and probably your week. It’s not worth it–don’t try.
  4. Accesses files, connects to databases, calls web services, etc. I mentioned this in the first post in the series, but that was in the context of saying that these things aren’t considered unit tests. Well, another issue here is that they’re also relatively brittle and long running. If you write tests that do these things, they’re going to fail at weird times and in unpredictable ways. You’ll be used to all of your tests passing and suddenly one fails and then passes again, and it turns out it’s because Bill from accounting bumped into the database server and its Nic card is a little “tricky.” If you have unit tests that fail for borderline-inconceivable reasons beyond your control, you will become discouraged.
  5. Code that triggers any of the above anywhere in the call stack. You don’t escape the problems of threading, global state, or externalities by not using them directly. If you trigger them, it’s the same difference.
  6. Classes that require crazy amounts of instantiation. If you want to test a method, but it has forty-five parameters, most of which are classes that are difficult or complex to create, forget it. That code sorely needs reworking, and creating massive, brittle tests for it this early in your career will be a world of pain. Chip away at making the design better before you tackle it.

Don’t worry–I’m not suggesting that you give up on a long timeline, and I’ll continue on with this series and discuss strategy for addressing these things later. But for now, just consider them signals that this code is out of bounds for testing. If you don’t, there’s a high likelihood that you’ll spin your wheels and get angry, frustrated, and irritable, making it more likely that you’ll give up. I can’t eliminate the frustration of being new at something like driving, but I can at least steer you away from six-way traffic lights and three-lane roundabouts.

Jun
16

By

Don’t Write Code You Don’t Need

Category: Language Agnostic Tags: , | 31 Comments

I was reviewing some code the other day, and I saw a quick-and-dirty logger implementation that looked something like this (I’m re-creating from memory and modifying a bit for illustrative purposes in this post):

public class Logger
{
    public string Path { get; set; }

    public Logger()
    {

    }

    public Logger(string path)
    {
        Path = path;
    }

    public void WriteEntry(string entry)
    {
        using (var writer = File.AppendText(Path))
        {
            writer.WriteLine(entry);
        }
    }
}

A few things probably jump out at you, depending on your C#, OOP, and code reviewing chops. I’d imagine that, at the very least, you think, “you could just delete the default constructor.” You might also wonder why the other constructor exists since you could just set Path whenever you wanted. There’s the lack of null/valid check on path before creating a stream writer with it, and the unnecessary, potentially problematic, and definitely not threadsafe creation of the stream writer over and over. These things are all valid to point out, but I’d say that they’re also all symptoms of two larger root causes that I want to talk about.

Overeager to Please

TooMuchCode

You can offer too much code. By this I mean something subtly but critically different from “you can write too much code,” as when you are needlessly complicated or verbose. Offering too much code means that you’re giving users of the public interface of your classes too many options. Before my TDD days, this was something with which I constantly struggled. To understand what I mean, consider the thinking that probably went into the creation of this class:

Well, let’s see. A file needs a path, so a file logger should probably also take a path as input. From there, it should handle the details of streams and all that other stuff so that all we have to do is give it stuff to put in the file.

So far, so good. This is pretty clever as far as abstractions goes, and whoever wrote this class has a good grasp on how to create abstractions that provide value. But here’s what probably came next.

So, for path, let’s make that a public property in case the user of the class wants to change the path later. For convenience, let’s add a constructor that lets the user specify the path, but let’s also let him know that he doesn’t have to.

That thinking is considerate and helpful, but it’s offering too much code. You need to take ownership of your abstractions and be a little forceful and unyielding with them. You provide what you provide, and if users don’t like your class, they can create an inheritor or write their own or something: “this is my class, take it or leave it.”

In this case, you have to make concrete decisions about how and when the path of the logger is set:

In this class, the path is set at instantiation time and only instantiation time. As long as you live with my logger, you will obey my rules! 

public class Logger
{
    public string Path { get; private set; }

    public Logger(string path)
    {
        Path = path;
    }

    public void WriteEntry(string entry)
    {
        using (var writer = File.AppendText(Path))
        {
            writer.WriteLine(entry);
        }
    }
}

Notice that we’ve already eliminated the problem of the pointless constructor by making this decision. We’re also well poised to handle the null/valid checking of the path since we can just do it in the constructor instead of doing it in the constructor and in the setter for path and worrying about duplication, when exactly to validate and how, what to do on invalid set when the last path was valid, etc. It’s also going to be very easy to take care of the issue with the needless creation of StreamWriter instances. Now that you can’t change the path once the instance is created, you can simply create the writer in the constructor and reuse it for the lifetime of the object by storing it as a private field. Now making this threadsafe becomes a pretty manageable task. In general, eliminating one conceptual option eliminates a lot of internal implementation complexity.

But what about the users who want to set the path at some point later in the object instance’s lifetime? Psst…that’s not a real common use case. And you know what? If they really want to do that, they can just instantiate a second logger. Don’t make your life really complicated because you think users of your code might want flexibility. Because you know what they want more than extra flexibility? Code that works. And it’s a lot harder to offer them code that works if you’re bending over backwards to handle every conceivable thing they might want to do.

Pointless or Speculative Mutability

The last section touches on this obliquely, but I’d like to bring this to the fore. Mutable, state-based code is a lot more prone to problems than immutable or functional code that has fixed or no state, respectively. Consider the refactoring path I’ve proposed here. In the last section, I chose to eliminate the public setter for Path instead of the constructor that took Path as an argument. Did that seem like a coin flip and pick one to you? Well, it wasn’t.

With the constructor injection of path, we have to do validity checking and initialization only once, to establish preconditions for the object’s existence. With the public setter for path, we have to maintain object invariants, which tends to be more complex. A good example of this is how we handle the transition from a valid path to the user setting an invalid or null path. Do we throw an exception? Revert to the previous path? You don’t have a discussion when there was no valid previous path as is the case with constructor injection.

This is just an example of a broader idea, which is that mutability creates state transition management tasks and that these tasks are harder to get right. To understand what I mean, imagine that you’re implementing the API for an array. You tell your users, “alright, when you create the array, you specify how many elements it will have, and then you can set and access the elements as you please by index.” The users come back and say, “well, we want to change the size of the array on the fly,” to which you respond, “oh, crap,” as you start to wonder what you’ll do if they want to resize it smaller than what they’ve used or if they try to make it negative or too big or something. You can feel the number of edge cases multiplying.

If you come from a background where you write a lot of procedural, script-based, or hacked-together utility code, this may seem like a weird thing to think about. You’re probably used to doing things like declaring boolean ‘flags’ somewhere in a method and setting them much later in that same method in order to keep track of something further down the line. And if you do this, you’re probably used to a lot of tweaking, guessing, and hoping for the best. But as it turns out, you’re doing things the hard way.

The easy way is to program without any state at all. That means no assignment and no variables, the way you might do if asked to write a method that took x and y and returned 4x + 2y. There’s no need to do any of that: just have “return 4x + 2y” and be done with it. This way of doing things is called functional programming, and its calling card is that output is always a pure function of the input and is entirely predictable.

If functional paradigm isn’t an option (and it often isn’t in the entirety of an application), the next easiest thing to deal with is immutable objects. These are objects that have state, but that state is not modifiable after creation time. An example of an object that can easily be immutable is something like an address. An address is just a handful of strings bound together with a class definition, so why have it be mutable? If you want a different one, just let the one you have go out of scope and create a new one. Then you don’t have to worry about questions like “what if I set a new address1, but not a new address2–that would never happen, right?” In immutable land, that’s simply not a consideration that you bother with.

The hardest way of doing things is with state transition management or mutable state. That’s the nature of the original logger at the top. Anything goes. The object has only transitive state, so it has either to constantly manage all permutations of state changes to check for accuracy or it has to risk being wrong in an invalid state. This is not a great choice and should be avoided if at all possible.

As you develop, you’ll find it’s possible a lot more often than you think. Objects that perform a service, such as a logger, generally have little reason to store mutable state, particularly in a nicely decoupled application. Even a lot of data objects, which represent the very core of what we’d think of as mutable, can be immutable a lot more often than you’d think (e.g. address). Mutability in your code is often best left in places where it’s unavoidable. If you don’t have the luxury of pass-through interaction with a database, you will generally maintain an in-memory domain model that needs to have mutable state because it’s representing the database which is, almost by definition, a whole gigantic system of files full of mutable state. A lot of objects that help you manage user interface will have mutable state (e.g. some class that stores things like “is checkbox X currently checked”). But unavoidable as it may be, you can certainly minimize and isolate it. Whatever you do, don’t introduce it where you don’t need it because you’re creating needless complexity, which means extra things that can go wrong.

Take-Away

The upshot of this exhaustive code-review that I’ve conducted is just advice to consider carefully where your classes fall on the functional-immutable-mutable spectrum and to keep them as far toward the functional side as possible. I’m not suggesting that you run out and rewrite existing code this minute or even that you vow to change how you do things. I’m just suggesting you be aware that mutable objects come with a heavy cost in terms of complexity and difficulty. This will help you in your programming travels in general, and especially if you’re looking to delve into things like architecture, test-driven development, or concurrent (multi-threaded) programming.

Jun
14

By

Introduction to Unit Testing Part 2: Let’s Write a Test

Category: .NET Tags: , , , | 5 Comments

In the last post in this series, I covered the essentials of unit testing without diving into the topic in any real detail. The idea was to offer sort of a guerrilla guide to what unit testing is for those who don’t know but feel they should. Continuing on that path and generating material for my upcoming presentation, I’m going to continue with the introduction.

In the previous post, I talked about what unit testing is and isn’t, what its actual purpose is, and what some best practices are. This is like explaining the Grand Canyon to someone that isn’t familiar with it. You now know enough to say that it’s a big hole in the earth that provides some spectacular views and that you can hike down into it, seeing incredible shifts in flora and fauna as you go. You can probably convince someone you’ve been there in a casual conversation, even without having seen it. But, you won’t really get it until you’re standing there, speechless. With unit testing, you won’t really get it until you do it and get some benefit out of it.

So, Let’s Write that Test

Let’s say that we have a class called PrimeFinder. (Anyone who has watched my Pluralsight course will probably recognize that I’m recycling the example class from there.) This class’s job is to determine whether or not numbers are prime, and it looks like this:

public class PrimeFinder
{
    public bool IsPrime(int possiblePrime)
    {
        return possiblePrime != 1 && !Enumerable.Range(2, (int)Math.Sqrt(possiblePrime) - 1).Any(i => possiblePrime % i == 0);
    }       
}

Wow, that’s pretty dense looking code. If we take the method at face value, it should tell us whether a number is prime or not. Do we believe the method? Do we have any way of knowing that it works reliably, apart from running an entire application, finding the part that uses it, and poking at it to see if anything blows up? Probably not, if this is your code and you needed my last post to understand what a unit test was. But this is a pretty simple method in a pretty simple class. Doesn’t it seem like there ought to be a way to make sure it works?

I know what you’re thinking. You have a scratchpad and you copy and paste code into it when you want to experiment and see how things work. Fine and good, but that’s a throw-away effort that means nothing. It might even mislead when your production code starts changing. And checking might not be possible if you have a lot of dependencies that come along for the ride.

But never fear. We can write a unit test. Now, you aren’t going to write a unit test just anywhere. In Visual Studio, what you want to do is create a unit test project and have it refer to your code. So if the PrimeFinder class is in a project called Daedtech.Production, you would create a new unit test project called DaedTech.Production.Test and add a project reference to Daedtech.Production. (In Java, the convention isn’t quite as cut and dry, but I’m going to stick with .NET since that’s my audience for my talk). You want to keep your tests out of your production code so that you can deploy without also deploying a bunch of unit test code.

Once the test class is in place, you write something like this, keeping in mind the “Arrange, Act, Assert” paradigm described in my previous post:

[TestMethod]
public void Returns_False_For_One()
{
    var primeFinder = new PrimeFinder(); //Arrange

    bool result = primeFinder.IsPrime(1); //Act

    Assert.IsFalse(result); //Assert
}

The TestMethod attribute is something that I described in my last post. This tells the test runner that the method should be executed as a unit test. The rest of the method is pretty straightforward. The arranging is just declaring an instance of the class under test (commonly abbreviated CUT). Sometimes this will be multiple statements if your CUTs are more complex and require state manipulation prior to what you’re testing. The acting is where we test to see what the method returns when we pass it a value of 1. The asserting is the Assert.IsFalse() line where we instruct the unit test runner that a value of false for result means the test should pass, but true means that it should fail since 1 is not prime.

Now, we can run this unit test and see what happens. If it passes, that means that it’s working correctly, at least for the case of 1. Maybe once we’re convinced of that, we can write a series of unit tests for a smattering of other cases in order to convince ourselves that this code works. And here’s the best part: when you’re done exploring the code with your unit tests to see what it does and convince yourself that it works (or perhaps you find a bug during your testing and fix the code), you can check the unit tests into source control and run them whenever you want to make sure the class is still working.

Why would you do that? Well, might be that you or someone else later starts playing around with the implementation of IsPrime(). Maybe you want to make it faster. Maybe you realize it doesn’t handle negative numbers properly and aim to correct it. Maybe you realize that method is written in a way that’s clear as mud and you want to refactor toward readability. Whatever the case may be, you now have a safety net. No matter what happens, 1 will never be prime, so the unit test above will be good for as long as your production code is around–and longer. With this test, you’ve not only verified that your production code works now; you’ve also set the stage for making sure it works later.

Resist the Urge to Write Kitchen Sink Tests

kitchensink

When I talked about a smattering of tests, I bet you had an idea. I bet your idea was this:

[TestMethod]
public void Test_A_Bunch_Of_Primes()
{
    var primes = new List() { 2, 3, 5, 7, 11, 13, 17 };
    var primeFinder = new PrimeFinder();


    foreach(var prime in primes)
        Assert.IsTrue(primeFinder.IsPrime(prime));
}

After all, it’s wasteful and inefficient to write a method for each case that you want to test when you could write a loop and iterate more succinctly. It’ll run faster, and it’s more concise from a coding perspective. It has every advantage that you’ve learned about in your programming career. This must be good. Right?

Well, not so much, counterintuitive as that may seem. In the first place, when you’re running a bunch of unit tests, you’re generally going to see their result in a unit test runner grid that looks something like a spreadsheet. Or perhaps you’ll see it in a report. If when you’re looking at that, you see a failure next to “IsPrime_Returns_False_For_12” then you immediately know, at a glance, that something went wrong for the case of 12. If, instead, you see a failure for “Test_A_Bunch_Of_Primes”, you have no idea what happened without further investigation. Another problem with the looping approach is that you’re masking potential failures. In the method above, what information do you get if the method is wrong for both 2 and 17? Well, you just know that it failed for something. So you step through in the debugger, see that it failed for 2, fix that, and move on. But then you wind up right back there because there were actually two failures, though only one was being reported.

Unit test code is different from regular code in that you’re valuing clarity and the capture of intent and requirements over brevity and compactness. As you get comfortable with unit tests, you’ll start to give them titles that describe correct functionality of the system and you’ll use them as kind of a checklist for getting code right. It’s like your to-do list. If every box is checked, you feel pretty good. And you can put checkboxes next to statements like “Throws_Exception_When_Passed_A_Null_Value” but not next to “Test_Null”.

There are some very common things that new unit testers tend to do for a while before things click. Naming test methods things like “Test_01” and having dozens of asserts in them is very high on the list. This comes from heavily procedural thinking. You’ll need to break out of that to realize the benefit of unit testing, which is inherently modular because it requires a complete breakdown into components to work. If nothing else, remember that it’s, “Arrange, Act, Assert,” not, “Arrange, Act, Assert, Act, Assert, Assert, Assert, Act, Act, Assert, Assert, Act, Assert, etc.”

Wrap-Up

The gist of this installment is that unit tests can be used to explore a system, understand what it does, and then guard to make sure the system continues to work as expected even when you are others are in it making changes later. This helps prevent unexpected side effects during later modification (i.e. regressions). We’ve also covered that unit tests are generally small, simple and focused, following the “Arrange, Act, Assert” pattern. No one unit test is expected to cover much ground–that’s why you build a large suite of them.

Jun
12

By

It’s a Work in Progress

Category: The Life of a Programmer Tags: |

I’ll have that for you next week. Oh, you want it to do that other thing that we talked about earlier where it hides those controls depending on what the user selects? I’ll have it for you next month. Oh, and you also want the new skinning and the performance improvements too? I’ll get started and we’ll set about six months out as the target date. Unless you want to migrate over to using Postgres. Then make it a year.

Have you ever had any conversations that went like this? Ever been the person making this promise or at least the one that would ultimately be responsible for delivering it? If so, I bet you felt like, “man, I’m totally just making things up, but I guess we’ll worry about that later.” Have you ever been on the hearing end of this kind of thing? I bet you felt like, “man, he’s totally just making things up, and none of this will ever actually happen.”

As you’ll know if you’ve been checking in at this blog for a while, my opinions about the “waterfall methodology” are no secret. It isn’t my intention to harp on that here but rather to point out that the so-called waterfall approach to software development is simply part of what I consider to be a larger fail: namely, putting a lot of distance between promises and deliveries.

I get the desire to do this–I really do. It’s the same reason that we root for underdogs in sports or we want the nice guy not to finish last. Having success is good, but sudden, unexpected success is awesome. It creates a narrative that is far more compelling than “guy who always delivers satisfactory results does it yet again.” Instead, it says, “I just pulled this thing called an iPod out of nowhere and now I’m going to be the subject of posthumous books and cheesy project manager motivational calendars in a decade.”

Companies have done things like this for a long, long time–particularly companies that sell tangible goods. It’s been the story of a consumer-based world where you innovate, patent, and then sell temporarily monopolized widgets, making a fortune by giving the world something new and bold. You probably wind up on the cover of various magazines with feel-good stories about the guy behind the revolutionary thing no one saw coming, except for one visionary (or a team of them) toiling away in anonymity and secrecy, ready to pull back the curtain and dazzle us with The Prestige.

Magician

But here’s the thing. The world is getting increasingly service oriented. You don’t really buy software anymore because now you subscribe and download it from “the cloud” or something. Increasingly, you rent tools and other things that perhaps you would previously have purchased. Even hardware goods and electronics have become relatively disposable, and there is the expectation of constant re-selling. Even the much-ballyhooed innovator, Apple, hasn’t really done anything interesting for a while. We’re in a world where value is delivered constantly and on a slight incline, rather than in sudden, massive eruptions and subsequent periods of resting on laurels.

What does the shifting global economic model have to do with the “waterfall” method of getting things done? Well, it pokes a hole in the pipe-dream of “we’ll go off and bury ourselves in a bunker for six months and then emerge with the software that does everything you want and also makes cold fusion a reality.” You won’t actually do that because no one really does anymore, and even back when people did, they failed a lot more often than they succeeded in this approach.

I love to hear people say, “it’s a work in progress.” That means that work is getting done and those doing it aren’t satisfied, and those two components are essential to the service model. These words mean that the speaker is constantly adding value. But more than that, it means that the speaker is setting small, attainable goals and that those listening can expect improvements soon and with a high degree of confidence. The person doing the work and the target audience are not going to get it perfect this week, next week, next month, or maybe ever. But they will get it better and better at each of those intervals, making everyone involved more and more satisfied. And having satisfaction steadily improve sure beats having no satisfaction at all for a long time and then a very large chance of anger and lawsuits. Make your work a work in progress, and make people’s stock in your labor a blue chipper rather than a junk bond.

Jun
10

By

Introduction to Unit Testing (Don’t Worry, Your Secret is Safe with Me)

Category: You Asked For It Tags: , | 8 Comments

Have you ever been introduced to someone and promptly forgotten their name? Or have you ever worked with or seen someone enough socially that you ought to know their name? They know yours, and you’re no longer in a position that it’s socially acceptable for you to ask them theirs. You’re stuck, so now you’re sentenced to an indefinite period of either avoiding calling them by name, avoiding them altogether, or awkwardly trying to get someone else to say their name. Through little to no fault of your own, you’re going to be punished socially for an indefinite period of time.

I think the feeling described strongly parallels the feeling that a developer has when the subject of unit testing comes up. Maybe it’s broached in an interview by the interviewer or interviewee. Perhaps it comes up when a new team member arrives or when you arrive as the new team member. Maybe it’s at a conference over drinks or at dinner. Someone asks what you do for unit testing, and you scramble to hide your dirty little secret–you don’t have the faintest idea how it works. I say that it’s a dirty little secret because, these days, it’s generally industry-settled case law that unit testing is what the big boys do, so you need either to be doing it or have a reason that you’re not. Unless you come up with an awesome lie.

The reasons for not doing it vary. “We’re trying to get into it.” “I used to do it, but it’s been a while.” “Oh, well, sometimes I do, but sometimes I don’t. It’s hard when you’re in the {insert industry here} industry.” “We have our own way of testing.” “We’re going to do that starting next month.” It’s kind of like when dentists ask if you’re flossing. But here’s the thing–like flossing, (almost) nobody really debates the merits of the practice. They just make excuses for not doing it.

I’m not here to be your dentist and scold you for not flossing. Instead, I’m here to be your buddy: the buddy with you at the party that knows you don’t know that guy’s name and is going to help you figure it out. I’m going to provide you with a survival guide to getting started with unit testing–the bare essentials. It is my hope that you’ll take this and get started following an excellent practice. But if you don’t, this will at least help you fake it a lot better at interviews, conferences, and other discussions.

(If you’re a grizzled unit-testing veteran this will be review for you, though you can feel free to read through and critique the finer points)

Let’s Get our Terminology Straight

One common pitfall for the unit-test-savvy faker is misuse of the term. Nothing is a faster giveaway that you’re faking it than saying that you unit test and proceeding to describe something you do that isn’t unit testing. This is akin to claiming to know the mystery person’s name and then getting it wrong. Those present may correct you or they may simply let you embarrass yourself thoroughly.

Unit testing is testing of units in your code base–specifically, the most granular units or the building blocks of your application. In theory, if you write massive, procedural constructs, the minimum unit of code could be a module. But in reality, the unit in question is a class or method. You can argue semantics about the nature of a unit, but this is what people who know what they’re talking about in the industry mean by unit test. It’s a test for a class and most likely a specific method on that class.

Here are some examples of things that are not unit tests and some good ways to spot fakers very quickly:

  1. Developer Testing. Compiling and running the application to see if it does what you want it to do is not unit testing. A lot of people, for some reason, refer to this as unit testing. You can now join the crowd of people laughing inwardly as they stridently get this wrong.
  2. Smoke/Quality Testing. Running an automated test that chugs through a laundry list of procedures involving large sections of your code base is not a unit test. This is the way the QA department does testing. Let them do their job and you stick to yours.
  3. Things that involve databases, files, web services, device drivers, or other things that are external to your code. These are called integration tests and they test a lot more than just your code. The fact that something is automated does not make it a unit test.

So what is a unit test? What actually happens? It’s so simple that you’ll think I’m kidding. And then you’ll think I’m an idiot. Really. It’s a mental leap to see the value of such an activity. Here it is:

[TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
public void Returns_False_For_1()
{
    var finder = new PrimeFinder();
    Assert.IsFalse(finder.IsPrime(1)); 
}

I instantiate a class that finds primes, and then I check to make sure that IsPrime(1) returns false, since 1 is not a prime number. That’s it. I don’t connect to any databases or write any files. There are no debug logs or anything like that. I don’t even iterate through all of the numbers, 1 through 100, asserting the correct thing for each of them. Two lines of code and one simple check. I am testing the smallest unit of code that I can find–a method on the class yields X output for Y input. This is a unit test.

I told you that you might think it’s obtuse. I’ll get to why it’s obtuse like a fox later. For now, let’s just understand what it is so that at our dinner party we’re at least not blurting out the wrong name, unprovoked.

What is the Purpose of Unit Testing?

Unit testing is testing, and testing a system is an activity to ensure quality. Right? Well, not so fast. Testing is, at its core, experimentation. We’re just so used to the hypothesis being that our code will work and the test confirming that to be the case (or proving that it isn’t, resulting in changes until it does) that we equate testing with its outcome–quality assurance. And we generally think of this occurring at the module or system level.

Testing at the unit level is about running experiments on classes and methods and capturing those experiments and their results via code. In this fashion, a sort of “state of the class” is constructed via the unit test suite for each unit-tested class. The test suite documents the behavior of the system at a very granular level and, in doing so, provides valuable feedback as to whether or not the class’s behavior is changing when changes are made to the system.

So unit testing is part of quality assurance, but it isn’t itself quality assurance, per se. Rather, it’s a way of documenting and locking in the behavior of the finest-grained units of your code–classes–in isolation. By understanding exactly how all of the smallest units of your code behave, it is straightforward to assemble them predictably into larger and larger components and thus construct a well designed system.

Some Basic Best Practices and Definitions

So now that you understand what unit testing is and why people do it, let’s look a little bit at some basic definitions and generally accepted practices surrounding unit tests. These are the sort of things you’d be expected to know if you claimed unit-testing experience.

  • Unit tests are just methods that are decorated with annotations (Java) or attributes (C#) to indicate to the unit test runner that they are unit tests. Each method is generally a test.
  • A unit test runner is simply a program that executes compiled test code and provides feedback on the results.
  • Tests will generally either pass or fail, but they might also be inconclusive or timeout, depending on the tooling that you use.
  • Unit tests (usually) have “Assert” statements in them. These statements are the backbone of unit tests and are the mechanism by which results are rendered. For instance, if there is some method in your unit test library Assert.IsTrue(bool x) and you pass it a true variable, the test will pass. A false variable will cause the test to fail.
  • The general anatomy of a unit test can be described using the mnemonic AAA, which stands for “Arrange, Act Assert.” What this means is that you start off the test by setting up the class instance in the state you want it (usually be instantiating it and then whatever else you want), executing your test, and then verifying that what happened is what you expected to happen.
  • There should be one assertion per test method the vast majority of the time, not multiple assertions. And there should (almost) always be an assertion.
  • Unit tests that do not assert will be considered passing, but they are also spurious since they don’t test anything. (There are exceptions to this, but that is an intermediate topic and in the early stages you’d be best served to pretend that there aren’t.)
  • Unit tests are simple and localized. They should not involve multi-threading, file I/O, database connections, web services, etc. If you’re writing these things, you’re not writing unit tests but rather integration tests.
  • Unit tests are fast. Your entire unit test suite should execute in seconds or quicker.
  • Unit tests run in isolation, and the order in which they’re run should not matter at all. If test one has to run before test two, you’re not writing unit tests.
  • Unit tests should not involve setting or depending on global application state such as public, static, stateful methods, singeltons, etc.

Lesson One Wrap-Up

I’m going to stop here and probably continue this as a series of posts. I’m going to be giving an “introduction to unit tests” talk, probably this week, so I’m doing double duty with blog posts and prepping for that talk. When finished, I’ll probably post the PowerPoint slides for anyone interested. Hopefully if you’ve been faking it all along with unit tests you can now at least fake it a little bit better by actually knowing what they are, how they work, why people write them, and how people use them. Perhaps together we can get you started adding some value by writing them, but at least for now this is a start.

« Older Posts
Newer Posts »