Rethinking Assert with Shouldly

I was doing a bit of work with Tweetdeck open, when I noticed this tweet.

Dear .Net world, please stop writing barely readable unit tests with the old fashioned Assert.IsTrue()/AreEqual()/etc. syntax

— jeremydmiller (@jeremydmiller) August 7, 2016

I’ve been using Assert.IsTrue() and its friends for years, so you might think I would take offense.  But instead, this struck me as an interesting and provocative statement.  I scanned through the conversation this started and it got me to thinking.

Over the years, I’ve evolved my unit tests heavily in the name of readability.  I’ve come to favor mocking frameworks on the basis of having fluent APIs and readable setup.  On a pointer from Steve Smith, I’ve adopted his philosophy and approach to naming unit test classes and tests.  My arrange and act inside of the tests have become highly readable and optimized for comprehension.

But then, there’s Assert.AreEqual.  Same as it ever was.

Rethinking Assert

I tend to walk a fine line between “if it ain’t broke, don’t fix it” and continuous improvement.  I juggle a ton of client work, so I have to pick my spots for improvement and adopting new techniques.  Jeremy’s tweet and my subsequent contemplation made me realize that just such an opportunity was staring me in the face, here.

Assert.AreEqual(expected, actual);

Classic stuff.  Any grizzled unit testing veteran will tell you that this is how you check equality.  But convention drives this — not comprehension.

Imagine reading this as prose.  “Assert are equal expect and actual.”  One can decipher it, but the cadence gives pause.  And a newbie will struggle with knowing whether actual or expected goes first, learning this only by rote memorization.

I remember first writing asserts this way in the early 2000s.  The rest of the unit test has evolved, so I concluded I’d try evolving the assert along with it.

Introducing Shouldly

The conversation mentioned two potential tools: Shouldly and Fluent Assertions.  At a quick glance, I found the semantics of Shouldly more appealing, so I started poking around there.  You install it simply by adding a Nuget package, and with minimal effort you can set about prettying your assertions immediately.

I’ll offer a quick example of what Shouldly does.  Below is a unit test taken from some code I’ve written to help with the codebase assessments that I do.  This unit test exercises an implementation that calculates a “testability” index of target code, though specifics here do not matter terribly.

[TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
public void Return_20_When_There_Are_25_Lines_Of_Code_In_Constructor()
{
    Constructor.Arrange(xtor => xtor.NbLinesOfCode).Returns(25);

    double calculationResult = Target.Calculate(TargetType);

    Assert.AreEqual<double>(20, calculationResult);
}

Fairly straightforward.  Arrange it so that the modeled code under test indicates a 25 line constructor, perform the testability calculation, and assert that the result is equal to 20.

Here’s the same test with Shouldly.

[TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
public void Return_20_When_There_Are_25_Lines_Of_Code_In_Constructor()
{
    Constructor.Arrange(xtor => xtor.NbLinesOfCode).Returns(25);

    double calculationResult = Target.Calculate(TargetType);

    calculationResult.ShouldBe(20);
}

I have more than a decade of experience that tells me the first snippet contains the ‘correct’ format for an equality assertion.  And yet, in spite of all of that ingrained memory, the second test immediately strikes me as more readable.  I have to imagine that will only become more true as time passes.

But There’s More

Let’s make this test fail and see what happens.  To do this, I will temporarily change my expectation to 0 and produce a failure.  One of the big drawbacks, historically, to writing my own assertion wrappers, was the way it was easy to mangle or obfuscate the test feedback.  But Shouldly handle this neatly.

Notice that what it was and what is should be figure prominently in the test failure message.  But also note that Shouldly aligns these vertically for comparison.  That doesn’t matter much for a couple of doubles, but think of those times where you compare two gigantic strings and they fail equality because a character or two in the middle don’t align.  Having them aligned vertically comes in pretty handy.

I don’t normally use the Visual Studio test runner much, so here’s my view of it using NCrunch.

Is This For Me?

Shouldly grants you readability and some smartness in your test failure feedback.  But, like any tool, it has its tradeoff considerations.  I am, by no means, an expert, but here are a couple that I noticed in my cursory research.

1. Shouldly requires a bit of extra consideration on a build machine.  As explained here, part of the magic is using PDB files to help with its error messages, so things get weird if you haven’t set those up.
2. Shouldly, like anything else, involves taking an extra dependency in your codebase.  As dependencies go, one for non-production code presents relatively little risk, but this is something you should always consider.

Beyond that, I’d say you have to figure out with your team whether it feels right or not.  That’s certainly what I’m going to spend some time doing myself.

But whether Shouldly works for you or not, I feel that the broader lessons here are important.  Finding ways to avoid complacency and continuously improve are important.  And the traditional assert patterns leave a great deal of room for improvement.  Put both of those considerations together to make sure you’re writing readable code.