DaedTech

Stories about Software

By

Rethinking Assert with Shouldly

I was doing a bit of work with Tweetdeck open, when I noticed this tweet.

I’ve been using Assert.IsTrue() and its friends for years, so you might think I would take offense.  But instead, this struck me as an interesting and provocative statement.  I scanned through the conversation this started and it got me to thinking.

Over the years, I’ve evolved my unit tests heavily in the name of readability.  I’ve come to favor mocking frameworks on the basis of having fluent APIs and readable setup.  On a pointer from Steve Smith, I’ve adopted his philosophy and approach to naming unit test classes and tests.  My arrange and act inside of the tests have become highly readable and optimized for comprehension.

But then, there’s Assert.AreEqual.  Same as it ever was.

WorkHarder

Rethinking Assert

I tend to walk a fine line between “if it ain’t broke, don’t fix it” and continuous improvement.  I juggle a ton of client work, so I have to pick my spots for improvement and adopting new techniques.  Jeremy’s tweet and my subsequent contemplation made me realize that just such an opportunity was staring me in the face, here.

Classic stuff.  Any grizzled unit testing veteran will tell you that this is how you check equality.  But convention drives this — not comprehension.

Imagine reading this as prose.  “Assert are equal expect and actual.”  One can decipher it, but the cadence gives pause.  And a newbie will struggle with knowing whether actual or expected goes first, learning this only by rote memorization.

I remember first writing asserts this way in the early 2000s.  The rest of the unit test has evolved, so I concluded I’d try evolving the assert along with it.

Introducing Shouldly

The conversation mentioned two potential tools: Shouldly and Fluent Assertions.  At a quick glance, I found the semantics of Shouldly more appealing, so I started poking around there.  You install it simply by adding a Nuget package, and with minimal effort you can set about prettying your assertions immediately.

I’ll offer a quick example of what Shouldly does.  Below is a unit test taken from some code I’ve written to help with the codebase assessments that I do.  This unit test exercises an implementation that calculates a “testability” index of target code, though specifics here do not matter terribly.

Fairly straightforward.  Arrange it so that the modeled code under test indicates a 25 line constructor, perform the testability calculation, and assert that the result is equal to 20.

Here’s the same test with Shouldly.

I have more than a decade of experience that tells me the first snippet contains the ‘correct’ format for an equality assertion.  And yet, in spite of all of that ingrained memory, the second test immediately strikes me as more readable.  I have to imagine that will only become more true as time passes.

But There’s More

Let’s make this test fail and see what happens.  To do this, I will temporarily change my expectation to 0 and produce a failure.  One of the big drawbacks, historically, to writing my own assertion wrappers, was the way it was easy to mangle or obfuscate the test feedback.  But Shouldly handle this neatly.

ShouldlyFailMessage

Notice that what it was and what is should be figure prominently in the test failure message.  But also note that Shouldly aligns these vertically for comparison.  That doesn’t matter much for a couple of doubles, but think of those times where you compare two gigantic strings and they fail equality because a character or two in the middle don’t align.  Having them aligned vertically comes in pretty handy.

I don’t normally use the Visual Studio test runner much, so here’s my view of it using NCrunch.

ShouldlyWithNCrunch

Is This For Me?

Shouldly grants you readability and some smartness in your test failure feedback.  But, like any tool, it has its tradeoff considerations.  I am, by no means, an expert, but here are a couple that I noticed in my cursory research.

  1. Shouldly requires a bit of extra consideration on a build machine.  As explained here, part of the magic is using PDB files to help with its error messages, so things get weird if you haven’t set those up.
  2. Shouldly, like anything else, involves taking an extra dependency in your codebase.  As dependencies go, one for non-production code presents relatively little risk, but this is something you should always consider.

Beyond that, I’d say you have to figure out with your team whether it feels right or not.  That’s certainly what I’m going to spend some time doing myself.

But whether Shouldly works for you or not, I feel that the broader lessons here are important.  Finding ways to avoid complacency and continuously improve are important.  And the traditional assert patterns leave a great deal of room for improvement.  Put both of those considerations together to make sure you’re writing readable code.

  Subscribe  
newest oldest most voted
Notify of
Chris Surfleet
Guest
Chris Surfleet

I’ve been using FluentAssertions for quite a while now. I keep finding more cool stuff in there. I don’t know if Shouldy has the same sort of exception handling assertions but if so check them out 🙂

Erik Dietrich
Guest

I had for years written (and later published to Nuget) my own exception handling assertions. I was pleased to see that, not only does Shouldly have them, but they are nearly identical, semantically, to what I had done. I felt pretty vindicated in my approach 🙂

dave falkner
Guest
dave falkner

It’s subtle and easy to miss, but Shouldly’s Should.Throw(Action a) returns the exception instance itself, upon which you can perform further assertions of its properties. The resulting assertion reads kind of funny, though.

Joseph N. Musser II
Guest
Joseph N. Musser II

Just to chime in, I’ve fallen in love with NUnit’s Assert.That(actual, Is.EqualTo(expected)) or Assert.That(collection, Is.Not.Empty).

I was forced to use NUnit because I needed dynamically parameterized tests, and when I moved I was shocked by how much I had been missing out on.

Erik Dietrich
Guest

Cool! Enough years have passed since I’ve used NUnit that I had no idea this was available. Sounds like I’ve missed some promising innovations since I last visited.

Kerry Patrick
Guest
Kerry Patrick

This is an unrelated question, but what do you use the test category of “Proven” for? I think I remember this from your ChessTDD series, too.

Erik Dietrich
Guest

That’s sort of vestigial. Many years ago, when I created the templates for my unit tests in CodeRush, I was introducing a team to unit tests as part of the build. Since a lot of these folks were testing newbies, I didn’t want to discourage them from writing tests, but didn’t want the build to fail because of ill-constructed unit tests. So the “proven” flag was there to indicate a code reviewed/blessed unit tests. The build would fail only on failing tests that were “proven.” I should probably take that out, but have never bothered, since it harms nothing.

Paulo Morgado
Guest

Using extension methods on a test framework is all nice until the API being developed introduces a method with the same signature.

NUnit’s approach seems to be more full proof.

Erik Dietrich
Guest

Have you seen these collisions happen before?

Paulo Morgado
Guest

Not in this specific case. But it doesn’t mean that it’s not a poorly designed API, given the possibility of it happening.

Extension methods must be used carefully. If you’re extending System.Object, then something is wrong or it is a very controlled environment, which unit tests aren’t.

That must be one of the reasons MSTest never went for it.

Joseph N. Musser II
Guest
Joseph N. Musser II

I feel the same about hanging extension methods off Object and I prefer NUnit, but practically speaking I can’t see this ever being an issue. We already have ways of dealing with extension method conflicts should we find ourselves in a world where Shouldly did conflict with another library.