MS Test Report Generator

I’ve been working on a side project for a while, and today I uploaded it to Source Forge. The project is a tool that takes XML results generated by MS Test runs and turns them into HTML-based reports. I believe that TFS will do this for you if you have fully Microsoft-integrated everything, but, in the environment I’m currently working in, I don’t.

So I created this guy.

Back Story

I was working on a build process with a number of constraints that predated my tenure on the process itself. My task was to integrate unit tests into the build and have the build fail if the unit tests were not all in a passing state. The unit test harness was MS Test and the build tool was Final Builder.

During the course of this process, some unit tests had been in a failing state for some period of time. Many of these were written by people making a good faith effort but nibbling at an unfortunate amount of global state and singletons. These tests fail erratically and some of the developers that wrote them fell out of the habit of running the tests, presumably due to frustration. I created a scheme where the build would only run tests decorated with the “TestType” attribute “Proven”. In this fashion, people could write unit tests, ‘promote’ them to be part of the build, and have it as an opt-in situation. My reasoning here is that I didn’t want to deter people who were on the fence about testing by having them be responsible for failing the build because they didn’t know exactly what they were doing.

After poking around some, I saw that there was no native final builder action that accomplished what I wanted– to execute a certain subset of tests and display a report. So I created my own batch scripts (not included in the project) that would execute the MS Test command line executable with filtering parameters. This produces an XML based output. I scan that output for the run result and, if it isn’t equal to “Passed”, I fail the build. From there, I generate a report using my custom utility so that people can see stats about the tests.

Design

During the course of my spare time, I decided to play around with some architectural concepts and new goodies added to C# in the .NET 4.0 release. Specifically, I added some default parameters and experimented with co-variance and contra-variance. In terms of architecture, I experimented with adding the IRepository pattern to an existing tiered architecture.

On the whole, the design is extensible, flexible, and highly modular. It covered about 99.7% by unit tests and, consequently, was complete overkill for a little command line utility designed to generating an HTML report. However, I was thinking bigger. Early on, I decided I wanted to put this on Source Forge. The reason I designed it the way I did was to allow for expansion of the utility into a GUI-driven application that can jack into a server database and maintain aggregate unit testing statistics on a project. Over the course of time, you can track and get reports on things like test passing rate, test addition rate, which developers write tests, etc. For that, the architecture of the application is very well suited.

The various testing domain objects are read into memory, and the XML test file and HTML output file are treated as just another kind of persistence. So in order to adapt the application in its current incarnation to, say, write the run results to a MySQL or SQL server database, it would only be necessary to add a few classes and modify main to persist the results.

Whether I actually do this or not is still up in there air, and may depend upon how much, if any, interest it generates on Source Forge. If people would find it useful, both in general and specifically on projects I work on, I’m almost certain to do it. If no one cares, I have a lot of projects that I work on.

How to Access

You can download a zipped MSI installer from SourceForge.

If you want to look at the source code, you can do a checkout/export on the following SVN address: https://mstestreportgen.svn.sourceforge.net/svnroot/mstestreportgen

That will provide read-only access to the source.