InStyle -- StyleCop and FxCop

After my post on CruiseControl.NET and Sandcastle, one of my many (okay, 29) Twitter followers asked "Do you look code quality or test coverage as part of your automated builds?" Test coverage is another post, but I'll talk a bit about code quality.

The old-skool way to bring up code quality is the dreaded code review. Dreaded because you have to print out the code, read it, and sit around a big conference table. And we don't have good donuts here at Scimatic. I think the big conference room table is passé when Guido writes Rietveld.

The new-skool way to bring up code quality is pair-programming. Never tried it, but would like to. It seems to work well for Obie and his team at Hashrocket, but then again they have those fly 30in monitors ...

Anyway, back on target ... In the build process, is there any way to proxy something for code quality? It turns out Microsoft gives us two options, StyleCop and FxCop.

StyleCop started out as an internal tool in Microsoft to ensure that all developers were coding to Microsoft's standards. And given that there are 90,000 88,600 85,000 people working at Microsoft, it's important that all their code be coded in the same way.

However, there are a few problems. First of all, StyleCop is not mandated through out Microsoft, and some teams there don't use it. So some of the code that you see coming out of Microsoft won't pass StyleCop's rules. Worse, some code automatically generated by Visual Studio does not pass StyleCop's rules (I'm looking at you, AssemblyInfo.cs). And typically the files that don't pass are the ones that you aren't supposed to edit by hand (they're handled through various Visual Studio wizards and forms).

Second issue is that you cannot selectively turn on/off StyleCop's rules within a piece of source code. For example, in the syntactically similar Perl::Critic module for checking perl best practice code quality, you can put a "## no critic" pragma to turn off the style checker for that part of the code

## no critic
# evil code here, for which you have a great reason to skip Perl::Critic
## use critic
# back onside

You cannot do this in StyleCop; you can only use a rule everywhere, or turn the rule off completely. This is a bad thing, as once you find a situation where you need to bend the rules (for example, comment formats on enums), you end up dropping the entire commenting rule.

So. bottom line, we're not using StyleCop right now. Too much pain.

On the other hand, we are using FxCop.

FxCop is not a source code analyzer. It is byte-code analyzer that looks at the output of the build. From the byte code, it can look for design, performance, localization and security issues. And because it's looking at the binary, it's really looking at the thing you care about, the bits that are going to the customer. Its rules are targeted toward being compliant Design Guidelines for Class library developers.

So, for our build process, we use a FxCop project file that tests the release build of our assemblies. We call FxCop from the command line in CruiseControl and use the existing xsl files to merge the reports into the build log. For our day to day development, we mostly build in debug mode. For that, we add a Post-build event in our project.

IF '$(ConfigurationName)'=='Debug'
  set FxCopConfigurationName=$(ConfigurationName) &&  "%ProgramFiles%\Microsoft FxCop 1.36\FxCopCmd.exe" /p:"$(SolutionPath).fxcop" /console

This only runs in the debug mode, so that it does not run when we run the build through devenv in CruiseControl, we do not run FxCop; it's run separately after unit testing. [Aside: yes, we will move to MSBuild in the future.]

This hack requires setting the project names in the .fxcop file to be bin/%FxCopConfigurationName%/<assembly name>; and to define %FxCopConfigurationName% = Release on our development boxes, so that if we run the FxCop GUI, our assemblies will load. It's not ideal.

However, we've already corrected five issues relating to speed and internationalization, so I think this is a win for us.