Setting up Sonar analysis for C# projects

In an effort to better understand some of the problematic areas of the C# codebase I work on, I recently setup an instance of the Sonar code analysis platform.  Sonar is originally written for Java analysis and later added C# support.  This posting walks you through my experience attempting to setup, configure and run the analysis.

Sonar Overview:
So why did I even do this?  Once up and running, Sonar provides some useful metrics for pointing out hotspots in your code that may be making it more difficult to maintain and extend your functionality. Through the web interface, you can drill-down on any of the metrics to the module, class, and method level, including full source code.  Some of the metrics provided for each C# project include (screenshots are from the "nemo" demo site mentioned below, my own project, or other sources):

• General analysis 
• Uses several rule-based static analysis tools
• FxCop
• StyleCop
• Gendarme (part of the Mono project)
• Details and statistics, with drill-downs, on rule violations

• Cyclomatic Complexity
• By method, class, file

• Comments:
• Percentage of code commented
• Percentage of public APIs that are (un)documented
• Duplication:
• Percentage of code that is duplicated
• Counts by duplicated lines of code, blocks, files

• Unit tests:
• Coverage
• Success/Failure statistics
• Time to run

• Counts:
• Lines of code
• Count of files
• Count of classes
• Count of methods

In addition, it will track changes over time, so you can see where issues are increasing/decreasing in your code.

Sonar is open sourced under the LGPL and free to use, however some of the plugins used to perform the analysis are only commercially available, and in some cases come with steep licencing fees.  For this blog, I focus on only freely available (ie: no fees) aspects of the product. Each of the tools I used are also freely available (FxCop, StyleCop, Gendarme, Gallio, PartCover, MySQL -- all have licenses that allow no-fee usage for most people).

There is a demo site provided by one of the commericial plugin providers to demo the system (including their not-free SQALE plugin) which shows analysis for several opensource Java projects and can give you a feel for the UI and the data that can be provided.  Beware -- this includes data from some of the commercial plugins, so don't expect to see everything on that site after following this posting.

General Sonar Technical Architecture:
Generally, the Sonar runner consumes the source code and analyzes it via plugins.  This information is published to the sonar database directly.

The Sonar server pulls the results from the database and provides a UI for reviewing them.

Step 1: Prerequisites and Assumptions:
You'll need a 32-bit machine to run your analysis on.  There are issues with the 64-bit versions of Gallio and PartCover which caused errors.  If you must run on 64-bit, there are sites with workarounds to getting this running, but I didn't go that route because I needed a valid 64-bit build environment for my day-to-day tasks.

In my case, I spun up a Windows 7 Virtual PC and installed everything on there, including my TeamCity build server.

This posting assumes you have a working Debug build of your codebase that compiles with no errors and generates .pdb files. You need access to the source code and the output, as they exists at the completion of your build.  In my case, I have a CI server (TeamCity -- free for small/mid-scale configurations) running and just added another build step at the end to kick-off the Sonar analysis from the command line.

You'll need to install the .Net Reference Assemblies, which are part of the Win7 SDK.  If you're using .Net 4, it seems the Reference Assemblies may only be available by installing Visual Studio 2010.  Note that the FxCop 10 installer is included (exclusively) with the Win7 SDK.

If you have any Silverlight components in your build, you will need to install the Silverlight SDK for the version of Silverlight you are targeting.


Step 2: The Sonar Server
Download the Sonar server. At the time of this posting, it was version 2.11.

This will give you a .zip file.  Decompress it into the location you want; I used C:/sonar-2.11

You can find the official installation notes here: http://docs.codehaus.org/display/SONAR/Install+Sonar

For the initial configuration/setup, I'd suggest running it at the command line until you know you have it fully configured.  To do that, open a command window and run bin\windows-x86-32\StartSonar.bat This will run in your current window (no real output, but the command blocks) until you hit Cntl-Z or Cntl-Break.

Again, I had trouble with the 64-bit version even running (no console output -- just died), so I'd stay with the 32-bit side.

Once you get things working the way you want, I'd suggest installing it as a Windows service.  To do this, first register Sonar as a service using the provided script: bin\windows-x86-32\InstallNTService.bat (There's an UnInstallNTService.bat script too).  Then, start the service from the Windows Services control panel, or using the provided "StartNTService.bat" script.  Personally, I would suggest just configuring it to be Automattic so it will service reboots, etc.

The server has a sonar.properties config file that drives some of it's functionality, including which database to use. You can find it in the \conf folder.  The only change I made to this file was for the database (see Step 2), but if you want to use the built-in Derby database, no changes are needed.

Step 2a: Verify and Change Password
At this point, you should be able to access the sonar server at http://localhost:9000 (or whatever port you set in the sonar.properties file). I would highly recommend changing the admin password at this time.

Step 3: The Database
Sonar server comes with a build-in Derby database.  While it is quick and easy (and pre-configured), I would suggest moving away from it almost immediately.  I had MAJOR performance problems using it.  For instance, for my solution of about 400K lines of code (50+ Visual Studio projects), when using the Derby database, it took about 2 hours to perform the analysis, 5 hours if the runner was on a differnt host. During that time, the server CPU would be pegged at 100% for 5 - 15 minutes per project as the runner reported it's results.  Once I moved to a MySQL database, it takes about 20 minutes.

So, I would suggest downloading and installing the free MySQL Communit Edition from Oracle.

Once installed, you'll need to create the sonar database instance.  This is pretty straightforward, but Sonar provides a script to make it even that much easier.  From the sonar installation folder, the script is in extras/database/mysql/create_database.sql. This will generate the sonar database instance, as well as grant access from localhost and remotely.  The sonar schema is created the first time you run the server.

You'll need to configure the Sonar server to use this db.  This is as easy as changing a couple of lines in the sonar.properties config file. Chances are, the lines you want are already in the file (commented out), as samples are there for MySQL, PostgreSQL, Oracle and MS SqlServer. (Note: MS SqlServer is not officially supported, although people on the web have posted ways to get it working. I did not venture down this path).

Update: While getting Sonar up and running in our production environment, the person doing the work was able to get up and running with MS SqlServer and provided me with this feedback. Note, he was using v.2.13.1, which is newer than what I was using:

I followed the link you had on your page (http://www.sezok.de/sonar/sonar.html) but differed in the following ways:

• The database must be setup with Case Sensitive and Accent Sensitive rules, so I created the database with collate option SQL_Latin1_General_CP1_CS_AS

• The jdbc drivers that come bundled with 2.13.1 actually work, and the ones listed in the above link don’t.

For me, I commented out the Derby lines and used these for my MySQL instance:

#----- MySQL 5.x/6.x
# Comment the embedded database and uncomment the following properties to use MySQL.
# The validation query is optional.
sonar.jdbc.url: jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8
sonar.jdbc.driverClassName: com.mysql.jdbc.Driver
#sonar.jdbc.validationQuery: select 1

Restart the sonar server, if it's running.

I should note that there isn't a quick-and-easy way to migrate from the Derby db to another, so expect any data collected in Derby will be lost when you move.  I found this page where someone found a way, if you really must try.


Step 3a: Verify and Change password
At this point, the sonar server should be up and running, albeit with no data.  I would suggest verifying you can reach the server at http://localhost:9000 (or whatever port you used in the sonar.properties file).

I would highly recommend changing the admin password at this time!

Step 4: The Runner
Sonar was build for Java, so the docs almost all assume you're using Maven.  You may be able to get this working with nAnt, but there's an easier route.  A small, java-based runner has been created that can be kicked-off from the command line. This is what I used from within my TeamCity builds.

To use this runner, you will need to do a few things:
1) Download the runner. This will come as a Zip file. Decompress it in an appropriate location. For me, I put it in C:/Sonar-2.11/runner

2) Install the Java JDK.  Yes, the *JDK*. It's big and ugly, I know. You're a .Net dev, I know. Do it anyway.

3) Set the JAVA_HOME environment variable to point to the JDK root folder.  Make sure to set this environment var where the runner will get it.  Typically, I would say set it as a System ENV var, but in my case, there is a bug in the current TeamCity build agent that does not properly pass system env vars when running as a remote agent, so while running in that config, I had to use User vars and run the agent as my user.  If you're running this all on the same machine, you should be fine with a System Environment Var.

4) Test that it works.  Open a command line to the sonar-runner/bin folder and run sonar-runner -h.  This will either show you the usage statement (if things are working fine), or blow up.  If it blows up, check you're JAVA_HOME is set correctly.

5) Once it works, set another environment var: SONAR_RUNNER_HOME to the root installation folder for sonar-runner. This is used when you want to run the sonar runner from a folder other than the /bin folder. (ie: when you run sonar-runner from your source folder).

6) Change the conf/sonar-runner.properties file. This has general configuration items used by the runner, some of which can be overridden in the project's config file (we'll get to that later).  At a minimum, you'll need to set the sonar.host.url to point to where you have your sonar server running, and the database config.  For the db, just copy/paste the lines from the sonar.properties file you created in Step 2. Again, example values are provided in the file (commented out) which will likely work just fine. Don't forget to comment out the Derby lines.

Step 5: The C# Plugins
You'll need to install the "C# Plugins Ecosystem" (ie: The plugins to analyze C# code).  This will be a zip file with each of the C# plugins .jar files. Decompress them and place them into the /extentions/plugins folder of your Sonar installation folder.  You can enable/disable each one individually, so no harm in putting them all in there at once.

The plugins consist of the following:

• C# Core (support for the C# language - general API used by other plugins -- required)
• C# Squid (support for parsing C# language -- required)
• FxCop support (general analysis)
• StyleCop support (formatting analysis)
• Gendarme support (general analysis)
• Gallio support (unit testing / coverage)

If you're going to use FxCop, you'll need to install FxCop 10 (the FxCop installer is installed as part of the Win7 SDK install).

If you're going to use StyleCop, you can either use a bundled version included with Sonar, or install you're own.

If you're going to use Gendarme, I've found that the version bundled with Sonar has a bug in it that caused it to error if you referenced System.Drawing from the .Net 4.0 library.  I had to install the newest version (2.11 snapshot).

If you're going to use Gallio, you'll need to install it as well as PartCover (or you can use NCover, if you have a license).

Step 6: The sonar-project.properties File
Each Solution will need to have it's own sonar-project.properties file.  This file will need to exist in the folder from which you execute the sonar-runner.  To make this easy, I would suggest putting the file in the same folder as your .sln file.

The file will have a few sections, which I will describe here. Note that some of these, I believe, can be put in the sonar-runner.properties file if you have the same settings for each project.

Important Note: Any folder names in the config file will need to either escape the backslash with another backslash (\\) or use a forward slash (/). I've chosen the latter.

Project Identification:
This section will provide the project key used by the Sonar server to group analysis results over time, as well as provide a useful name in the UI, etc.  This should be unique across projects.  The project version can be used to track different branches, etc.

# Project identification
sonar.projectKey=Jwright:DemoApp
sonar.projectVersion=trunk
sonar.projectName=DemoApplication

Then, describe the source code layout. The "sources" field points to the top-level folder where the source code exists.  If you're .sln and .csproj files have relative paths internally, then this should be the top-level folder.  Assuming you don't have any strange layouts, this will likely be the same folder as your .sln file (which is likely where your .properties file exists), so can just be ".".  Additionally, you need to denote that the language is C# using the "cs" value.

# Info required for Sonar
sources=.
sonar.language=cs  

C#-specific settings:
Here, you'll need to provide information about where the .sln file is located and where key libraries are located, and which version of .Net you're using.

#Core C# Settings
sonar.dotnet.visualstudio.solution.file=DemoApp.sln
sonar.silverlight.4.mscorlib.location=C:/Program Files/Reference Assemblies/Microsoft/Framework/Silverlight/v4.0
sonar.dotnet.excludeGeneratedCode=true
sonar.dotnet.4.0.sdk.directory=C:/WIndows/Microsoft.NET/Framework/v4.0.30319
sonar.dotnet.version=4.0 

Plug-in Specific Sections:
For each plugin, there is a "mode" setting.  If blank, then the plugin will run. If you want to skip/not run a plugin, set the mode to "skip".

For Gallio, you can stipulate if you want to use PartCover (free) or NCover (not free). You can also stipulate the runner mode.  I had trouble using anything other than "Local".  You will also need to stipulate the naming pattern (regular expressions, I believe) for the Visual Studio projects that include unit tests. You can have multiple patterns, seperated by semicolons.


Very Important: Note that there is a bug/typo in the Gallio plugin where the key for the Test Project Pattern does not have a "t" in "sonar.donet.visualstudio.testProjectPattern".  You'll also need to provide the installation directory for PartCover.

#Gendarme
sonar.gendarme.mode=  

# Gallio
sonar.gallio.mode=
sonar.gallio.coverage.tool=PartCover
sonar.gallio.runner=Local
sonar.donet.visualstudio.testProjectPattern=*UnitTest*;Testing*
sonar.partcover.installDirectory=C:/Program Files/PartCover/PartCover .NET 4.0/ 

# FXCop
sonar.fxcop.mode=   

#StyleCop
sonar.stylecop.mode=

Step 7: Running an Analysis
From the folder with the sonar-project.properties file, run the command
$SONAR_RUNNER_HOME/bin/sonar-runner

You'll see the runner start up, listing some details like the working folder, etc, then it will kick off the source code parsing, then the plugins (such as running the unit tests).  If there are any errors, a Java exception will be thrown. Sometimes these contain enough details to troubleshoot, sometimes not, so you may need to run with the -X command line argument to get additional details when errors occur.

After a successful run of the analysis, you can see the results in the sonar server webpage. Unless you have changed the port in the sonar.properties file, this will be at http://localhost:9000/

The homepage will have a link for each project (the value provided as sonar.projectName), which will take you to the project overview.  Clicking on any of the links on the overview page will take you to the drill-down data for that metric.

References:

I used this page to start my journey, in addition to the Sonar project docs: http://www.ifunky.net/Blog/post/Install-and-Configure-Sonar-on-Windows-2008.aspx

The C# Plugins page: http://docs.codehaus.org/display/SONAR/C-Sharp+Plugins+Ecosystem

The Sonar installation page: http://docs.codehaus.org/display/SONAR/Install+Sonar

Update:
The Sonar Source page, complete with mailing lists, issue trackers, etc. http://www.sonarsource.org/support/support 

Hope this helps!