Sunday, August 10, 2014

How to use MsTest and NUnit Initialisation Methods

How to use the available initialisation methods in both MSTest and NUnit can be a little confusing when learning how to write effective tests. Effective tests are designed, just like production code, shared code should be in one place (DRY) and initialisation should be separate so to make tests less brittle.

Imagine if every test constructs an object like so:

var subject = new Foo();

Also imagine that the Foo class has decent number of methods to test.  One day a developer adds a parameter to the Foo constructor.  Now many tests are broken and require the same change. If the initialisation code is in one place this would have been a fast fix:- these tests are less brittle.

It would be handy to run some code before each test, some only once per class, and sometimes you need to initialise something before you run any tests.

First up its important to note some semantic behavour of all xUnit frameworks.
Every test will run in its own instance of the test class.  No two tests share the same instance.  When a test is finished running the instance is available for garbage collection. This means that any instance fields or properties are not shared between the tests in a class.  This is by design and cannot be configured. This is mostly to isolate one test from another, but another reason for this is to support running tests in parallel on multiple threads, although MsTest and NUnit do not run tests in parallel by default, although they both can.

MsTest

In MsTest here's the low-down on the available initialisation methods:

  • TestInitialize
  • ClassInitalize
  • AssemblyInitialize
  • TestCleanUp
These attributes are only recognised inside a class decorated with [TestClass].

TestIntialize

[TestInitialize]
public void TestInitialise() 
{
    // Your initialisation code here
}

This method will run before every test. This is normally the best place to set up test data, mocks, and instantiate the subject under test.  Sometimes, if possible, execution of the subject method can be done here as well so that each test only contains an assert. This has very similar to class's constructor, and both can be used, but its considered best practice to use the TestInitialise and not use constructors on Test Classes.

ClassInitialize


[ClassInitialize]
public static void ClassInitialise(TestContext context)
{
    // Your initialisation code here
}


This method will run once per class only, its very similar to a Type-Constructor. This can be used to initialise any static fields and properties in the test class. The TestContext argument primarily allows output to the MsTest console log.

AssemblyInitialize


[TestClass]
public class Global
{
    [AssemblyInitialize]
    public static void AssemblyInitialise(TestContext context)
    {
        // Your initialisation code here
    }
}

This method will run only once per Unit Test Assembly per test run.  Only one AssemblyInitialize method can exist per unit test assembly (or an exception is thrown when a test run starts). It is considered best practice to put this method into an easy to find Global class with no other tests. The TestContext argument primarily allows output to the MsTest console log.  I personally find this handy for registering AutoMapper configuration before running any tests involving mappers.

TestCleanUp

[TestCleanup]
public void TestCleanUp()
{
    // Your clean up code here
}

This is useful when you need to dispose any instance fields or properties, or reset anything back to a known state. Use with caution though, use if this method could be a "smell" that your tests are not isolated from dependencies like database and third-party services.
Along with TestCleanUp you can also use the ClassCleanUp attribute.  I'm not a fan of this, I've never had a good reason to use it.  To me, it indicates use of a static dependency and that doesn't sit well with my style.

Here's how the Sequence unfolds

Lets use a test to show the sequence of method calls.


    [TestClass]
    public class Global
    {
        [AssemblyInitialize]
        public static void AssemblyInitialise(TestContext context)
        {
            Debug.WriteLine("Assembly Initialise");
        }
    }

    [TestClass]
    public class SemanticTest
    {
        private int counter;
        private Guid id;

        public SemanticTest()
        {
            this.id = Guid.NewGuid();
        }

        [ClassInitialize]
        public static void ClassInitialise(TestContext context)
        {
            Debug.WriteLine("Class Initialise");
        }

        [TestMethod]
        public void Test1()
        {
            Debug.Write("Test1 ");
            this.counter++;
            Debug.WriteLine("Counter: {0} Id:{1}", this.counter, this.id);
        }

        [TestMethod]
        public void Test2()
        {
            Debug.Write("Test2 ");
            this.counter++;
            Debug.WriteLine("Counter: {0} Id:{1}", this.counter, this.id);
        }

        [TestMethod]
        public void Test3()
        {
            Debug.Write("Test3 ");
            this.counter++;
            Debug.WriteLine("Counter: {0} Id:{1}", this.counter, this.id);
        }

        [TestMethod]
        public void Test4()
        {
            Debug.Write("Test4 ");
            this.counter++;
            Debug.WriteLine("Counter: {0} Id:{1}", this.counter, this.id);
        }

        [TestMethod]
        public void Test5()
        {
            Debug.Write("Test5 ");
            this.counter++;
            Debug.WriteLine("Counter: {0} Id:{1}", this.counter, this.id);
        }

        [TestCleanup]
        public void TestCleanUp()
        {
            Debug.WriteLine("TestCleanUp " + this.counter);
        }

        [TestInitialize]
        public void TestIninitalise()
        {
            Debug.WriteLine("Test Initialise");
        }
    }

The output is:
Assembly Initialise
Class Initialise
Test Initialise
Test1 Counter: 1 Id:fb05338c-b64e-4f30-bdbb-22f4c35749b1
TestCleanUp 1
Test Initialise
Test2 Counter: 1 Id:c207ef4b-83c1-4909-828b-10a7a43ef653
TestCleanUp 1
Test Initialise
Test3 Counter: 1 Id:07e5ddd0-6f3d-4e32-bd2d-2a9544e8bd51
TestCleanUp 1
Test Initialise
Test4 Counter: 1 Id:73e703a9-9c6b-4b02-8cb5-433c8c64167e
TestCleanUp 1
Test Initialise
Test5 Counter: 1 Id:37124412-e593-4ba2-8d28-6421d32117c8
TestCleanUp 1

Here its plain to see all tests ran in their own instance of the SemanticTest class.

NUnit

All the above behaves identically in NUnit. Here are the equivalent attributes in NUnit as compared to MsTest.
  • SetUp == TestIntialize
  • TestFixtureSetUp == ClassInitialize
  • SetUpFixture == AssemblyInitialize
    Although in NUnit, SetUpFixture is scoped to a namespace not an entire assembly, which allows multiple SetUpFixture per assembly, but only one per namespace.
  • TearDown == TestCleanUp



See Also

Friday, January 17, 2014

Using Performance Counters in .NET


Why use performance counters?

I have good logging you say, excellent traceability in my database, if need be I can attach Ants Profiler, I don't need performance counters.  Not true. Every application that runs on a server and serves multiple users should be load tested.  A few definitions first:
Load testing is not using the Performance Analyser in Visual Studio or Ants Profiler.  This is merely looking for memory leaks and bad performance for a single user.  Still good to do, but doesn't replace load testing.
To be able to load test and understand the results you need to be able to measure throughput and the most likely individual performance of specific tasks while the rest of the system is under load.
Performance counters are the easiest way to monitor throughput and performance of a running system in real time.  It could be achieved with logging to files or database, but why bother? Most load testing tools capture performance monitor data and later analysis, and previous test data doesn't pollute later tests.

What else are they good for? 

Even for single user applications performance counters can be a convenient way to measure individual intensive tasks within the application.  Its also a good way to measure if running tasks in parallel is better than not.

Why not, they're easy to use and there's some free stuff

There's a huge array of standard performance counters that come with the OS and with .NET, that are very useful.  Counters to measure processor utilisation, number of logical threads created, memory usage, GC stats, and ASP.Net requests per second, and % of WCF throttles used.  Basically only code that you have written inside your app won't be measured.  Writing your own performance counters is dead easy, and there isn't much code that invades into your source to make it happen. 


var perfCounter = new PerformanceCounter("My App Name", "Counter 1", false);
perfCounter.Increment();

Watching Performance Counters

To view performance counters and watch live data coming through use PerfMon. This is part of Windows OS.


Gotchas:

  1. You may need to reboot after installing a performance counter with the OS.  It may appear in the PerfMon list, but no data comes through.  A reboot will likely fix this, it did for me.
  2. Admin priviledges will be required to install custom performance counters into the OS and delete them.  This works in code, but isn't a great idea to run / require production apps to use admin credentials.  To update performance counter data the app only needs to be a member of the Performance Log Users Group.
  3. There are some random gaps sometimes in the counters, but in my experience never more than a few minutes.

Download my sample code

Other References:

Azure gets PCI DSS Compliance

From Scott Guthrie's blog:
http://weblogs.asp.net/scottgu/archive/2014/01/16/windows-azure-staging-publishing-support-for-web-sites-monitoring-improvements-hyper-v-recovery-manager-ga-and-pci-compliance.aspx



Windows Azure Now Validated for PCI DSS Compliance

We are very excited to announce that Windows Azure has been validated for compliance with the Payment Card Industry (PCI) Data Security Standards (DSS) by an independent Qualified Security Assessor (QSA).
The PCI DSS is the global standard that any organization of any size must adhere to in order to accept payment cards, and to store, process, and/or transmit cardholder data. By providing PCI DSS validated infrastructure and platform services, Windows Azure delivers a compliant platform for you to run your own secure and compliant applications. You can now achieve PCI DSS certification for those applications using Windows Azure.
To assist customers in achieving PCI DSS certification, Microsoft is making the Windows Azure PCI Attestation of Compliance and Windows Azure Customer PCI Guide available for immediate download.
Visit the Trust Center for a full list of in scope features or for more information on Windows Azure security and compliance.

Summary

Today’s release includes a bunch of great features that enable you to build even better cloud solutions.  If you don’t already have a Windows Azure account, you can sign-up for a free trial and start using all of the above features today.  Then visit the Windows Azure Documentation Centerto learn more about how to build apps with it.


Sunday, January 12, 2014

Check your own security

Checks how good your current client is at using a secure SSL/TLS connection.
https://www.howsmyssl.com/
Checks if your name has been published as having been inappropriately accessed by unauthorized persons.
https://haveibeenpwned.com

How secure is your password? How long would it take for a standard desktop PC to brute force your password?
https://howsecureismypassword.net/