Unit Testing 101 Guidelines Part 4

Also see See also Unit Testing Guidelines Terminology, Part1, Part2, Part3, Part4

Standard Practices & Guidelines

Code Coverage Levels

·         DO have at least 60% code coverage with any new Microsoft.Net code written.

·         CONSIDER aiming for 80% + (Remember it is not possibly to unit test ALL code, so don't beat yourself up if you can't get to 100%).

·         DO analyse your code using the Code Coverage panel in Visual Studio. This will show you the code paths not tested.

Structure of Test Fixtures and Projects

·         DO write at least one Test Fixture per production class.

·         CONSIDER writing more than one Test Fixture per production class. Test Fixtures with fewer methods and with clear themes are easier to maintain.

·         CONSIDER creating one test project per Visual Studio solution, not one test project per production project.

·         DO NOT add references to test libraries in the production projects. (ie NunitFramework.dll, RhinoMocks.dll, YourTestProject.dll etc).

·         DO give your test fixtures strong names with no acronyms.  Better it be long and descriptive than short and cryptic.

·         DO put Integration Tests and Unit Tests in different namespaces within your test project. This will make it easier for people to bypass the longer larger integration tests if they choose to do so.

·         DO NOT inter-mingle unit tests and integration tests in the same test fixture.

·         AVOID using constructors in test fixtures. Use methods decorated with the [Setup] attribute instead.

·         DO NOT use IDisposable or destructors in test fixtures instead use the [Teardown] attribute on your clean up method.

·         DO put all your accessors in a sub-folder and different namespace within your test project.

·         DO put all your mocks and stubs in a sub-folder and different namespace within your test project.

Single Test Design

·         CONSIDER the theme of your test.  Each test should be testing a specific use case or code path. Better to the have lots of simple tests than few complex difficult to maintain tests.

·         AVOID designing your tests to be slower than 500ms each.

·         CONSIDER optimising until they are running in less than 100ms each.

·         CONSIDER keeping your tests to 50 lines of code or less.

·         DO NOT let your test exceed 100 lines of code.

·         DO give your tests strong names with no acronyms.  Better it be long and descriptive than short and cryptic.

·         CONSIDER giving all your tests descriptions using the [Test] attribute. For example: [Test(Description = “My test that tests the something method and looks for this...”)].

·         DO NOT write tests that could affect the running of other parallel tests. For example avoid using class level or static variables, except if they are readonly.  Tests should always be able to run in parallel; in fact most tools do this by default (Resharper and NUnit Gui included).

·         DO NOT write tests that fail periodically and pass sometimes. Tests should always be consistent.

·         AVOID writing tests that access more than one public method or entry point into a class. Write more tests.

·         DO always add the [Timeout] attribute to all your tests.  Give it at most a 500ms timeout. This will prevent failing tests waiting for internal timeouts to expire to fail the test. (You will need to remove the attribute to debug the test).

·         CONSIDER using a Mock framework like RhinoMocks to automatically generate your mocks and stubs. Writing them by hand is tedious and time consuming.

·         CONSIDER extensively commenting your tests.  Tests are commonly used for explaining how to use a particular class.  It also increases maintainability of the test.

·         DO use a wide range of input values for a test. Always test fringe data like null, empty, zero, and negatives where appropriate.

·         CONSIDER using the [Values] attribute rather than copying and pasting the same test over and over but with different input data for the targeted method.

·         AVOID using try...catch to catch expected exceptions in tests.  For example if you want to ensure a NullArgumentException is thrown when a consumer tries to pass null to a method, use the [ExpectedException] attribute rather than wrapping the test in try...catch.

·         DO test all exceptions thrown by a code path. If the method being tested has code that specifically throws exceptions, write tests for these use cases.

·         DO check the state of the class as well as return values of functions.  A function could change the fields within the class as well as return a value.

Dependencies

·         DO mock all references the class under test has to other classes.  Exceptions are listed below in this section.

·         AVOID mocking/stubbing dependencies of type ICommand.  Commands are one of the few very simple standard objects that can be exempted.  This only applies to Commands of type RoutedCommand and RelayCommand. If you have a custom implementation then it will need to be mocked.

·         AVOID mocking/stubbing dependencies that are Data Transfer Objects (DTOs).  These are objects that are used to transfer data to and from a WCF service.  These objects are basically structs and do not contain any logic.

·         CONSIDER using Strict mocks when using Rhino Mocks whenever possible as they catch a much wider range of issues with much less code.

·         AVOID using a string based private accessor. Prefer instead a strongly typed private accessor like those generated by the PrivateAccessorGenerator class.

What to test

·         DO test all public methods of a class. Most often the non-public members will be called by the public members. If the non-public members are not being covered by testing only the public members consider testing them too.

·         CONSIDER testing all paths through a method of code. Use the Code Coverage analysis tool inside Visual Studio to show you the paths currently not being tested (they will be highlighted in pink).

·         DO test all main points of entry into the class. An entry point might be a public method, or protected or internal. 

·         DO test the constructors of a class to ensure the class is in a predictable defined state upon instantiation.

·         CONSIDER testing internal and protected members if the public methods of a class do not call all internal and protected methods.

·         DO NOT test properties, fields, structs, and enums. These members should not contain any logic.  The only exception to this rule is the properties who raise the INotifyPropertyChanged.PropertyChanged event.

·         DO test properties that intentionally throw exceptions.  This technique is sometimes used to validate property setters in WPF. These setters should be tested with some known valid and invalid cases.

Design for Testing

·         DO NOT write tests that require user intervention.

·         DO NOT add test code or test concerns to production code. Unit testing is supposed to be unobtrusive into the production code and leave no footprint on the production code.  This means no test libraries should be referenced by production assemblies. There should be no if statements or compiler switches for testing versus production. By looking at the production code you should not be able to tell unit tests are being used against a class.

·         DO use interfaces for references to other classes.

·         CONSIDER using a basic abstract class instead of an interface but prefer interfaces.

·         DO use the following dependency injection techniques in this order or preference:
1)            Interfaced properties that use a backing field, that if null creates the production instance of that type.  A private setter can be used to inject a test mock.

private IMessageBoxService backingMessageBoxService; public IMessageBoxService MessageBox {     get {         return this.backingMessageBoxService ?? (this.MessageBox = new WpfMessageBoxService());         }         private set {             this.backingMessageBoxService = value;         }     } }

This is best when only a test mock and a production type are used. If there could be more then rather use an Object Factory to get the instance.

2)            Constructor injection. The number of constructor arguments should be no more than 3 or 4.

public FormController(IContactDataService contactDataService) { … }

3)            Use an Object Factory to get an instance of an interface.

private IMessageBoxService backingMessageBoxService; public IMessageBoxService MessageBoxService {     get {         if (this.backingMessageBoxService == null) {             this.backingMessageBoxService = StructureMap.ObjectFactory.Container.GetInstance<IMessageBoxService>();         }           return this.backingMessageBoxService;         }     } }

·         DO NOT use property injection. It is very slow in comparison to constructor injection and object factory look-up.

·         DO NOT call the static class MessageBox directly. Instead see this example: Message Box Usage

·         DO NOT show Windows directly. Ie Do not call Window.Show().  Instead see this example:

·         Showing Another Window From a Controller

·         AVOID static properties or fields in a class. Rather ask the object factory to return you a singleton reference.  The key piece is the .Singleton() method call.

ObjectFactory.Initialize(init => {     init.For<ITestSingleton>().Singleton().Use<TestSingleton>(); }); var singleton = ObjectFactory.GetInstance<ITestSingleton>();

In this example the same instance will be returned whenever someone asks for an instance of ITestSingleton.

Bugs

·         DO write at least one test every time you find a bug or are given a bug report to resolve. This ensures the exact same bug cannot reoccur. The test should reproduce the bug and fail.  You can then write code to make the test pass. Ensure your test(s) covers all scenarios documented in the bug report.

·         CONSIDER writing more tests if you can see more than one way the bug could manifest itself.

·         DO write only the code necessary to make your tests pass.