TypeMock Isolator war stories

Like many other developers out there, most of the time we are working with some form of legacy code that when written was not designed to be tested.  Changing existing code to be unit testable can be hard particularly if there really wasn't much care taken with the design originally, or there's been years of hacky maintenance of the code.  I was always curious to give Typemock's Isolator a try to see if it helped making testing of legacy code easier or quicker.

Background

I had two completely different projects in mind.  One a legacy project that is around 7 years old and has been submitted to constant time constrained projects that led to bad coding practices.  This project is essentially a standard 3 tier architecture, although I focus on the WCF / Remoting service layer and below.  I'll refer to this project as the legacy project. 

The other is purely a green fields project.  It is a standalone Windows executable that has been designed and built recently and makes extensive use of interfaces, abstract classes, and virtual methods. I'll refer to this project as the modern project.

I'll try to compare these two distinct scenarios to test if Typemock's Isolator is best suited to one or the other. 

What is Typemock Isolator

By now most modern developers have had some exposure to unit testing, so I won't go into definitions or how-to's here.  There are many unit testing frameworks and utilities that help in unit testing.  Firstly you'll need a unit test runner (nUnit, MsTest, xUnit etc). In all but the most basic of unit tests you will also need a mocking framework (RhinoMock, Moq, NMock, and now with Visual Studio 2012 Microsoft Fakes). Isolator falls into this category and comes packages with other utilities that integrate into Visual Studio to assist in unit testing.  

Immediate Differences on the Surface

Ease of use is gauged simply as: easy, medium, hard

	Isolator Basic Edition	Isolator Essential Edition	RhinoMocks	Moq	Microsoft Fakes
Client License Price	Free	US$799	Free	Free	Requires Visual Studio 2012Ultimate
Server License Price	Free	US$2499	Free	Free	Free
Ease of use (imho)	Medium	Medium	Easy-Medium *1	Easy	Hard *2
Mock Interfaces/Abstracts/Virtuals	Yes	Yes	Yes	Yes	Yes
Mock Concrete classes	No	Yes	No	No	Yes
Mock Static usage	No	Yes	No	No	Yes

*1 - There are alternative usage styles in RhinoMocks. The simple way works most of the time, but sometimes you may need to dig deeper to write more advanced tests. Using the more advanced features does increase the difficulty. (This is based on feedback I get from other devs I work with; I think its a good trade-off for a feature rich framework).
*2 - This may be because my exposure to it is a little limited, however the syntax does feel a little awkward and verbose.

Biggest difference is the price.  Being able to exercise .NET black magic and mock the unmockable doesn't come for free.

Working with the Legacy Project

Legacy code never has just one issue, there are always multiple nasty problems intertwined.  Don't be naive and think that writing Isolator is a magic bullet that will make it easy. It won't. It does make it easier for sure. But it won't help you much with issues like badly designed methods and classes with thousands of lines of code.

Have a look at this for examples of working with Legacy code tests using Isolator.

The temptation is to write tests that ensure every interaction which normally results in brittle tests.  As soon as you refactor to a better design the method will be doing less.  Its normally more effective to focus on use cases for a class rather than focusing on the legacy methods.

So is this an effective strategy on legacy code? Is it good bang for buck - time spent versus quality and robustness of tests going forward?  Can the same be achieved by integration testing with SOAPUI or Selenium for example?

My experience has been you seldom come across legacy code that is well design, but just doesn't have tests. Writing good tests involves focusing on behaviour not tests line by line a method.  This is very difficult with legacy code as the Single Responsibility Principle has most likely been violated as well.  Behaviours will be spread across multiple classes in a not necessarily logical manner. So writing tests straight from the legacy code is a bad idea, it should be refactored as well. The best way to do this is use integration tests (either coded or using SoapUI, Selenium etc) and then refactor the internal code before writing focused unit tests.

Working with the Modern Project

So if you can afford it, does it mean that writing straight-forward code that does not use interfaces, abstracts, or virtuals is no longer necessary? No. No. No. Good design principals still apply.  Just because unit testing is the first consumer of abstractions and extension points in your code does not mean there will be no other future consumers.  Maintaining code that has been SOLID-ly designed is proven to be easier (easier == cheaper) over and over.

So given that you want to write good loosely coupled SOLID code, do you need Isolator?  No, its like say do you need to take vitamins with a balanced diet.  However it still can help.  Isolator really shines when you need to test code that you have little control over, like where your code interfaces into a third party API.  Also there might be times when using its ability to mock Static method calls provides a cleaner way to test something, rather than writing wrappers and interfaces.

Utilities

Do all the fancy shiney utilities and helpers really matter?  Well, it depends.  I personally do not like too many utilities creating noise in my IDE.  But I can see how the Isolator utilities can guide your test design, ensuring your tests are focused by giving a warning if more than one assert is used etc.

Conclusion

If good design principals and cheap maintenance and extension (read as low cost of ownership) are important to you, then abandoning SOLID techniques is a bad idea. That said, you can still use Isolator and write well designed SOLID code.

For modern projects if you are lucky to have Visual Studio Ultimate, no brainer, you don't need Isolator. If you don't have Ultimate its a slightly more difficult decision: It does seem hard to justify spending the money when the other open-source frameworks will definitely serve you well.  The utilities in my humble opinion are nice to haves; these are not compelling enough to drive me to spend the money.  The Isolator utilities do not enforce anyone to follow their advise so there are no guarantees. My opinion is that because Isolator would be difficult to be abstracted away from or switched out once you have a significant number of tests, I don't like the idea of being locked in when I don't feel a compelling need to use it. How about the free edition of Isolator? To be honest, (subjective warning) I prefer a more fluent style like Rhino or Moq. You may also be at the mercy of features being pulled into the paid versions.  Again, this is just my personal preference, not really based on scientific reasons.

Using it to test UI code (ie controllers and code behind) has no benefit in my opinion. This code will change often and you're tests will begin to test ASP.NET and other third parties that are out of scope for your test. UI code always has high churn, so testing individual components starts to offer low bang for buck.  I would prefer integration style testing of UI's (like SpecFlow or Selenium).

For legacy projects, again if you're lucky to have Visual Studio Ultimate, it will be hard to justify this expense.  Otherwise, I strongly believe that if you are refactoring code you should refactor it to a better design including interfaces etc.   However, introducing interfaces and proper design will take time. Legacy code is never simple to cover in tests, Isolator will definitely help you and has some nice features that will help show progress to stakeholders. The counter argument is that integration style testing with tools like Selenium and SoapUI might serve better less brittle tests.  I believe both approaches are important, integration tests ensure behaviour, unit tests ensure good code design and micro-behaviour that is focused.

One final thing:  Using Isolator (or Ms Fakes) to test and validate third party API's is quite effective.  This allows you to keep third party vendors honest that when they make claims about backwards compatibility etc.

Using the Factory Method Pattern

The factory method pattern is a creational software design pattern that encapsulates how to construct a dependency into method.  The method could return any number of concrete implementations. However the consuming code only accesses the dependency through an interface or abstract base class.  This pattern is useful when there is distinct logic to return one concrete class over another.  It is also an effective pattern when working with legacy code, as it is easier to introduce a new method than a whole new interface.

The essence of the pattern is simply a virtual method that contains logic to create and/or return an instance of a dependency used by the consuming class.

Here's an example using the pattern to allow for unit testing substitution.

namespace FactoryMethodDemo
{
    public class ProductMaintenance
    {
        public void EditProduct(int id, string name, decimal price)
        {
            var dataAccess = this.CreateDataAccess();
            var product = dataAccess.GetProductById(id);
            product.Name = name;
            product.Price = price;
            dataAccess.Save(product);
        }

        public virtual DataAccessLayer CreateDataAccess()
        {
            return new DataAccessLayer();
        }
    }

    public class DataAccessLayer 
    {
        public virtual Product GetProductById(int id)
        {
            // Expensive database call...
            // Omitted
            throw new NotImplementedException();
        }

        public virtual void Save(Product product)
        {
            // Omitted
            throw new NotImplementedException();
        }

    }

    public class Product
    {
        public string Name { get; set; }
        public int Id { get; set; }
        public decimal Price { get; set; }

    }

}

In this example the CreateDataAccess method is the factory method pattern. Granted this is a very simple example, the method could contain a great deal of initialisation logic, or logic to determine which DataAccessLayer to return if there is more than one data store.  This could be useful when the database is partitioned (sharded) across multiple servers.

Things to note:

• Always make factory methods virtual. This always for testing as well as other variations in other sub-classes.
• Prefer to name the method with clear indicators about how it behaves.  I have chosen the word "Create" to inform consumers that each call to the method will create a new DataAccessLayer instance.
• Prefer to make the return type an interface. This may not be feasible with legacy code, hence my demo here uses virtual methods. This is easier to introduce into an existing code base than an interface. Although if its possible introduce an interface.

Here's the test code to test the ProductMaintenance class.

namespace TestProject1

{
    using FactoryMethodDemo;
    using Microsoft.VisualStudio.TestTools.UnitTesting;
    using Rhino.Mocks;

    public class ProductMaintenanceTestHarness : ProductMaintenance
    {
        private DataAccessLayer mock;

        public ProductMaintenanceTestHarness(DataAccessLayer mockDataAccessLayer)
        {
            this.mock = mockDataAccessLayer;
        }

        public override DataAccessLayer CreateDataAccess()
        {
            return this.mock;
        }
    }

    [TestClass]
    public class ProductMaintenanceTest
    {
        public TestContext TestContext { get; set; }

        [TestMethod]
        public void EditProductTest()
        {
            var productTestData = new Product
            {
                Id = 1,
                Name = "Bar",
                Price = 89.95M,
            };

            var mockDataAccessLayer = MockRepository.GenerateMock<DataAccessLayer>();  // Able to mock because methods are virtual
            mockDataAccessLayer.Expect(m => m.GetProductById(1)).Return(productTestData);
            mockDataAccessLayer.Expect(m => m.Save(productTestData));

            var subject = new ProductMaintenanceTestHarness(mockDataAccessLayer);
            subject.EditProduct(1, "Foo", 99.99M);

            mockDataAccessLayer.VerifyAllExpectations();
            Assert.AreEqual(99.99M, productTestData.Price);
        }
    }

}

My Type Visualiser Project

I've always like to be able to visualise software designs in diagram form. When looking at code its a constant struggle to filter out the detail and only take notice of the important stuff. Most of the time learning a new code base or refactoring code the details are not important immediately. I find just exploring the dependencies of a type and other types it consumes a good place to start understanding a class. Easier said than done.

Visualising a type and its dependencies and relationships to other types, is useful to me for two reasons:
1) Assessing what this subject type is doing, and is it well structured.
2) How difficult would it be to get the subject type under test.

Enter the Type Visualiser.

This tool is intended to target one .NET type and visualise its associations.  Different relationships are modelled as different coloured lines and arrow heads.  Line thickness indicates how often a type is consumed by the subject or how many different fields or properties use other types.  The style is loosely based on UML, but is by no means intended to strictly follow UML.

The tool targets compiled .NET libraries (and executables) and presents a list of types it can find inside the chosen library.

The tool is intended to give a strategic overview of a type and how it relates to other types, so members themselves are not shown.  Only statistics and totals are given to provide a sense of how big and complex a type is. Nearly everything has a tooltip to indicate the various symbols and lines.

I'm intending on adding a Testibility Index to indicate how "testable" a type is. The more strong relationships it has (to sealed or concrete classes as opposed to interfaces and abstracts) the higher the index. The more lines of IL code and members the higher the index.  I haven't really ironed out the details of how this feature will work yet.

If there's strong interest I might open source the project.

These diagrams help me assess how much time/effort is required to get an existing piece of code under test and refactor it as well as add new features.  Obviously the more complex the diagram the more this efforts any estimates I give.  For example, this monster class shown below is definitely a huge concern in terms of complexity and the number of strong types is "welded" to.

Clearly, this subject type is broken.

Technical Debt vs Code Debt

I really like the distinction Steve Johnson makes in his article about Technical versus Code Debt.  Its a worthwhile distinction I think because it hints at the type or person required to do the clean up.  He defines Technical Debt as sloppy or too quick architecture or design that results in poor performance (scalability, throughput or extensibility).  Code Debt is defined as implementation problems of an otherwise adequate architecture and design. This code mean poor understanding, communication or lack of experience and skill. Or even more simply, lack of adequate code review.

One remedy for this is finding quick and easy analysis and metrics tools.  For example, NDepend, or Visual Studio's (Premium and better) Code Analysis and Metrics.  Visual Studio Ultimate also has a neat auto-diagram generation tool to show Type Sequence Diagrams (sometimes called type interaction diagrams but gets confused with UX Interaction Diagrams).  I'm also on the hunt for a good class diagram generator, (currently looking at Vs Ultimate class diagrams, but seems a little manual; and also ModelMaker).

Steve also suggests creating a "System Persona" to add value to the debt story and help you evaluate ROI.

Technical Debt

I re-read this article the other day, its still absolutely current describes the common problem of "Technical Debt".

Paying back Technical Debt