Rhino Mock - Running tests in parallel - not thread safe

I came across this issue while diagnosing why some unit tests intermittently fail.  It appears to be that Rhino Mock (v3.6.1) is not thread safe.

There does appear to be some chatter on this topic when you google it.  Someone has even customised their own version of the Rhino code to be thread safe here.

First off, hats off to Ayende and crew for creating Rhino mocks its a fantastic premium mocking library that I have enjoyed using.  Back when it was written running tests in parallel was fringe at best.

My environment:

• MsTest using a .TestSettings file set to run two tests in parallel.
• Visual Studio 2012 Premium (not that I think this is relevant).
• 8 core processor.
• I'm using the Visual Studio Test Runner to run the tests.

The Code:

Full test code that exposes the issue can be downloaded here.

In Test1 you can see both test methods look fine. 

    [TestClass]
    public class Test1
    {
        [TestMethod]
        public void IsTrivialAssociationShouldReturnFalse()
        {
            var dependency1 = MockRepository.GenerateStub<IDependency1>();
            var dependency2 = MockRepository.GenerateStub<IDependency2>();

            dependency1.Stub(m => m.DependentMethod("System.String")).Return(false);

            dependency2.Stub(m => m.Kind).Return(TypeKind.Foo);
            dependency2.Stub(m => m.StringProperty).Return("System.String");

            var target = new TestSubject(dependency1, dependency2);
            Assert.IsFalse(target.SubjectMethod());
        }

        [TestMethod]
        public void IsTrivialAssociationShouldReturnTrue()
        {
            var dependency1 = MockRepository.GenerateStub<IDependency1>();
            var dependency2 = MockRepository.GenerateStub<IDependency2>();

            dependency1.Stub(m => m.DependentMethod("System.String")).Return(true);

            dependency2.Stub(m => m.Kind).Return(TypeKind.Foo);
            dependency2.Stub(m => m.StringProperty).Return("System.String");

            var target = new TestSubject(dependency1, dependency2);
            Assert.IsTrue(target.SubjectMethod());
        }
    }

The problem only seems to manifest when using the same interface type multiple times in the same test class.
When this runs either one of these tests will intermittently fail given the test settings I am using (running 2 or more in parallel).

You'll either get an Assert failure even though clearly it should not.

A few times I got another weird "System.InvalidOperationException: This action is invalid when the mock object is in replay state."

Its more apparent when you throw many threads at it:

    [TestClass]
    public class Test2
    {
        [TestMethod]
        public void BruteForceTest()
        {
            var threads = new Thread[30];
            for (int i = 0; i < threads.GetUpperBound(0); i++)
            {
                threads[i] = new Thread(this.ThreadAction);
                threads[i].Start();
            }

            for (int i = 0; i < threads.GetUpperBound(0); i++)
            {
                threads[i].Join();
            }
        }

        private void ThreadAction()
        {
            var dependency1 = MockRepository.GenerateStub<IDependency1>();
            var dependency2 = MockRepository.GenerateStub<IDependency2>();

            dependency1.Stub(m => m.DependentMethod("System.String")).Return(false);

            dependency2.Stub(m => m.Kind).Return(TypeKind.Foo);
            dependency2.Stub(m => m.StringProperty).Return("System.String");

            var target = new TestSubject(dependency1, dependency2);
            target.SubjectMethod();
        }
    }

I tried solving this by using the old Rhino syntax of declaring a method local repository with no luck.  I also tried creating dedicated fields for all dependencies for each test and initialising them in the constructor, but this didn't work either.

Solution:

I tested a threadsafe modification to Rhino and it resolved the issues shown in the code example. But I didn't test it extensively.  You can find the blog post here and the DLL download here.  I'm not sure if this is a good idea or not.

Another alternative is to place all your mock expectation calls in the ClassInitialise() of the test class and create separate static fields for each dependency for each test.  This ensures that all tests have had their set up done first.  You should not put this code in a TestInitialise() as this will run prior to every test, and what you're after is running the expectation code exactly once.

    [TestClass]
    public class Test3
    {
        private static IDependency1 dependency1a;

        private static IDependency2 dependency2a;

        private static IDependency1 dependency1b;

        private static IDependency2 dependency2b;

        [ClassInitialize]
        public static void ClassInitialise(TestContext context)
        {
            // For Test a
            dependency1a = MockRepository.GenerateStub<IDependency1>();
            dependency2a = MockRepository.GenerateStub<IDependency2>();

            dependency1a.Stub(m => m.DependentMethod("System.String")).Return(false);

            dependency2a.Stub(m => m.Kind).Return(TypeKind.Foo);
            dependency2a.Stub(m => m.StringProperty).Return("System.String");

            // For Test b
            dependency1b = MockRepository.GenerateStub<IDependency1>();
            dependency2b = MockRepository.GenerateStub<IDependency2>();

            dependency1b.Stub(m => m.DependentMethod("System.String")).Return(false);

            dependency2b.Stub(m => m.Kind).Return(TypeKind.Foo);
            dependency2b.Stub(m => m.StringProperty).Return("System.String");
        }

        [TestMethod]
        public void Test3a()
        {
            var target = new TestSubject(dependency1a, dependency2a);
            Assert.IsFalse(target.SubjectMethod());
        }

        [TestMethod]
        public void Test3b()
        {
            var target = new TestSubject(dependency1b, dependency2b);
            Assert.IsFalse(target.SubjectMethod());
        }
    }

Test 3a and 3b will always pass.

You could also use a lock, although this feels a bit dirty in unit testing, its the least amount of code.

So just a word of warning, you will need to be aware of this only if you take advantage of running tests in parallel.