Sunday, December 20, 2009

Unit Testing 101 Guidelines Part 1

See also Unit Testing Guidelines Terminology, Part1, Part2, Part3, Part4

What is Unit Testing?

A unit test is a focused automated test for one member in a class.  This means it’s a small piece of .Net code designed to test one member of a class.  Given control of the input to a method the output can be predicted (with knowledge of the code or business function) and then compared to a predetermined constant. If it does not meet this expectation outcome the test fails.  

For example if you have a method Add(x, y) that takes two integers you know (based on the specification) that passing in 2 and 3 should return 5.

Unit tests are built over time and stored in large libraries of tests. These libraries are then used to test a lot of code very quickly and covering a good deal of code.  This fast high code coverage testing can then be used to gauge the success of new code (has a newly added feature somehow broken existing code aka ripple-effect).

The style of inputs generally fall into two categories: common use case input, and fringe input intended to see if the method gracefully handles unexpected data.

Unit tests are intended to be isolated and independent from one another. This means that one test running should not interfere with others also running at the same time or subsequently.  For the sake of speed, applications that run unit tests quite often run tests in parallel on multiple threads; doing so should not fail the tests. Making use of static state variables will cause tests running in parallel to interfere with each other.

Unit testing is strictly targeting development and developers.  However unit tests should always be automated into the master build process.  This gives an indication of build quality and ensures QA testers do not start testing a broken build.  A broken build is further described as a build that does not compile or a build in which not all tests are passing.

Unit testing is only possible on code that has been designed to be tested.  Designing for testing essentially means that class design allows for ways to provide alternatives (stubs and mocks) for all dependencies of the class.

When testing a method within a class you go through a process of “mocking out” dependencies so the class under test does not try and call other production code outside the class. For example to ensure a class does not call a live service or database. This creates an isolated test environment that focuses the testing to the one class and method. Only code in the class under test can fail the test.


What is Test Driven Development (TDD)? 

Test Driven Development is sometimes used in conjunction with Unit Testing. Unit testing can be fully utilised without following a Test Driven Development process. However by following TDD high code coverage is achieved more rapidly and code tends to be more concise and less complex.

WIKIPEDIA: It’s an approach focused on writing only the code necessary to pass tests, resulting in cleaner and clearer designs than is achieved by other methods. It cuts out extra functionality that is outside of the current scope. This definitely results in simpler, easier to implement and follow code, but means that extensibility becomes a problem for future projects unless extensibility is part of the original specification and scope.

The main principles are:

  • —The main principal is to KISS YAGNI. 
  • Keep It Simple Stupid, You Ain’t Gunna Need It. 
  • Developers are given Use Cases with the software specifications. 
  • Write failing tests first that match Use Cases. 
  • Write code only to make tests pass. Results in a clean uncomplicated easy to use application.

Tuesday, December 15, 2009

NZ Home Loans Review - Extremely Poor Service


15-December-2009

I know this is off-topic but I simply must have a rant and warn anyone who will listen about this company and their evident incompetence and lack of interest in righting their mistakes.


I was very shocked disappointed with the results and with the advice I have received.  I was given advice without proper disclosure of conditions, risks, and with disregard for jeopardizing a great deal of my money. My complaint centers around quality of advice and a verbal pre-approval with the single condition that the valuation is to be more than the purchase price at auction.  

I originally approached NZ Home Loans on the 1st of October, and was impressed by their friendliness and informality.  During this first meeting we told our broker that we wished to attend an auction on the 21st of October. We were told it would be easy to turn the application into an approval before then.  After various chasing phone calls I finally received word on the 16th of October, that I had been approved but with a condition that they required a valuation of the property to be completely unconditional.  This was disappointing as there now was not enough time to organise a valuation before the auction on the following Wednesday.  

We discussed this, and I was advised that the valuation is only about ensuring the purchase price is not more than the valuation.  I agreed to take the risk of going to auction with only a verbal conditional pre-approval with the understanding there was only one condition. This condition was simply disclosed to me as obtaining a valuation for the property to ensure the value is more or equal to the purchase price at auction.  The broker arranged a valuer friend to quickly and informally preview the property prior to auction to give some confidence where the value will be. At no time was I made aware that the formal valuation could raise 'other' potential 'declineable' issues. There was no disclosure that despite the valuation price being well clear of our purchase price it could still be declined. In my opinion these were hidden and undisclosed conditions that I was completely unaware of. This is the exact reason why we seek advise from experienced professional people to begin with is it not?

After reading my email trail again, I also cannot help but question the speed of handling our application.  The loan application documents were hand delivered on the 7th of October. The broker was aware of our interest in attending the auction on the 21st of October.  I was notified by telephone on Friday the 16th of pre-approval for my application, but that one condition as stated above. We were originally told prior, that the requirement of a valuation is "unlikely".  So at this point there was not enough time to organise a valuation, or for this valuation to be processed by NZ Home Loans.  So on advice I proceeded with plans to attend the auction knowing that his valuer had previewed the property and was comfortable it was easily well clear of what my personal budget ceiling.  After winning the auction I waited 4 weeks to get this final response. If this was a standard settlement period I would have been forced to default on the settlement. I chased this up during this period with phone calls and emails, usually with no response. One response was "we seem to have lost the documents, can you please resend"!

On Tuesday the 17th I was advised that they had been forced to decline my application!  Then, on the Wednesday I was then told this decision wasn't a 'No' after all, but if we obtain a structural builder's report and moisture report there may be a chance. Failing that, the suggestion was to max out our credit cards to fund an additional 30k deposit!

At that point I decided to ditch NZ Home Loans, and take the risk of finding funding elsewhere. Incidentally, our approval with our new lender was turned around in 36 hours completely unconditional. Big Thanks to Westpac New Zealand. After complaining to NZ Home Loans head office I get a phone call from the manager asking to discuss the situation.  Which I happily did 6 days later and told them exactly how I felt about the whole foul up.  During this meeting, his way of making things right was to offer me an approval for funding; a little late I think to say the least.

I wish I lived in the States where suing these clowns would be much easier. However I haven't ruled out contacting the Ombudsman. At the very least I will be vocal about my experiences to my family, friends, and in my tweets and blog posts. I think there was a severe failure in procedure and it seems to me that competency levels are well short of what they should be.

Please think twice before risking their service. Happy home ownership :-)

Saturday, November 14, 2009

Activator.CreateInstance(...) Performance



Recently I have been looking into performance profiling different methods of implementing an
 Object Factory pattern
.  Something that can choose an appropriate concrete type and create it when a requested interface is given.  
A fundamental portion of this is to be able to register types with the factory, and in my case this needs to be done by a xml configuration file.
 Therefore the type is ultimately created by calling Type.GetType("FullyQualifiedTypeNameHere, AssemblyName").  This has bothered
me for some time and I have been waiting until I have time to performance test it.




I have an existing Object Factory that can create types when a requested interface is given.  You can register a concrete type against
an interface either by code or configuration, but mostly this needs to be done by the configuration xml file.  I wrote a test program
to time different methods of creating objects and was a little surprised by the results.




Here's the test program code:


namespace ActivatorCreateInstanceAlternatives {
    using System.Collections.Generic;
    using System.Diagnostics;
    using System.Linq;
    using System.Reflection;
    public class Program {
        private static ConstructorInfo _constr;
        private static Type _type;
        private static Dictionary<Guid, Client> allClients = new Dictionary<Guid, Client>();
        public static void Main(string[] args) {
            Console.WriteLine("Test One - Loop thru each 10000 times (x,x,x,x,x.... y,y,y,y,y.... z,z,z,z....).");
            var objectFactoryTime = ObjectFactoryPerfTests();
            var newCreateTime = NewPerfTests();
            var activatorTime = ActivatorCreatePerfTests();
            var activatorCachedTime = ActivatorCreateCachedTypePerfTests();
            var cachedConstructorTime = CachedConstructorPerfTests();

            Console.WriteLine("Baseline 'new' construct " + newCreateTime.ToString("N"));
            Console.WriteLine("Activator.CreateInstance " + activatorTime.ToString("N"));
            Console.WriteLine("Cached Activator.Create  " + activatorCachedTime.ToString("N"));
            Console.WriteLine("Cached constructor       " + cachedConstructorTime.ToString("N"));
            Console.WriteLine("Object Factory           " + objectFactoryTime.ToString("N"));
            Console.WriteLine(" ");
            Console.WriteLine("Test Two - For each 10,000 use each creation method (x,y,z, x,y,z, ...)");

            var objectFactoryTime2 = new List<double>();
            var newCreateTime2 = new List<double>();
            var activatorTime2 = new List<double>();
            var activatorCachedTime2 = new List<double>();
            var cachedConstructorTime2 = new List<double>();

            for (int index = 0; index < 10000; index++) {
                newCreateTime2.Add(NewPerfTest());
                activatorTime2.Add(ActivatorCreatePerfTest());
                activatorCachedTime2.Add(ActivatorCreateCachedTypePerfTest());
                cachedConstructorTime2.Add(CachedConstructorPerfTest());
                objectFactoryTime2.Add(ObjectFactoryPerfTest());
            }

            Console.WriteLine("Baseline 'new' construct " + newCreateTime2.Average(x => x).ToString("N"));
            Console.WriteLine("Activator.CreateInstance " + activatorTime2.Average(x => x).ToString("N"));
            Console.WriteLine("Cached Activator.Create  " + activatorCachedTime2.Average(x => x).ToString("N"));
            Console.WriteLine("Cached constructor       " + cachedConstructorTime2.Average(x => x).ToString("N"));
            Console.WriteLine("Object Factory           " + objectFactoryTime2.Average(x => x).ToString("N"));
            Console.WriteLine(" ");
        }

        private static double NewPerfTests() {
            var results = new List<double>();
            for (int index = 0; index < 10000; index++) {
                results.Add(NewPerfTest());
            }

            return results.Average(x => x);
        }

        private static double NewPerfTest() {
            var stopwatch = Stopwatch.StartNew();
            var client = new Client();
            Debug.Assert(client != null);
            stopwatch.Stop();
            allClients.Add(client.Id, client);
            return stopwatch.ElapsedTicks;
        }

        private static double ActivatorCreateCachedTypePerfTests() {
            var results = new List<double>();
            if (_type == null) {
                _type = Type.GetType("ActivatorCreateInstanceAlternatives.Client");
           }

            for (int index = 0; index < 10000; index++) {
                results.Add(ActivatorCreateCachedTypePerfTest());
            }

            return results.Average(x => x);
        }

        private static double ActivatorCreateCachedTypePerfTest() {
           var stopwatch = Stopwatch.StartNew();
           var client = Activator.CreateInstance(_type) as Client;
           Debug.Assert(client != null);
           stopwatch.Stop();
           allClients.Add(client.Id, client);
           return stopwatch.ElapsedTicks;
        }

        private static double ActivatorCreatePerfTests() {
            var results = new List<double>();
            for (int index = 0; index < 10000; index++) {
               results.Add(ActivatorCreatePerfTest());
            }

            return results.Average(x => x);
        }

        private static double ActivatorCreatePerfTest() {
            var stopwatch = Stopwatch.StartNew();
            var client = Activator.CreateInstance(Type.GetType("ActivatorCreateInstanceAlternatives.Client")) as Client;
            Debug.Assert(client != null);
            stopwatch.Stop();
            allClients.Add(client.Id, client);
            return stopwatch.ElapsedTicks;
        }

        private static double CachedConstructorPerfTests() {
            var results = new List<double>();
            if (_constr == null) {
                Type type = Type.GetType("ActivatorCreateInstanceAlternatives.Client");
                _constr = type.GetConstructor(new Type[] { });
            }

            Debug.Assert(_constr != null);
            for (int index = 0; index < 10000; index++) {
                results.Add(CachedConstructorPerfTest());
            }

            return results.Average(x => x);
        }

        private static double CachedConstructorPerfTest() {
            var stopwatch = Stopwatch.StartNew();
            var client = _constr.Invoke(new object[] { }) as Client;
            Debug.Assert(client != null);
            stopwatch.Stop();
            allClients.Add(client.Id, client);
            return stopwatch.ElapsedTicks;
        }

        private static double ObjectFactoryPerfTests() {
            var results = new List<double>();
            for (int index = 0; index < 10000; index++) {
                results.Add(ObjectFactoryPerfTest());
            }

            return results.Average(x => x);
        }

        private static double ObjectFactoryPerfTest() {
            var stopwatch = Stopwatch.StartNew();
            var client = ObjectFactory.Current.CreateInstance<IClient>() as Client;
            Debug.Assert(client != null);
            stopwatch.Stop();
            allClients.Add(client.Id, client);
            return stopwatch.ElapsedTicks;
        }
    }
 }




Test Overview
Let me summerise this code into what I am trying to profile.  
There are two test sets, each will create 10,000 objects using each technique of creating an object.  
The first test set creates 10,000 using one technique then moves onto the next technique.  
The second set creates one object using each technique 10,000 times. This is just to check consistency of the times.




New keyword
The 'new' keyword is used as a baseline control for the test, I wouldn't expect anything to be consistently faster than this.

Activator.CreateInstance
The next test is Activator.CreateInstance(Type.GetType("SomeTypeNameHere")).  
This I am expecting to be consistently the slowest, as it has to find the type then create an instance of it for each of the 10,000 iterations.

Activator.CreateInstance with cached Type
This test will use Activator.CreateInstance without refetching the type each iteration (the type is cached into a class level field).

Cached Constructor
This test will simply get a ConstructorInfo object, cache it into a class level field and Invoke for each iteration through the loop.

Object Factory
This is my existing object factory.  This isn't really a fair comparison as the object factory is doing a little more work than
simply creating the object, (like argument preconditions etc) but it shouldn't be a million miles off the others.

Running the program 3 times I get the following results:
Run1:
Run2:
Run3:

Summary of results:
 Baseline 'new' construct 5.26ms 1
 Activator Type cached 5.72ms 2
 Cached Constructor 9.23ms 3
 Object Factory 9.57ms 4
 Activator without caching the type 26.43 5
Its safe to say (despite the slightly high results from run1) that using the 'new' keyword to create an instance of a known type is the fastest, as you would expect.

But in second place is Activator.CreateInstance when using an instance of a known Type cached into a class level field.  So, because my object factory uses this technique already (with the addition of some checks etc) it is not that far behind. 

Conclusion
I don't think my existing Object Factory is broken.  This proves the cached type when using Activator.Create is not that slow.  Considering the alternatives (Reflection.Emit, Dynamic Languages and others) and the complexity they can introduce I think I'll leave it as is.  If you can see any serious flaws in this logic let me know.
-Ben

Monday, November 9, 2009

C# vs Vb.Net.... Fight!

16-November-2009


I need to have a bleet about some of my experiences converting Vb.Net 7 to C#3.5 and unfortunately to Vb.Net 9 as well. The conversion is for a massive existing Vb.Net solution in .Net1.1 that for various reasons must remain in Vb. In a perfect world redeveloping using new language constructs and in C# would be my preference. The good news is that there are some satellite web services and components that are small enough to justify conversion to C# and to drag the design kicking and screaming into this decade.


The debate rages between C# and Vb.Net fans and usually most people (who actually know what they are talking about) are polarised to one side or the other. I have used both, and for some time, have a slight preference for C#, but also there are some very nice positive aspects to Vb.Net. I coded exclusively in Vb and Vb.Net for about 7 years, and more recently 3 years in C#.






One major asset I have been a fan of in the Vb.Net camp has is good layman readability. I have taken advantage of this repeatedly in the past and used Vb.Net code to show clients and BA's to help explain behaviours etc.

This can also be taken advantage of by learner developers, as the syntax seems a little more approachable and less intimidating than C#.

On this subject, a downside is that sometimes Vb's syntax is very ambiguous and difficult to analyse with automated (handcrafted or thirdparty) tools.
C# is often praised for its concise unambiguous syntax; one line of code can only be interpreted one way.

There is an abundance of tools available for C#, and analytical tools are easier to write for C# source code. Vb.net also has tools available, but not as many as C#, and they are a little harder to come by.



In my personal opinion I like the Vb.Net syntax of generic type modifiers, apart from the excessive use of parenthesis in Vb and the 'Of' keyword. But I do like the 'As' modifier syntax directly after the generic declaration. For example compare these:
Public Sub MethodName(Of T As {Class, New, IDisposable})()
public void MethodName<T>() where T : class, new, IDisposable { }

Lambdas in C# are really nice. Vb's syntax is simply awful. Compare these:
list.ForEach(Function(x) Process(x))
list.ForEach(x => Process(x));
In actual fact this won't compile in Vb. Unless I'm missing something you cannot write a Lambda action (one that does not return a value) in Vb. Using Lambda actions is very handy for event handling in C# without having to write loads of methods to handle events. 

And in a similar vein, there are no multi-line lambdas in Vb.Net. Stink.
set.All(x => { Process(x); return true; }); 
I tend to use this syntax a lot (when changing the method to invoke is not an option), on anything other than List<T> where there is no .ForEach method. 

Historically using Vb.Net without turning on Option Explicit has been frowned upon; at least in my circle or contacts.  Now with Vb.Net usage of implicitly typed variables is not possible with this on.  Which seems odd to me. I need to investigate this further as I thought Option Explicit only prevent implicit casting from one type to another not type inference during declaration. So right now as far as I am aware in my conversions I have not been able to replicate this in Vb:
var results = from x in someList where condition(x) select new {Name = x.ToString(), From = x.GetType().Name};

This one really made me tear my hair out. It seems you cannot invoke an Action delegate that is stored in property. Consider this:
Public Class Class1
    Public Property MyDelegate As Action(Of object)
End Class
Inside some other method...
Public Sub Method1(director as Class1)
    director1.MyDelegate(new Object()) ' Will not work
    'You must do this...
    director1.MyDelegate()(New Object()) 'this will work - but is a very nasty confusing syntax - looks like a method call or array access
    'Or
    Dim action As Action(Of Object) = director.MyDelegate
    action(new Object()) ' this will work to.
End Sub
Annoying! - C# score's another point here for sure.



The built in support for XML in Vb.Net is quite convenient, if a little weird. Its definitely more concise and readable than C#.

Dim testXml = <test>
                  <product title="Prod1"/>
                  <product title="Prod2"/>
                  <product title="Prod3"/>
              </test>
Dim queryA = From t In testXml...<product> _
             Select New With {.Title = t.@title} '... is any descendant

Auto-Properties quite simply rock. For example:
public class Class1 {
    public string MyString {get; set;}
    public int MyInteger {get; set;}
           }
In comparison to the Vb conversion:
Public Class Class1 
    Private _myString as String
    Private _myInt as Integer
    Public Property MyString as String
        Get
            Return _myString
        End Get
    End Property
    ...


I really like Optional method arguments in Vb.  It means the number of overloads for a method is reduced, resulting in more concise code. Optional arguments are not support currently in C#, and can only be implemented by implementing an overload.

Public Function Foo(x As Integer, Optional y as Integer = 1) as Integer ...
Note: I believe optional parameters are coming in C# 4.
It has to be said that I hate the excessive use of parenthesis in Vb. You can almost hear the chief Vb designer say if in doubt throw in a bracket.  A crazy example of this is Properties that can take arguments!  Why?! Whats the difference then between a property and a method? Its commonly accepted that the whole point of a property is a simple state attribute of an object, it should not change state, and should not contain a significant amount of work. C# definitely wins a clarity point here.



I cannot help but like the convenience of in-line Vb dates:
Dim MyDate As Date = #2/29/2008#
Compared to C#
DateTime myDate = new DateTime(2008, 2, 29); // Or
DateTime myDate2 = new DateTime.Parse("2008/2/29");

C# has array initialisers.
var theList = new List<string>() {"abc", "def", "ghi"};
Vb has no such convenience :-(. Although I believe this is coming in Vb 10.
In Summary, and again this is based on my own personal "frustration-Richter-scale". C# tops Vb with 8 points to 5.  Hence my personal preference for C#.  Each to their own.