Monday, November 9, 2009

C# vs Vb.Net.... Fight!

16-November-2009


I need to have a bleet about some of my experiences converting Vb.Net 7 to C#3.5 and unfortunately to Vb.Net 9 as well. The conversion is for a massive existing Vb.Net solution in .Net1.1 that for various reasons must remain in Vb. In a perfect world redeveloping using new language constructs and in C# would be my preference. The good news is that there are some satellite web services and components that are small enough to justify conversion to C# and to drag the design kicking and screaming into this decade.


The debate rages between C# and Vb.Net fans and usually most people (who actually know what they are talking about) are polarised to one side or the other. I have used both, and for some time, have a slight preference for C#, but also there are some very nice positive aspects to Vb.Net. I coded exclusively in Vb and Vb.Net for about 7 years, and more recently 3 years in C#.






One major asset I have been a fan of in the Vb.Net camp has is good layman readability. I have taken advantage of this repeatedly in the past and used Vb.Net code to show clients and BA's to help explain behaviours etc.

This can also be taken advantage of by learner developers, as the syntax seems a little more approachable and less intimidating than C#.

On this subject, a downside is that sometimes Vb's syntax is very ambiguous and difficult to analyse with automated (handcrafted or thirdparty) tools.
C# is often praised for its concise unambiguous syntax; one line of code can only be interpreted one way.

There is an abundance of tools available for C#, and analytical tools are easier to write for C# source code. Vb.net also has tools available, but not as many as C#, and they are a little harder to come by.



In my personal opinion I like the Vb.Net syntax of generic type modifiers, apart from the excessive use of parenthesis in Vb and the 'Of' keyword. But I do like the 'As' modifier syntax directly after the generic declaration. For example compare these:
Public Sub MethodName(Of T As {Class, New, IDisposable})()
public void MethodName<T>() where T : class, new, IDisposable { }

Lambdas in C# are really nice. Vb's syntax is simply awful. Compare these:
list.ForEach(Function(x) Process(x))
list.ForEach(x => Process(x));
In actual fact this won't compile in Vb. Unless I'm missing something you cannot write a Lambda action (one that does not return a value) in Vb. Using Lambda actions is very handy for event handling in C# without having to write loads of methods to handle events. 

And in a similar vein, there are no multi-line lambdas in Vb.Net. Stink.
set.All(x => { Process(x); return true; }); 
I tend to use this syntax a lot (when changing the method to invoke is not an option), on anything other than List<T> where there is no .ForEach method. 

Historically using Vb.Net without turning on Option Explicit has been frowned upon; at least in my circle or contacts.  Now with Vb.Net usage of implicitly typed variables is not possible with this on.  Which seems odd to me. I need to investigate this further as I thought Option Explicit only prevent implicit casting from one type to another not type inference during declaration. So right now as far as I am aware in my conversions I have not been able to replicate this in Vb:
var results = from x in someList where condition(x) select new {Name = x.ToString(), From = x.GetType().Name};

This one really made me tear my hair out. It seems you cannot invoke an Action delegate that is stored in property. Consider this:
Public Class Class1
    Public Property MyDelegate As Action(Of object)
End Class
Inside some other method...
Public Sub Method1(director as Class1)
    director1.MyDelegate(new Object()) ' Will not work
    'You must do this...
    director1.MyDelegate()(New Object()) 'this will work - but is a very nasty confusing syntax - looks like a method call or array access
    'Or
    Dim action As Action(Of Object) = director.MyDelegate
    action(new Object()) ' this will work to.
End Sub
Annoying! - C# score's another point here for sure.



The built in support for XML in Vb.Net is quite convenient, if a little weird. Its definitely more concise and readable than C#.

Dim testXml = <test>
                  <product title="Prod1"/>
                  <product title="Prod2"/>
                  <product title="Prod3"/>
              </test>
Dim queryA = From t In testXml...<product> _
             Select New With {.Title = t.@title} '... is any descendant

Auto-Properties quite simply rock. For example:
public class Class1 {
    public string MyString {get; set;}
    public int MyInteger {get; set;}
           }
In comparison to the Vb conversion:
Public Class Class1 
    Private _myString as String
    Private _myInt as Integer
    Public Property MyString as String
        Get
            Return _myString
        End Get
    End Property
    ...


I really like Optional method arguments in Vb.  It means the number of overloads for a method is reduced, resulting in more concise code. Optional arguments are not support currently in C#, and can only be implemented by implementing an overload.

Public Function Foo(x As Integer, Optional y as Integer = 1) as Integer ...
Note: I believe optional parameters are coming in C# 4.
It has to be said that I hate the excessive use of parenthesis in Vb. You can almost hear the chief Vb designer say if in doubt throw in a bracket.  A crazy example of this is Properties that can take arguments!  Why?! Whats the difference then between a property and a method? Its commonly accepted that the whole point of a property is a simple state attribute of an object, it should not change state, and should not contain a significant amount of work. C# definitely wins a clarity point here.



I cannot help but like the convenience of in-line Vb dates:
Dim MyDate As Date = #2/29/2008#
Compared to C#
DateTime myDate = new DateTime(2008, 2, 29); // Or
DateTime myDate2 = new DateTime.Parse("2008/2/29");

C# has array initialisers.
var theList = new List<string>() {"abc", "def", "ghi"};
Vb has no such convenience :-(. Although I believe this is coming in Vb 10.
In Summary, and again this is based on my own personal "frustration-Richter-scale". C# tops Vb with 8 points to 5.  Hence my personal preference for C#.  Each to their own. 

Thursday, November 5, 2009

Dynamic Unit Test Data

5-November-2009

Ever written a test and found yourself copying and pasting it multiple times to pass in different pieces of test data?  Particularly if you have multiple classes that transform the same input data.  I am currently working on some client side libraries that work off transfered Data Transfer Objects (dtos).  So most of the client classes take these dtos as input.  One such class is a Json Adaptor that converts any dto to a string and back again (internally it uses the .Net Json serialiser see DataContractJsonSerializer with some more goodness injected).

To test this class I need to give it a wide range of test data. Including many different types of data, and with each type instantiated with a wide range of values.  I created a library of test data in the Data namespace, with each type in this namespace offering many different instances of the type.


namespace TestData
    public static class AddressDtoDataLibrary {
        private static readonly IList<TestDataGroup<Dto.AddressDto>> Data = new List<TestDataGroup<Dto.AddressDto>>();

        static AddressDtoDataLibrary() {
            Guid id = Guid.NewGuid();
            var serialised = new Dictionary<OutputFormat, string>() {
                { OutputFormat.Xml, GetInput1Xml(id) }, 
                { OutputFormat.Json, GetInput1Json(id) }
            };
            Data.Add(new TestDataGroup<Dto.AddressDto>(GetInput1(id), serialised));

            id = Guid.NewGuid();
            serialised = new Dictionary<OutputFormat, string>() {
                { OutputFormat.Xml, GetInput2Xml(id) },
                { OutputFormat.Json, GetInput2Json(id) },
            };
            Data.Add(new TestDataGroup<Identity.AddressDto>(GetInput2(id), serialised));

            // etc
        }

        public static TestDataGroup<Dto.AddressDto>[] DataArray {
            get {
                return Data.ToArray();
            }
        }

        internal static Identity.AddressDto GetInput1(Guid id) {
            return new Identity.AddressDto() {
                City = "Auckland",
                Country = "New Zealand",
                CreatedOn = new DateTime(2000, 2, 29),
                Id = id,
                Line1 = "24 Rodney Street",
                Line2 = string.Empty,
                Sid = string.Format("{0}@{1}", id, typeof(AddressDtoDataLibrary).Name),
                State = "Auckland",
                Suburb = "Birkenhead",
                Updated = new DateTime(2007, 12, 31, 7, 41, 59),
                Version = id,
                Zip = "2801"
            };
        }
        internal static string GetInput1Json(Guid id) {
            return @"{""City"":""Auckland"",""Country"":""New Zealand"",""CreatedOn"":""\/Date(951735600000+1300)\/"",""Id"":""" + id + @""",""Line1"":""24 Rodney Street"",""Line2"":"""",""Sid"":""" + id + @"@AddressDto"",""State"":""Auckland"",""Suburb"":""Birkenhead"",""Updated"":""\/Date(1199040119000+1300)\/"",""Version"":""" + id + @""",""Zip"":""2801""}";
        }
        internal static string GetInput1Xml(Guid id) {
            return @"<AddressDto>
                <City>Auckland</City>
                <Country>New Zealand</Country>
                <CreatedOn>2000-02-29T00:00:00</CreatedOn>
                <Id>" + id + @"</Id>
                <Line1>24 Rodney Street</Line1>
                <Line2></Line2>
                <Sid>" + id + @"@AddressDto</Sid>
                <State>Auckland</State>
                <Suburb>Birkenhead</Suburb>
                <Updated>2007-12-31T07:41:59</Updated>
                <Version>" + id + @"</Version>
                <Zip>2801</Zip>
            </AddressDto>";
        }
    }
}
I could use the NUnit [Values] attribute, but this requires me to hard code each set of data for each test method. Not to mention its an uglier syntax. Maybe acceptable for basic scalar types and strings, but not for objects imho. If I add more test data, it won't be applied to existing tests.
If I did go down this track it would look like this:


[Test]
public void TestJsonSerialiser(
    [Values(TestData.AddressDtoDataLibrary.DataArray[0].DtoObject)] IDto dtoObject, 
    [Values(TestData.AddressDtoDataLibrary.DataArray[0].Serialised[OutputFormat.Json])] string serialised) {
    /// test code here
}

Using TestDataSource attribute allows you to use a method to gather the testdata programmatically.

Here's the test fixture:



    [TestFixture]
    public class HelperTest {
        private static IEnumerable<Type> dataSourceClasses = new[] {
             typeof(Data.AddressDto), 
             typeof(Data.ClientIdentityAddressDto),
             typeof(Data.ClientIdentityDto),
             typeof(Data.CreateIdentityResponseDto),
             typeof(Data.PingResponseDto),
             typeof(Data.ScreenNameAvailabilityDto),
             typeof(Data.UpdateResponseDto)
        };
        private static object[] TestDataSourceArray;

        static HelperTest() {
            TestDataSourceArray = PopulateTestDataArray();
        }

        private static object[] PopulateTestDataArray() {
            var list = new List<object[]>();
            dataSourceClasses.All(t => {
                list.AddRange(from index in Enumerable.Range(0, HowMuchTestDataDoesThisClassHave(t)) select new object[] { t, index });
                return true;
            });
            return list.ToArray();
        }

        private static int HowMuchTestDataDoesThisClassHave(Type type) {
            var property = type.GetProperty("DataArray", BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static);
            if (property == null) {
                return 0;
            }

            var array = property.GetValue(null, new object[] { }) as object[];
            if (array == null) {
                return 0;
            }

            return array.Length;
        }

        [Test]
        [TestCaseSource("TestDataSourceArray")]
        public void TestJsonSerialise(Type dtoType, int index) {
            var dto = Data.TestDataHelper.GetTestDataItem(dtoType, index);
            Assert.IsNotNull(dto);
            var serialiser = new JsonSerialiser();
            var result = serialiser.Serialise(dto);
            Debug.WriteLine(result);
            Assert.IsNotNullOrEmpty(result);
        }
    }

Suffice to say the GetTestDataItem method in the Test gets the actual test data input from the type passed in. This type argument has a static property that presents an array of test data sets.  Each set contains the object, the matching xml serialisation string, and the matching json string.


public static class TestDataHelper {
        public static IDto GetTestDataItem(Type type, int index) {
            if (type == null) {
                return null;
            }

            if (index < 0) {
                return null;
            }

            try {
                var array = type.GetProperty("DataArray", BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static).GetValue(null, new object[] { }) as object[];
                if (array == null) {
                    return null;
                }

                if (index >= array.Length) {
                    return null;
                }

                var testDataGroup = array[index];
                return testDataGroup.GetType().GetProperty("DtoObject", BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic).GetValue(testDataGroup, new object[] { }) as IDto;
            } catch (Exception ex) {
                throw new NotSupportedException("The test data dto object " + type.Name + " does not have a DataArray or DtoObject property implemented.", ex);
            }
        }
    }

A little crude using reflection, (not cool in heavily used production code) but it works like a charm for testing.

Killed more than two birds with one test (especially since the test data was already written :-) ...

The test output from Nunit looks like this:

Friday, October 23, 2009

Json Serialization

23-Oct-2009
I've currently been experimenting with performance tuning some REST services.  I was surprised how easy it was to change XML serialisation to Json serialisation.  Just a simply matter of changing System.Xml.Serialization.XmlSerializer to System.Runtime.Serialization.Json.DataContractJsonSerializer.

Seems to be about 30% smaller in size over xml.

Serialisation:
            using (var stream1 = new MemoryStream()) {
                var serializer = new DataContractJsonSerializer(dto.GetType());
                serializer.WriteObject(stream1, dto);
                stream1.Flush();
                stream1.Position = 0;
                var reader = new StreamReader(stream1);
                return reader.ReadToEnd();
            }

Deserialisation:
            using (var stream2 = new MemoryStream(Encoding.Unicode.GetBytes(serialisedInstance)) {
                var ser = new DataContractJsonSerializer(type);
                return ser.ReadObject(stream2);
            }

Size comparison:

<AddressDto>
    <City>Auckland</City>
    <Country>New Zealand</Country>
    <CreatedOn>2008-02-29T00:00:00</CreatedOn>
    <Id>c32288fc-5037-4f22-aba5-fca6d10000b3</Id>
    <Line1>24 Rodney Street</Line1>
    <Line2></Line2>
    <Sid>c32288fc-5037-4f22-aba5-fca6d10000b3@AddressDto</Sid>
    <State>Auckland</State>
    <Suburb>Birkenhead</Suburb>
    <Updated>2007-12-31T07:41:59</Updated>
    <Version>c32288fc-5037-4f22-aba5-fca6d10000b3</Version>
    <Zip>2801</Zip>
</AddressDto>
499 bytes.


{"City":"Auckland","Country":"New Zealand","CreatedOn":"\/Date(951735600000+1300)\/","Id":"36322616-0b72-45b4-aa51-4e7119103f27","Line1":"24 Rodney Street","Line2":"","Sid":"36322616-0b72-45b4-aa51-4e7119103f27@AddressDto","State":"Auckland","Suburb":"Birkenhead","Updated":"\/Date(1199040119000+1300)\/","Version":"36322616-0b72-45b4-aa51-4e7119103f27","Zip":"2801"}
367 bytes.

Performance Testing:
I've done a comparison of this Xml to Json change and now have some hard data on how much smaller and faster Json is.

The test involves calling a REST service to first check for a randomly created string LogOnId availability; this is a HTTP GET call. LogOnId must be unique and a user may choose their preferred handle. After the uniqueness check comes back successful, it calls a create client service call. This is a HTTP POST. Finally it calls GET to return the newly created client.

Each call is time individually. 10 threads will be used to create 1,000 new clients. The REST service has been configured to use an in-memory database to minimuse random Database and network delays. All up 10,000 clients should be created.

The test creating the 10,000 clients for Xml and Json were repeated 3 times each to give an average.


Thursday, October 8, 2009

REST Service Mock

8-Oct-2009
Unit testing a class that has a dependency on a external service, can be annoying.  The last thing you want to do is send live calls.  The other options appear to be setting up a test service with a test database, or writing an Echo style service that simply accepts any incoming data and echoes it back.  Neither of these are cool, and require far too much work and plumbing (not to mention slow). The ideal would seem to be the ability to quickly configure and run up a mock http listener to receive requests and give back programmable responses all isolated to the test.

I stumbled across a partial solution here: http://weblogs.asp.net/pglavich/archive/2005/09/04/424392.aspx

I customised, and perhaps simplified for my use case to this: (apologies for the bad copy and paste of my IDE style)

Start at the start, the TestMethod:
        [TestMethod]
        public void SendNotificationTest() {
            var mockService = new MockRestService();
            var subjectUnderTest = new SendNotificationJob() {
                Identity = GetTestData(Guid.NewGuid(), CreateUniqueLogOnId()),
                Url = new Uri(mockService.ListenOnUrl + "TestMethodName")
            };
            mockService.StartAsync();
            subjectUnderTest.SendNotification();
            mockService.Stop();
            Assert.IsTrue(mockService.RequestUrlsReceived.Count == 1);
            Assert.AreEqual(mockService.ListenOnUrl + "TestMethodName"mockService.RequestUrlsReceived[0]);
        }



Secondly the MockRestService class:
    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Net;
    using System.Threading;

    public class MockRestService {
        private readonly HttpListener testServer;
        private bool keepListening = true;
        private Thread listeningThread;

        public MockRestService() {
            string uniqueServiceName = Guid.NewGuid().ToString().Replace("-"string.Empty);
            string serviceUrl = "http://localhost:60340/" + uniqueServiceName + "/";
            this.testServer = new HttpListener();
            this.testServer.Prefixes.Add(serviceUrl);
            this.ListenOnUrl = new Uri(serviceUrl);
            this.testServer.Start();
            this.ResponsesToReturn = new Queue<string>();
            this.StatusesToReturn = new Queue<HttpStatusCode>();
            this.RequestUrlsReceived = new List<string>();
        }

        public Uri ListenOnUrl { getprivate set; }

        public IList<string> RequestUrlsReceived { getprivate set; }

        public Queue<string> ResponsesToReturn { getprivate set; }

        public Queue<HttpStatusCode> StatusesToReturn { getprivate set; }

        public void Stop() {
            this.keepListening = false;
            this.listeningThread.Join();
        }

        public void StartAsync() {
            this.listeningThread = new Thread(() => this.Start());
            this.listeningThread.Start();
        }

        private void Start() {
            do {
                var context = this.testServer.GetContext();
                if (context.Request != null && context.Request.Url != null && !string.IsNullOrEmpty(context.Request.Url.ToString())) {
                    this.RequestUrlsReceived.Add(context.Request.Url.ToString());
                    var writer = new StreamWriter(context.Response.OutputStream);
                    if (this.ResponsesToReturn.Count == 0) {
                        writer.WriteLine("Ok");
                    } else {
                        writer.WriteLine(this.ResponsesToReturn.Dequeue());
                    }

                    writer.Close();
                    if (this.StatusesToReturn.Count == 0) {
                        context.Response.StatusCode = (int)HttpStatusCode.OK;
                    } else {
                        context.Response.StatusCode = (int)this.StatusesToReturn.Dequeue();
                    }

                    context.Response.Close();
                }
            } while (this.keepListening);
        }
    }

Thats basically the crux of it.  If anyone's interested I'll post complete code.


Ben Rees - Oct 25, 2009 12:19 AM
A much tidier revision. Added support for optionally adding headers, and used a event to handle incoming requests.

using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Threading;
using Collections;

/// <summary>
/// A mock to remove dependencies on external REST services. Designed to listen on a Url and Port number using standard Http.
/// </summary>
public sealed class MockRestService : IDisposable {
private readonly HttpListener testServer;
private HttpListenerContext context;

private bool isActive;

private bool keepListening = true;

/// <summary>
/// Initializes a new instance of the <see cref="MockRestService"/> class.
/// </summary>
public MockRestService() {
string uniqueServiceName = Guid.NewGuid().ToString().Replace("-", string.Empty);
string serviceUrl = "http://localhost:60340/" + uniqueServiceName + "/";
this.testServer = new HttpListener();
this.testServer.Prefixes.Add(serviceUrl);
this.ListenOnUrl = new Uri(serviceUrl);
this.testServer.Start();
this.ResponsesToReturn = new EventedQueue<string>();
this.ResponsesToReturn.ItemAdded += (s, e) => {
if (this.isActive) {
throw new InvalidOperationException("You cannot add to the Response Queue after the service has been started.");
}
};
this.StatusesToReturn = new EventedQueue<HttpStatusCode>();
this.StatusesToReturn.ItemAdded += (s, e) => {
if (this.isActive) {
throw new InvalidOperationException("You cannot add to the Status Queue after the service has been started.");
}
};
this.RequestUrlsReceived = new List<string>();
this.ResponseHeaders = new Dictionary<int, KeyValuePair<string, string>>();
}

/// <summary>
/// Gets the URL that has been setup to listen on. The Url is auto-generated to be unique every time this class is instantiated.
/// </summary>
/// <value>The listen on URL.</value>
public Uri ListenOnUrl { get; private set; }

/// <summary>
/// Gets the request urls that have been received. Used to check and assert test case pass.
/// </summary>
/// <value>The request urls received.</value>
public IList<string> RequestUrlsReceived { get; private set; }

/// <summary>
/// Gets the response headers. Keyed by an int zero-based index to indicate which response should be given then header.
/// </summary>
/// <example>
/// To add a header to the second response:
/// using (var serviceMock = new MockRestService) {
/// serviceMock.ResponseHeaders.Add(1, new KeyValuePair("HeaderName", "Some Header content goes in here."));
/// serviceMock.StartAsync();
/// }
/// </example>
/// <value>The response headers.</value>
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Design", "CA1006:DoNotNestGenericTypesInMemberSignatures", Justification = "Assessed and is appropriate here")]
public IDictionary<int, KeyValuePair<string, string>> ResponseHeaders { get; private set; }

/// <summary>
/// Gets the responses to return for each request. Responses are given out in queue order.
/// </summary>
/// <value>The responses to return.</value>
public EventedQueue<string> ResponsesToReturn { get; private set; }

/// <summary>
/// Gets the statuses to return. Statuses are given out in queue order.
/// </summary>
/// <value>The statuses to return.</value>
public EventedQueue<HttpStatusCode> StatusesToReturn { get; private set; }

/// <summary>
/// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
/// </summary>
public void Dispose() {
this.Dispose(true);
}

/// <summary>
/// Starts the listener. This will spin up a asynchronous thread to listen for incoming requests.
/// </summary>
public void StartAsync() {
this.isActive = true;
this.testServer.BeginGetContext(this.RequestReceived, null);
}

/// <summary>
/// Releases unmanaged and - optionally - managed resources
/// </summary>
/// <param name="cleanUpManaged"><c>true</c> to release both managed and unmanaged resources; <c>false</c> to release only unmanaged resources.</param>
public void Dispose(bool cleanUpManaged) {
if (cleanUpManaged) {
this.keepListening = false;
this.isActive = false;
if (this.testServer.IsListening) {
this.testServer.Close();
}
}
}

private void RequestReceived(IAsyncResult result) {
if (!this.keepListening) {
return;
}

this.context = this.testServer.EndGetContext(result);
if (this.context.Request != null && this.context.Request.Url != null && !string.IsNullOrEmpty(this.context.Request.Url.ToString())) {
this.RequestUrlsReceived.Add(this.context.Request.Url.ToString());
int requestNumber = this.RequestUrlsReceived.Count - 1;
var writer = new StreamWriter(this.context.Response.OutputStream);
if (this.ResponsesToReturn.Count == 0) {
writer.WriteLine("Ok");
} else {
writer.WriteLine(this.ResponsesToReturn.Dequeue());
}

if (this.ResponseHeaders.ContainsKey(requestNumber)) {
var header = this.ResponseHeaders[requestNumber];
this.context.Response.AddHeader(header.Key, header.Value);
}

if (this.StatusesToReturn.Count == 0) {
this.context.Response.StatusCode = (int)HttpStatusCode.OK;
} else {
this.context.Response.StatusCode = (int)this.StatusesToReturn.Dequeue();
}

writer.Close();
this.context.Response.Close();
}

if (this.keepListening) { // Could be improved to ensure against race-condition (keepListening may be set to false before all calls can be processed).
this.testServer.BeginGetContext(this.RequestReceived, null);
}
}
}