Serialising a Linq Query to Json

IEnumerable<Employee> empJson = from emp in employees
                                where emp.Department == "IT Department"
                                select emp;
var ser = new DataContractJsonSerializer(typeof(IEnumerable<Employee>));
using (stream = new MemoryStream()) {
    ser.WriteObject(stream, empJson);
    string json = Encoding.Default.GetString(stream.ToArray());
    return json;

}

Linq References

Handy Linq References:

MSDN Linq 101

LinqPad

LINQ Cookbook

Parallel Extensions

31-August-2010

The new Task Parallel Library included in .NET4 is an incredibly easy way to parallelise processing, that would otherwise have to be done with such devices as SpinLock, Semaphores, Monitors and others.  Unfortunately those previous devices were crazy easy to get wrong, and remember how they worked after 6 months has elapsed.

One of my favourite sessions at TechEd this year showed in some detail how to Parallise various code fragments using the new constructs.  

Parallel Loops

Loops are the easiest place to start (shortly followed thereafter with reconsidering all LINQ statements).

namespace TPLTest1
{
    using System;
    using System.Threading;
    using System.Threading.Tasks;

    public static class Program
    {
        public static void Main(string[] args)
        {
            Action a1 = () =>
            {
                Thread.Sleep(500);
                Console.WriteLine("Action1");
            };
            Action a2 = () =>
            {
                Thread.Sleep(500);
                Console.WriteLine("Action2");
            };
            Action a3 = () =>
            {
                Thread.Sleep(500);
                Console.WriteLine("Action3");
            };
            Action a4 = () =>
            {
                Thread.Sleep(500);
                Console.WriteLine("Action4");
            };
            Parallel.Invoke(a1, a2, a3, a4);
            Console.WriteLine("Finished");

            Parallel.For(0, 4, index =>
                {
                    Thread.Sleep(500);
                    Console.WriteLine("Enumerator " + index);
                });
            Console.WriteLine("Finished For");
        }
    }
}

The handy thing about these new For constructs are the threads will be synchronised back into the main thread after the loop. Here's the output:







Action1

Action2

Action3

Action4

Finished

Enumerator 2

Enumerator 1

Enumerator 0

Enumerator 3

Finished For

Press any key to continue . . .







PLINQ

To be able to effectively and safely filter a collection and copy results into a new collection you used to have to do something like this:







IEnumerable<RaceCarDriver> drivers = …;
var results = new List<RaceCarDriver>();
int partitionsCount = Environment.ProcessorCount;
int remainingCount = partitionsCount;
var enumerator = drivers.GetEnumerator();
try {
    using (var done = new ManualResetEvent(false)) {
        for(int i = 0; i < partitionsCount; i++) {
            ThreadPool.QueueUserWorkItem(delegate {
                while(true) {
                    RaceCarDriver driver;
                    lock (enumerator) {
                        if (!enumerator.MoveNext()) break;
                        driver = enumerator.Current;
                    }
                    if (driver.Name == queryName &&
                        driver.Wins.Count >= queryWinCount) {
                            lock(results) results.Add(driver);
                    }
                }
                if (Interlocked.Decrement(ref remainingCount) == 0) done.Set();
            });
        }
        done.WaitOne();
        results.Sort((b1, b2) => b1.Age.CompareTo(b2.Age));
    }
}
finally { if (enumerator is IDisposable) ((IDisposable)enumerator).Dispose(); }





Now you do this:





var results = from driver in drivers
              where driver.Name == queryName &&
                    driver.Wins.Count >= queryWinCount              
              orderby driver.Age ascending
              select driver;


Crazy easy.



One of the few options you might need to consider when using PLINQ is the partitioning algorithm used.  Here's a great slide from Ivan Towlson session at TechEd NZ showing the different algorithms in action:





I believe the Chunking algorithm is the default, and usually is an ok choice for most things. Except if you are looking for a certain grouping of data, for example searching a list of people and processing the first one of a series of duplicates, then you should use the Hash algorithm.



Tasks

Best described by two more slides from the same session:









Awesome.

PLINQ

Here's a summary of some great new (and some improved) features in LINQ and most importantly PLINQ.

Source: http://msdn.microsoft.com/en-us/magazine/ee819128.aspx

PLINQ will actually be slower than LINQ to objects if there is no expensive filtering, or projection process.  Here are some examples when using PLINQ is a good idea.

Where selecting has a time consuming process:

IEnumerable<int> src = ...
var query = 
      src.AsParallel()
      .Where(x => x % 2 == 0)
      .Select(x => Foo(x));

Using ForAll with an expensive projection function and possibly an expensive loop:

int[] src = Enumerable.Range(0, 100).ToArray();
var query = src.AsParallel()
             .Select(x => ExpensiveFunc(x));

int resultSum = query.ForAll(
      x => Console.WriteLine(x)
);

Using a Partitioner to ensure load balancing:

int[] src = Enumerable.Range(0, 100).ToArray();
var query = Partitioner.Create(src, true).AsParallel()
             .Select(x => ExpensiveFunc(x));

foreach(var x in query)
{
      Console.WriteLine(x);
}

An expensive filter:

int[] src = Enumerable.Range(0, 100).ToArray();
var query = src.AsParallel()
             .Where(x => ExpensiveFilter(x));

foreach(var x in query)
{
      Console.WriteLine(x);
}

Sequence zipping (takes two elements from two input arrays and outputs one resulting element):

int[] arr1 = ..., arr2 = ...;
int[] results =
      arr1.AsParallel().AsOrdered()
      .Zip(
           arr2.AsParallel().AsOrdered(),
           (arr1Elem, arr2Elem) => ExpensiveFunc(arr1Elem, arr2Elem))
      .ToArray();

A reduction extension method:

public static double Average(this IEnumerable<int> source)
{
      return source.Aggregate(
             () => new double[2],
             (acc, elem) => {
                   acc[0] += elem; acc[1]++; return acc;
             },
             (acc1, acc2) => {
                   acc1[0] += acc2[0]; acc1[1] += acc2[1]; return acc1;
             },
             acc => acc[0] / acc[1]);
}