Using Performance Counters in .NET

Why use performance counters?

I have good logging you say, excellent traceability in my database, if need be I can attach Ants Profiler, I don't need performance counters.  Not true. Every application that runs on a server and serves multiple users should be load tested.  A few definitions first:
Load testing is not using the Performance Analyser in Visual Studio or Ants Profiler.  This is merely looking for memory leaks and bad performance for a single user.  Still good to do, but doesn't replace load testing.
To be able to load test and understand the results you need to be able to measure throughput and the most likely individual performance of specific tasks while the rest of the system is under load.
Performance counters are the easiest way to monitor throughput and performance of a running system in real time.  It could be achieved with logging to files or database, but why bother? Most load testing tools capture performance monitor data and later analysis, and previous test data doesn't pollute later tests.

What else are they good for? 

Even for single user applications performance counters can be a convenient way to measure individual intensive tasks within the application.  Its also a good way to measure if running tasks in parallel is better than not.

Why not, they're easy to use and there's some free stuff

There's a huge array of standard performance counters that come with the OS and with .NET, that are very useful.  Counters to measure processor utilisation, number of logical threads created, memory usage, GC stats, and ASP.Net requests per second, and % of WCF throttles used.  Basically only code that you have written inside your app won't be measured.  Writing your own performance counters is dead easy, and there isn't much code that invades into your source to make it happen. 

var perfCounter = new PerformanceCounter("My App Name", "Counter 1", false);
perfCounter.Increment();

Watching Performance Counters

To view performance counters and watch live data coming through use PerfMon. This is part of Windows OS.

Gotchas:

1. You may need to reboot after installing a performance counter with the OS.  It may appear in the PerfMon list, but no data comes through.  A reboot will likely fix this, it did for me.
2. Admin priviledges will be required to install custom performance counters into the OS and delete them.  This works in code, but isn't a great idea to run / require production apps to use admin credentials.  To update performance counter data the app only needs to be a member of the Performance Log Users Group.
3. There are some random gaps sometimes in the counters, but in my experience never more than a few minutes.

Download my sample code

Other References:

• Code Project Introduction Article
• MSDN Performance Counter Object
• Using Performance Counters Article

WCF Service Diagnostics Part 2

In the previous post on this topic I gave an outline of the most common tools used to look into problems with WCF.  In this post I'll give some more information on more advanced tools and approaches for more insidious problems.

SOAP Xml verfication.
It is quite common to use WCF services as the boundaries between different parties or boundaries between responsibilities of teams.  Just like defining an interface so two developers can work on either side in parallel, one developing the interface implementation and the other the consumer; so can service WSDL be used.  In this scenario parties often swap SOAP xml requests and responses as files and check them against their own code.

Tools that can be used are:

• SOAP UI. Basically this tool allows you to copy and paste SOAP Xml into its UI and then send it to a service.  It will also allow you to see the response SOAP.  Obviously however it is specific to the SOAP protocol and HTTP. It does appear to have loads of other functionality that could be useful particularly for JAVA based diagnostics.
  
• Fiddler can be used as well.  I have used it to look and record the HTTP traffic before, but apparentely you can also use it as a proxy to return preconfigured responses as well (although I personally haven't tried this).
• WsdlSoapXmlValidator. This is a rough tool I have written to take request and response SOAP xml and verify them against the embedded XSD schemas within WSDL (it assumes the WSDL includes Schema and will not work otherwise) and then host a dummy service compliant with the WSDL and using the request and response xml against the service to test actual WCF deserialisation of the samples.
  
• WireShark is another tool I have reached for on odd occasions to view network traffic. This is useful for TCP/IP and binary based protocols.
• WCF Load Test Tool. This is a free codeplex tool written by the Visual Studio ALM Rangers.

WCF Service Diagnostics Part 1

Sometimes its useful to be able to examine the raw soap messages coming into and out of a service.  Especially when trying to interface into a third party service or client. Quite often all you get is WSDL and not much else.

Back in the days of ASMX Web Services you had to write your own SoapExtension to be able get access to the SOAP/XML. Including all the tedious IO mechanics of writing the log information to an external source. WCF has a built in logging mechanism that just needs to be turned on.  In addition WCF also has its own log viewer.

Enabling WCF Logging
This requires a few modifications to your web.config.  Alternatively you can use the WCF Service Configuration Editor tool from Visual Studio's Tools menu (or located on your disk at: <Program Files (x86)>\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools)

<?xml version="1.0"?>
<configuration>
    <system.diagnostics>
        <sources>
            <source 
                name="System.ServiceModel.MessageLogging" 
                switchValue="Off,ActivityTracing">
                <listeners>
                    <add 
                        type="System.Diagnostics.DefaultTraceListener" 
                        name="Default">
                        <filter type="" />
                    </add>
                    <add name="ServiceModelMessageLoggingListener">
                        <filter type="" />
                    </add>
                </listeners>
            </source>
        </sources>
        <sharedListeners>
            <add 
                initializeData="C:\web_messages.svclog" 
                type="System.Diagnostics.XmlWriterTraceListener, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
                name="ServiceModelMessageLoggingListener" 
                traceOutputOptions="Timestamp">
                <filter type="" />
            </add>
        </sharedListeners>
        <trace autoflush="true" />
    </system.diagnostics>
    
    <system.web>
        <compilation debug="true" targetFramework="4.0" />
    </system.web>
    
    <system.serviceModel>
        <diagnostics>
            <messageLogging 
                logEntireMessage="true" 
                logMalformedMessages="false"
                logMessagesAtServiceLevel="true" 
                logMessagesAtTransportLevel="false" />
        </diagnostics>

...

Microsoft Service Trace Viewer
To view a log file double-clicking the file will open it if you leave the extension as *.svclog, The viewer is installed as part of Visual Studio 2010 and will be located on your disk <Program Files (x86)>\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools).

The complete soap message envelope is shown inside the application data element.  On the left pane use the messages tab to show only messages and look for the items that have a named action. This is the action name used in the OperationContract attributes. Or if left blank it will be as shown above using the TempUri namespace.

Recommended Settings for Production and Development Debugging etc are recommended by Microsoft here:
http://msdn.microsoft.com/en-us/library/aa702726.aspx


WCF Test Client
Located at <Program Files (x86)>\Microsoft Visual Studio 10.0\Common7\IDE) and is installed as part of Visual Studio. This is a great simple utility for quickly testing services of any kind.  In fact it will auto start when using the Visual Studio WCF Application project template.

Its great for quick testing of services and viewing request and response xml.

Packet Sniffing
http://www.pocketsoap.com/tcptrace/
This is useful for sniffing packets at a lower level particularly when you are writing a client that is interfacing into a third party non-Http service.

For Http services it will probably be easier using Fiddler.

References:

http://msdn.microsoft.com/en-us/library/ms730064.aspx

http://weblogs.asp.net/nmarun/archive/2011/04/13/service-trace-viewer-how-to-use.aspx

http://geekswithblogs.net/mnf/archive/2008/10/03/use-wcf-message-logging.aspx

http://msdn.microsoft.com/en-us/library/aa702726.aspx

Memory Leak Diagnostics

More often than not the occurance of memory leaks seem to be related to bad coding practises and sloppy style.  (For example not fastidiously unsubscribing to events when you're finished with them). Prevention is always better than an ambulance at the bottom of a cliff for sure.  Obviously though, no one is perfect and mistakes will be made, get through peer review and be checked in.

It is surprising though how good the GC actually is, I'm certain that there is loads of sloppy code checked in all the time and more often than not the GC does a better than fair job cleaning up. Isn't managed code grand, allowing us to spend more time writing product. ;-)  Sometimes though, it is going to be unable to detect an object or graph of objects are no longer needed.

When this happens you need to be able to profile what is going on a great detail inside memory and the GC.
SOS.DLL is an assmebly provided by the .NET framework for this purpose exactly. Check out this article on Code Project to find out how to use it.

http://www.codeproject.com/KB/dotnet/Memory_Leak_Detection.aspx

Thanks Marjorie.