Handy LINQ & Lambda Methods and Extensions (Part I)

The System.Linq namespace contains a fantastic set of utility extension methods for filtering, ordering & manipulating the contents of your collections and objects. In the following posts I’ll go through some of the most useful ones (in my humble opinion) and how you might use them in your C# solutions

This is part 1 in a series of posts on Linq & Lambda capabilities in C# 

Before we start, here’s a handy static method to print your resulting collections to the console so you can quickly verify the results.

public class SuperConsole
{
    public static void WriteLine<T>(IEnumerable<T> list, bool includeCarriageReturnBetweenItems =false)
    {
        var seperator = includeCarriageReturnBetweenItems ? ",\n" : ", ";
        var result = string.Join(seperator, list);
        Console.WriteLine(result);
    }
}

Enumerable

The System.Linq.Enumerable type has 2 very useful static methods on it for quickly generating a sequence of items. Enumerable.Range & Enumerable.Repeat. The Range method allows you to quickly generate a sequential list of integers from a given starting point for a given number of items.

IEnumerable<int> range = Enumerable.Range(1, 10);
SuperConsole.WriteLine(range);
//prints "1, 2, 3, 4, 5, 6, 7, 8, 9, 10"

So why is this useful, well you could use it to quickly generate a pre-initialised list of integers rather than new’ing up a list and then iterating over it to populate it. Or you could use it to replicate for(;;) behavior. e.g.

for (int i = 1; i <= 10; i++) 
{     
    //DoWork(i); 
} 

Enumerable.Range(1, 10).ToList().ForEach(i =>
    {
        //DoWork(i)
    });

Repeat is similar but is not limited to integers. You can generate a Sequence of a given length with the same default value in every item. Imagine you wanted to create a list of 10 strings all initialised with a default string of “ABC”;

var myList = Enumerable.Repeat("ABC", 10).ToList();

Item Conversion

There are also a few handy ways to convert/cast items built into the System.Linq namespace. The Cast<T> extension method allows you to cast a list of variables from one type to another as long as a valid cast is available. This can be useful for quickly changing a collection of super types into their base types.

var integers = Enumerable.Range(1, 5);
var objects = integers.Cast<object>().ToList();

Console.WriteLine(objects.GetType());
SuperConsole.WriteLine(objects);

//prints
//System.Collections.Generic.List`1[System.Object]
//1, 2, 3, 4, 5

But what if a valid implicit cast isn’t available. What if we wanted to convert our collection of integers into a collection of strings with a ‘:’ suffix. Thankfully Linq has us covered with it’s ConvertAll Method on List

var integers = Enumerable.Range(1, 5);
var converter = new Converter<int, string>(input => string.Format("{0}: ", input));
var results = integers.ToList().ConvertAll(converter);

SuperConsole.WriteLine(results, true);
/*prints
    1:
    2:
    3:
    4:
    5:
    */

In the next post, we’ll look at some the lazy & deferred execution capabilities of LINQ and some useful methods for performing quick calculations and manipulations on our collections.

~Eoin C

Implementing HTML Formatted Emails in the Enterprise Library Logging Block

Enterprise LibraryThe Microsoft Patterns & Practices Enterprise Library contains a number of useful applications blocks for simplifying things like DataAcces, Logging & Exception Handling in your .NET Applications. Recently we had a requirement to add HTML based formatting to the Email TraceListener in the Logging Application Block, something that’s unfortunately missing from the base functionality. Thankfully, Enterprise Library is an open source code plex project so implementing a custom solution is a relatively trivial task. The email tracelistener functionality is contained in 3 main files: EmailTraceListener – The actual listener which you add to your configuration EmailTraceListenerData – The object representing the configuration settings EmailMessage – The wrapper object around a LogMessage which gets sent via email. Unfortunately because of the the way these classes are implemented in the EnterpriseLibrary Logging Block, they are not easily extended due to dependencies on Private Variables and Internal classes in the EnterpriseLibaray Logging Assembly so they need to be fully implemented in your own solution.

Implementing a Solution

Step 1 was to take a copy of these three files and place them in my own Library Solution. I prefixed the name of each of them with Html; HtmlEmailTraceListener, HtmlEmailTraceListenerData and HtmlEmailMessage. Other code needed to be cleaned up including removing some dependencies on the internal ResourceDependency attributes used to decorate properties within the class & tidying up the Xml Documentation Comments. The main change was then to enable the IsBodyHtml flag on the mail message itself. This was done in the CreateMailMessage method of the HtmlEmailMessage

protected MailMessage CreateMailMessage()
{
	string header = GenerateSubjectPrefix(configurationData.SubjectLineStarter);
	string footer = GenerateSubjectSuffix(configurationData.SubjectLineEnder);

	string sendToSmtpSubject = header + logEntry.Severity.ToString() + footer;

	MailMessage message = new MailMessage();
	string[] toAddresses = configurationData.ToAddress.Split(';');
	foreach (string toAddress in toAddresses)
	{
		message.To.Add(new MailAddress(toAddress));
	}

	message.From = new MailAddress(configurationData.FromAddress);

	message.Body = (formatter != null) ? formatter.Format(logEntry) : logEntry.Message;
	message.Subject = sendToSmtpSubject;
	message.BodyEncoding = Encoding.UTF8;
	message.IsBodyHtml = true;

	return message;
}

Using your new solution

Once implemented it’s simply a matter of reconfiguring your app/web.config logging sections to use the new types you’ve created instead of the original Enterprise Library types. You need to change the type and listenerDataType properties of your Email Listener in the &gl;listeners@gt; section of your config.

<listeners>
      <!-- Please update the following Settings: toAddress, subjectLineStarter, subjectLineEnder-->
      <add name="EmailLog"
           toAddress="toAddress@example.com"
           subjectLineStarter="Test Console - "
           subjectLineEnder=" Alert"
           filter="Verbose"
           fromAddress="fromAddress@example.com"
           formatter="EmailFormatter"
           smtpServer="smtp.gmail.com"
           smtpPort="587"
           authenticationMode="UserNameAndPassword"
           useSSL="true"
           userName="fromAddress@example.com"
           password="Password"
           type="YourLibrary.YourNamespace.HtmlEmailTraceListener, YourLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=0000000000000000"
           listenerDataType="YourLibrary.YourNamespace.HtmlEmailTraceListenerData,  YourLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=0000000000000000"
           traceOutputOptions="Callstack" />
    </listeners>

You’ll also need to ensure that you’ve escaped your Html formatted textFormatter template in the formatters section of your code. i.e. replacing <html> with &lt;html&gt;

<formatters>
      <add name="EmailFormatter" 
              type="Microsoft.Practices.EnterpriseLibrary.Logging.Formatters.TextFormatter, Microsoft.Practices.EnterpriseLibrary.Logging, Version=5.0.414.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" 
              template="
                  &lt;html&gt;
                  &lt;body&gt;
                  &lt;table border=&quot;1&quot; style=&quot;border: solid 1px #000000; border-collapse:collapse;&quot;&gt;
                  &lt;tr&gt;&lt;td&gt;&lt;b&gt;Message&lt;/b&gt;&lt;/td&gt;&lt;td&gt;&lt;b&gt;{message}&lt;/b&gt;&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Local TimeStamp&lt;/td&gt;&lt;td&gt;{timestamp(local)}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Timestamp&lt;/td&gt;&lt;td&gt;{timestamp}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Title&lt;/td&gt;&lt;td&gt;{title}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Severity&lt;/td&gt;&lt;td&gt;{severity}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Category&lt;/td&gt;&lt;td&gt;{category}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Priority&lt;/td&gt;&lt;td&gt;{priority}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;EventId&lt;/td&gt;&lt;td&gt;{eventid}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Local Machine&lt;/td&gt;&lt;td&gt;{localMachine}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;AppDomain&lt;/td&gt;&lt;td&gt;{appDomain}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;LocalDomain&lt;/td&gt;&lt;td&gt;{localAppDomain}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Local Process Name&lt;/td&gt;&lt;td&gt;{localProcessName}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Local Process&lt;/td&gt;&lt;td&gt;{localProcessId}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Win32ThreadId&lt;/td&gt;&lt;td&gt;{win32ThreadId}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;ThreadName&lt;/td&gt;&lt;td&gt;{threadName}&lt;/td&gt;&lt;/tr&gt;
                  &lt;tr&gt;&lt;td&gt;Extended Properties&lt;/td&gt;&lt;td&gt;
                  &lt;table border=&quot;1&quot; style=&quot;border: solid 1px #000000; border-collapse:collapse;&quot;&gt;
                  {dictionary(&lt;tr&gt;&lt;td&gt;{key}&lt;/td&gt;&lt;td&gt;{value}&lt;/td&gt;&lt;/tr&gt;)}
                  &lt;/table&gt;&lt;/td&gt;&lt;/tr&gt;
                  &lt;/table&gt;
                  &lt;/body&gt;
                  &lt;/html&gt;
" />

    </formatters>

All done. Now you can happily send email log messages in HTML format via your Application Logging calls.

~EoinC

Using Signlar to Publish Dashboard Data

SignalR
SignalR

Recently David Fowler announced the release of the Signlar 1.1.0 Beta Release. So I decided to do some dabbling to get a prototype application up and running. The solution is pretty simple. It uses a SignlaR hub to broadcast the current Processor % usage, and renders it in a nice visual graph using HighCharts.

 

[important]The completed solution can be found on GitHub at https://github.com/eoincampbell/signalr-processor-demo [/important]

First things first we’ll need a bare bones web application which we can pull in the relevant nuget packages into. I started with a basic empty web application running under .NET 4.5. Installing the signlar & highcharts packages is a breeze. Open up the PowerShell Nuget Console and run the following commands. HighCharts gets installed as a solution level package so you’ll need to manually copy the relevant JavaScript files to your scripts directory in your application.

Install-Package HighCharts
Install-Package Microsoft.AspNet.SignalR

The Hub

Signalr relies on a “Hub” to push data back to all the connected Clients. I’ve created a “ProcessorDataHub” which implements the Signalr Base Hub to manage this process. It contains a constructor for Initializing a static instance of my ProcessorTicker class, and a start method to start the thread within the ticker. The HubName attribute specifies the name which the hub will be accessible by on the Javascript side.

[HubName("processorTicker")]
public class ProcessorDataHub : Hub
{
    private readonly ProcessorTicker _ticker;

    public ProcessorDataHub() : this(ProcessorTicker.Instance) { }

    public ProcessorDataHub(ProcessorTicker ticker)
    {
        _ticker = ticker;
    }

    public void Start()
    {
        _ticker.Start(Clients);
    }
}

The ProcessorTicker

The heavy lifting is then done by the ProcessorTicker. This is instantiated with a reference to the Clients object, a HubConnectionContext which contains dynamic objects allowing you to push notifications to some or all connected client side callers. The implementation is fairlly simple using a System.Thread.Timer which reads the current processor level from a peformance counter once per second, and Broadcasts that value to the client side.

Since the Clients.All connection is dynamic, calling “updateCpuUsage” on this object will work at runtime, so long as the relevant client side wiring up to that expected method has been done correctly.

Clients.All.updateCpuUsage(percentage);

The Client Side

One change since the previous version of SignalR is the requirement for the developer to manually & explicity wireup the dynamically generated Javascript endpoint where SignalR creates it’s javascript. This can be done on Application Start by calling the RouteTable..Routes.MapHubs() method

protected void Application_Start(object sender, EventArgs e)
{
RouteTable.Routes.MapHubs();
}

Finally we’re ready to consume these published messages on our Client Page. Signlar requires the following javascript includes in the Head Section of your page.

<script type="text/javascript" src="/Signalr/Scripts/jquery-1.6.4.js"></script>
<script type="text/javascript" src="/Signalr/Scripts/jquery.signalR-1.1.0-beta1.js"></script>
<script type="text/javascript" src="/Signalr/signalr/hubs"></script>

With those inplace, we wire up our own custom Javascript function to access our ProcessorTicker, start the Hub on a button click, and begin receiving and processing the

    <script type="text/javascript">
    $(function () {
        var ticker = $.connection.processorTicker;

        //HighCharts JS Omitted..

        ticker.client.updateCpuUsage = function (percentage) {
            $("#processorTicker").text("" + percentage + "%");

            var x = (new Date()).getTime(), // current time
                y = percentage,
                series = chart.series[0];

            series.addPoint([x, y], true, true);
        };

        // Start the connection
        $.connection.hub.start(function () {
            //alert('Started');
        });

        // Wire up the buttons
        $("#start").click(function () {
            ticker.server.start();
        });
    });
</script>

The result is that I can fire up a number of separate browser instances and they’ll all get the correct values published to them from the hub over a persistent long running response. Obviously this an extremely powerful system that could be applied to Live Operations Systems where dash boards have traditionally relied on polling the server at some regular interval.

Live Processor Data to Multiple Browsers via SignalR
Live Processor Data to Multiple Browsers via SignalR

~Eoin C

When’s a Deep Dive not a Deep Dive ?

Global Windows Azure Bootcamp

This weekend, I attended the Global Windows Azure Deep Dive conference in the National College of Ireland, Dublin. Microsoft in conjunction This was a community organised event where Local & National IT Organisations, Educational Institutions & .NET Communities were running a series of events in parallel in a number of cities around the world. The purpose; Deep Dive into the latest technology available on Microsoft as well as take part in a massively parallel lab where participants from all over the world would spin up worker roles to contribute to 3D graphics rendering based on depth data from a KINECT. Alas, Deep it was not, and Dive we didn’t.

I suppose I can’t complain too much. You get what you pay for and it was a free event but I’d have serious reservations about attending this type of session again. Don’t get me wrong, I don’t want to sound ungrateful, and fair dues to the organisers for holding the event but if you’re going to advertise something as a “Deep Dive” or a “Bootcamp” then that has certain connotations that there would actually be some Advanced Hands-on learning.

Instead the day would barely have qualified as a Level 100 introduction to 2 or 3 Windows Azure technologies interspersed with Sales Pitches, Student Demo’s of their project work and filler talks relating to cloud computing in general. Probably most disappointingly we didn’t actually take part in the RenderLab experiment which kinda torpedoed the “Global” aspect of the day as well. You can see the agenda below. I’ve highlighted the practical aspects in Red.

Time Topic
0930 Welcome Dr Pramod – Pathak, Dean, School of Computing, NCI
0935 Schedule for the day  – Vikas Sahni, Lecturer, School of Computing,NCI
0940 How ISIN can help – Dave Feenan, Manager, ISIN
0945 Microsoft’s Best Practice in Data Centre Design – Mark O’Neill, Data Center Evangelist, Microsoft
1000 Virtual Machines – Demo and Lab 1 – Vikas Sahni, Lecturer, School of Computing, NCI
1100 Careers in the Cloud – Dr Horacio Gonzalez-Velez, Head, Cloud Competency Center, School of Computing, NCI
1110 Graduates available today – Robert Ward, Head of Marketing, NCI
1120 Break
1135 Web Sites – Demo and Lab 2 – Vikas Sahni, Lecturer, School of Computing, NCI
1235 Building the Trusted Cloud – Terry Landers, Regional Standards Officer for Western Europe, Microsoft
1300 Lunch
1400 Tools for Cloud Development – Colum Horgan, InverCloud
1410 Windows Azure Mobile Services – Overview and Showcase –  Vikas Sahni, Lecturer, School of Computing, NCI and Students of NCI
1440 Developing PaaS applications – Demo – Michael Bradford, Lecturer, School of Computing, NCI
1530 Break
1545 Windows Azure – The Big Picture – Vikas Sahni, Lecturer, School of Computing, NCI
1645 Q&A

Alas even the practical aspects of the day were extremely basic and the kinda of thing that most people in the room had done/could do in their own spare time.

  • During the Virtual Machines Lab, we spun up a Virtual Machine from the Windows Azure Gallery and remote desktop connected into it.
  • During the Websites Lab, we deployed a WordPress install… unless you were feeling brave enough to do something else. To be fair I hadn’t done a hands on GitHub Deploy of the code so that was interesting.
  • During the PaaS Application Demo… well it was supposed to be a Hello World web/worker role deployment but god love the poor chap he was out of his depth with Visual Studio and had a few technical hiccups and it was just a bad demo. Upshot was we ran out of time before there was an opportunity for any hands on time in the room.

At 15:30 we left… I didn’t have another lecture in me, although at least we’d had the common courtesy to stay that long. Half the room didn’t come back after lunch.

The takeaways; I know that alot of time and effort goes into these events, and particularly when they are free, that time and effort is greatly appreciated. But you need to make sure you get your audience right. If you advertise Advanced and deliver basic, people will be disappointed. That was clear from the mass exodus that occured during the day… I’m kinda curious to know if there was anyone around for the Q&A at all. I’ll be sure as heck checking the agenda on these type of events before committing my time to them in future. We aren’t currently using Windows Azure in our company yet, and embarrassingly I had been promoting it internally and had convinced several of my colleagues to give up their Saturday for it.

~Eoin C

Could not load type ‘System.Runtime.CompilerServices. ExtensionAttribute’ from assembly mscorlib when using ILMerge

Works On My Machine
Works On My Machine

I ran into a pretty horrible problem with ILMerge this week when attempting to build and deploy a windows service I’d been working on. While the merged executable & subsequently created MSI worked fine on my own machine, it gave the following rather nasty problem when run on a colleagues machine.

[error]

Could not load type ‘System.Runtime.CompilerServices.ExtensionAttribute’ from assembly mscorlib

[/error]

It turns out that between .NET 4.0 & .NET 4.5; this attribute was moved from System.Core.dll to mscorlib.dll. While that sounds like a rather nasty breaking change in a framework version that is supposed to be 100% compatible, a [TypeForwardedTo] attribute is supposed to make this difference unobservable.

Unfortunately things breakwhen ILMerge is used to merge several assemblies into one. When I merge my .NET 4.0 app, with some other assemblies on the machine with .NET 4.5 installed, it sets the targetplatform for ILMerge to .NET 4.0. This in turn looks into C:\windows\Microsoft.NET\Framework\v4.0.30319 to find the relevant DLLs. But since .NET 4.5 is an in place upgrade, these have all been updated with their .NET 4.5 counter parts.

Breaking Changes
“Every well intended change has at least one failure mode that nobody thought of”

You need to specific that ILMerge should use the older .NET 4.0 reference assemblies which are still available in C:\Program Files\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0. (or program files x86) if your on a 64-bit box). There’s more info on the stackoverflow question where I finally found a solution and in a linked blog post by Matt Wrock.

http://stackoverflow.com/questions/13748055/could-not-load-type-system-runtime-compilerservices-extensionattribute-from-a

and

http://www.mattwrock.com/post/2012/02/29/What-you-should-know-about-running-ILMerge-on-Net-45-Beta-assemblies-targeting-Net-40.aspx

To override this behavior you need to specify this target platform directory as part of your ILMerge command. e.g.

"C:\Path\To\ILMerge.exe"
    /out:"$(TargetDir)OutputExecutable.exe"
    /target:exe
    /targetplatform:"v4,C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0"
      "$(TargetDir)InputExecutable.exe"
      "$(TargetDir)A.dll"
      "$(TargetDir)B.dll"
I had previously been using the ILMerge.MSBuild.Tasks tool from nuget but unfortunately, this library doesn’t currently support specifying the TargetPlatform. There’s an unactioned open item on their issue tracker in google code.
~EoinC