Long running processes: C# 5, Visual Studio 11, async and await; updating a Winforms UI asynchronously

I recently downloaded Visual Studio 11 Beta in order to take a look at the new 4.5 .Net Framework and play around with its new features. I decided to take the opportunity to build a "quick" app that I've been meaning to build for a few years; a single form Windows Client that allows me to send out an email merge to the 1,400 agents I have in my outlook contacts list, to let them know I'm available for work. Before now, every time I've needed to send an "I'm available!" email, I've copied + pasted the same email a number of times, attached my CV and BCC'd 100 or so agents at a time (too many and Outlook just falls over). Not only is it a massive waste of my time, but it's very error prone and not a very friendly way of doing things. Unfortunately, the mail merge feature of Word has its limitations, so I've had to look in to my own solution.

Anyone who's worked with bulk email systems before knows fine well that any kind of mass emailing (solicited or unsolicited) is seen as "spam" by many email servers and ISPs. Some servers are clever enough to spot a mail merge, but most still work on limits. My current server host allows me to send 300 email per hour, which is way more than I intend to send, but I've still got to consider the receiving servers; I have 100+ agent email addresses per agency / domain, so I'm likely to be black listed if I sent them all at once. So, with this in mind, I decided to build an email system that I could configure to fire off emails asynchronously, over a configurable period of time. Here's a quick screenshot - it's not exactly pretty, but it does what I need it to :)

The Email Template drop down requires a little bit of explanation; I'd already written a emailer class for my RustyShark project that accepts XML templates and replaces place holders (wrapped in {curly brackets}) with any data I so wish. All I had to do was build the forms, add in some configuration options and write the threading code.

In my app, I wanted to be able to select a CSV of email addresses and names, then click a button to send the selected template to everyone in the list at various increments. All of this is pretty each to achieve, but as always, long running processes (the app will be running over a number of hours; 1400 contacts x 20 second intervals = 7.8 hours) lock up the UI thread, meaning it's very difficult to monitor their progress. Obviously there's the System.Threading namespace, and the various BeginInvoke() methods scattered around the Framework, but I downloaded the VS11 Beta to learn something, and learn something I did!

Enter the new "async" and "await" C# language features. I spent a while in the Visual Studio Developer Centre watching videos, reading whitepapers and trying examples, then I delved in to my specific requirement. Asynchrony is not an easy task in any language or framework, but the new keywords (as long as you understand what they mean) do help to simplify your code, and help with debugging. Let's jump straight in to an example.

When I click "Send" at the bottom right of the app, I wanted to fire off a foreach() loop, which sends an email, updates the DataGrid, the label next to the send button, and a progress bar, which is not visible in the screenshot above. Though my form has a lot more code (error handling, logging etc), normal, synchronous code would look like this:

private void btnSend_Click(object sender, EventArgs e) {
    // Set up progress bar
    progressBar.Maximum = recipients.Count;
    progressBar.Value = 0;
    progressBar.Step = 1;
    progressBar.Visible = true;
    // Disable form
    ToggleForm(false);

    foreach (var r in recipients) {
        try {
            // Send email
            var msg = new Email();
            if (!msg.Send()) {
                // Email Failure
                r.Error = "Unknown";
                continue;
            }
            // Update recipient
            r.Success = true;
        } catch (Exception ex) {
            r.Error = ex.Message;
            continue;
        } finally {
            // Update progress
            progressBar.PerformStep();
            progressBar.Refresh();
            // Refresh grid
            r.Processed = true;
            dgvRecipients.Refresh();
            // Update label
            UpdateSummaryMessage();
        }

        // Do not wait if there are no more recipients to process
        if (recipients.All(x => x.Success)) continue;

        // Wait this thread for the specified amount of time
        var waitTime = (int)(udSeconds.Value*1000);
        Thread.Sleep(waitTime);
    }
    // Enable form
    ToggleForm(true);

    progressBar.Visible = false;
}

Recipients is a List<Recipient>, which is data bound to the DataGridView. It contains the contact information loaded from the contacts CSV. udSeconds is the Up Down Control on the form that selects the number of seconds to wait between emails. The var msg = Email() line uses my custom Email templating class, and actually needs a lot more code to work, but you get the general idea.

As you can see, I'm using Thread.Sleep() to wait the thread for the number of seconds I've specified with the Up Down Control. I experimented with a timer at this point, but using one would have pretty much negated the need for the async / await keywords I wanted to play with (and required me to use callbacks and invokes to update the UI thread), so I avoided it.

UPDATE: 2nd May 2011 - Since I wrote this post, I found out that Thread.Sleep() should not be used for exact timing. The thread is blocked for a minimum of {n} seconds, after which, control is then passed back to the OS to be scheduled back in for execution. Each version of each OS has it's own idea of what a processor time slice is, so there is no guarantee that the thread will continue execution after the time you have specified. My examples below are still valid, but only as examples. Neither Thread.Sleep(), await Task.Delay() (also new to the Developer Preview) or Task.Wait() should be used for precise scheduling. Contrary to my comment above, there are various timers available in the .Net Framework that are much more reliable than trying to control the thread directly, and should be used for this type of scenario.

In my example, each call to Thread.Sleep() took exponentially longer than the last. As you can imagine, over a number of hours, this built up to quite a lot of additional time. This would be totally unacceptable in a mission critical application.

When the above code is implemented, the UI hangs for however long the task is running (in this case, nearly 8 hours). Even for a small, personal application, this is an unacceptable situation. Making this code asynchronous would normally require quite a bit of refactoring, but the new async / await keywords really speed it up:

private async void btnSend_ClickAsync(object sender, EventArgs e) {
    // Set up progress bar
    progressBar.Maximum = recipients.Count;
    progressBar.Value = 0;
    progressBar.Step = 1;
    progressBar.Visible = true;
    // Disable form
    ToggleForm(false);

    await Task.Run(() => {
        foreach(var r in recipients) {
            try {
                // Send email
                var msg = new Email();
                if(!msg.Send()) {
                    // Email Failure
                    r.Error = "Unknown";
                    continue;
                }
                // Update recipient
                r.Success = true;
            } catch(Exception ex) {
                r.Error = ex.Message;
                continue;
            } finally {
                // Update progress
                r.Processed = true;
                Report(r);
            }

            // Do not wait if there are no more recipients to process
            if(recipients.All(x => x.Success)) continue;

            // Wait this thread for the specified amount of time
            var waitTime = (int)(udSeconds.Value * 1000);
            Thread.Sleep(waitTime);
        }
    });
    // Enable form
    ToggleForm(true);

    progressBar.Visible = false;
}

"That's it?!" I hear you scream? Well, almost. Most examples you'll see online just show you the addition of the two new keywords, and usually have a second method (tagged with the async keyword), called in place of my await Task.Run() line. Task.Run() is new in the .Net 4.5 Developer Preview, and is basically a shortcut to the Task.Factory.StartNew() method from .Net 4.0, only much simpler to use.

So, what's this await keyword all about? In order for any method to use the await keyword within its body, it must be awaitable. In order to make a  method awaitable, you have to tag the async keyword on to the method signature. If you tag async on to a method and do not include the await keyword in the body, the method is always executed synchronously. The await keyword simply tells the compiler that asynchronous code is to be executed, somewhere within the body of the method. Because the code is asynchronous, the method immediately returns to the UI thread and the code inside the Task.Run() block is started on another thread. The interesting thing to understand here, is that the rest of the btnSend_ClickAsync method waits for the asynchronous code to be executed before it is executed itself. This is immensely clever, considering everything looks perfectly synchronous, there are no callbacks specified, and we've hardly changed any code.

I've not only used Task.Run() because it has a cool lambda implementation and I dislike superfluous methods, I've used it to demonstrate that the await keyword can be applied to any method that returns a Task, as well as those that use the async keyword. The Task.Run() method does not use the async keyword, it simply returns a Task. The Task class implements IAsyncResult, which proves that the compiler still uses callbacks in the background, even though you don't have to deal with them yourself.

So, where's this callback? Well, the compiler puts everything that comes after the await Task.Run() line in to a callback (everything under the // Enable form comment), so it's only executed once the asynchronous code is finished. I don't fully understand this bit yet, but notice that the code I've written accesses the UI controls directly, as if they were being executed in the UI thread; the code looks synchronous, but I imagine the callback that's created by the compiler also rewrites these UI calls in a thread-safe manner (possibly using invokes).

The "Async" suffix on the method name does not seem to be required, but is used consistently in examples, blogs and whitepapers. Of course, this makes sense, as you may want to implement both synchronous and asynchronous methods for a single task.

It's also worth briefly mentioning that you can use the result of the Task.Run() method in the callback, to determine a return value from the asynchronous code. Apparently, you're supposed to be able to return a Task<T>, but I've not yet been able to figure out how to do that. The best I've been able to figure out is that I can return a value from the Task.Run() method. Take a look at the following simple example:

    var asyncResult = await Task.Run(() => {
        foreach(var r in recipients) {
            try {
                // Send email
                var msg = new Email();
                if(!msg.Send()) {
                    // Email Failure
                    r.Error = "Unknown";
                    continue;
                }
                // Update recipient
                r.Success = true;
            } catch(Exception ex) {
                r.Error = ex.Message;
                continue;
            } finally {
                // Update progress
                r.Processed = true;
                Report(r);
            }
            // Do not wait if there are no more recipients to process
            if(recipients.All(x => x.Success)) continue;

            // Wait this thread for the specified amount of time
            var waitTime = (int)(udSeconds.Value * 1000);
            Thread.Sleep(waitTime);
        }
        return true;
    });

    if(asyncResult) {
        // Enable form
        ToggleForm(true);

        progressBar.Visible = false;
    }

I've no use for it in my example, but I'm sure I'll need it in the future :)

So, on to the rest of the changes I've made to the above example. The UI update code that was in the finally() block has been moved to the Report() method. This method is provided by the IProgress<T> interface, which is another new interface available in .Net 4.5 Developer Preview. I'm using it here to post to a SynchronizationContext object I created on the main form. I might be missing something, but I think it's provided for convenience and separation only; I don't *think* you need to use the IProgress<T> interface to update the SynchronizationContext , it just makes it a bit neater. An example of the implementation is below:

public partial class AsyncTest : Form, IProgress<Recipient> {
    private readonly SynchronizationContext sc;

    public AsyncTest() {
        InitializeComponent();
        sc = SynchronizationContext.Current;
    }

    public void Report(Recipient value) {
        sc.Post(x => {
            value.Success = true;
            progressBar.PerformStep();
            progressBar.Refresh();
            // Refresh grid
            dgvRecipients.Refresh();
            // Update label
            UpdateSummaryMessage();
        }, value);
    }
}

SynchronizationContext has been in .Net since version 2.0, and allows you to post (asynchronous) or send (synchronous) information back to the UI thread. Report() is being called in place of the UI update code (which would have broken at runtime if we'd left it within the Task.Run() block), which in turn posts the data. I'm fairly confident that you could remove the IProgress<T> implementation and put the sc.Post() code in to the finally() block without it causing too much trouble, though I've not tried it yet.

Within .Net 4.5 there are a number of new Async methods built in to classes for potentially long-running operations, such as reading a Stream or sending an email with System.Net.Mail.SmtpClient(), which are all welcome additions. I imagine they're all just wrappers, though in theory you can create your own Async method for any method in the framework, which is quite exciting.

So, here's a full example of the code above. Please bear in mind that this is a cut-down version of my form code, so it won't work out of the box. It's intended to show you how to implement an asynchronous Winform button using the new async and await keywords, and carry out progress updates without having to specify callbacks:

public partial class AsyncTest : Form, IProgress<Recipient> {
    private readonly SynchronizationContext sc;
    private List<Recipient> recipients;

    public AsyncTest() {
        InitializeComponent();
        sc = SynchronizationContext.Current;
    }

    public void Report(Recipient value) {
        sc.Post(x => {
            value.Success = true;
            progressBar.PerformStep();
            progressBar.Refresh();
            // Refresh grid
            dgvRecipients.Refresh();
            // Update label
            UpdateSummaryMessage();
        }, value);
    }


    private void UpdateSummaryMessage() {
        // update summary message code
    }

    private void ToggleForm(bool enabled) {
        // enable / disable relevant form controls
    }

    private async void btnSend_ClickAsync(object sender, EventArgs e) {
        // Set up progress bar
        progressBar.Maximum = recipients.Count;
        progressBar.Value = 0;
        progressBar.Step = 1;
        progressBar.Visible = true;
        // Disable form
        ToggleForm(false);

        var asyncResult = await Task.Run(() => {
            foreach(var r in recipients) {
                try {
                    // Send email
                    var msg = new Email();
                    if(!msg.Send()) {
                        // Email Failure
                        r.Error = "Unknown";
                        continue;
                    }
                    // Update recipient
                    r.Success = true;
                } catch(Exception ex) {
                    r.Error = ex.Message;
                    continue;
                } finally {
                    // Update progress
                    r.Processed = true;
                    Report(r);
                }
                // Do not wait if there are no more recipients to process
                if(recipients.All(x => x.Success)) continue;

                // Wait this thread for the specified amount of time
                var waitTime = (int)(udSeconds.Value * 1000);
                Thread.Sleep(waitTime);
            }
            return true;
        });

        if(asyncResult) {
            // Enable form
            ToggleForm(true);

            progressBar.Visible = false;
        }
    }
}

I'm afraid I'm no expert with asynchrony, and my understanding is limited - still, I've done my research, so I'm hoping that all of the information I've posted is accurate. If you spot anything that's incorrect, please let me know and I'll update it.

My next task is to put an example together for MVC and / ASP.Net Webforms, to see how useful it'll be. I've been using AJAX for asynchrony for years, so I'm not sure how these methods will fit in the web world at the moment.