Threading

An example of speculative execution

Ryan Schreiber — Thu, 19 Nov 2009 17:53:39 GMT

Speculative execution is a term applied to the execution of code that you may not need the results of. With the abundance of spare processing power on servers these days (and workstations for that matter), you can easily make an application be far more responsive to the user or make processing tasks on a server application quicker.

A contrived example that I've come up: Consider a process that runs periodically on a server that:

Finds a list of all your customers from the SQL Server-based accounting system that have purchased an item from your online store in the past 24 hours (quick DB call) (1 sec in total)
For each customer, sum up the values of all their transactions for the past 6 months - it's a longer process because you have to fetch the data from data warehouse tables on a different SQL Server (2 sec per call)
For customers who have purchased more than ZAR100*, check if they have received a cheesy gift from the online loyalty app that you outsource to. (2 sec per call).
If they haven't received the gift yet, make another call to the online loyalty app to request that a gift be delivered (2 sec)

Most likely you have these running sequentially; looking at the tasks that need to be performed, they all seem to be dependant on the previous step. A quick tally based on a made of set of results returned in each step:

5 customers are returned. Total running time elapsed: 1 sec
Sum up the values of each customer @ 2 seconds/call. Total running time elapsed: 11 sec
Let's say only 2 make it to this step that need to be checked. Total running time elapsed: 15 sec
One has already received a gift, so it's only 1 call left. Total running time elapsed: 16 sec

Here's an example of what the code may look like:

IList<Customer> customers = GetListOfCustomers();

foreach (var customer in customers)

    if (GetPurchaseTotalForSixMonths(customer) > 100)

        if (!HasCustomerReceivedGift(customer))

            RequestGiftForCustomer(customer);

Considering that in every step from the process your server application is waiting for a response from another server, it's most likely idling away. So let's run some of the tasks in parallel. We can run each call made in (2) at the same time as the call that may or may not have been made initially in (3). We can make the call to the online loyalty app, and if the result from the call in two requires the data from the call in (3), you use it. If not, too bad - you just discard the results. Your code may look something like this:

        private static void Speculative()

            IList<Customer> customers = GetListOfCustomers();

            foreach (var customer in customers)

                var c = customer;   //**

                bool meetsPurchaseLimit=false;

                bool hasReceivedGift=false;

                Fork.Begin()

                    .Call(() => meetsPurchaseLimit = (GetPurchaseTotalForSixMonths(c) > 100))

                    .Call(() => hasReceivedGift = HasCustomerReceivedGift(c))

                    .End();

                if (meetsPurchaseLimit && !hasReceivedGift)

                    RequestGiftForCustomer(customer);

The Fork code that you see here is the same as what I've described in the article "An easy to use asynchronous fork".

Re-doing the calculation on the elapsed time, but now using speculative execution on the call made in (3), we get:

5 customers are returned. Total running time elapsed: 1 sec
Sum up the values of each customer @ 2 seconds/call, but at the same time check if the customer received a gift. Total running time elapsed: 11 sec
Marrying up the results from the parallel calls, make the single call for the gift. Total running time elapsed: 12 sec

Back to the example: As you can see, you have a 25% saving in time. Yes, this is a contrived example with naive timing, but the general idea is what matters most. Don't feel bad about using your CPU's idle time to get work done, even if you're going to throw it away later.

* ZAR100 is equivalent to about USD500 or GBP300. Ok, it isn't so. Check here if you really want to.

** You may or may not be wondering why I have the line:

                var c = customer;

It's to remove the warning I get from ReSharper - it complains about a possible "access to a modified closure". Have a look at this article from Eric Lippert explaining it.

An easy to use asynchronous fork

Ryan Schreiber — Fri, 24 Apr 2009 07:15:00 GMT

I’ve been making use of Roger Alsing’s Async Fork for a few months as an alternative to the “for testing purposes only” Microsoft Parallel Extensions. With the release of C# 4.0 on the horizon, I decided to modify the internal workings of the Fork class to make use of the parallel extensions, which would result in very few modifications to my existing code. My reason for the modifications are to take advantage of the inner working of parallels extensions.

Here is the modified version of the Fork class:

    public class Fork

        private readonly IList<Action> _Actions ;

        private Fork()

            _Actions = new List<Action>();

        /// <summary>

        /// Starts an async fork

        /// </summary>

        /// <returns>Returns a new fork instance</returns>

        public static Fork Begin()

            return new Fork();

        /// <summary>

///

        /// </summary>

        /// <param name="action">Delegate that should be executed async</param>

        /// <returns>Returns self</returns>

        public Fork Call(Action action)

            _Actions.Add(action);

            return this;

        /// <summary>

        /// Executes all the calls async and waits untill all of them are finished

        /// </summary>

        public void End()

            var taskManager = new TaskManager();

            Parallel.Invoke(_Actions.ToArray(), taskManager, TaskCreationOptions.Detached);

And making use of the exact same example in the original article, you can use it like this:

            //declare the variables we want to assign

            string str = null;

            int val = 0;

            //start a new async fork

            //assign the variables inside the fork

            Fork.Begin()

                .Call(() => str = CallSomeWebService(123, "abc"))

                .Call(() => val = ExecSomeStoredProc("hello"))

                .End();

            //the fork has finished

            //we can use the variables now

            Console.WriteLine("{0} {1}", str, val);