Making 3rd-party API Libraries more Robust

In this post we'll be combining Generics, Functions and Actions with the Decorator Pattern to solve the following problem:

"Network requests are flaky"

Every time you make a call "across the wire" you've got to provision for errors. Everything from gateway timeouts, server faults right down to your toddler pulling the lan cable out. If it can happen, at some point, it probably will.

Most solutions are very ad-hoc and resemble this:

using (var wc = new WebClient())
{
  for (int i = 0; i < 5; i++)
  {
    try
    {
      wc.DownloadFile(source, target);
      break;
    }
    catch (Exception e)
    {
      Thread.Wait(200);
    }
  }
}

This approach is fine if you only have a couple of calls in your application. In today's hyper-connected world, however, it's more likely that you'll be making lots of these requests with wildly differing signatures. Very quickly your code will become unreadable as you battle to trap all the possible exceptions and manage the retries. What we need is a pattern for easily applying this retry mechanism to any call whilst keeping our client code clean.

Real-world Example

There are a myriad of 3rd party libraries that allow you to target platform APIs e.g. Twitter, Facebook etc. At PhotoPivot.com we rely heavily on Sam and Tim's great FlickrNet library to perform an enormous number of calls to Flickr. Here's a greatly simplified bit of production code from our server-side application:

public class FlickrPhotoDataDownloadService
{
  private readonly IFlickr _flickr;

  public FlickrPhotoDataDownloadService(IFlickr flickr)
  {
    _flickr = flickr;
  }

  public void GetPhotoDataForUser(string userId)
  {
    var photoCollection = _flickr.PhotosGetAllNotInSet();

    foreach (var photo in photoCollection)
    {
      var photoInfo = _flickr.PhotosGetInfo(photo.PhotoId);
      var exifs = _flickr.PhotosGetExif(photo.PhotoId);

      // Do something with the data...
    }
  }
}

Potentially, any of the calls to the IFlickr interface can result in a sporadic network error. This is a serious pain if it happens inside the foreach loop when the routine is 90% through a list of 10,000 photos. If we implemented the ad-hoc solution at the start of this post the GetPhotoDataForUser method would become very untidy and a developer's ability to read it would lessen. So, let's look at some of the related classes. Below you can see the interface that the service uses to describe its requirements. We then used the Adapter Pattern to "bring in" the interface to the main FlickrNet class.

public interface IFlickr
{
  PhotoCollection PhotosGetAllNotInSet();
  PhotoInfo PhotosGetInfo(photo.PhotoId);
  ExifTagCollection PhotosGetExif(photo.PhotoId);
}
public class FlickrEx : FlickrNet.Flickr, IFlickr
{
}

The client code that uses the service looks like this:

var flickr = new FlickrEx();
var service = new FlickrPhotoDataDownloadService(flickr);
service.GetPhotoDataForUser(userId);

So, how do we encapsulate all the methods, or at least the methods we need, in the main Flickr class? We can't rely on just extending the class as it, or its methods, might be sealed. Let's start by creating a class that can keep track on failed attempts and whether a retry is possible.

using System;
using System.Threading;

internal abstract class RetryableBase
{
  private readonly int _maxAttempts;
  private readonly TimeSpan _delay;
  private int _attempts;

  internal RetryableBase(int maxAttempts, TimeSpan delay)
  {
    _maxAttempts = maxAttempts;
    _delay = delay;
  }

  protected bool CanRetry(Exception e)
  {
    _attempts++;

    Thread.Sleep(_delay);

    return (_attempts < _maxAttempts);
  }
}

Next we'll extend this and, by leveraging C#'s Func<> capabilities, create a couple of classes that can wrap a function in some standardised retry routines.

using System;

internal class RetryableFunc<TResult> : RetryableBase
{
  private readonly Func<TResult> _func;

  public RetryableFunc(Func<TResult> func, int maxAttempts, TimeSpan delay)
  : base(maxAttempts, delay)
  {
    _func = func;
  }

  public TResult Call()
  {
    TResult result = default(TResult);

    bool success = false;

    while (!success)
    {
      try
      {
        result = _func();
        success = true;
      }
      catch (Exception e)
      {
        if (!CanRetry(e))
          throw;
      }
    }

    return result;
  }
}

internal class RetryableFunc<T, TResult> : RetryableBase
{
  private readonly Func<T, TResult> _func;

  public RetryableFunc(Func<T, TResult> func, int maxAttempts, TimeSpan delay)
  : base(maxAttempts, delay)
  {
    _func = func;
  }

  public TResult Call(T t)
  {
    TResult result = default(TResult);

    bool success = false;

    while (!success)
    {
      try
      {
        result = _func(t);
        success = true;
      }
      catch (Exception e)
      {
        if (!CanRetry(e))
          throw;
      }
    }

    return result;
  }
}

You will, unfortunately, need to create a new class for each signature function call needed. For instance, the one's above will work for calls that match Func<TResult> or Func<T, TResult> only. Func<T1, T2, TResult> will need another class. Functions that don't return anything are Actions<>. You'll need separate classes for these also. Once done, though, the ROI will be huge.

We now have the capability to make calls robust but how do we 'invisibly' add this to someone else's class? We'll use the Decorator Pattern to create a new class that "is a" IFlickr but also "has a" IFlickr. By initially abstracting our service's needs to an interface we have enabled ourselves to do this very easily.

public class RetryableFlickrEx : IFlickr
{
  private readonly IFlickr _flickr;

  public RetryableFlickrEx(IFlickr flickr)
  {
    _flickr = flickr;
  }

  public PhotoCollection PhotosGetAllNotInSet()
  {
    var retry = new RetryableFunc<PhotoCollection>(_flickr.PhotosGetAllNotInSet);

    return retry.Call();
  }
  public PhotoInfo PhotosGetInfo(string photoId)
  {
    var retry = new RetryableFunc<string, PhotoInfo>(_flickr.PhotosGetInfo);

    return retry.Call(photoId);
  }
  public ExifTagCollection PhotosGetExif(string photoId)
  {
    var retry = new RetryableFunc<string, ExifTagCollection>(_flickr.PhotosGetExif);

    return retry.Call(photoId);
  }
}

Now that we have all our pieces in place the ONLY change we need to make to our entire application is one line in the client. The service still gets the IFlickr it requires and simply doesn't care about the implementation:

var flickrBase = new FlickrEx();

// New line: Use the Decorator Pattern to extend functionality at runtime...
var flickr = new RetryableFlickrEx(flickrBase);

var service = new FlickrPhotoDataDownloadService(flickr);
service.GetPhotoDataForUser(userId);

Please leave comments if you can see a way of enhancing this idea.