Nonblocking Pool Class
November 20th, 2005
This is not an original idea but I thought I would post/explain it anyway. This is a generalized version of a pattern I have been using for a while. I’m not sure where I first picked it up but I’ve seen it used in several places.
The purpose of this class is to pool instances of a particular type in a server application. The assumptions I am making about the problem are:
- It is both possible and worthwhile to reuse instances of a certain type. Types that may fit this criteria are large arrays of primitive types, types that hold unmanaged or scarce resources such as connections, et al. Not all types fit this criteria, obviously.
- It is more undesirable to have a thread enter a waiting state (fail to acquire a lock, in other words) than it is to create a new instance of the type being reused. That would be the case if the instances are somewhat cheap but the average request or call time to your server is relatively long.
The nice thing about this pool class is that it handles the second case gracefully. It will reuse objects as much as possible, but it won’t block a thread in the case that the attempt fails. If it didn’t, you might end up introducing massive contention in your attempt to increase throughput with a different, locking pool.
The class provides very lightweight synchronization using atomic operations - there’s no use of critical sections (the lock keyword).
/// <summary>
/// Provides and reuses objects of type <typeparamref name="T"/>.
/// </summary>
/// <typeparam name="T">
/// The type that is pooled. Must provide a default constructor.
/// </typeparam>
public class NonBlockingPool<T>
where T : new()
{
// Contains the pooled items.
private Stack<T> _stack;
// The maximum size of _stack.
private int _max;
// This reference is used to ensure that only one thread
// calls methods on _stack at a time.
private object _lock = new object();
/// <summary>
/// Gets or sets the maximum size of the pool.
/// </summary>
public int MaximumSize
{
get { return _max; }
set { _max = value; }
}
/// <summary>
/// Gets a pooled instance of type <typeparamref name="T"/>,
/// or yields a new instance.
/// </summary>
public T Get()
{
// If two threads enter this method at the same time,
// only one will acquire _lock (the other will be given
// null). The caller that fails to acquire _lock will
// be returned a new instance of T.
T ret = default(T);
object obj = Interlocked.Exchange(ref _lock, null);
try
{
if (obj != null && _stack.Count > 0)
{
ret = _stack.Pop();
}
else
{
ret = new T();
}
}
finally
{
if (obj != null)
{
_lock = obj;
}
}
return ret;
}
/// <summary>
/// Reuses an instance of type <paramref name="T"/> in a
/// subsequent request or call whenever possible.
/// </summary>
public void Reuse(T t)
{
// If two threads enter this method at the same time,
// only one will acquire _lock (the other will be given
// a null reference). The instance of T provided by
// the losing thread will just be collected and not
// reused.
object obj = Interlocked.Exchange(ref _lock, null);
try
{
if (obj != null && _stack.Count < _max)
{
_stack.Push(t);
}
}
finally
{
if (obj != null)
{
_lock = obj;
}
}
}
/// <summary>
/// Constructor.
/// </summary>
/// <param name="max">
/// The maximum number of instances of
/// <typeparamref name="T"/> to hold in the pool.
/// </param>
public NonBlockingPool(int max)
{
if (max < 0)
{
throw new ArgumentOutOfRangeException("max");
}
_stack = new Stack<T>(max);
_max = max;
}
}
Here’s a (contrived) minimal example of a consumer of such a pool. This server class makes a context object available to each thread for the duration of each request. This object is stored in a slot unique to each thread (specified with the ThreadStaticAttribute) while a ProcessRequest function is called. The instance is returned to the pool in a finally block after that call is finished.
public class SampleServer
{
[ThreadStatic]
private static ServerContext _context;
private NonBlockingPool<ServerContext> _pool;
public static ServerContext Context
{
get { return _context; }
}
internal void ProcessRequest(IServerApp app)
{
try
{
_context = _pool.Get();
app.ProcessRequest();
}
finally
{
if (_context != null)
{
_context.Reset();
_pool.Reuse(_context);
// We want the context to be collected if it isn't
// actually reused by the pool.
_context = null;
}
}
}
}
A more concrete example might be an IHttpModule or a remoting server channel sink. As I said once already, it’s important to consider 1) the type of resource you are pooling and 2) the amount of load your application is expecting before committing yourself to a pattern such as this one.
