Pooling example
The idea is to build an executor backed by a pool of
QueuedExecutor: this will show how Spring.NET
provides some useful low-level/high-quality reusable threading and
pooling abstractions.
This executor will provide parallel executions (in our case
grep-like file scans). Note: This example
is not in the 1.0.0 release to its use of classes in the Spring.Threading
namespace scheduled for release in Spring 1.1. To access ths example
please get the code from CVS (instructions) or from the download section of the
Spring.NET website that contains an .zip with the full CVS tree.
Some information on QueuedExecutor is helpful to
better understand the implementation and to possibly disagree with it.
Keep in mind that the point is to show how to develop your own
object-pool.
A QueuedExecutor is an executor where
IRunnable instances are run serialy by a worker
thread. When you Execute with a
QueuedExecutor, your request is queued; at some
point in the future your request will be taken and executed by the
worker thread: in case of error the thread is terminated.
However
this executor recreates its worker thread as needed.
Last but not least, this executor can be shut down in
a few different ways (please refer to the Spring.NET SDK documentation).
Given its simplicity, it is very powerful.
The example project Spring.Examples.Pool provides
an implementation of a pooled executor, backed by n instances of
Spring.Threading.QueuedExecutor: please ignore
the fact that Spring.Threading includes already a
very different implementation of a PooledExecutor:
here we wanto to use a pool of QueuedExecutors.
This executor will be used to implement a parallel
recursive grep-like console executable.
Implementing <literal>Spring.Pool.IPoolableObjectFactory</literal>
In order to use the SimplePool implementation,
the first thing to do is to implement the IPoolableObjectFactory
interface. This interface is intended to be implemented by objects
that can create the type of objects that should be pooled.
The SimplePool
will call the lifecycle methods on IPoolableObjectFactory interface
(MakeObject, ActivateObject, ValidateObject, PassivateObject, and DestroyObject)
as appropriate when the pool is created, objects are borrowed and returned to the pool, and when
the pool is destroyed.
In our case, as already said, we want to to implement a pool
of QueuedExecutor. Ok, here the declaration:
public class QueuedExecutorPoolableFactory : IPoolableObjectFactory
{
the first task a factory should do is to create objects:
object IPoolableObjectFactory.MakeObject()
{
// to actually make this work as a pooled executor
// use a bounded queue of capacity 1.
// If we don't do this one of the queued executors
// will accept all the queued IRunnables as, by default
// its queue is unbounded, and the PooledExecutor
// will happen to always run only one thread ...
return new QueuedExecutor(new BoundedBuffer(1));
}
and should be also able to destroy them:
void IPoolableObjectFactory.DestroyObject(object o)
{
// ah, self documenting code:
// Here you can see that we decided to let the
// executor process all the currently queued tasks.
QueuedExecutor executor = o as QueuedExecutor;
executor.ShutdownAfterProcessingCurrentlyQueuedTasks();
}
When an object is taken from the pool, to satisfy a client request,
may be the object should be activated. We can possibly implement the
activation like this:
void IPoolableObjectFactory.ActivateObject(object o)
{
QueuedExecutor executor = o as QueuedExecutor;
executor.Restart();
}
even if a QueuedExecutor restarts itself as
needed and so a valid implementation could leave this method empty.
After activation, and before the pooled object can be succesfully
returned to the client, it is validated (should the object be
invalid, it will be discarded: this can lead to an empty unusable
pool
You may think that we can provide a smarter
implementation and you are probably right. However, it is not so
difficult to create a new pool in case the old one became unusable.
It could not be your preferred choice but surely it leverages
simplicity and object immutability
).
Here we check that the worker thread exists:
bool IPoolableObjectFactory.ValidateObject(object o)
{
QueuedExecutor executor = o as QueuedExecutor;
return executor.Thread != null;
}
Passivation, symmetrical to activation, is the process a pooled
object is subject to when the object is returned to the pool. In our
case we simply do nothing:
void IPoolableObjectFactory.PassivateObject(object o)
{
}
At this point, creating a pool is simply a matter of creating an
SimplePool as in:
pool = new SimplePool(new QueuedExecutorPoolableFactory(), size);
Being smart using pooled objects
Taking advantage of the using keyword seems
to be very important in these c# days, so we
implement a very simple helper (PooledObjectHolder)
that can allow us to do things like:
using (PooledObjectHolder holder = PooledObjectHolder.UseFrom(pool))
{
QueuedExecutor executor = (QueuedExecutor) holder.Pooled;
executor.Execute(runnable);
}
without worrying about obtaining and returning an object from/to the
pool.
Here is the implementation:
public class PooledObjectHolder : IDisposable
{
IObjectPool pool;
object pooled;
/// <summary>
/// Builds a new <see cref="PooledObjectHolder"/>
/// trying to borrow an object form it
/// </summary>
/// <param name="pool"></param>
private PooledObjectHolder(IObjectPool pool)
{
this.pool = pool;
this.pooled = pool.BorrowObject();
}
/// <summary>
/// Allow to access the borrowed pooled object
/// </summary>
public object Pooled
{
get
{
return pooled;
}
}
/// <summary>
/// Returns the borrowed object to the pool
/// </summary>
public void Dispose()
{
pool.ReturnObject(pooled);
}
/// <summary>
/// Creates a new <see cref="PooledObjectHolder"/> for the
/// given pool.
/// </summary>
public static PooledObjectHolder UseFrom(IObjectPool pool)
{
return new PooledObjectHolder(pool);
}
}
Please don't forget to destroy all the pooled istances once you have
finished! How? Well using something like this in
PooledQueuedExecutor:
public void Stop ()
{
// waits for all the grep-task to have been queued ...
foreach (ISync sync in syncs)
{
sync.Acquire();
}
pool.Close();
}
Using the executor to do a parallel <literal>grep</literal>
The use of the just built executor is quite straigtforward but a
little tricky if we want to really exploit the pool.
private PooledQueuedExecutor executor;
public ParallelGrep(int size)
{
executor = new PooledQueuedExecutor(size);
}
public void Recurse(string startPath, string filePattern, string regexPattern)
{
foreach (string file in Directory.GetFiles(startPath, filePattern))
{
executor.Execute(new Grep(file, regexPattern));
}
foreach (string directory in Directory.GetDirectories(startPath))
{
Recurse(directory, filePattern, regexPattern);
}
}
public void Stop()
{
executor.Stop();
}
public static void Main(string[] args)
{
if (args.Length < 3)
{
Console.Out.WriteLine("usage: {0} regex directory file-pattern [pool-size]", Assembly.GetEntryAssembly().CodeBase);
Environment.Exit(1);
}
string regexPattern = args[0];
string startPath = args[1];
string filePattern = args[2];
int size = 10;
try
{
size = Int32.Parse(args[3]);
}
catch
{
}
Console.Out.WriteLine ("pool size {0}", size);
ParallelGrep grep = new ParallelGrep(size);
grep.Recurse(startPath, filePattern, regexPattern);
grep.Stop();
}