One of the common strategies used today in multi-threaded programming is to allocate a thread to handle each incoming request thereby, multiple requests can be handled at the same time. When a new request arrives, the server creates a new thread dedicated to the incoming request. The thread is destroyed when processing is finished.
The above approach is problematic: creating and destroying threads is not optimum. If the operations performed by the thread are not complex, the extra time it takes to create and destroy threads can severely affect performance. Another problematic point if the impact of all those threads in stress conditions. Having each request executing in a new thread will cause the CPU to reach 100% utilization in stress conditions, because most of the time would be sent in context switching. This often results in an exponential increment of response time as the number of requests increases.
The preferred implementation is use a thread pool. When a request arrives, the application adds it to a request queue. A group of pre-created threads retrieve requests from this queue (first in, first out) and process them. When a thread finishes processing, it will begin processing another request from the queue, if one exists. This scheme is shown below:
Given the diagram above, the basic operation of a thread pool is as follows: When requests arrive, they are immediately placed in a queue waiting to be processed. Because initially all threads are free, the first three requests are immediately processed. Once processing of any of these requests is done, a free thread takes the fourth request and processes it. In this scenario, there is no need to create and destroy threads for each request - the threads are reused between requests. If the thread pool implementation is efficient, it can create/destroy threads and adjust the pool size for best performance. For example, if all three threads are processing requests and CPU usage is below 50% (because each request is reading from the disk, for example), the thread pool can detect this and create a new thread to process any pending requests. And in the opposite case, the thread pool can detect if its CPU usage is above 90% and decrease the number of thread to avoid the overhead of excessive context switching.
Based on the above, it is critical that enterprise applications have an effective thread pool implementation. The .NET Framework comes ready with a thread pool ready for use. This thread pool is ready to use not only by applications that want to benefit from thread pooling but it is also integrated with most classes in the .NET Framework. For example, .NET Remoting uses the thread pool to process requests for remote objects. The .NET Framework also uses thread pooling for asynchronous calls, System.Net socket connections, asynchronous I/O, timers, registered wait operations, and others..
When a managed application is executed, the runtime offers a thread pool that will be created the first time the code accesses it. This pool is associated with the physical process where the application is running. This is an important point to remember when you use .NET 's functionality to run different applications (called application domains) within the same process. A bad application monopolizing the pool can affect the entire process because they all share one pool.
The thread pool itself can be accessed using the ThreadPool class in System.Threading namespace. Note that all the members of this class are static and there are no constructors. This is so because there is one pool per process and we cannot create a new one. The purpose of this limitation is to centralize all asynchronous programming in the same pool so that a third party component cannot create a parallel pool that cannot be managed.
You use the thread pool by asking for a function to be executed on the pool, and this is accomplished by calling ThreadPool.QueueUserWorkItem passing in that function.
public static bool QueueUserWorkItem( WaitCallback function, object state );
Where the first parameter specifies the function you want to execute on the pool. The signature of the function must match that of the WaitCallback delegate. The second parameter allows you to pass parameters to the function. Class CTestQueueUserWorkItem in project ThreadPooling shows the basic steps to run a function on the thread pool. The following is the output from that class:
Note how the third request, looking up part TMS320C50 (A Digital Signal Processor by Texas Instruments!) is not started until the request is finished, and more importantly, how this third request uses the same thread used to service request one (the Hash code, or thread ID is 70 in both cases) The reason that the third request was not started because the CPU was at 100% when it was servicing the first two request and it, correctly, decided to wait until a thread is free, instead of creating a new one. This was, there is context switching, and less overhead.
Applications often have a need for certain tasks to occur at regular time intervals. Such a service is implemented by a timer, which is an object that calls back to a pre-determined function at regular intervals. Required processing then takes places inside this function. For example, you may use a timer from anything to showing available disk space to showing stock quotes. A more common use of timers is to implement a watchdog which can periodically check the status of registered components for end-of-trial period, or check the status of hardware devices (for example disconnected printer or even print jobs).
Win32 applications often use the SetTimer API to implement a timer where actions can be performed on a timely basis, perhaps to send a heartbeat to a remote server telling it that the application is up and running. There are a couple of problems encountered with this approach: First, and most important, the timer is based on receiving a Windows message, WM_TIMER, which often leads to inaccurate frequency of receiving timer events., Second, you need to have a window (with its implicit message loop), which means that console applications would have to implement their own message loop with GetMessage and TranslateMessage, etc.
Another approach was to create a class whose thread would sleep for a predetermined amount of time before it calls back a function. This approach made away with the Windows messages, and provided more accurate results, however, each timer needs to create a separate thread that waits the specified amount of time before calling back a function. The cost of this implementation is very high, as each timer needs to have its own thread. Having 10 timers means having 10 threads, one for each.
Another problematic approach would be to use the .NET thread pool to run functions that sleep for a pre-determined amount of time. before calling back a function. In other words, use the approach just described above but make use of the .NET thread pool. Again, this approach has problems: If the pool is full, your request will not be processed, hence the timer will not be accurate. Second, if several timers are created, then several threads from the pool need to be dedicated to those timers, hence affecting the availability of threads in the pool.
So what is the best solution? .NET comes to the rescue by providing time-dependent requests. Here we can have hundreds of timers without using any threads from the pool. It is the pool itself that will process these timers once a timer expires. This functionality is available in three different classes:
All three mechanisms comply with the same set of generic requirements, allowing the application using the timer to:
It is important to understand the difference between these two classes and between the System.Windows.Forms.Timer class. This last is optimized for use in windows, and must be used in a window.
System.Timers.Timer is used by supplying the Elapsed member event with one or more handler functions. System.Timers.Timer will call these handler methods at specified timer intervals using a thread from the thread pool. You can have multiple timers call the same handler method - the sender parameter in the handler can be used to distinguish between different timers. The following code illustrates:
private void btnTimersTimer_Click(object sender, System.EventArgs e)
{
// Create a timer
System.Timers.Timer timer = new System.Timers.Timer(1000);
// Assign the timer a callback function
to handle the Elapsed event
timer.Elapsed += new ElapsedEventHandler( System_Timers_Timer_Handler );
// Start timer (timer function will be called on a different thread)
timer.Start();
// Block to illustrate how the timer function gets called (also verified
// that the timer handler is called on a different thread)
Thread.Sleep( 5000 );
// Stop and close timer
timer.Stop();
timer.Close();
}
// This handler will be called from
the thread pool
private void System_Timers_Timer_Handler( object sender, ElapsedEventArgs args )
{
bool bThreadPool = Thread.CurrentThread.IsThreadPoolThread;
Trace.WriteLine( "Timer handler: " + args.SignalTime.ToLongTimeString() +
" Thread pool: " + bThreadPool.ToString());
}
Output:
Timer handler: 18:53:56 Thread pool: True
Timer handler: 18:53:57 Thread pool: True
Timer handler: 18:53:58 Thread pool: True
Timer handler: 18:53:59 Thread pool: True
On particular interest is System.Timers.Timer.SynchronizingObject property. It allows you to specify an object implementing ISynchronizeInvoke to be used by the timer to callback into the application, instead of directly calling by using a thread from the pool. In other words, the event handler for the timer will be called from the thread associated with the synchronizing object. For example, when the Elapsed event is handled by a Windows Forms, you must make sure that the handler is called by the thread on which the form is running, and not from a thread pool. By setting SynchronizingObject to the Windows Forms object, you ensure that the Elapsed event handler will be called from the same thread that the Form was created on:
public class MyForm : Form
{
public void foo()
{
System.Timers.Timer timer = new System.Timers.Timer(1000);
timer.Elapsed += new ElapsedEventHandler( System_Timers_Timer_Handler );
timer.SynchronizingObject = this;
// Start timer (timer function will be called on a different thread)
timer.Start();
...
}
}
// This handler will be called from
the same thread that carated MyForm
private void System_Timers_Timer_Handler( object sender, ElapsedEventArgs args )
{
bool bThreadPool = Thread.CurrentThread.IsThreadPoolThread;
Trace.WriteLine( "Timer handler: " + args.SignalTime.ToLongTimeString() + " Thread pool: " + bThreadPool.ToString());
}
Output
Timer handler: 19:07:22 Thread pool: False
Timer handler: 19:07:23 Thread pool: False
Timer handler: 19:07:24 Thread pool: False
Timer handler: 19:07:25 Thread pool: False
By default, SynchronizingObject will be set to null, which means that the timer will use threads from the pool. Also note that VisualStudio.NET has built-in designer support for using System.Timers.Timer. Simply drag and drop a Timer from the Components toolbox on to the form - be it Windows Forms, ASP.NET Form or a Web Service Form.
System.Threading.Timer is similar to System.Timers.Timer in terms of using the thread pool to call back on a handler whenever a specified time period elapses. The main difference is that System.Threading.Timer provides fine-grained and advanced control - you can set the timer's due (when it is due to start), and you can pass any generic information to the callback method. If you want to stop the timer you must call its Dispose() method. If you want to restart it, you must create a new timer object. The following example illustrates:
private void btnThreadingTimer_Click(object sender, System.EventArgs e)
{
// Create and start a new timer. Third
parameter indicates that it will start in 0 msec and
// fourth (last) indicates a timer period of 1000 msec
System.Threading.Timer timer = new System.Threading.Timer( new TimerCallback(System_Threading_Timer), this, 0, 1000 );
// Block to illustrate how the timer function gets called (also verifies
// that the timer handler is called on a different thread)
Thread.Sleep( 5000 );
// Stop the timer
timer.Dispose();
}
private void System_Threading_Timer( object sender )
{
bool bThreadPool = Thread.CurrentThread.IsThreadPoolThread;
Trace.WriteLine( "Timer handler: " + DateTime.Now.ToLongTimeString() +
" Thread pool: " + bThreadPool.ToString());
}
Output:
Timer handler: 08:49:28 Thread pool: True
Timer handler: 08:49:29 Thread pool: True
Timer handler: 08:49:30 Thread pool: True
Timer handler: 08:49:31 Thread pool: True
Timer handler: 08:49:32 Thread pool: True
Timer handler: 08:49:33 Thread pool: True
System.Windows,Forms.Timer does not use the thread pool to call back into the Windows Forms application. Instead of using a thread from the thread pool a System.Windows,Forms.Timer posts a WM_TIMER message to the message queue of its current thread at specified intervals.
Because all callbacks are dispatched on the main UI thread, there will be no need to managed concurrency when using Windows Forms timers. On other hand, this may pose problems if the timer's processing is extensive as the user interface will not be responsive.
Class CTestTimers in project ThreadPooling shows the basic steps to use a timer controlled by the thread pool:
Global or shared resources in a multi-threaded environment must have their access synchronized using synchronization objects. Without a thread pool in a multithreaded environment, threads would block waiting for synchronization objects to become signaled. This will ultimately increase the number of threads in a system, leading to performance degradation.
The .NET thread pool supports execution of functions based on synchronization objects. Specifically, the thread pool allows us to queue requests that will only be executed when a particular synchronization object becomes signaled. Optimization is achieved by not allocating any threads to the function while that particular event is not signaled.
This functionality is achieved using the System.Threading.ThreadPool.RegisterWaitForSingleObject function. The first parameter to this function can be any object derived from WaitHandle such as Mutex, ManualResetEvent, or AutoResetEvent. This means that only system synchronization objects can be used - you cannot use any other mechanism like monitors or read-write locks.
Class CTestSyncObject in project ThreadPooling shows how to execute a function in the thread pool based on synchronization objects. Output from this class is shown below:
The most common scenario for using the thread pool is I/O. Most applications need to wait while reading/writing from/to the HD, sending/receiving data on sockets, Internet connections, serial communication and so on. All of these operations share the common limitation (from performance perspective) that they do not use the CPU while they are performed.
All I/O classes in .NET offer the possibility of invoking their methods asynchronously. When the operation is finished, a user-specified function (the call back function) is executed on the thread pool. The performance with this feature is much better, especially if the server has several threads performing I/O operations.
Class CTestIO in project ThreadPooling shows how to perform I/O operations asynchronously. It illustrates how to write a file to the HD asynchronously using FileStream.BeginWrite() by specifying a user-defined function that will be execute on the thread pool once the operations is completed. Note that the IAsyncResult interface can be accessed at any time to query the status of the asynchronous operation. Output from this test class is shown below:
Using the thread pool with Socket I/O is more important because its I/O operations are usually slower that disk I/O. The overall programming pattern for sending/receiving is similar to File I/O, except that Sockets has few additional methods:
Always prefer to use these methods if your server application uses sockets to communicate with other clients. This simplifies the programming model considerably, because instead of needing a thread for each connected client, all the operations will be performed asynchronously on the thread pool.
In the previous disk I/O examples, we did not need any results from the asynchronous operation. The situation is different when accessing the Web. Now we need the response from the Web server when the operations has completed. In order to retireve this information, all .NET classes that offer I/O operations have a pair of methods for each one. In the case of HttpWebRequest, which is the class used to access the Web, the pair is BeginGetResponse and EndGetResponse.
The Endxxx version allows you to retrieve the results of asynchronous operations. In our example, we use EndGetResponse to get the server's response, but we do not call this method at any time. We call it when we know that our asynchronous operations has completed, i.e., when the callback has been called. If EndGetResponse is called before that, the operation will block until it finishes.
Class CWebIO in project ThreadPooling shows how to perform Web operations asynchronously. The following shows a response from class CWebIO.
The ThreadPool class has two methods to monitor the status of the thread pool. Method GetAvailableThreads() allows to retrieve the number of available worker threads and completion port threads. These two types of threads are described below:
Class CTestPoolMonitoring in project ThreadPooling shows how to monitor the thread pool by asynchronously opening a connection to the Web and then sending a GET request. From the screen snapshot below note that connection with a socket uses a worker thread, while sending data uses a completion port thread.
Note that the above also shows that the maximum number of threads for each type is 25. If this maximum is reaches, no more threads will be created and requests will be queued. Note that class ThreadPool offers no way to change this max thread count. This is because the thread pool is a unique shared resource per process.
An incorrect implementation of asynchronous functions executed on the pool can cause the application to hang. Imagine the following scenario: Your code attempts to open a socket with BeginConnect() and then waits for the connection to be established with the EndConnect() method as follows:
public void Connect()
{
...
Socket socket = new Socket( ... );
IAsyncResult ar = socket.BeginConnect( ... );
socket.EndConnect( ar );
...
}
What happens if this function is called on the thread pool? For now, just imagine that the pool size is 2 and that we attempt to connect to a socket using the above function. With both functions executing in the thread pool, there is no room for additional requests until the functions are finished. If the above function somehow ends up launching another asynchronous function, then it will be queued on the thread pool waiting for any thread to be freed. This could never happen because the functions using the pool could be waiting for the queued functions to finish.
In general, we have a deadlock whenever a pool thread is waiting for an asynchronous operation to finish. So if you want to avoid deadlocks in your application do not ever block a thread executed in the thread pool that is waiting for another function in the pool. Note that the last statement implies two more rules:
Finally, to detect a deadlock, always check the number of available threads in the thread pool using ThreadPool.GetAvailableThreads() function. A clear sign of a deadlock is the lack of available threads and a CPU utilization near 0%.
The ThreadPool class has two functions that relate to security: UnsafeQueueUseWorkItem and UnsafeRegisterWaitForSingleObject.They are unsafe with respect to .NET security as they do no maintain the call stack for the asynchronous functions they perform. Unsafe versions are faster but the callback functions will be executed only within the current assembly security policy, losing all permissions applied to the call stack of the assemblies.
Use these functions in extreme caution and only in situations where performance is very important and the security is controlled. Never uses these functions if you are developing a class library that can be used by a third party.