Maximizing performance in asynchronous programming
Task and Task<TResult>
The Task and Task<TResult> types were introduced in .NET 4.0 as part of the Task Parallel Library (TPL) in 2010, which provided a new model for writing multithreaded and asynchronous code.
For demonstration purposes, let’s build an example:
using System;
using System.Diagnostics;
using System.Threading.Tasks;
public class Program
{
public static void Main()
{
long limit = 2;
var stopwatch = new Stopwatch();
Console.WriteLine("Press 'C' key to exit the loop...");
while (true)
{
if (Console.KeyAvailable && Console.ReadKey(true).Key == ConsoleKey.C)
{
break;
}
limit++;
stopwatch.Start();
// Call the asynchronous method 2_147_483_600
Task<long> maxPrimeTask = MaxPrimeLessThanAsync(limit);
// Wait for the task to complete to prevent the program from exiting prematurely
maxPrimeTask.Wait();
stopwatch.Stop();
// Print the result
Console.Write($"Max prime less than {limit,10} is {maxPrimeTask.Result,10} ");
Console.Write($"task took {stopwatch.ElapsedMilliseconds,5} ms.\r");
}
}
public static Task<long> MaxPrimeLessThanAsync(long number)
{
if (number < 2_000_000_000)
{
return Task.FromResult(MaxPrimeLessThan(number));
}
else
{
return Task.Run(() => MaxPrimeLessThan(number));
}
}
private static long MaxPrimeLessThan(long n)
{
for (long i = n - 1; i >= 2; i--)
{
if (IsPrime(i))
{
return i;
}
}
return -1; // Return -1 if no prime found (i.e., when n is less than 2)
}
private static bool IsPrime(long n)
{
if (n < 2) return false;
for (int i = 2; i * i <= n; i++)
{
if (n % i == 0) return false;
}
return true;
}
}
The example demonstrate how to use the Task class for asynchronous programming. Specifically, it's computing prime numbers, a CPU-intensive operation, asynchronously.
The main method runs an loop, incrementing a limit variable on each iteration. Within the loop, it calls an asynchronous method MaxPrimeLessThanAsync, waits for its result, and then prints this result. It also measures and prints the time taken for each computation and the current memory usage.
MaxPrimeLessThanAsync method uses Task.FromResult or Task.Run to start an asynchronous operation that calculates the maximum prime number less than a given number. It uses Task.Run for larger numbers, and Task.FromResult for smaller ones to optimize performance since smaller ones don’t take long.
The async and await keywords were introduced in .NET 4.5 to simplify the process of working with tasks and asynchronous code. Here is the async version of the example:
public static async Task Main()
{
long limit = 2;
var stopwatch = new Stopwatch();
Console.WriteLine("Press 'C' key to exit the loop...");
while (true)
{
if (Console.KeyAvailable && Console.ReadKey(true).Key == ConsoleKey.C)
{
break;
}
limit++;
stopwatch.Start();
// Call the asynchronous method 2_147_483_600
long maxPrime = await MaxPrimeLessThanAsync(limit);
stopwatch.Stop();
// Print the result
Console.Write($"Max prime less than {limit,10} is {maxPrime,10} ");
Console.Write($"task took {stopwatch.ElapsedMilliseconds, 5} ms.\r");
}
if (enteredNoGCRegion)
{
GC.EndNoGCRegion();
}
}
The method MaxPrimeLessThanAsync(limit) is now awaited, which means the function will be executed asynchronously, and the Main method will be suspended until the task completes. Once the task completes, the Main method resumes execution. The result of the task is directly assigned to the maxPrime variable, instead of being accessed via the Result property of the task.
When you call an async function with await in C#, the compiler does some behind-the-scenes work to manage the call stack and allow your asynchronous code to run without blocking the calling thread. The process involves transforming your async function into a state machine, allowing the function to be paused and resumed at specific points without losing its state.
Here’s a high-level overview of how this works with the stack:
- When you call an async function, the compiler generates a state machine for that function. This state machine includes a structure that holds all the local variables, method parameters, and other necessary information for the function. This structure effectively takes the place of the traditional call stack, allowing the state machine to maintain the function’s state even when it’s paused.
- When the async function reaches an
await expression, it evaluates the awaited task to see if it has already completed. If the task is already completed, the async function continues executing on the same thread and stack, just like a synchronous method call.
- If the awaited task has not yet completed, the state machine saves the current state of the async function, including the values of local variables and the position in the code where the execution is paused. This information is stored in the state machine’s structure, allowing the function to be resumed later without losing its state.
- The calling thread is released, allowing it to continue executing other work while the awaited task is still running. Since the async function’s state is saved in the state machine, there’s no need for the thread to maintain its stack for the async function.
- Once the awaited task completes, the async function is resumed by the state machine, typically on a different thread, but not always. The state machine restores the saved state, including local variables and execution position, and the function continues executing from where it left off.
By using a state machine and saving the state of the async function, the await keyword effectively offloads the management of the function's state from the thread's stack to the state machine. This allows the calling thread to be released and prevents the thread from being blocked while waiting for the asynchronous operation to complete.
Memory
Stack and heap are two types of memory used in program execution. However, the way they operate and their roles within your program’s architecture differ significantly:
Stack
In the Windows operating system, each thread is allocated a fixed amount of stack memory. This allocation occurs upon thread creation. The management of this stack memory is achieved through the use of a stack pointer, which is responsible for controlling the movement of data onto and off of the stack. Specifically, when local variables are introduced in function calls, or when values are returned from functions, the stack pointer adjusts accordingly. This involves merely shifting the stack register pointer up or down, which circumvents the need for allocating or deallocating memory, thereby resulting in more efficient operations.
This system is particularly advantageous because it eliminates the overhead associated with dynamic memory allocation, which typically involves more complex operations like searching for appropriately sized blocks of memory, updating memory allocation tables, and more.
However, the size of the stack is limited, which restricts the amount of data that can be stored on the stack. Large objects or arrays should generally be stored on the heap instead. Also, data created on the stack will be deallocated when a function returns, so stack data is local in nature.
Heap
Contrarily, heap memory is allocated for the entire process. The management of heap memory necessitates a more complex approach. It involves dynamically allocating memory on the heap as required, keeping track of this memory, and deallocating it when it’s no longer needed. In .NET, this deallocation is handled by the Garbage Collector (GC).
This system of memory management is considerably more intricate and typically less performant than stack memory management due to the overhead involved in tracking and managing allocated blocks of memory.
However, the heap provides certain advantages over the stack. Firstly, it’s more spacious, allowing it to accommodate larger objects or arrays. Secondly, since the deallocation is managed explicitly, data on the heap can persist for as long as necessary, making it highly useful for long-lived objects. Finally, because heap memory is allocated for the entire process, it can be shared between threads, offering greater flexibility for multi-threaded applications.
Value Types and Reference Types
In C#, data types are divided into two categories: value types and reference types. These two categories primarily differ in how they are stored and managed in memory.
Value Types
Value types directly contain their data. They are stored in the stack memory, and when you assign one value type variable to another or pass them to methods, the system creates a separate copy of the data.
The basic data types such as int, float, double, char, bool, decimal, byte, struct, and all enumerations (enum) are examples of value types. Also, Nullable types (Nullable<T>) are considered value types.
Reference Types
Reference types, on the other hand, store a reference to the value’s memory address. They are stored in the heap memory, and when you assign one reference type variable to another, they both point to the same memory location. So, if you change one variable, it changes the value at the memory location, which affects all variables pointing to that memory location.
The class, interface, delegate, array, string, and dynamic types are examples of reference types.
The Problem
As the program runs, it continuously allocates memory on the heap. This allocation occurs because each invocation of the method MaxPrimeLessThanAsync causes a new Task object (a reference type) to be created and placed on the heap. In a real-world application, this kind of continuous allocation and potential frequent garbage collection cycles could lead to performance issues.
public static Task<long> MaxPrimeLessThanAsync(long number)
{
if (number < 2_000_000_000)
{
return Task.FromResult(MaxPrimeLessThan(number));
}
else
{
return Task.Run(() => MaxPrimeLessThan(number));
}
}
However, within the MaxPrimeLessThanAsync function, if the input parameter number is less than 2,000,000,000, it actually executes synchronously rather than asynchronously. In this case, it's not necessary to return a Task object, which is essentially a promise to the caller that they can retrieve the result when the computation is complete. We could simply return the value right away since the result is already available at that point.
If we could return a value (a value type) instead of an object (a reference type), that means the return can be managed through the stack instead of the heap. This approach would avoid heap allocations every time the function returns, provided it’s running synchronously.
This leads us to consider a new .NET feature: ValueTask.
ValueTask and ValueTask<TResult>
ValueTask is a structure in C# introduced in .NET Core 2.1 that represents an asynchronous operation. It serves a similar purpose as the Task class, but with some differences in terms of memory allocation and performance. While Task is a reference type that is always allocated on the heap, ValueTask is a value type that can be allocated on the stack, resulting in less memory overhead and better performance in certain scenarios.
public readonly struct ValueTask<TResult> : IEquatable<ValueTask<TResult>>
We can adjust the MaxPrimeLessThanAsync method to return a ValueTask<long> instead of Task<long>. This modification is beneficial in scenarios where the result of the operation is often already available when the method is called, as ValueTask can help avoid unnecessary memory allocations.
public static ValueTask<long> MaxPrimeLessThanAsync(long number)
{
if (number < 2_000_000_000)
{
return new ValueTask<long>(MaxPrimeLessThan(number));
}
else
{
return new ValueTask<long>(Task.Run(() => MaxPrimeLessThan(number)));
}
}
However, the advantage of ValueTask<T> won't extend to the part of the method that runs asynchronously.
return new ValueTask<long>(Task.Run(() => MaxPrimeLessThan(number)));
When the operation is truly asynchronous, meaning the result isn't readily available and a separate thread is needed to complete the operation, we can't simply return a value. The value can still change as the operation executes on a different thread. In such cases, ValueTask<T> wouldn't offer any performance improvement over Task<T>.
Although the return type is ValueTask<T>, it essentially acts as a wrapper around the Task<T> object when the operation is truly asynchronous. In this case, the ValueTask<T> behaves like a reference or a pointer pointing to the actual Task<T> object. The actual computation and the result still reside in the Task<T>.
Now, is there a way to optimize the asynchronous part? That’s where IValueTaskSource comes into play.
IValueTaskSource and IValueTaskSource<TResult>
IValueTaskSource<T> is an interface that allows for the creation of ValueTask<T> and ValueTask instances that can be manually controlled. In other words, it provides a mechanism for you to create tasks that you can signal for completion, error, or cancellation. This can be useful in high-performance scenarios where you need to minimize allocations or need more control over the task lifecycle. By using IValueTaskSource<T>, we can return a value task that allows for the recycling of objects, significantly reducing memory allocations when many operations complete synchronously.
The ManualResetValueTaskSourceCore<T> is a structure in .NET that helps implement the IValueTaskSource<T> or IValueTaskSource interfaces. These interfaces are used in creating user-defined, task-like types that wrap arbitrary operations, such as reading from a network, database calls, or other I/O operations.
The ManualResetValueTaskSourceCore<T> structure is a reusable and resettable core that helps manage the lifecycle of an operation, including triggering completion, retrieving results, and resetting for reuse. It can be used as a field in a class that implements IValueTaskSource<T> or IValueTaskSource, delegating the interface method calls to it.
A conceptual implementation of our example might look like this:
...
public static ValueTask<long> MaxPrimeLessThanAsync(long number)
{
if (number < 2_000_000_000)
{
return new ValueTask<long>(MaxPrimeLessThan(number));
}
else
{
var vts = new ValueTaskSource<long>();
Task.Run(() =>
{
var result = MaxPrimeLessThan(number);
vts.SetResult(result);
});
return new ValueTask<long>(vts, vts.Version);
}
}
...
public class ValueTaskSource<T> : IValueTaskSource<T>
{
private ManualResetValueTaskSourceCore<T> _core; // mutable struct; do not make this readonly
public short Version => _core.Version;
public void SetResult(T result)
{
_core.SetResult(result);
}
public ValueTaskSourceStatus GetStatus(short token)
{
return _core.GetStatus(token);
}
public T GetResult(short token)
{
return _core.GetResult(token);
}
public void OnCompleted(Action<object> continuation, object state, short token, ValueTaskSourceOnCompletedFlags flags)
{
_core.OnCompleted(continuation, state, token, flags);
}
}
The Limitation of ValueTask
There are certain limitations associated with the usage of ValueTask and ValueTask<TResult>:
Should only be awaited
ValueTask and ValueTask<TResult> are designed to be consumed through the await keyword. That's because these types represent operations that are expected to complete asynchronously. Other methods of consumption, such as polling the result or manually manipulating the task, are not recommended as they may lead to incorrect behavior.
Should be awaited only once
A key difference between ValueTask (and ValueTask<TResult>) and the standard Task is that ValueTasks are not designed to be awaited multiple times. They are meant to represent a single operation that will complete in the future, and once that operation is completed and the result is consumed, the ValueTask should not be used again. This is different from Task, which allows for multiple await operations.
Use .AsTask() if you need to break these limitations
If you find that you need to break either of the above two limitations (for example, if you need to await the ValueTask multiple times), you should convert the ValueTask to a Task using the .AsTask() method. This will provide a Task object that can be used according to the usual semantics of Task, which includes support for multiple await operations. However, please note that this may introduce additional overhead as a new Task object needs to be allocated, which negates the performance benefits of using ValueTask in the first place.