A Short Detour: Executors

A few weeks ago, the member of the C++ standardisation committee Felix Petriconi wrote me an E-Mail. He said my article about std::future Extensions is quite dated. Honestly, he is right. The future of the futures changed mainly because of executors.

Before is write about the future of the futures, I have to introduce the concepts of executors.  Executors have quite a history in C++. The discussion began at least 8 years ago. For the details, Detlef Vollmanns gives in his presentation " Finally Executors for C++ " a great overview.

This post is mainly based on the proposals to the design of executors P0761 , and to their formal description P0443 . This post also refers to the relatively new "Modest Executor Proposal" P1055 .

First of all. What are Executors?

Executors

Executors are the basic building block for execution in C++ and fulfil a similar role for execution such as allocators for the containers in C++. At June 2018 many proposals were written for executors, and many design decisions are still open. The expectation is that they are part of C++23 but can be used much earlier as an extension of the C++ standard.

An executor consists of a set of rules about where , when , and how to run a callable. A callable can be a function, a function object, or a lambda function.

• Where : The callable may run on an internal or external processor and that the result is read back from the internal or external processor.
• When : The callable may run immediately or just be scheduled.
• How : The callable may run on a CPU or GPU or even be executed in a vectorised way.

Because the executors are the building blocks for execution, the concurrency and parallelism features of C++ heavily depend on them. This holds for the new concurrency features inC++20/23 such as extended futures, latches and barriers, coroutines, transactional memory, and task blocks. This holds for the extensions for networking, but also for the  parallel algorithms of the STL .

First Examples

Using an Executor

Here are a few code snippets showing the usage of the executor my_excutor :

• The promise std::async

// get an executor through some means
my_executor_type my_executor = ...

// launch an async using my executor
auto future = std::async(my_executor, [] {
    std::cout << "Hello world, from a new execution agent!" << std::endl;
});

• The STL algorithm std::for_each

// get an executor through some means
my_executor_type my_executor = ...

// execute a parallel for_each "on" my executor
std::for_each(std::execution::par.on(my_executor),
              data.begin(), data.end(), func);

Obtaining an Executor

There are various ways to obtain an executor.

• From the execution context static_thread_pool

// create a thread pool with 4 threads
static_thread_pool pool(4);

// get an executor from the thread pool
auto exec = pool.executor();

// use the executor on some long-running task
auto task1 = long_running_task(exec);

• From the system executor

This is the default executor that usually uses a thread for the execution. It is used if not another one is specified.

• From an executor adapter

// get an executor from a thread pool
auto exec = pool.executor();

// wrap the thread pool's executor in a logging_executor
logging_executor<decltype(exec)> logging_exec(exec);

// use the logging executor in a parallel sort
std::sort(std::execution::par.on(logging_exec), my_data.begin(), my_data.end());

logging_executor is in the code snippet a wrapper for the pool executor.

Goals of an Executor Concept

What are the goals of an executor concept according to the proposal P1055 ?

1. Batchable : control the trade-off between the cost of the transition of the callable and the size of it.
2. Heterogenous : allow the callable to run on heterogeneous contexts and get the result back.
3. Orderable : specify the order in which the callables are invoked. The goal includes ordering guarantees such as LIFO ( L ast I n, F irst O ut),  FIFO   ( F irst  I n,  F irst  O ut) execution, priority or time constraints, or even sequential execution.
4. Controllable : the callable has to be targetable to a specific compute resource, deferred, or even cancelled.
5. Continuable : to control an asynchronous callable signals are needed. These signals have to indicate, whether the result is available, whether an error occurred, when the callable is done or if the callee wants to cancel the callable. The explicit starting of the callable or the stopping of the staring should also be possible.
6. Layerable : hierarchies allow capabilities to be added without increasing the complexity of the simpler use-cases.
7. Usable : ease of use for the implementer and the user should be the main goal.
8. Composable : allows a user to extend the executors for features that are not part of the standard.
9. Minimal : nothing should exist on the executor concepts that could be added externally in a library on top of the concept.

Execution Functions

An executor provides one or more execution function for creating execution agents from a callable. An executor has to support at least one of the six following functions.

Each execution function has two properties: cardinality and direction.

• Cardinality :
  • single : creates one execution agent
  • bulk : creates a group of execution agents
• Direction :
  
  • oneway : creates an execution agent and does not return a result
  • twoway : creates an execution agent and does return a future that can be used to wait for execution to complete
  • then : creates an execution agent and does return a future that can be used to wait for execution to complete. The execution agent begins execution after a given future becomes ready.

Let me explain the execution functions more informally.

First, I refer to the single cardinality case.

• A oneway execution function is a fire and forget job. It's quite similar to a fire and forget future , but it does not automatically block in the destructor of the future.
• A twoway execution function returns you a future which you can use to pick up the result. This behaves similarly to a std::promise that gives you back the handle to the associated  std::future .
• A then execution it is a kind of continuation. It gives you back a future, but the execution agent runs only if the provided future is  ready .

Second, the bulk cardinality case is more complicated. These functions create a group of execution agents, and each of these execution agents calls the given callable. They return the result of a factory and not the result of a single callable f invoked by the execution agents. The user is responsible for disambiguating the right result via this factory.

execution::require

How can you be sure that your executor supports the specific execution function?

In the special case, you know it.

void concrete_context(const my_oneway_single_executor& ex)
{
    auto task = ...;
    ex.execute(task);
}

In the general case, you can use the function execution::require to ask for it.

template <typename Executor>
void generic_context(const Executor& ex)
{
    auto task = ...;

    // ensure .twoway_execute() is available with execution::require()
    execution::require(ex, execution::single, execution::twoway).twoway_execute(task);
}

In this case, the executor ex has to be a single cardinality and twoway direction executor.

What's next?

In the next post, I will continue my detour from the C++ core guidelines. The future of the futures changed mainly because of the executors; therefore I will write about the futures.

Get your e-book at leanpub:

The C++ Standard Library

Concurrency With Modern C++

Get Both as one Bundle

With C++11, C++14, and C++17 we got a lot of new C++ libraries. In addition, the existing ones are greatly improved. The key idea of my book is to give you the necessary information to the current C++ libraries in about 200 pages.

C++11is the first C++ standard that deals with concurrency. The story goes on with C++17 and will continue with C++20 .

I'll give you a detailed insight in the current and the upcoming concurrency in C++. This insight includes the theory and a lot of practice with more the 100 source files.

Get my books " The C++ Standard Library " (including C++17) and " Concurrency with Modern C++ " in a bundle.

In sum, you get more than 550 pages full of modern C++ and more than 100 source files presenting concurrency in practice.