resumable functions - async and await
published at 18.08.2013 22:46 by Jens Weller
Save to Instapaper Pocket
While I did my series about the papers for Bristol, there was one paper, which I personally found a bit weird. This paper was about resumable functions, and at that time it was just another paper full of ideas for C++ to me. At C++Now suddenly, I got a better insight to what the use of resumable functions could be. I wasn't expecting Hartmut Kaiser to talk about resumable functions, after all his talk was about HPX, at least I expected this. But at the end, he gave a very nice overview and use case for resumable functions. So finally I could understand the topic to its full extend:
As time passed, BUILD conference came, and Herb Sutter was giving his talk about the future of C++. This talk was full of nice examples of C++11 and C++14. And then again suddenly out of nowhere, resumable functions. Herb is one of the authors of the papers regarding std::future and resumable functions, so more surprising to me was how much time of his talk he spend on it, and that Visual Studio 2013 will have this feature as an addition. Maybe not from the start, but his talk at BUILD made clear, that is a feature to expect much sooner than later in Visual Studio.
Also seeing his presentation gave me more insights at the topic, and I remembered Hartmuts talk, back then in Aspen. I also thought, that it would be nice and worth to blog about this feature, as it's pushed by Microsoft, and would be a nice addition to the language. I'll start with a short spoiler: this is at least a C++1y feature, it won't be in C++14, but there will be a TS for concurrency and parallelism, which resumable functions could become part of. So, this feature might be available on other compilers too, currently its Microsoft stepping ahead with its implementation. That is not a coincidence, as the feature is inspired by async/await from C#.
What are resumable functions?
That is after all the key question, which people need to understand. Before I start to explain what this could be, and how it is defined by the current paper of N3650, I have to take a short stop at futures, as this paper is based on the assumption that at least the .then feature exists for std::future, as N3634 suggests. A future is kind of the result of or a value calculated by a thread or concurrent operation. It is a very basic concept of asynchronous programming. A future returns a placeholder to a value calculated asynchronously by a thread or a threadpool. You then can either call get() to wait for its return, or register a handler with .then, that is executed after it is calculated. One of the errors in C++11 is, that std::future exists without .then. C++14 will hopefully add .then and some other helpfull functions to std::future. This proposal builds up on the assumption that std::future has a .then method. C++11 also brought lambdas to C++, so in combination, this allows for chaining lambdas, or how you also could name it, callbacks in C++. In production code, it is now possible to start a future, and then react through .then to it once calculated. This can lead to chaining callbacks, first read the server response, then parse it, then react to it. With errorchecking and logging in between. This is already common in other languages, as of now it is new to C++. So to some people, this style of building chains of callbacks and asynchronous code is the new goto. Yes, this article deals with C#, but its a very good read to understand why async/await could be a real game changer here.
A short code snipped to illustrate std::future:
std::future<int> f_int = make_dummy_future(42);
int i = f_int.get()//wait for the calculation of the value
f_int.then([](std::future<int> i){/* deal with it */})//register a handler
The idea of resumable functions is, to let the compiler take care and build this chain of futures linked together by continuations and calls to .then. To achieve this, the authors of N3650 propose the introduction of resumable functions. This includes two new keywords: async and await. Please note, that this has nothing to do with std::async, this is not a library solution. The proposal tries to deal with the problem of chaining asynchronous code at the language level. A resumable function is marked with the keyword async, after its functionhead but before its exception specification:
void resumable_function(int i) async
So now the compiler knows, hey that is a resumable function! And the fun starts. While a resumable function is a function, it is also a quite restricted function. You should think of a resumable function as a function specialization for concurrency. The first restriction is the return type, it is not just any returntype, only void and std::future/std::shared_future are allowed. Maybe this could also allow types which are convertible to std::(shared_)future. But implicit conversions aren't the best, so a strict rule for the returntype is maybe favored by the committee. Currently the paper also allows to return T instead of std::future<T>, which then will be converted implicitly to a std::future<T>. Inside of a resumable function things get a little different. Using the keyword await, one can now spawn expressions or functions in a future, which will calculate the expression or call the function in a different thread, using std::future. The keyword await is handled here as a unary operator, which binds with the same rules as the !operator does.
This is, where it gets interesting with resumable functions. The first occurrence from await also lets return the resumable function with a std::future of its later calculated outcome. You can use await as often as you like inside of a resumable function, each time a std::future will be spawned for the expression or functioncall, executing code in parallel. So, the proposal for resumable functions aims at easing the way people can create asynchronous code. Let's see a little example, in his talk, Hartmut Kaiser used fibonacci as an example:
std::future<uint64_t> fibonacci(uint64_t n) async
{
if (n < 2) return std::make_ready_future(n);
std::future<uint64_t> lhs = std::async(&fibonacci, n-1);
std::future<uint64_t> rhs = fibonacci(n-2);
return await lhs + await rhs;
}
This is how a resumable function would look like in code. The wrapping of lhs in a std::future is actually not needed, you can call any function with await, and the compiler will wrap it for you in a std::future. As I wrote earlier, a resumable function is a special type of function, as the first await returns also a future to the caller, things do get complicated. One thing is, that the implementation has to provide a little bit more than just a function stack, which would be destroyed normally by the first await. The implementation will need to ensure that the resumable function is correctly framed for each await, so that all local variables and function parameters are still accessible. But for the caller and the programmer implementing the resumable function this should always stay an implementation detail, which is left to the compiler.
The library solution
I just introduced the idea of resumable functions, and one of my thoughts reading about this the first time was: "can that not be done without introducing new keywords and constraints on the language?" I guess I share this thought with most C++ programmers. And the answer is yes, it can be achieved. There is (almost) no benefit in resumable functions for performance that could not be achieved with a library solution. Key to this is understanding how resumable functions work, and to wrap this into a library solution. As far as I understand, at least HPX has done this. Thomas Heller, one of last years speakers at Meeting C++, has come up with a library solution for HPX. This proves that it is doable without implementing resumable functions as a language feature. Again, as an example fibonacci:
std::future< uint64_t> fibonacci(uint64_t n)
{
if (n < 2) return std::make_ready_future(n);
std::future<uint64_t> lhs_future = std::async(&fibonacci, n-1); //.unwrap();
std::future<uint64_t> rhs_future = fibonacci(n-2);
return
dataflow(
unwrapped([](uint64_t lhs, uint64_t rhs)
{
return lhs + rhs;
})
, lhs_future, rhs_future
);
}
This is how a library solution to resumable functions could look like. But please note, that dataflow will only have the semantics of await if placed as the last statement in a function. Only in this case it can return a future representing the overall result right away, without waiting for all futures to get ready. So with C++11 or C++14, this is already available in libraryland.
So, why change the language and introduce something funky like resumable functions?
As I wrote earlier, there is no direct visible performance advantage for the language solution, still it is a little more elegant, and has its clear advantages. I had a conversation with Hartmut Kaiser about this feature while doing research on it, and he clearly is backing resumable functions, as a good solution. He points out, that a language solution would be able to reuse its stack:
There clearly is a performance advantage for the language based solution, at least in the general case. The language based solution will allow to reuse the stack segment the resumable function was executed on. A library based solution is always required to allocate a separate stack (the exception is the dataflow() construct above when used as the last statement in a function – only in this case it can emulate the callcc semantics implemented by await.)
The advantage of having resumable functions
It isn't only the speed or performance what makes resumable functions so nice to have, as the HPX example of dataflow shows. It is more the syntax and advantages that come with language level solutions what make resumable functions sexy. Asny/await allows for asynchronous function invocations to be embedded into the normal control flow in C++ (if/else, for etc.). The code becomes much clearer, as this example from N3650 shows, first with using std::future:
future<int> f(shared_ptr str)
{
shared_ptr<vector> buf = ...;
return str->read(512, buf)
.then([](future<int> op)// lambda 1
{
return op.get() + 11;
});
}
future<void> g()
{
shared_ptr s = ...;
return f(s).then([s](future<int> op) // lambda 2
{
s->close();
});
}
And with resumable functions:
future<void> f(stream str) async
{
shared_ptr<vector> buf = ...;
int count = await str.read(512, buf);
return count + 11;
}
future g() async
{
stream s = ...;
int pls11 = await f(s);
s.close();
}
So the code using resumable functions becomes shorter, and its much better readable what the code is trying to achieve. But the real advantage comes into play, when combining the asynchronous code with control structures such as if or while. Herb Sutter showed this in his talk at BUILD, I'll show you a simple example he provided in his talk:
std::string read( std::string file, std::string suffix ) {
std::istream fi = open(file).get();
std::string ret, chunk;
while( (chunk = fi.read().get()).size() )
ret += chunk + suffix;
return ret;
}
So, this is a simple example of reading a file asynchrounsly, using future::get() to wait on the async operation in std::future. To speed things up, it would be nice to be able to use .then() instead of .get(). Well, lets see how the code evolves:
task<std::string> read( std::string file, std::string suffix ) {
return open(file)
.then([=](std::istream fi) {
auto ret = std::make_shared<std::string>();
auto next =
std::make_shared<std::function<task()>>(
[=]{
fi.read()
.then([=](std::string chunk) {
if( chunk.size() ) {
*ret += chunk + suffix;
return (*next)();
}
return *ret;
});
});
return (*next)();
});
}
In order to use .then() correctly here, the loop becomes a little challenge. The lambda next calls more or less it self recursive. But Herb explains this in his talk a bit better, then I ever could. The solution using await + .then is much easier to understand, and here resumable functions come to shine to their full beauty, they just feel a bit more naturally in the code then the above construct:
task<std::string> read( std::string file, std::string suffix ) __async {
std::istream fi = __await open(file);
std::string ret, chunk;
while( (chunk = __await fi.read()).size() )
ret += chunk + suffix;
return ret;
}
For both solutions the return value has to be task<std::string> (afaik ppl::task, so it could also be a future), as the returned value is possibly still being calculated. The version using await is a lot clearer as the version using .then(). So, while its possible to this in the library version to, the language version shows that its possible to do this with out the otherwise needed complexity. This code uses the implementation __async and __await, how its going to be integrated into Visual Studio later.
Let's get back to your production code. As production code is, your job is only maintaining it, somebody else might have written it. Looking at a chain of std::future, auto and .then expressions filled with lambdas isn't what you signed up for maybe, but this could be very well where you end up. If the code is well written, it will not be less performant as the same thing done with resumable functions. Still, certain things the compiler could do for you with resumable functions, such as framing and wrapping arguments, needs to be done by you in the library solution. Some parameters and variables need to be created on the heap, as there is no shared stack between the futures. So, it is now at least 1:0 for resumable functions, as otherwise in a library solution, you'll need to take care of things the compiler could do.
Still, for most of us, this would be worth the work doing. So while there is now a clear advantage for resumable functions, it is not the case that one might change the language of C++ for having this. There must be more. And there is. Actually the whole point of resumable functions is to let the compiler do things for you, you otherwise would need a library to do. The library solution, while being as performant as the language solution, has its cons too. Debugging it will be quite hard, and as not all code on earth is perfect, debugging is an important issue. Depending on your code quality, your debugger might ask for a bucket to puke in for debugging the library solution. The compiler has done its work, but a problem can be, that information is lost between the context switches and chains of futures. The debugger might not be able to point you at the error you are looking for in your library solution. While the debugging wizards among us will feel challenged by such situation, the rest of us will fear it. Tracking down errors in such an environment could be quite hard. This could be very well callback hell.
Also, which might already sell resumable functions to most of us, the code using the language feature is shorter and much clearer, then the otherwise possible library solution. One positive result also is that the code is better to maintain. And as the above paragraph shows, the compiler can generate much more information, so that the debugger is able to do a much better job.
As I wrote earlier, a resumable function is a in a certain way restricted function. Especially the return value is restricted to std::(shared_)future or void. This isn't the best thing, as it would be also nice to use boost::future or in Hartmuts case, hpx::future. Maybe this could be achieved over concepts, but with the current proposal resumable functions are restricted to std::(shared_)future or void. Another restriction is that a resumable functions can't use VArgs, for using VArgs a wrapper function would be needed. I'm not sure if this is also the case for variadic templates. Also, the value held by the future must adhere the restrictions std::future does to its possible value types, in practice this means that T in std::future<T> should be copy and/or move constructible.
Future plans
As I wrote earlier this feature will not be part of C++14. It would be such a killer feature for C++14, but (un)fortunately C++14 will not contain any killer features at the language level. C++14 will focus on improving C++11 and adding a few improvements. So this is a C++1y feature. The next big step for resumable functions is to be part of a technical specification (TS), the responsible subgroup in the Committee is WG21. As of now, it seems that there will be one TS for concurrency and two for parallelism. Also, while the syntax and the restrictions to resumable functions is quite an easy one to understand, the implementation at the compiler level isn't. It is debatable, which backend or solution resumable functions should utilize. At the end, this will most likely be implementation defined. As I wrote earlier, a first implementation is going to ship with a CTP for Visual Studio maybe this year. This implementation will use __async and __await as keywords.
Also, and very important to understand this topic is, that this is work in progress. This proposal depends on .then for future, and await could be seen as std::future::get. Also, this feature would make use of the executor pattern, as suggested in N3562.
Opinion
A few words about this from my point of view. I'm not a concurrency guy, so there are smarter people to decide on this. I like the proposal for being in my view elegant in adding parallelism as a language feature. A different take on this could be how Cilk is doing it, which also is a proven way to do concurrency. Still it is my opinion that this way is a little better and hides a little more from the programmer. So this approach promises less code, better maintainability, and maybe less bugs + better debugging. Also the await keyword is fitting nice into the controlflow of C++, enabling the ability to embed concurrency into your code. Of course new keywords could also break existing code, the authors of N3650 have searched the STL and boost for this, and found no case of await breaking code. And async is unique in its position after the functionhead.
So for me currently, the pro site is much better. Still I'm not quite happy, as a resumable function to me is not a full function, its kind of a specialization, a different take. Currently the proposal only does minimal changes to the C++ language itself, so that only defining a resumable function is special, maybe also calling it should be special. Also there is a whole zoo of things that come with functions, which might need to be asked if they are also true for resumable functions. For example, should there be resumable lambdas? I know, that is a mean one. If you are interested in the topic, I'll suggest reading the meeting minutes of WG21 from the July meeting and the current proposal for resumable functions N3650. As I see it, most concerns are currently about implementing usable backends for this. The follow-up paper on this topic will most likely reflect this. So it is work in progress, and Visual Studio will let you play around with a first version. Have fun!
Update from Paper N3722
At the end of August a new paper was published updating the proposal for resumable functions. The first change is that the async keyword is now being replaced with the keyword resumable. This is actually nice, as a resumable function is now also named that way. The meaning of await has not changed.
Also there is now a paragraph about using other future<T> types then std::future. The paper defines that the return-type s<T> must have the interface of future<T>:
- a parameter less get() function which returns T or throws an exception.
- a .then function taking a callable object with a parameter of either a s<T>, s<T>& or const s<T>. The value held by this parameter must be immidiatley available to get().
- an optional bool is_ready() method, returning the state of the future
Further more the authors think, that such a type should define a s<T>::promise_type, which would be exposed to the resumable function implementation. Such a type must provide a set_value(T) and a set_exception(exception_ptr) method. There must be an implicit conversion between s::promise_type and s.
Generators
The paper now also includes a concept for so called generator functions. While STL Algorithms apply to all elements of a sequence and return then, the generator will return immidiately before the first element is being applied. A generator function returns an object of sequence<T>, which the caller can then iterate over. Each iteration invokes the calculation of the next value. The producer will only produce whats and when its data is needed. To achieve this the new keyword yield is introduced:
sequence<int> range(int low, int high) resumable
{
for(int i = low; i <= high; ++i)
{
yield i;
}
}
yield will calculate the value of i when queried by sequence<int>. Each iteration of sequence<int> will execute the function until the next yield is hit. This introduces no concurrency, so range is not being invoked in a different thread. The paper suggests, that yield and await could be combined to achieve concurrency. In this case, await is used to pause the function to wait for generating data, and yield is used to pause for the querying/waiting of/for data.
The paper now includes also the possible wording for the standardization, the proposed wording for an upcoming Technical Specification.
Join the Meeting C++ patreon community!
This and other posts on Meeting C++ are enabled by my supporters on patreon!