By Lieven De Cock, C++ consultant, coach and trainer at CppDriven. Contributor to Code::Blocks. | Edited by Saqib Jan - Read the full editorial note on this write-up at the tail end.

Inside C++ Coroutines How They Really Work

4.54MB ∙ PDF file

Download

Lieven's slides from the session. The full recording is above if you would rather watch it.w

Download

I have something like thirty plus years of experience in C++ development, in different areas and different roles. I have been following the evolution of C++ closely, and at some point I changed the goal of my career to helping others tag along with that evolution. That is why I am now an independent consultant and coach, teaching not just the bare language and library but also the tooling and ecosystem around it, so teams can write more efficient and cleaner code. That is my mission for the rest of my career.

C++20 had the big four. Modules, ranges, concepts, and coroutines. There was a lot of fuss, and everybody had high expectations.

If we imagine coroutines as a nice book cabinet where you can store your books, that is what we were expecting. The reality is that in C++20 we got the build-it-yourself kit. That is one of the things a lot of people do not understand. Coroutines in C++20 are a language fundamental feature which allows you to build things, and that is also what we are going to do here. We are going to build up several coroutines, and we will see that we create a lot of boilerplate we would rather avoid.

How do we avoid that boilerplate in future? We use libraries that have support for coroutines. Boost.Asio, for example. We will look at a little example of that at the end.

So buckle up. We have some construction work to do.

An assembled bookcase beside the same five panels lying flat and disassembled, showing what C++20 coroutine users expected against the kit they received — flat-pack

Coroutines are not a multithreading feature

First, some myths. If you mention coroutines, people start anxiously jumping up and down saying yes, multithreading, asynchronous programming, that is what this is all about.

That is not true. Asynchronous work is one of the areas where coroutines shine, as we will see at the end. But it is just like an integer type, which also has nothing to do with multithreading, and which we still use in a multithreaded environment. Nearly all the examples here will be single threaded. Multithreading and asynchronous programming by themselves could take up another one or two sessions.

Before we go further, it is worth separating concurrency from parallelism, because the distinction is what makes coroutines interesting.

Picture a cook who is either chopping the carrot or stirring the pot. Chopping a bit, stirring a bit. That is concurrency. Both the chopping and the stirring make forward progress, but neither is happening at the same time. If we switch quickly enough between them, an observer might think both are progressing simultaneously, while they are not.

If, however, somebody is chopping and stirring at the very same time, a single person cannot do that. It would require a second cook, which is to say a second CPU, a second core. Then we have parallelism, where both are genuinely making progress at the same instant.

Every generation of this problem has been solved by making the switch cheaper

Let me go back in time, and this might tell my age.

In the mid eighties I got my first computer, a nice machine with big floppy disks, and I could run one program at a time. I inserted the floppy and started my word processing. It was not Microsoft Word back then, the king of the hill was WordPerfect. I would be editing text and get bored, and I would want to play a game. So I had to stop WordPerfect, insert another floppy disk, run Out Run, and go racing. Then when I had wasted enough time I stopped the game and started the word processor again. Nobody would call that concurrency. The swapping was far too slow to give any impression that both were progressing.

Then Windows came along, and by the mid nineties Windows 95. Now I was playing Pac-Man in one window, writing text in another, and the clock in the system tray was ticking the seconds away. It felt like everything was happening at once. It was the operating system switching the CPU between three processes, and PCs then were single core, so parallelism was not even possible. That was real concurrency, and switching between processes is something that in computer land takes a huge amount of time compared to running a single C++ statement. A completely different order of magnitude.

We had another problem in those days. If I filled in an input field, pressed calculate, and the calculation took a minute or two, then switched to my game and came back, I got a frozen GUI. The program had code to draw the interface, but the program was single threaded and the thread was doing the calculation.

That is what threads solved. Now the scheduler was not just handing the CPU to process one and then process two. It was handing the CPU to thread five of process one, then taking it away preemptively and giving it to thread one of process ten. Within my program I had a calculation thread and a GUI thread, and the GUI could refresh while the calculation continued. Switching between threads was much faster than switching between processes. But compared to a regular C++ statement, it is still extremely slow.

That is where coroutines come in. A coroutine runs a bit, then suspends, and something else can happen, maybe another coroutine, maybe the caller. That switch is of a completely different magnitude from a thread context switch. Way, way smaller. Way more efficient.

Coroutines are a collaboration. If a coroutine decides never to pause, it is not willingly giving up the CPU for anyone else, and you are back in the world where the scheduler eventually says you took enough time and takes the CPU away. But if it collaborates nicely, we get very quick switching between different pieces of the program.

A coroutine is a function that can be paused and resumed, which turns out to mean it is an object

What is a coroutine? It is a function that can be paused, suspended, and resumed. That is an absolutely correct definition. But what does it actually mean?

A regular function starts, does a job, and ends. It returns. With a coroutine you are saying that we start, and midway I want to pause, and later I want to continue where I left off. These are challenges we need to solve. The C++ coroutines ecosystem solves them, but it needs our help. That help is the part where we put the IKEA book cabinet together ourselves.

Here is the flow. Main is executing statements, and at some point it calls a coroutine. The coroutine starts, and at some point says it is going to suspend. We go back to the statement after the call, main continues, and then main resumes the coroutine. We pick up exactly where we left off, run more statements, suspend again, go back to main, and at some point the coroutine ends and hands control back completely.

Unless we put in effort for it to be otherwise, this is all on the same thread. By default the coroutine runs on the thread of the caller. It does not need to.

There is another way of looking at this. When we suspend, we do not have to return to our caller. We can go somewhere else entirely. That is the flow used in asynchronous environments, and we will come back to it.

Control flow between a caller and a C++ coroutine, showing execution alternating between the two while the coroutine frame stays alive through every suspension — suspend and resume flow

So when is a function a coroutine? The moment one of three keywords appears in the function body. co_return, co_yield, or co_await. From that point the compiler knows this is a coroutine and does its magic, with the assistance of the programmer.

A quick note for later. co_yield you could read as here is a value to my caller, and then co_await. So co_yield is really here is the value, now I pause.

There are two sides to the story. There is the compiler-facing side, where the compiler recognizes the coroutine and needs information from us. And there is the user-facing side, where somebody uses that coroutine. We will fold these two angles together and meet somewhere in the middle.

A radio station, and why a regular function does not cut it

Let us build a first silly but educational example. We are going to create a radio station. We tell it what type of music we like and how many songs we want, and a radio station emerges that plays songs.

We ask the radio station to prepare a song. The DJ looks for the record, puts it on the turntable, and when it is done the radio station gives the song to us. Then it suspends. We listen to it, and afterwards we tell the coroutine to resume and put the next record on.

If we model this as a regular function, we pass in the style and the number of songs, we loop, and we play. The annoying thing is that the loop just continues. We get all the songs at once. If you are a DJ mixing, you might like that. If you want to listen, it is not a good user experience. So a regular function does not cut it.

What does it mean to model this as a coroutine? We create it, and that results in something. Different names exist in the literature. The coroutine interface, the coroutine API, the coroutine remote control, whatever you want to call it. After we create it, we get something we can interact with. Resume, for example. In our case that is please play the next song. Are there still songs to play, because if all the requested songs have been played it makes no sense to ask for another. Or maybe I am midway through song two and I realize I have to catch my train, and I want to stop.

We are professional programmers. Whatever we allocate, we want to deallocate. If we register, we unregister. If we create, we destruct.

That sounds like an object. You create it, it provides methods to interact with, and if you want to stop, the destructor takes care of it. So our function that can pause and resume is no longer just a simple function. It is an object. That is already a very interesting observation.

The state cannot live on the stack, so the burden comes back

This function has state. We pass in the type of music and the number of songs, and it needs those for the whole body. It gives us songs in a loop, so it needs to know which iteration it is on. When the coroutine is suspended, all of this needs to remain stored somewhere. It cannot be discarded, because resuming needs it.

Let us take a step back and look at what happens when we call a regular function. The caller puts the return address on the stack, so we know the next statement to execute when we come back. We put room on the stack for the return value. We put the arguments on the stack. We call the function, and its local variables go on the stack too. The stack keeps growing.

Then the function returns. It does not matter whether that is an early return, the closing brace, or an exception. Unwinding happens, everything is cleaned up, and the stack is back exactly as it was before the call.

Now imagine we want to resume after that has happened. We are in serious trouble, because we have learned there is state that needs to be preserved. So this state can no longer live on the stack, because it would all be gone.

If we cannot store it on the stack, what else do we have? The heap. Obvious choice. That is why C++ coroutines are called stackless. They store their state on the heap.

And we all know what using the heap means. We know we need to deallocate. That is the programmer’s responsibility. Not too early, and do not forget it.

A coroutine stores its state on the heap, and then it says, I have this coroutine frame here, dear developer, I want to give you a handle to it, and now it is up to you to free it at the correct time.

We were so used to smart pointers that we do not worry about this anymore. Well, ladies and gentlemen, this burden is back on our shoulders.

It is possible for compilers in certain circumstances to eliminate the heap allocation. If they can see enough of what is happening and other conditions hold, they can put it on the stack instead. We are not going into that, it is too much detail. But sometimes you might need to go there, because you do not have a heap. That can happen.

A regular function stack frame growing and then vanishing when the function returns, compared with a C++ coroutine frame on the heap that stays unchanged and is kept alive by a handle — frame stack vs heap

Writing the coroutine is easy, and then the work starts

Here is our radio station as a coroutine.

RadioStation radioStation(int style, int songs)
{
  for (int i = 0; i < songs; ++i)
  {
     const auto idx = i % Songs;
     co_yield( style ? electronic[idx] : rap[idx] );
  }
}

Spot the two differences from the regular version. We use co_yield instead of printing. And the return value is a RadioStation, that coroutine object.

There is something interesting to notice. You are telling me it returns a RadioStation, so why is there no return statement? There is a closing brace and we are not returning anything, so we return void. Which is it?

The answer is both. A function used to return one thing. Coroutines actually return two things. They return that coroutine object, and they can return a value. In our case the radio station returns nothing at the end. It returns void.

Now, from the user’s perspective. We are going to create the radio station, so it needs a constructor, and indirectly, because the compiler calls it for us. We will call the destructor, because the object is in our scope. We want to ask whether we are done. We want to ask for the next song.

And there is one more question. When the radio station emerges, does it immediately start playing? Or is creating it just getting the DJ into the booth with his records, sitting ready for the first request? In the second case the coroutine is lazy. In the first it is eager. The compiler needs to know which, and it cannot guess. We have to tell it.

The promise type is where the compiler asks its questions

The compiler wants answers from us. Do we suspend at startup? Symmetry being a good thing, do we suspend at the end? If an exception escapes the body uncaught, what should happen? And how exactly do I create that return object?

These questions are answered through a concept called the promise type. This has nothing to do with std::promise from std::async. It is a C++20 concept defining a range of methods, and depending on what your coroutine does, some of them need to be implemented as members of the promise type, which is just a class or a struct.

initial_suspend takes no arguments. Return std::suspend_always{} for lazy, std::suspend_never{} for eager. We chose lazy.

final_suspend is the same shape and must be noexcept. Most of the time you want to suspend at the end. There are situations where you do not, and you will see later why suspending matters for our examples.

unhandled_exception takes no arguments and returns void, and it is the fallback if an exception escapes. Over to you, the compiler says, you solve this problem we have. Let us take the easy route and terminate. In production code you probably want something more sane.

When the closing brace is hit, if the coroutine returns void the compiler calls return_void. If it returns a value, it calls return_value taking that type. We return void, so it is easy to implement. Open brace, close brace.

Do we need to write this for every coroutine that returns nothing? Yes. Again and again. If you want a lot of coroutines, you have a lot of boilerplate. At least it is not hard.

Then there is the value we are generating. co_yield produces a song, and the compiler asks where to put it. It wants to store it in the promise type. So if we yield a value of type T, we implement yield_value taking a const reference. Until now the promise type had no state. Now it needs some.

auto initial_suspend()
{
    return std::suspend_always{};
}

void unhandled_exception()
{
    std::terminate();
}

auto final_suspend() noexcept
{
    return std::suspend_always{};
}

void return_void() {}

auto yield_value(const std::string& valueIn)
{
    value = valueIn;
    return std::suspend_always{};
}

std::string value;

Notice yield_value stores the value and then returns suspend_always. Remember, co_yield is here is the value, and then co_await. That suspend_always is what the co_await part is doing.

This still fits on one slide. The font has decreased a little, but it is rather trivial. This is not rocket science.

The handle is how we reach into the frame

One piece of the puzzle is still missing, get_return_object, and to understand it we need the hierarchy.

The coroutine frame lives on the heap. A lot of things live in it. The arguments, so which music and how many songs. The loop index, so where we are. And the promise type, because the promise type has state.

We need access to this frame, because at some point we have to destroy it. So we create a handle to it. You could call it a smart handle, that is probably the best way to look at it. The handle is templated on the promise type, and it provides methods that make sense for a smart handle. Resume. Destroy. Are you done. Is there still a handle, because if the handle has been destroyed it will be false. And because the promise type lives in the frame that the handle points at, we can ask the handle for access to the promise.

The return object, our RadioStation, gets the handle at construction time. So the constructor of RadioStation takes a handle, and the compiler passes it in.

class [[nodiscard]] RadioStation
{
public:
    struct promise_type;
    using CoroHandle = std::coroutine_handle<promise_type>;

    RadioStation(auto handle) : mHandle{handle}
    {
    }

    ~RadioStation()
    {
        if (mHandle)
        {
            mHandle.destroy();
        }
    }

    bool nextSong() const
    {
        if (!mHandle || mHandle.done())
        {
            return false; // we are done
        }
        mHandle.resume();
        return !mHandle.done();
    }

    std::string value() const
    {
        return mHandle.promise().value;
    }

private:
    CoroHandle mHandle;
};

nextSong is our resume. We can only resume if there is a handle and we are not done, otherwise we return false and there is no next song, stop calling us. If we can, we tell the handle to resume. That resumption might be the one that moves us from the final suspend to really done, which is why we check again afterwards.

value fetches the song. The compiler called yield_value in the promise type, which stored the string in its state. We have the handle, we ask the handle for the promise, and we read the member. That is how the song gets out of the coroutine and into user code.

For get_return_object, the compiler calls it and we need to produce the return object. There is a factory method on the coroutine handle, from_promise, and we pass in ourselves. So get_return_object lives in the promise type, creates the handle from the promise, and calls the RadioStation constructor with it.

And we forbid copying. How do you copy a handle? I am not even sure. I did not try it out, so I need to be honest, I do not know the answer. Moving might make sense, if you were managing several radio stations in a container. Copying, I would say do not go there.

A C++ coroutine frame on the heap holding the function arguments, the loop state and the promise type, with the coroutine handle reaching in from the returned object to access the promise — the hierarchy

The while loop works, and then we want a range-based for

We have implemented our first coroutine. Writing the coroutine was one slide. The using code was one slide. The boilerplate was two more.

But look at the using code. It is a while loop. We do not want while loops. A coroutine generating things is a range. In our case it ends after seven songs. It could be infinite. One of the poster children of coroutines is a Fibonacci generator, which never ends.

So we would like a range-based for, which gives us the value directly without calling value() and nextSong() separately. I think we can agree that is much nicer code.

For that we need an iterator, and there is none. More boilerplate.

A range-based for needs begin and end. The iterator needs to be incrementable, dereferenceable, and comparable, either with another iterator of the same type or with a sentinel since C++20. Dereferencing means give us the yielded value. Incrementing means resume.

The iterator needs the handle, so we pass it at construction. And since the handle is a kind of pointer, the end iterator is simply the iterator holding a null pointer. operator++ resumes the handle, then checks whether we are done, and if so sets its handle to null so it compares equal to end. For comparison we do not even need to write it, since C++20 gives us = default.

end is trivial, an iterator with a null pointer. begin returns an iterator with a null pointer if there is no handle or it is already done, so begin immediately equals end. Otherwise it takes the handle, resumes once, and returns.

Most of this belongs in a library, not in your code

So what can we conclude? The coroutine function itself was easy. The user code was easy. The boilerplate, the RadioStation class and the promise type, was not hard either. It is annoying that we have to do it, and if I write another coroutine I have to do it again.

The question is whether this is production ready.

That is the discussion around C++20, and many people agree the answer is no. Everybody reading this is now an expert, we have seen how it works. But if you write such a coroutine with the boilerplate, will your colleague tomorrow, looking at that code in a review, be able to understand it without proper training?

That is why people say C++20 coroutines are a language feature which is a building block for others to build upon. You could reasonably say, I do not want to write coroutines. I want to use libraries that have implemented coroutines and which make my life easier.

We were promised library support in C++23. We got something. Unfortunately, only one thing.

What we implemented was a generator of strings. So with std::generator, our radio station is nothing more than a std::generator<std::string>, the coroutine body stays exactly as it was, and we are done. We have been talking for nearly an hour to implement a coroutine, and in C++23 it boils down to one slide.

In case what you are doing is a generator.

A pinball machine, when you are not generating anything

Let us do an example that is not a generator. A pinball machine, with two players, me and the coroutine. We take turns. When either of us plays a ball we are not producing any value, we just do our thing and pause. Over to you.

We do not want to return anything but we do want to pause, so we use co_await. Remember co_yield is here is the value and then co_await. Here we just want the pause, so co_await std::suspend_always{}.

Writing the coroutine is very easy. It takes how many turns, it loops, it awaits. The using code is a while loop again, as long as the machine is playing, it plays, then I play.

The promise type goes quicker this time, because we have experience. We are not yielding anything, so no yield_value at all. Everything else is the same as before. The Pinball class is the same shape as RadioStation without the value fetch, because nothing is yielded.

Is there something like std::generator for a coroutine that only awaits, in C++23? No. So if this is your use case, boilerplate time.

Now, a stroke of genius or a stroke of stupidity, somewhere in the middle. Let us cheat the system. Maybe playing pinball is a generator of integers. Let us yield zero every time and ignore it.

cpp

using Pinball = std::generator<int>;

Pinball pinball(int turns)
{
    for (int i = 0; i < turns; ++i)
    {
        std::cout << "     Your turn to play.\n";
        co_yield 0;
    }
    std::cout << "       You loose.\n";
}

int main()
{
    auto pball = pinball(4);
    for (const auto& turn : pinball)
    {
        (void)turn;
        std::cout << "My turn to play.\n";
    }
    std::cout << "I win.\n";
    return 0;
}

We have implemented the pinball machine with std::generator. Is this stupidity? Is this genius? I do not know. It is a way out where you generate an int nobody cares about, but you can use the standard type. Your decision.

Doing work in chunks, and co_return

Third example, and the third keyword. A coroutine that does work in chunks. Say we are calculating an average over a very large set of values, and doing it in one go would take unacceptably long. So when we call the coroutine it adds a few inputs, we resume, it adds a few more, and on the final resumption it divides by the number of elements and co_returns the average.

That is the collaboration. I know I have a lot of work to do, but I will do it in little bits and pause so that someone else can also do some work, and I am not monopolizing whatever resource.

Where does the returned value get stored? The same place as before, the promise type. So we implement return_value instead of return_void, the promise type gets state again, and our Average class gets a getResult method that reaches through the handle into the promise.

The coroutine is a for loop adding one entry per iteration with a co_await between, then a co_return of the average. Again no rocket science.

And again, is there a std::generator for this? No. And again, we can cheat. We can make it a generator of std::optional<int>, yielding an empty optional each time round the loop and yielding the filled one containing the average at the end. The user code loops until the range ends, and prints when the optional is not empty. Normally it should be the last iteration, otherwise we have a bug.

Awaiters are the second configuration point

The promise type configures the coroutine towards the compiler. There is a second concept that configures coroutines, and that is the awaitable. Awaitables are the operand of co_await, and an awaiter is a specific way to implement one. It comes into play whenever co_await or co_yield is used.

An awaiter has three methods. If your struct has these three, it is an awaiter and it can be the operand of co_await. It can have a zillion other methods too, that does not matter.

await_ready is called just before the suspension happens, while the coroutine is still active. It returns a boolean, and if it returns true the coroutine does not suspend. Typically you return false, because suspending was the intention. But changing your mind becomes useful in the asynchronous world. co_await on something, and the question is whether that something is ready. A socket, I would like to read some data. Oh, I have data already, here it is, no need to suspend. Or, I do not have data yet, I will launch an asynchronous read, so go ahead and suspend.

await_suspend is called immediately after the coroutine suspends, but before control returns to the caller. It receives the handle of the coroutine that has just been suspended, and that is very important. From here we can change our mind again and not suspend, we can suspend and let the flow go back to the caller, or we can suspend and go somewhere else entirely.

await_resume is called when the coroutine is resumed, and it can return a value. That is the value the co_await or co_yield expression evaluates to. It does not have to return anything, which is why we write auto.

We already know two awaiters. std::suspend_always and std::suspend_never. In both, await_suspend and await_resume are empty. The only difference is await_ready. Suspend always means I really want to suspend, so it returns false. Suspend never says, suspending, are you crazy, I am ready, and returns true.

Getting a value back into the coroutine

Here is a use for a custom awaiter. Until now information flowed from the coroutine to the caller. We get a song. But what if we want to resume the coroutine and say, I have some information for you, take it into account. In our silly example, we give the song we just heard a score, and the coroutine prints it out.

The score goes in through the promise type. We add a score method to RadioStation that reaches through the handle and stores it in a new promise member. The calling code is easy, we like all songs and give them ten out of ten. The coroutine side is easy too, co_yield returns something, we store it in a local variable and print it.

The hard part is how the co_yield expression produces that value, and suspend_always is not going to cut it. That is where the custom awaiter comes in. Instead of returning suspend_always from yield_value, we return our own awaiter templated on the handle type.

Our awaiter holds a handle, null at construction. await_ready returns false, we do want to suspend. await_resume asks the handle for the promise, reads the score, and returns it, which is what makes the co_yield expression evaluate to the score.

But how does the awaiter get the handle? await_suspend. It is called just after suspension and it receives the handle of the coroutine that was just suspended. That is exactly the handle we want, so we store it. Then we return void, because we are not changing our mind. On resumption, await_resume runs, we have the handle, and our plan worked.

A coroutine calling another coroutine

Now the case somebody asked about during the session. An outer coroutine calling an inner one, where from the outside the caller cannot tell which of them suspended. That is an implementation detail of the outer coroutine.

Calling the inner coroutine directly does not work, because it returns its coroutine object which we do not store, so it dies on the same line. Looping over the inner coroutine inside the outer one does not work either, because then the outer coroutine does everything at once from its caller’s perspective, which is not what we wanted.

Awaitables solve this. If the outer coroutine’s promise type could store the handle of the inner coroutine, then resume becomes simple. Am I done? If so everything is done. If not, the handle I resume is my own by default, unless there is a sub-handle that is not done, in which case I resume that one instead. The last check still returns whether my own handle is done, because after the sub coroutine finishes the outer coroutine still has its own work.

So the whole problem is getting the inner handle into the outer coroutine. And this is where it clicks. If the inner coroutine’s object is itself an awaitable, then when the outer coroutine says co_await innerCoroutine, the inner awaitable’s await_suspend is called with the handle of the coroutine that just suspended, which is the outer one. So the inner coroutine now has the outer coroutine’s handle, can ask it for its promise, and can store its own handle there.

await_suspend has three possible return types. Void, meaning we are not changing our mind and we continue suspending, thank you for the handle. Bool, where true continues the suspension and false cancels it. Or another coroutine handle, and in that case we are not going back to our caller at all. That is the coroutine we are going to resume. That is how you chain coroutines one after another, and it is called symmetric transfer. An interesting use is at the final suspend. This coroutine is finished, what is the next thing to do? Start the next task.

We are running out of time, so we will not go deeper there.

Where coroutines actually shine

Now the asynchronous world. When you launch an asynchronous operation with Boost.Asio you pass in a completion handler, which is also a form of continuation. I do an async read on a socket, and when the bytes arrive, please call my read handler.

That is one of the drawbacks. Say we want an echo server. We accept, then we async read until the whole message has arrived, and then the read handler is called. Suddenly we are in a completely different part of the code, where we write those bytes back on the socket. Then the write handler says I want to async read again to see if there are more messages. So we are jumping around in the code base wondering where the flow is going.

If it were synchronous it would be connect, and wait. Read, and wait. Write, and wait. The benefit was a very easy recipe to follow. Connect, read, write, loop. Of course it does not perform, because we cannot serve anyone else.

This is where coroutines shine. Code that got spread all over the place with completion handlers suddenly looks like synchronous code again. Connect, loop, read, write, done.

C++20 brings the fundamental building blocks, and typically they are not for mere mortals. They are for library vendors, and Boost writes that boilerplate for us. In a Boost.Asio echo server, the coroutine does not return a RadioStation or a Pinball or an std::generator. It returns a boost::asio::awaitable<void>. We co_await on async_accept with use_awaitable instead of a completion handler, and Asio knows we are working in the coroutine ecosystem. The line suspends, and when a connection arrives it resumes and the line returns. Then we loop, co_await async_read_some, check the error, co_await async_write.

Now to the multithreading question. Assume multiple threads are calling context.run(), so we have a thread pool helping process the work posted on that IO context. The asynchronous operation may well happen on a completely different thread from the one this coroutine was running on, or is suspended on, or will be resumed on.

This is not a problem, and it is worth seeing why. The buffer is a local variable. Either we are suspended waiting for bytes, in which case we are not touching the buffer, or the async read has returned and is no longer touching it, and we read it out. Then we call async write and suspend again, so we stop touching it. Only one party is ever working with that buffer, and it is the only one who can be.

So we did not need any mutexes. That is one of the reasons coroutines are such lightweight things in this kind of scenario. We are in the asynchronous world, our code looks linear again, and context switching is cheap.

Imagine having to implement all of that yourself. You would learn a great deal about networking and threading and what you can do inside coroutines. But as a mere mortal, I do not want to know. I want to use coroutines. I write this little coroutine and the library vendor does the heavy lifting for me.

Why it is called co_await

One last thing.

In all our generator examples the coroutine stopped and went back to main, and we always looked at it from main’s side. Main calls the coroutine and blocks until the coroutine says, I have a song for you, and suspends. So you could say main was waiting.

But look at it from inside the coroutine. The coroutine suspends, and the coroutine is waiting. It is waiting until somebody resumes it. It is co_awaiting. Come on, please resume me.

The same in the asynchronous world. From inside the coroutine, on that accept line, I am waiting for a connection to occur. I am waiting. Please, somebody resume me.

That is why it is called co_await. And that is why there are papers arguing let us co_await everything. You can look at it from the caller’s side, but you can also look at it from the inside. I am the coroutine, and I am waiting until somebody finally resumes me.

That is all I wanted to share.

Session notes

A few things I flagged as out of scope on the day and did not cover here. Avoiding the heap allocation is possible in certain circumstances but it is detailed work. Custom allocators are possible by overloading operator new in the promise type, which I confirmed after the session. Symmetric transfer deserves more room than we had.

On copying a coroutine object, my advice is do not go there. Moving may make sense if you are managing several in a container. If you do go there, you have homework to do to make sure it happens correctly.

On mutexes, I am not saying they are out of the question, it depends on how much state ends up shared. But if you have a use case where you still need one, check whether your design can avoid it first. If it cannot, solve it as you did before.

If you have further questions you can reach me on LinkedIn and I will be happy to help.

Inside C++ Coroutines How They Really Work

4.54MB ∙ PDF file

Download

Download

This deep dive is adapted from Lieven De Cock’s Deep Engineering Live session, Inside C++ Coroutines, How They Really Work.

How this deep dive was edited
Lieven’s session ran two and a half hours and built three complete coroutines from scratch across 110 slides. This piece is edited from the session transcript and stays in his own words throughout. The spoken delivery has been tightened for reading, the three worked examples compressed to their decisive moments, and roughly forty code slides reduced to the three that carry the most weight. Nothing has been added that he did not say on the day.
Topics he flagged as out of scope during the session, including heap elision, custom allocators and the full detail of symmetric transfer, remain out of scope here. Two diagrams in his deck use images credited to Nicolai Josuttis and Hana Dusíková, so the illustrations in this piece are original rather than reproductions.
The complete deck and the full recording are above, and both are worth your time if you want the parts that did not fit.

Building a C++ Coroutine by Hand, and Why You…