<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
        "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
	<title>Ruminations on networking and executors</title>

	<style>
	p {text-align:justify}
	li {text-align:justify}
	blockquote.note
	{
		background-color:#E0E0E0;
		padding-left: 15px;
		padding-right: 15px;
		padding-top: 1px;
		padding-bottom: 1px;
	}
	ins {color:#00A000}
	del {color:#A00000}
	</style>
</head>
<body>

<address align=right>
Document number: P2464
<br/>
Audience: WG21, LEWG
<br/>
<br/>
<a href="mailto:ville.voutilainen@gmail.com">Ville Voutilainen</a><br/>
On behalf of SFS (Finland)<br/>
2021-09-28<br/>
</address>
<hr/>
<h1 align=center>Ruminations on networking and executors</h1>

<h2>Credits</h2>

<p>
  Many thanks to Chris Kohlhoff and Niall Douglas for answering
  my questions about ASIO, the Networking TS, and its work-executing
  model. Huge thanks to Eric Niebler for answering my thousands
  of questions about Senders and Receivers with stoic patience.
</p>
<p>
  Massive thanks to Kirk Shoop for correcting my colossal misunderstandings
  of how the NetTS model works. To some extent, those misunderstandings
  indicated that the problems would be much worse than they actually are, correcting
  of which was laudable, considering that Kirk is not a proponent
  of that design. That shows true class as an engineer, and I can't
  quite fully express my appreciation for it.
</p>

<h2>Abstract</h2>

<p>
  This paper makes the plain observation that we should not standardize
  the Networking TS as it's currently designed. The problem is the use
  of a P0443 executor. This paper makes the strong recommendation
  that we shouldn't, in fact, standardize pretty much anything that
  employs a P0443 executor as a significant component of the design, if it
  can even be described as a component.
</p>
<p>
  A P0443 executor is not an executor. It's a work-submitter. It doesn't
  provide any means of finding out whether the work submitted succeeded,
  or whether
  an error occurred. Such error handling is fundamental and crucial.
  Leaving it out from a facility that abstracts the execution of work
  simply doesn't work as a building block for libraries, larger systems,
  and applications for an audience as wide as the audience of a C++
  standard. The only thing that knows whether the work succeeded
  is the same facility that accepted the work submission, and trying
  to entertain the fantasy that the work submission and result observation
  are separable concepts is just <em>wrong</em>.
</p>
<p>
  It certainly gave me no pleasure to write this paper. The original
  plan about a week earlier than the actual writing of this paper
  was to write a paper that makes a case for standardizing
  the Networking TS in C++23, and how that's fine even if we eventually
  end up with multiple models for executing asynchronous work. The subsequent
  homework and analysis, however, lead to something completely different.
  I had high hopes that the Networking TS doesn't actually suffer from
  the problems of P0443 executors, but that is not the case.
</p>

<h2>Change history</h2>

<ul>
  <li>R0: Initial version.</li>
</ul>


<h2>The problem</h2>

<p>
  Please read <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2235r0.html">P2235</a> for some background material for this.
</p>

<p>A P0443 executor does not facilitate generic error handling. It doesn't
  allow writing generic code that can realize that some sort of an error
  occurred, and it provides no way to report errors that occur after
  work submission but before invoking a continuation.
</p>
<p>
  This means it doesn't scale. It doesn't work as a component of writing
  asynchronous code.
</p>

<p>
  The proponents of the Networking TS claim that this is not a problem
  in the design and model of the TS, or ASIO. But what they fail to
  see is something that we don't seem to manage to communicate across
  the disagreeing parties of this matter:
  <ul>
    <li>the Networking TS effectively requires that the executors never
      fail between work submission and continuation invocation,</li>
    <li>but if we standardize an executor abstraction, that's what
      programmers will use, beyond their limited use in the Networking
      model and its APIs.</li>
    <li>C++ programmers will write *thousands* of these executors. Yes,
      you read that correctly. The users of the Networking TS have maybe
      dozens of them, on a good day. An asynchronous executor will
      have orders of magnitude more implementations, many not related
      to or enjoying the limitations and stricter requirements of networking.
      </li>
  </ul>
</p>

<h2>The counter-arguments, and why they don't solve the problem</h2>

<p>
  The proponents of the Networking TS persistently suggest that this is not
  a problem. They either suggest that programmers can just add error handling
  support to a refined executor, or that programmers can just compose
  programs out of completion handlers and the executor is an insignificant
  part of this bigger picture.
</p>
<p>
  None of that works at all. The completion handlers in the Networking TS
  provide an associated executor, not a refined executor. There is no
  API to get a refined executor, or to write code that would
  work across thousands of different refined executors. The suggested
  approaches for how to get the error handling to happen in the Networking
  TS model, or how to compose completion handlers have thus far been
  very specific small-scale programming solutions that do not scale
  across a large amount of different implementations of the "concept",
  because it's not a useful concept that was designed with such programming
  in mind.
</p>
<p>
  We are going to have thousands of 'executors'. Then we are going to
  need to solve the same problems all over again, such as simply being
  able to tell whether the task given to the executor failed, for any
  reason, including errors that occurred between submission and continuation
  invocation. No matter of wrapping into completion handlers and asynchronous
  agents is going to change that; that concept would be the only way
  to abstract work-runners, and it doesn't provide generic error handling,
  or generic error detection. That's why all those 'executors' need to
  be schedulers instead, including the ones that are deep down
  in the guts of the Networking TS model.
</p>
<p>
  Because they are not deep down, they're not hidden, they're exposed - they
  are the customization mechanism for users for running continuations
  for i/o events. The problem of understanding this seems three-fold:
  <ol>
    <li>An i/o operation can report success/failure to a completion handler,
      but the executor can fail on its own, and there's no mechanism
      or channel for reporting that.</li>
    <li>The task runner that runs a continuation can be arbitrarily complex.
      From the perspective of the Networking TS, one might think it's just
      a simple threadpool that never fails to run the continuation, so the
      complexity is not noticeable for people who write just library wrappers
      on top of the Networking TS, still doing just networking, and other
      parts of their application use something else for whatever asynchronous
      operations. But the executor *is* that "something else", or would be,
      if it's a scheduler. It can, in addition to just invoking the continuation
      passed to it, execute an arbitrarily complex preamble or postamble.
      And many of them will.</li>
    <li>In order to work with multiple different task runners, we need
      to perform a from-specific-to-generic translation. The continuation
      runner is generic, it models a single concept. It can wrap our
      continuation into a generic continuation. We do not know the specific
      types of the continuation runner and the continuation, but we do know the specific i/o operation we want to run. The rest of the exercise is combining
      these operations, and that doesn't happen by writing more completion
      handlers, or by somehow relying on a more refined continuation runner or
      a more refined continuation for the wrapping.
  </ol>
</p>

<p>
  Let's try a pseudo-code example:
</p>
<blockquote><pre><code>void run_that_io_operation(scheduler auto sched,
                           sender_of&lt;network_buffer&gt; auto wrapping_continuation)
{
      launch_our_io();
      auto [buffer, status] = fetch_io_results_somehow(maybe_asynchronously);
      run_on(sched, queue_onto(wrapping_continuation,
                               make_it_queueable(buffer, status)));
}
</code></pre>
</blockquote>

<p>
  I have no idea what the sched's concrete type is. I have no idea what
  the wrapping_continuation's concrete type is. They're none of my business,
  I'm performing a specific i/o operation and passing the results up,
  for consumption by whatever client. I convert the data I got from my
  specific i/o operation into a form that can be wrapped in the
  generic typed wrapper, I perform the wrapping, and then I toss
  the wrapped continuation over to the scheduler to run.
</p>
<p>
  The continuation wrapper has its own continuation that I don't know about,
  and it does whatever the calling client wanted to happen when I pass it
  the i/o result. I don't decide it, the client does. Where that continuation
  runs, i.e. the scheduler, is not something I decide, the client does.
</p>
<p>The client can perform more of these wrappings and queueings
  and other transformation, doing so in a similar fashion, going
  from specific-to-generic, using the same generic approach.
  Its clients can do the same. The scheduler is a Strategy pattern implementation,
  and the sender is a Decorator pattern implementation, the client
  can operate on the decorator in a generic fashion, adding decorator
  layers to it, and all this can be passed to the generic execution
  strategy without any specific knowledge of what the upper layers
  concretely did or passed to us, but at the end the upper layers run
  whatever continuations they needed, translated from our network result.</p>

<p>How many different implementations of this function do I need to
  support 200 different execution strategies and 10,500 different
  continuation-decorations(*)?</p>
<p>The 'solutions' suggested by the proponents of the Networking TS
  model seem to say the answer is "many".</p>
<p>With Senders and Receivers, the answer is "one, you just wrote it."</p>

<p>A P0443 executor can't do this, and it can't interoperate with it.
  The execution strategy is arbitrarily complex, and the wrapping continuation
  needs to know if the _strategy_ had an error in invoking the continuation.
  It also needs to know whether our i/o operation failed, but that's not
  the only thing that can fail in this whole shebang.
</p>

<p>In the Networking TS, the incoming execution strategy would be an
  executor. That doesn't work, that doesn't scale, that doesn't integrate
  to the more-generic, which is also higher-abstraction-level and more-complex.
  It's not just a simple thread-pool that never fails to run the continuation,
  because the execution strategy is arbitrarily complex and so is the wrapping
  continuation.
</p>

<p>(*)Those numbers of execution strategies and continuation wrappers
  are not hyperbolic jokes. They are completely arbitrary but they represent
  different kinds of event loops and task runners that pass asynchronous
  work results from runners to other runners to event loops to other
  runners to other event loops, queuing lower-level operation results
  onto higher-level result handling logic that translates a lower-level
  result to a higher-level result, ultimately invoking arbitrary application
  logic.</p>
<p>The algorithms that queue and translate low-level results to
  higher-level results
  and low-level errors to high-level results are generic, and the
  same across layers
  of different abstraction level. The algorithms that transfer the
  results to run as continuations on other runners and other event loops
  are generic, and the same across layers of different abstraction level.
  There are adapters that facilitate specific translations, but the algorithms
  are the same.
</p>
<p>
  Senders and Receivers are designed to be able to do that. A P0443
  executor can't work with this. It always requires ignoring the whole
  pseudo-concept of an executor, and getting at a concrete implementation
  to fetch the error information, and turning the payload data
  into something that's composable. The whole concept is completely
  useless, there's no composed code you can write with it.</p>

<h2>On the "need" of executor "guarantees" in the Networking TS</h2>

<p>
  It has been said that Networking TS needs a guarantee that once
  an execution context for a continuation has been established,
  that context is used for all invocations of the continuation,
  whether an error or a success. Two problems:
  <ol>
    <li>I don't see how the networking library needs this guarantee. Whether a single context is to be used for all invocations of a continuation seems
      domain-specific and application-specific to me, and nailing it down
      at the "framework" level seems like a hard-coded choice.</li>
    <li>A P0443 executor does NOT establish this guarantee. It can decide not to invoke the continuation at all. It can decide to invoke it from different contexts based on what the result of the operation was. It has no such semantics of guaranteeing how the continuation is invoked, all it establishes is an interface for submitting work. Which is not sufficient for establishing semantics for
      how the work is executed.</li>
  </ol>
</p>
<p>
  With Senders and Receivers, one can apply an algorithm on the chain/nested-pile of work executed to ensure this. And that algorithm can be applied
  on the higher abstraction layers that know what the actual user needs.
</p>

<h2>Rephrasing the problem, a summary</h2>

<p>Composing P0443 executors is an ad-hoc exercise. It's actually
  worse than that, because P0443 executors don't compose, you need
  to compose something more refined and more specific. Composing
  Networking TS completion handlers or asynchronous operations
  (which aren't even a thing in the NetTS APIs, so how would I be
  able to compose those other than in an ad-hoc manner?)
seems like an ad-hoc and non-generic
  exercise. Whether they were designed for or ever deployed in larger-scale
  compositions that scale from low-level to high-level with any sort
  of uniformity, I don't know. Such a uniform model could perhaps
  be built on the completion handlers, but it can't be built on
  executors, and the problem with that is that everybody will build
  their own incompatible composition model.</p>

<p>Senders and Receivers were designed to facilitate the generic
  and scalable composition from the get-go. It's their driving
  design goal, to separate the work from how it's executed, to
  translate the results separate from what the work is, to queue
  both the work and its continuations to different contexts, all concern-separated
  from the work itself and its continuations, with a uniform model
  that allows generic algorithms to do the heavy lifting but yet
  be generic across different abstraction levels and layers. The design
  recognizes that there are arbitrarily many concrete implementations
  of a scheduler and a sender, but yet provides common facilities
  as algorithms that work with all of those concrete implementations,
  in a composable manner.</p>

<h2>On the question of standardising existing practice</h2>

<p>"This is not a problem, ASIO has been shipping this for years,
and you can find a similar executor in java.util.concurrent."</p>

<p>
Right. The first one doesn't seem to provide a composition model at all,
so maybe these questions haven't come up. Maybe ASIO hasn't been
used with an executor that can fail to run the continuation, like
a deadline scheduler that might not be able to lazily acquire the
resources needed for running the continuation. Maybe ASIO hasn't
been used in an environment where such failures are an issue.
</p>
<p>
For the java executors, they always have an escape hatch of performing
a dynamic cast down to a specific executor that can tell you whether
it can report such errors to you. I don't know whether anyone does that,
or whether such executors just haven't been used.
</p>
<p>
Such existing practice has a huge problem that it just plain can't
handle errors that occur after submission. That's an irrecoverable
data loss, and we know how to do better, because we have other
existing practice where this is a solved problem, in libunifex.</p>


<a name="#Conclusion"/><h2>Conclusion, i.e. "so what should we do?"</h2>

<h3>A minimal suggestion</h3>

<p>At minimum, we need to axe the P0443 executor from the model
  of the Networking TS, and replace it with a P2300 scheduler.
  That way our Networking TS i/o code can fit into the scalable
  model. There are various impedance mismatches and efficiency
  concerns about that, so this may be an iffy exercise, and it will
  take time to design and implement.</p>

<h3>A bold suggestion</h3>
<p>
Here's a rather straightforward suggestion:
<ol>
  <li>Stop spending energy on standardizing the Networking TS for C++23.</li>
  <li>Focus on making progress with Senders and Receivers.
</ol>
</p>
<p>Let's be bold. Let's make a decision. Various analyses lead into this
  outcome. We have a scalable model, let's later standardize a networking library
that fits straight into that scalable model.</p>

</body>
</html>
