<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>

<meta http-equiv="Content-Type" content="text/html;charset=US-ASCII">

<style type="text/css">

body {
  color: #000000;
  background-color: #FFFFFF;
}

del {
  text-decoration: line-through;
  color: #8B0040;
}
ins {
  text-decoration: underline;
  color: #005100;
}

p.example {
  margin: 2em;
}
pre.example {
  margin: 2em;
}
div.example {
  margin: 2em;
}

code.extract {
  background-color: #F5F6A2;
}
pre.extract {
  margin: 2em;
  background-color: #F5F6A2;
  border: 1px solid #E1E28E;
}

p.function {
}

p.attribute {
  text-indent: 3em;
}

blockquote.std {
  color: #000000;
  background-color: #F1F1F1;
  border: 1px solid #D1D1D1;
  padding: 0.5em;
}

blockquote.stddel {
  text-decoration: line-through;
  color: #000000;
  background-color: #FFEBFF;
  border: 1px solid #ECD7EC;
  padding: 0.5em;
}

blockquote.stdins {
  text-decoration: underline;
  color: #000000;
  background-color: #C8FFC8;
  border: 1px solid #B3EBB3;
  padding: 0.5em;
}

table {
  border: 1px solid black;
  border-spacing: 0px;
  margin-left: auto;
  margin-right: auto;
}
th {
  text-align: left;
  vertical-align: top;
  padding: 0.2em;
  border: none;
}
td {
  text-align: left;
  vertical-align: top;
  padding: 0.2em;
  border: none;
}

</style>

<title>C++ Concurrent Queues</title>
</head>
<body>
<h1>C++ Concurrent Queues</h1>

<p>
ISO/IEC JTC1 SC22 WG21 N3434 = 12-0124 - 2012-09-23
</p>

<p>
Lawrence Crowl, crowl@google.com, Lawrence@Crowl.org
</p>

<p>
<a href="#Introduction">Introduction</a><br>
<a href="#Conceptual">Conceptual Interface</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#basic_operations">Basic Operations</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#trying">Trying Operations</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#nonblock">Non-Blocking Operations</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#push_back">Push Back Operations</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#closed_queues">Closed Queues</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#empty_queue">Empty Queues</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#queue_order">Queue Ordering</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#queue_names">Queue Names</a><br>
<a href="#Concrete">A Concrete Buffer Queue</a><br>
<a href="#Tools">Conceptual Tools</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#front_back">Fronts and Backs</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#iterators">Iterators</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Binary">Binary Interfaces</a><br>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Managed">Managed Indirection</a><br>
<a href="#Implementation">Implementation</a><br>
</p>

<p>
This paper revises N3353 = 12-0043 - 2012-01-14.
</p>


<h2><a name="Introduction">Introduction</a></h2>

<p>
The existing <code>deque</code> in the standard library
is an inherently sequential data structure.
Its reference-returning element access operations
cannot synchronize access to those elements
with other queue operations.
</p>

<p>
Concurrent pushes and pops on queues
requires a different interface to the queues.
</p>

<h2><a name="Conceptual">Conceptual Interface</a></h2>

<p>
We provide basic queue operations,
and then extend those operations to cover other important issues.
</p>


<h3><a name="basic_operations">Basic Operations</a></h3>

<p>
The essential solution to the problem
is to shift to a value-based operations, 
rather than reference-based operations.
</p>

<p>
The waiting operations are:
</p>

<dl>

<dt><code>void
<var>queue</var>::push(const <var>Element</var>&amp;);</code></dt>
<dt><code>void
<var>queue</var>::push(<var>Element</var>&amp;&amp;);</code></dt>
<dd>
<p>
Push the <code><var>Element</var></code> onto the queue.
If the operation cannot be completed immediately,
wait until it can be completed.
</p>
</dd>

<dt><code><var>Element</var> <var>queue</var>::value_pop();</code></dt>
<dd>
<p>
Pop the <code><var>Element</var></code> from the queue.
If the operation cannot be completed immediately,
wait until it can be completed.
The element will be moved out of the queue, if possible.
</p>
</dd>

</dl>

<h3><a name="trying">Trying Operations</a></h3>

<p>
Of course, there are operations to attempt a push or pop,
and execute that operation only if it can be completed soon.
</p>

<dl>

<dt><code>queue_op_status
<var>queue</var>::try_push(const <var>Element</var>&amp;);</code></dt>
<dt><code>queue_op_status
<var>queue</var>::try_push(<var>Element</var>&amp;&amp;);</code></dt>
<dd>
<p>
Try to immediately push the <code><var>Element</var></code> onto the queue.
If successful, return <code>queue_op_status::success</code>.
If the queue was full, return <code>queue_op_status::full</code>.
</p>
</dd>

<dt><code>queue_op_status
<var>queue</var>::try_pop(<var>Element</var>&amp;);</code></dt>
<dd>
<p>
Try to immediately pop the <code><var>Element</var></code> from the queue.
If successful, return <code>queue_op_status::success</code>.
If the queue was empty, return <code>queue_op_status::empty</code>.
The element will be moved out of the queue, if possible.
</p>
</dd>

</dl>


<h3><a name="nonblock">Non-Blocking Operations</a></h3>

<p>
The above <code>try_push</code> operations
will avoid waiting only when the queue is full.
If the queue is not full,
but contention requires the operation to wait for a mutex to discover that,
the operation will block.
The above <code>try_pop</code> operations behave similarly.
</p>

<p>
We could change these operations to also report failure to acquire the mutex,
perhaps by returning <code>queue_op_status::busy</code>.
These operations could then form the core of
a non-blocking conceptual interface.
</p>

<p>
That non-blocking interface could be extended with the basic operations,
provided that the basic operations
also be permitted to throw <code>queue_op_status::busy</code>.
</p>


<h3><a name="push_back">Push Back Operations</a></h3>

<p>
Occasionally, one may wish to return a popped item to the queue.
We can provide for this with a <code>push_back</code> operations.
These operations may have performance implications for some queues,
so these operations should be carefully considered.
</p>

<dl>

<dt><code>void
<var>queue</var>::push_back(const <var>Element</var>&amp;);</code></dt>
<dt><code>void
<var>queue</var>::push_back(<var>Element</var>&amp;&amp;);</code></dt>
<dd>
<p>
Push the <code><var>Element</var></code> onto the back of the queue,
i.e. in at the end of the queue that is normally popped.
If the operation cannot be completed immediately,
wait until it can be completed.
When successful, return <code>queue_op_status::success</code>.
</p>
</dd>

<dt><code>queue_op_status
<var>queue</var>::try_push_back(const <var>Element</var>&amp;);</code></dt>
<dt><code>queue_op_status
<var>queue</var>::try_push_back(<var>Element</var>&amp;&amp;);</code></dt>
<dd>
<p>
Try to immediately push the <code><var>Element</var></code>
onto the back of the queue,
i.e. in at the end of the queue that is normally popped.
If successful, return <code>queue_op_status::success</code>.
If the queue was full, return <code>queue_op_status::full</code>.
</p>
</dd>

</dl>

<p>
Should these operations be adopted,
renaming the <code>...push</code> operations
to <code>...push_front</code> operations
seems most consistent with the current library.
</p>


<h3><a name="closed_queues">Closed Queues</a></h3>

<p>
Threads using a queue for communication
need some mechanism to signal when the queue is no longer needed.
Rather than require an out-of-band signal,
we chose to directly support such a signal in the queue itself,
which considerably simplified coding.
</p>

<p>
To achieve this signal, a thread may <dfn>close</dfn> a queue.
Once closed, no new elements may be pushed onto the queue.
Push operations on a closed queue
will either return <code>queue_op_status::closed</code>
(when they have a status return)
or throw <code>queue_op_status::closed</code>
(when they do not).
Elements already on the queue may be popped off.
When an queue is empty and closed,
pop operations will either
return <code>queue_op_status::closed</code>
(when they have a status return)
or throw <code>queue_op_status::closed</code>
(when they do not).
</p>

<p>
The operations are as follows.
</p>

<dl>

<dt><code>void <var>queue</var>::close();</code></dt>
<dd>
<p>
Close the queue.
</p>
</dd>

<dt><code>bool <var>queue</var>::is_closed();</code></dt>
<dd>
<p>
Return true iff the queue is closed.
</p>
</dd>

<dt><code>queue_op_status
<var>queue</var>::wait_push(const <var>Element</var>&amp;);</code></dt>
<dt><code>queue_op_status
<var>queue</var>::wait_push(<var>Element</var>&amp;&amp;);</code></dt>
<dd>
<p>
Push the <code><var>Element</var></code> onto the queue.
If successful, return <code>queue_op_status::success</code>.
If the queue was closed, return <code>queue_op_status::closed</code>.
Otherwise, wait for either of the above to become true.
</p>
</dd>

<dt><code>queue_op_status
<var>queue</var>::wait_push_back(const <var>Element</var>&amp;);</code></dt>
<dt><code>queue_op_status
<var>queue</var>::wait_push_back(<var>Element</var>&amp;&amp;);</code></dt>
<dd>
<p>
Push the <code><var>Element</var></code> onto the back of the queue,
i.e. in at the end of the queue that is normally popped.
If successful, return <code>queue_op_status::success</code>.
If the queue was closed, return <code>queue_op_status::closed</code>.
Otherwise, wait for either of the above to become true.
Note that this operation
implies a less efficient single-writer single-reader queue.
</p>
</dd>

<dt><code>queue_op_status
<var>queue</var>::wait_pop(<var>Element</var>&amp;);</code></dt>
<dd>
<p>
Try to immediately pop the <code><var>Element</var></code> from the queue.
If successful, return <code>queue_op_status::success</code>.
If the queue was closed, return <code>queue_op_status::closed</code>.
Otherwise, wait for either of the above to become true.
The element will be moved out of the queue, if possible.
</p>
</dd>

</dl>


<h3><a name="empty_queue">Empty Queues</a></h3>

<p>
It is sometimes desirable to know if a queue is empty.
</p>

<dl>
<dt><code>bool <var>queue</var>::is_empty();</code></dt>
<dd>
<p>
Return true iff the queue is empty.
</p>
</dd>
</dl>

<p>
This operation is useful only during intervals when
the queue is known to not be subject to pushes and pops
from other threads.
Its primary use case is assertions on the state of the queue
at the end if its lifetime.
</p>


<h3><a name="queue_order">Queue Ordering</a></h3>

<p>
The conceptual queue interface does not specify an order of elements.
Concrete queues should specify this ordering, if any.
</p>

<p>
A push operation <em>synchronizes with</em>
the pop operation that obtains that element.
</p>


<h3><a name="queue_names">Queue Names</a></h3>

<p>
It is sometimes desirable for queues to be able to identify themselves.
This feature is particularly helpful for run-time diagnotics,
particularly when 'ends' become dynamically passed around between threads.
See <a href="#Managed">Managed Indirection</a> below.
There is some debate on this facility,
but I see no way to effectively replicate the facility.
</p>

<dl>
<dt><code>const char* <var>queue</var>::name();</code></dt>
<dd>
<p>
Return the name string provided as a parameter to queue construction.
</p>
</dd>
</dl>


<h2><a name="Concrete">A Concrete Buffer Queue</a></h2>

<p>
We provide a concrete concurrent queue
in the form of a fixed-size <code>buffer_queue</code>.
It provides the conceptual operations above,
construction of an empty queue,
and construction of a queue from a pair of iterators.
Constructors take a parameter
specifying the maximum number of elements in the buffer.
Constructors may also take a parameter
specifying the name of the queue.
If the name is not present, it defaults to the empty string.
</p>

<p>
The <code>buffer_queue</code>
deletes the default constructor, the copy constructor,
and the copy assignment operator.
We feel that their benefit might not justify their potential confusion.
</p>


<h2><a name="Tools">Conceptual Tools</a></h2>

<p>
There are a number of tools that support use of the conceptual interface.
</p>


<h3><a name="front_back">Fronts and Backs</a></h3>

<p>
Restricting an interface to one side of a queue
is a valuable code structuring tool.
This restriction is accomplished with
the classes <code>generic_queue_front&lt;Queue&gt;</code>
and <code>generic_queue_back&lt;Queue&gt;</code>.
These act as pointers
with access to only the front or the back of a queue.
</p>

<pre class="example">
<code>void send( int number, generic_queue_front&lt;buffer_queue&lt;int&gt;&gt; f );</code>
</pre>

<p>
These fronts and backs
are also able to provide <code>begin</code> and <code>end</code>
operations that produce iterators
because there is no ambiguity
on whether the iterators should be input or output iterators.
</p>


<h3><a name="iterators">Iterators</a></h3>

<p>
In order to enable the use of existing algorithms with concurrent queues,
they need to support iterators.
Output iterators will push to a queue
and input iterators will pop from a queue.
Stronger forms of iterators
are in general not possible with concurrent queues.
</p>

<pre class="example">
<code>void iterate(
    generic_queue_back&lt;buffer_queue&lt;int&gt;&gt;::iterator bitr,
    generic_queue_back&lt;buffer_queue&lt;int&gt;&gt;::iterator bend,
    generic_queue_front&lt;buffer_queue&lt;int&gt;&gt;::iterator fitr,
    generic_queue_front&lt;buffer_queue&lt;int&gt;&gt;::iterator fend,
    int (*compute)( int ) )
{
    while ( bitr != bend &amp;&amp; fitr != fend )
        *fitr++ = compute(*bitr++);
}</code>
</pre>

<p>Note that with suitable renaming,
the existing standard front insert and back insert iterators
could work as is.
However, there is nothing like a pop iterator adapter.
</p>


<h3><a name="Binary">Binary Interfaces</a></h3>

<p>
The standard library is template based,
but it is often desirable to have a binary interface
that shields the client from the concrete queue implementation.
We achieve this capability with type erasure.
</p>

<p>
We provide a <code>queue_base</code> class template
parameterized by the value type.
Its operations are virtual.
This class provides the essential independence
from the queue representation.
</p>

<p>
We also provide <code>queue_front</code> and <code>queue_back</code>
class templates parameterized by the value types.
These are essentially
<code>generic_queue_front&lt;queue_base&lt;Value_type&gt;&gt;</code> and
<code>generic_queue_front&lt;queue_base&lt;Value_type&gt;&gt;</code>,
respectively.
Indeed, they probably could be template aliases.
</p>

<p>
To obtain a pointer to <code>queue_base</code>
from an non-virtual concurrent queue,
construct an instance the <code>queue_wrapper</code> class template,
which is parameterized on the queue
and derived from <code>queue_base</code>.
Upcasting a pointer to the <code>queue_wrapper</code> instance
to a <code>queue_base</code> instance
thus erases the concrete queue type.
</p>

<pre class="example">
<code>extern void seq_fill( int count, queue_front&lt;int&gt; f );

buffer_queue&lt;int&gt; body(1, "body");
queue_wrapper&lt;buffer_queue&lt;int&gt;&gt; wrap(&amp;body);
seq_fill(1, &amp;wrap);</code>
</pre>


<h3><a name="Managed">Managed Indirection</a></h3>

<p>
Long running servers may have the need to
reconfigure the relationship between queues and threads.
The ability to pass 'ends' of queues between threads
with automatic memory management eases programming.
</p>

<p>
To this end, we provide
<code>shared_queue_front</code> and
<code>shared_queue_back</code> template classes.
These act as reference-counted versions
of the <code>queue_front</code> and
<code>queue_back</code> template classes.
These shared versions
act on a <code>queue_counted</code> class template,
which is a counted version of <code>queue_base</code>.
</p>

<p>
The concrete counted queues are
the <code>queue_owner</code> class template,
which takes ownership of a raw pointer to a queue,
and the <code>queue_object</code> class template,
which constructs and instance of the implementation queue within itself.
Both class templates are derived from <code>queue_counted</code>.
</p>

<pre class="example">
<code>queue_owner&lt;buffer_queue&lt;int&gt;&gt; own( new buffer_queue&lt;int&gt;(1, "body") );
seq_fill(1, &amp;own);</code>
</pre>

<p>
The <code>share_queue_ends(Args ... args)</code> template function
will provide a pair of
<code>shared_queue_front</code> and <code>shared_queue_back</code>
to a dynamically allocated <code>queue_object</code> instance
containing some implementation queue.
When the last of these fronts and backs are deleted,
the queue itself will be deleted.
Also, when the last of the fronts or the last of the backs is deleted,
the queue will be closed.
</p>

<pre class="example">
<code>auto x = share_queue_ends&lt;buffer_queue&lt;int&gt;&gt;(kSmall);
shared_queue_front&lt;int&gt; f(x.first);
shared_queue_back&lt;int&gt; b(x.second);
f.push(3);
assert(3 == b.value_pop());</code>
</pre>


<h2><a name="Implementation">Implementation</a></h2>

<p>
A free, open-source implementation of these interfaces
is avaliable at the Google Concurrency Library project
at 
<a href="http://code.google.com/p/google-concurrency-library/">
http://code.google.com/p/google-concurrency-library/</a>.
The concrete <code>buffer_queue</code> is in
<a href="http://code.google.com/p/google-concurrency-library/source/browse/include/buffer_queue.h">
..../source/browse/include/buffer_queue.h</a>.
The corresponding implementation of the conceptual tools is in
<a href="http://code.google.com/p/google-concurrency-library/source/browse/include/queue_base.h">
..../source/browse/include/queue_base.h</a>.
</p>


</body>
</html>
