<BASE HREF="/home/vollmann/winuser/orgs/wg21/parproc/n1834.html">

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN""http://www.w3.org/TR/html4/loose.dtd">
<HTML
><HEAD
><TITLE
>A Pleading for Reasonable Parallel Processing Support in C++</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.79"></HEAD
><BODY
CLASS="ARTICLE"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#0000FF"
VLINK="#840084"
ALINK="#0000FF"
><DIV
CLASS="ARTICLE"
><DIV
CLASS="TITLEPAGE"
><H1
CLASS="TITLE"
><A
NAME="AEN2"
>A Pleading for Reasonable Parallel Processing Support in C++</A
></H1
><SPAN
CLASS="FIRSTNAME"
>Detlef&nbsp;</SPAN
><SPAN
CLASS="SURNAME"
>Vollmann<BR></SPAN
><DIV
CLASS="AFFILIATION"
><SPAN
CLASS="ORGNAME"
>vollmann engineering gmbh<BR></SPAN
></DIV
><SPAN
CLASS="ADDRESS"
>      <A
HREF="mailto:dv@vollmann.ch"
>dv@vollmann.ch</A
>
    </SPAN
><TABLE
><TR
><TD
ALIGN="RIGHT"
><B
>Document Number: </B
></TD
><TD
>JTC 1/SC22/WG21/N1834 J16/05-0094</TD
></TR
><TR
><TD
ALIGN="RIGHT"
><B
>Date: </B
></TD
><TD
>2005-06-24</TD
></TR
><TR
><TD
ALIGN="RIGHT"
><B
>Project: </B
></TD
><TD
>     JTC1.22.32<BR>
     Programming Language C++
     <BR>Evolution Working Group
    </TD
></TR
><TR
><TD
ALIGN="RIGHT"
><B
>Reference: </B
></TD
><TD
>ISO/IEC IS 14882:2003(E)</TD
></TR
></TABLE
><HR></DIV
><DIV
CLASS="NOTE"
><BLOCKQUOTE
CLASS="NOTE"
><P
><B
>Preliminary: </B
>This paper presents the personal opinion of the author.
The statements in this paper might seem provocative to some, but
are based on quite a lot of personal experience.
  </P
><P
>This paper does not present a final statement, but there must
be strong reasons for the author not to follow the conclusions
given at the end of this paper.
  </P
></BLOCKQUOTE
></DIV
><DIV
CLASS="SECTION"
><HR><H2
CLASS="SECTION"
><A
NAME="INTRODUCTION"
>Introduction</A
></H2
><P
>Parallel processing is in modern computers a given.
More and more you have hardware support for parallel processing,
like Hyperthreading, multiple cores or multiple CPUs.
But even without real parallel hardware support, modern operating
systems give you a quasi-parallel execution environment even in
pretty small embedded systems.
  </P
><P
>Given that reality, there is probably a general agreement in the
C++ standardization committee that C++0x needs to address parallel
processing.  Where there is no general agreement yet is how such
support should look like.
  </P
><P
>This paper argues that fixing the execution model for C++ is a
must, while having a low-level "threading" or "locking" library
might be a bad idea.
  </P
></DIV
><DIV
CLASS="SECTION"
><HR><H2
CLASS="SECTION"
><A
NAME="TERMINOLOGY"
>Threads vs. Parallel Processing</A
></H2
><P
>A general note on terminology and concepts: some papers and authors
use the term <I
CLASS="FIRSTTERM"
>multi-threading</I
>
when addressing parallel processing,
and some use <I
CLASS="FIRSTTERM"
>parallel processing</I
> when
addressing multi-threading.
  </P
><P
>This paper uses the term <I
CLASS="FIRSTTERM"
>thread</I
> for a separate
thread of control
in completely the same address space (without the possible exception of
specifically declared thread-local storage).  This was common practice in
operating systems like PC-DOS where all tasks shared the same
address space.
  </P
><P
>This paper uses the term <I
CLASS="FIRSTTERM"
>parallel processing</I
>
generally for separate
thread of controls.  In modern operating systems this often
provides different address spaces, but allow sharing some common
resources (memory and others).
  </P
><P
>A lot of current literature proposes threads as a solution for
parallel processing needs, but this is almost always a
bad idea.  Any standard C++ feature that promotes threads instead
of general parallel processing means would be a bad service to
the C++ community.
  </P
></DIV
><DIV
CLASS="SECTION"
><HR><H2
CLASS="SECTION"
><A
NAME="CORE"
>Core Level Support</A
></H2
><P
>Paper <A
HREF="#MEMMODELBIB"
>N1777</A
>
and Andrei Alexandrecu's presentation in Redmond and
Hans Boehm's presentation in Lillehammer showed clearly that
the current execution model of C++ is underspecified for
parallel executing tasks accessing the same memory.
  </P
><P
>Some (including the author) believe that in that context the keyword
<CODE
CLASS="FUNCTION"
>volatile</CODE
> should play a role to mark memory
that is accessed
from different tasks in parallel.  But if volatile gets some
useful meaning (whatever), there are several parts of the C++
standard that need some revisiting (e.g. does an implementation
provide a volatile op=, do standard library classes (like
complex&#60;&#62;) support volatile operations, etc.).
  </P
></DIV
><DIV
CLASS="SECTION"
><HR><H2
CLASS="SECTION"
><A
NAME="LIBRARY"
>Library Support</A
></H2
><P
>Fixing the execution model of C++ helps for those using implementation
specific functionality.  But without some standard library support
there is no way for a C++ user to use parallel processing in a
standard way.
While paper <A
HREF="#STRATPLANTHREADSBIB"
>N1815</A
> simply states
<SPAN
CLASS="QUOTE"
>"The standard must define some mechanism for locking."</SPAN
>,
this paper proposes a different approach.
  </P
><P
>For standard library extensions for parallel processing support
exist different ideas.  These can be categorized into different
domains:
   <P
></P
><UL
><LI
><P
>high-level classes for shared resources and asynchronously executed procedures
     </P
></LI
><LI
><P
>low-level synchronization classes
     </P
></LI
><LI
><P
>thread starting and control
     </P
></LI
></UL
>
  </P
><DIV
CLASS="SECTION"
><HR><H3
CLASS="SECTION"
><A
NAME="HIGHLEVEL"
>High-Level Library</A
></H3
><P
>A high-level library to support parallel processing would provide
classes for shared resources (mainly memory, but
possibly also I/O streams) and means for asynchronously executed
functions.
   </P
><P
>Shared resources would be provided by pure container classes (strings,
fixed sized sequences, dynamic sequences, associative containers)
and <A
HREF="#MONITORBIB"
>monitors</A
> that combine shared
data storage with synchronized methods.
   </P
><P
>Asynchronously executed procedures would allow starting functions
asynchronously, and controlling them: requesting their status, wait
for them, and collect any results
(e.g. <A
HREF="#FUTUREBIB"
>Futures</A
>).
   </P
><P
>The definition of such a library must not be thread specific.
On the contrary for shared resources there should be a requirement
that they also work when accessed from different processes, and
for asynchronous functions there should be at least strong
encouragement for implementations favouring different processes
instead of different threads.  Whether such different implementations
should be transparent to the user is an open question.
   </P
><P
>Such a high-level library could be used by application programmers
without much risk of introducing the usual synchronization problems
(deadlock and races) and would therefore be useful to a major
portion of the C++ user community.
   </P
></DIV
><DIV
CLASS="SECTION"
><HR><H3
CLASS="SECTION"
><A
NAME="LOWLEVEL"
>Low-Level Library</A
></H3
><P
>A low-level library for parallel processing support would provide
classes for synchronization mechanisms like locks, semaphores and
conditional events.
Such mechanisms provide detailed control mechanisms for
fine-grained synchronization.
But such mechanisms also introduce a high risk for problems like
deadlock and race conditions.
   </P
><P
>Again, if such a library is specified, it must not be specific
to threads but must
also provide synchronization across process/address space boundaries.
   </P
><P
>A really useful low-level synchronization library must provide
mechanisms to wait for all kind of events (mutex availability,
process completion, I/O operations, signals, etc.)
As those events depend highly on
the underlying operating system, a really useful low-level
synchronization library is out of the scope of C++0x.
   </P
></DIV
><DIV
CLASS="SECTION"
><HR><H3
CLASS="SECTION"
><A
NAME="THREADLIB"
>Thread Library</A
></H3
><P
>A synchronization library is generally only useful if there are
means to start other threads of control that execute asynchronously.
A thread library would allow starting and controlling threads.
But such a library would need to support threads as well as processes.
To define such a library with natural
implementations on different operating systems is quite a challenge
(location of program executable, program arguments).
   </P
></DIV
></DIV
><DIV
CLASS="SECTION"
><HR><H2
CLASS="SECTION"
><A
NAME="CONCLUSION"
>Conclusion</A
></H2
><P
>All support for parallel processing in C++0x must work for
processes.  Support for threads might also be useful, but has
lower priority.
  </P
><P
>C++0x must provide an execution model that generally supports
asynchronous access to objects.
  </P
><P
>If library support for parallel processing is provided, a high-level
library has absolute priority.
  </P
><P
>A low-level library that is really useful requires a major effort
and is therefore probably out of scope for C++0x.
A half-baked low-level library would be a bad service to the
C++ community.
A half-baked low-level library without a high-level library would
be disastrous, as this would give the message "Use this dangerous
library, because it is standard, instead of a safer non-standard
library."
  </P
><P
>Generally, a C++ library approach along the lines of
<A
HREF="#POSIXBIB"
>POSIX threads</A
> is very problematic,
as that specification is focused on low-level support, while an approach
along the lines of <A
HREF="#OPENMPBIB"
>OpenMP</A
> looks much
more promising, as that specification focuses on high-level constructs
while still providing some low-level functionality.
  </P
><P
>Just as a side note: we currently have a similar situation with
the C++ I/O streams library.  This supports generally safe and easy to
use high-level abstractions instead of the low-level OS-specific
functionality like memory-mapped files and asynchronous I/O
that is often required for system programming.
  </P
></DIV
><H1
><A
NAME="AEN84"
>References</A
></H1
><DIV
CLASS="BIBLIOENTRY"
><A
NAME="MEMMODELBIB"
></A
><P
>[N1777]&nbsp;<SPAN
CLASS="AUTHOR"
>Andrei Alexandrescu, </SPAN
><SPAN
CLASS="AUTHOR"
>Hans Boehm, </SPAN
><SPAN
CLASS="AUTHOR"
>Kevlin Henney, </SPAN
><SPAN
CLASS="AUTHOR"
>Ben Hutchings, </SPAN
><SPAN
CLASS="AUTHOR"
>Doug Lea, </SPAN
><SPAN
CLASS="AUTHOR"
>and Bill Pugh</SPAN
>, C++ Standard Committee, WG21/N1777=J16/05-0037, <I
>Memory model for multithreaded C++: Issues</I
>,     <A
HREF="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1777.pdf"
TARGET="_top"
>http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1777.pdf</A
>
   .</P
><DIV
CLASS="BIBLIOENTRYBLOCK"
STYLE="margin-left: 0.5in"
></DIV
></DIV
><DIV
CLASS="BIBLIOENTRY"
><A
NAME="STRATPLANTHREADSBIB"
></A
><P
>[N1815]&nbsp;<SPAN
CLASS="AUTHOR"
>Lawrence Crowl</SPAN
>, C++ Standard Committee, WG21/N1815=J16/05-0075, <I
>ISO C++ Strategic Plan for Multithreading</I
>,     <A
HREF="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1815.html"
TARGET="_top"
>http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1815.html</A
>
   .</P
><DIV
CLASS="BIBLIOENTRYBLOCK"
STYLE="margin-left: 0.5in"
></DIV
></DIV
><DIV
CLASS="BIBLIOENTRY"
><A
NAME="MONITORBIB"
></A
><P
>[Monitor]&nbsp;<I
>2nd ACM Conference on the History of Programming Languages</I
>, "Monitors and Concurrent Pascal: A personal history", <SPAN
CLASS="AUTHOR"
>Per Brinch Hansen</SPAN
>, 1993, pp. 1-25.</P
><DIV
CLASS="BIBLIOENTRYBLOCK"
STYLE="margin-left: 0.5in"
></DIV
></DIV
><DIV
CLASS="BIBLIOENTRY"
><A
NAME="FUTUREBIB"
></A
><P
>[Future]&nbsp;<I
>ACM Transactions on Programming Languages and Systems</I
>, "Multilisp - A Language for Concurrent Symbolic Computation", <SPAN
CLASS="AUTHOR"
>Robert H. Halstead</SPAN
>, 1985, pp. 501-538.</P
><DIV
CLASS="BIBLIOENTRYBLOCK"
STYLE="margin-left: 0.5in"
></DIV
></DIV
><DIV
CLASS="BIBLIOENTRY"
><A
NAME="POSIXBIB"
></A
><P
>[POSIX]&nbsp;ISO, ISO/IEC 9945-1:2003, <I
>Information technology
           -- Portable Operating System Interface (POSIX)</I
>,     <A
HREF="http://www.unix.org/version3/iso_std.html"
TARGET="_top"
>http://www.unix.org/version3/iso_std.html</A
>
   .</P
><DIV
CLASS="BIBLIOENTRYBLOCK"
STYLE="margin-left: 0.5in"
></DIV
></DIV
><DIV
CLASS="BIBLIOENTRY"
><A
NAME="OPENMPBIB"
></A
><P
>[OpenMP]&nbsp;  <A
HREF="http://www.openmp.org/"
TARGET="_top"
>http://www.openmp.org/</A
>
 .</P
><DIV
CLASS="BIBLIOENTRYBLOCK"
STYLE="margin-left: 0.5in"
></DIV
></DIV
></DIV
></BODY
></HTML
>