<html>
<head>
<title>Concerns with changing existing types in Technical
Specifications</title>
</head>
<body>
<div>
Document number: N4041<br>
Date: 2014-05-23<br>
Project: Programming Language C++, Library Working Group<br>
Reply-to: Jonathan Wakely &lt;cxx&#x40;kayari.org&gt;<br>
</div>


<h1>Concerns with changing existing types in Technical
Specifications</h1>

<h2>Introduction</h2>

<p>
In Issaquah there were proposals targeting a TS which enhanced
existing types in the Standard Library, in particular N3857 which
modifies <tt>std::future</tt>, N3916 which modifies
<tt>std::function</tt>, <tt>std::promise</tt> and
<tt>std::packaged_task</tt>, and N3920 which modifies
<tt>std::shared_ptr</tt>.
</p>

<p>
The Library and Library Evolution Working Groups met in a joint session
to discuss whether a TS should specify changes to the C++ standard, as a
list of additions to the base specification, or whether to copy the
whole specification into the TS and create a duplicate type in namespace
<tt>std::experimental</tt> which could then be modified independently.
</p>

<p>
Following straw polls the decision was taken to allow a TS to change
existing types in namespace std, so that users who opt in to use the TS
(via some implementation defined means) get the enhanced versions
without needing to change any code.
</p>

<h2>Impact on Standard Library implementations</h2>

<p>
The straw polls taken in Issaquah were done with very few of the
standard library implementers in the room (most of them were in another
room doing ballot resolution for the Filesystem TS, which ironically
could have benefited from the decision had it been made months earlier,
by allowing a <tt>std::fstream</tt> to be constructed from a
<tt>std::experimental</tt>::path object).
</p>

<p>
The decision made following the straw polls means for an implementation
to ship a TS they must change existing classes in their implementation,
which has the potential to make silent ABI changes to users' programs.
An implementation can still choose to not declare <tt>std::function</tt>
when users enable support for a TS, instead declaring a different type
in an inline namespace, for example <tt>std::__libfund::function</tt>.
This ensures that some uses of <tt>std::function</tt> will result in
different name mangling and so it will be detectable when two
translation units have been compiled inconsistently (one with TS support
enabled and one without).  However, name mangling does not typically
encode anything about a function's return type or about the type of
global variables, and the mangled name of a class is not affected by the
type of one of its members changing from <tt>std::function</tt> to
<tt>std::__libfund::function</tt> due to the use of a compiler switch or
macro definition. This means when users opt in to the TS they can
introduce silent ODR violations that typically result in run-time
(rather than compile-time or link-time) errors.
</p>

<p>
This affects linking to libraries that use the changed types in their
API, possibly including the Standard Library itself. On some platforms
libstdc++ uses an extern <tt>std::function&lt;void()&gt;</tt> object in
the implementation of <tt>std::call_once</tt>. If a user called
<tt>std::call_once</tt> in a translation unit built with TS support
enabled the resulting object file would attempt to invoke member
functions on the global variable as though it was of type
<tt>std::__libfund::function</tt>, whereas the actual variable would be
of type <tt>std::function</tt>, resulting in an ODR violation.  This
could be worked around by the library implementer, for example by using
a different global variable when TS support is enabled, and possibly a
different <tt>std::call_once</tt>, but this means the changes are not
restricted to just the components altered by the TS, but to other,
seemingly unrelated parts of the implementation.  It also doesn't avoid
all ODR violations anyway, e.g. if the use of <tt>std::call_once</tt>
is in an inline function or a template that is used from two translation
units, one with TS support enabled and one without.
</p>

<p>
These issues can make it complicated for implementers to provide
TS support, so that users have to wait longer before they have access to
an implementation, which then makes it less likely the committee will
get useful and timely feedback on the TS contents.
</p>

<h2>Impact on third-party implementations</h2>

<p>

If implementing a TS requires changes to namespace <tt>std</tt> then it
can only be done by the "owner" of namespace <tt>std</tt>, i.e. the
standard library implementation. That makes it impossible to have
conforming third-party implementations from groups such as Boost. There
is no way for Boost to inject a modified <tt>std::future</tt> into the
implementation's <tt>&lt;future&gt;</tt> header, so it's necessary to
replace the entire header, but that might break other standard headers
which rely on implementation details expected to be found in
<tt>&lt;future&gt;</tt>. This was a real problem for the Boost.TR1
implementation, and the author of that library has expressed a very
strong preference for not requiring changes to <tt>std</tt> in any
specification except the C++ standard itself.
</p>

<p>

It is widely acknowledged that it's very difficult to provide
third-party implementations of the Standard Library because a number of
C++11 library features rely on compiler support, meaning the compiler
and library are more tightly coupled. There is no need for that to be
the case for a TS though. It makes a lot of sense to allow, and even
encourage, third-party implementations of experimental features. Boost
already implements the changes to <tt>std::shared_ptr</tt> that are
specified in the Library Fundamentals TS already and has non-standard
extensions to its <tt>std::future</tt> implementation similar to those
in the Concurrency TS. It is impossible for Boost to provide these
as conforming implementations of the  TS if the changes must be made to
types in <tt>std</tt> rather than as separate types in namespace
<tt>std::experimental</tt>.
</p>

<p>
If it is harder for third-party implementations to provide a working
version of the TS then that also makes it harder for users to get their
hands on an implementation and again makes it less likely we will get
timely feedback on the TS.
</p>



<h2>Impact on users</h2>

<p>
If a Standard Library implementation changes the definition of a type
when a compiler option or macro is used that is arguably no worse than
other ABI breaking options such as <tt>-fpack-struct</tt>, but the
difference is that we're encouraging users to try out TS features in the
hope we'll get feedback to help decide whether to standardise the
features later.
</p>

<p>
<tt>std::function</tt> is a vocabulary type, so it's bad to have two
separate versions of it, but the very fact it is a vocabulary type makes
it more likely to have been used in APIs and so the potential for ODR
violations makes it necessary for some users to recompile their entire
codebase.  The larger the codebase the more likely it is that something,
somewhere will be affected by an ABI change in non-obvious ways (i.e.
not resulting in a compile-time or link-time error). However recompiling
the entire codebase is just not feasible for most large code bases, and
feedback from large scale users is valuable.
</p>

<p>
Users can still get similar ODR violations if the TS requires a new type,
but then they must make source code changes. Code changes mean build tools
will typically cause recompilation of all affected files, or the inability
to make changes to a third-party library will make it obvious that the
library isn't compatible with the new type. Requiring code changes (not
just recompilation) makes opt in more explicit and requires a more
conscious decision. It's important to note that users can't benefit from
these new TS features without some code changes anyway.
</p>


<h2>Conclusion</h2>

<p>
I don't really have one, but I think the question of whether to require
changes to existing types deserves more attention. If altering existing
types makes it harder for implementors to safely ship the TS, or for
users to experiment with the contents of namespace
<tt>std::experimental</tt> in real code, then the committee and the
community miss out on feedback about the contents of the TS.
</p>


<h2>Acknowledgements</h2>

<p>
Thanks to John Maddock, Alisdair Meredith, and Jeffrey Yasskin for
sharing their thoughts on this subject.
</p>
</body>
</html>
