﻿<html>
<head>
    <title>Improving shared_ptr for C++0x, Revision 2</title>
    <meta content="http://schemas.microsoft.com/intellisense/ie5" name="vs_targetSchema" />
    <meta http-equiv="Content-Language" content="en-us" />
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
</head>
<body bgcolor="#ffffff">
    <address>
        Document number: N2351=07-0211</address>
    <address>
        Programming Language C++, Library Subgroup</address>
    <address>
        &nbsp;</address>
    <address>
        Peter Dimov, &lt;<a href="mailto:pdimov@pdimov.com">pdimov@pdimov.com</a>&gt;</address>
    <address>
        Beman Dawes, &lt;<a href="mailto:bdawes@acm.org">bdawes@acm.org</a>&gt;</address>
    <address>
        &nbsp;</address>
    <address>
        2007-07-19</address>
    <h1>
        Improving shared_ptr for C++0x, Revision 2</h1>
    <ul>
        <li><a href="#overview">Overview</a></li>
        <li><a href="#allocator">Allocator Support</a></li>
        <li><a href="#aliasing">Aliasing Support</a></li>
        <li><a href="#creation">Object Creation</a></li>
        <li><a href="#move">Move Support</a></li>
        <li><a href="#atomic">Atomic Access</a></li>
        <li><a href="#cycles">Cycle Collection</a></li>
    </ul>
    <h2>
        Changes in Revision 1</h2>
    <ul>
        <li>Expanded the Allocator Support section with more rationale.</li>
        <li>Clarified that the proposed changes are source- and binary-compatible.</li>
        <li>Added rationale for the <code>make_shared</code> name.</li>
        <li>Changed the <code>make_shared</code> proposed wording to use variadic templates
            and changed the Throws clause to reflect the semantics. Remarks added.</li></ul>
    <h2>
        <a name="overview">I. Overview</a></h2>
    <p>
        While <code>shared_ptr</code> has already proven its utility time and again, we
        have observed frequent requests for enhancements in several legitimate and key areas:</p>
    <ul>
        <li>Ability to control its internal allocations;</li>
        <li>Aliasing support;</li>
        <li>A factory function that eliminates the need for an explicit <code>new</code> expression;</li>
        <li>Reduced reference counting overhead when using <code>shared_ptr</code> in "mostly
            unique" ownership situations such as containers;</li>
        <li>A variant of <code>shared_ptr</code> that is <em>atomic</em>, that is, safe to be
            manipulated from multiple threads without synchronization; </li>
        <li>A way to reclaim cyclic <code>shared_ptr</code> structures; </li>
        <li>A variant of <code>shared_ptr</code> that is confined to a single thread and uses
            unsynchronized reference count updates. </li>
    </ul>
    <p>
        This document proposes additions to the C++0x standard to address the first six.
        Some of the proposed additions are essentially a subset of those presented in <a
            href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1851.pdf">N1851</a>
        by Kliatchko and Rocha. We refer the interested reader to that paper for a more
        extensive rationale.</p>
    <p>
        This proposal makes use of variadic templates and rvalue references. These C++0x
        features markedly improve the usability and effectiveness of several of the proposed
        additions.</p>
    <h2>
        <a name="allocator">II. Allocator Support</a></h2>
    <p>
        The default behavior of <code>shared_ptr</code> is to allocate its control block
        using <code>new</code>. This precludes its use in contexts where uncontrolled dynamic
        allocations are not allowed. The proposed addition allows the user to supply an
        allocator that <code>shared_ptr</code> will use.</p>
    <p>
        Boost users have repeatedly been asking for the ability to control the internal
        control block allocation; one particularly amusing occurence was the desire to use
        a <code>shared_ptr</code> in the implementation of <code>::operator new</code>. Typical environments
        where <code>shared_ptr</code> cannot be used as-is include embedded systems and
        computer and video games. Paul Pedriana (one of the primary authors of the Electronic
        Arts STL implementation) writes in <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2271.html">
            N2271</a> that</p>
    <blockquote>
        EASTL's shared_ptr/weak_ptr allow the user to specify an allocator instead of implicitly
        using global new</blockquote>
    <p>
        and explains that</p>
    <blockquote>
        As described in the <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2271.html#game_software_issues">
            game software issues</a> section, global new usage is often verboten in game
        software development, at least for console platforms. Thus any library facility
        which uses global operator new or any memory allocation that cannot be controlled
        by the user is unacceptable.</blockquote>
    <p>
        Asked to comment on the proposed addition, Paul kindly offered the following quote:</p>
    <blockquote>
        Certainly we (I can speak for all of Electronic Arts here) are in support of your
        proposal and you can document it as such. You could probably get other game developers
        and embedded developers to agree as well.
    </blockquote>
    <p>
        Vladimir Kliatchko and Ilougino Rocha offer additional arguments in favor of this
        functionality in <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1851.pdf">
            N1851</a>, complete with proposed wording. The suggested changes in this paper
        are a subset of those.</p>
    <p>
        The key part of this proposal is that <strong>the allocator is not made part of <code>
            shared_ptr</code>'s type</strong>. It is instead supplied as a parameter to
        the <code>shared_ptr</code> constructor in the same manner as the deleter is. This
        is made possible by the already existing deleter support infrastructure. Coupled
        with the fact that <code>shared_ptr</code> only allocates memory once in its constructor,
        and only deallocates it when the last instance in an ownership group is destroyed,
        this allows us to avoid all common sources of allocator problems. The allocator
        used for the creation of a particular <code>shared_ptr</code> instance remains purely
        an implementation detail for the user of that <code>shared_ptr</code>.</p>
    <p>
        Why use the <code>std::allocator</code> interface if it's widely perceived as "broken"?
        The simple answer is that it works for our (and our users') purposes and is already
        in the Standard.</p>
    <p>
        [Acknowledgments: this allocator interface has been independently suggested by Doug
        Gregor, Joe Gottman, Greg Colvin and others on the Boost list, as well as in N1851.]</p>
    <h3>
        Impact:</h3>
    <p>
        This feature extends the interface of <code>shared_ptr</code> in a backward-compatible
        way, allowing its broader use, and is therefore strongly recommended to be added
        to the C++0x standard. It does not impact existing uses, nor does it introduce binary
        compatibility issues.</p>
    <h3>
        Proposed text:</h3>
    <p>
        Add to <code>shared_ptr</code> [util.smartptr.shared] the following constructor:</p>
    <blockquote>
        <pre>template&lt;class Y, class D, class A&gt; shared_ptr( Y * p, D d, A a );</pre>
    </blockquote>
    <p>
        and the following member function:</p>
    <blockquote>
        <pre>template&lt;class Y, class D, class A&gt; void reset( Y * p, D d, A a );</pre>
    </blockquote>
    <p>
        Change the section:</p>
    <blockquote>
        <pre>template&lt;class Y, class D&gt; shared_ptr( Y * p, D d );</pre>
        <p>
            <em>Requires:</em> <code>p</code> shall be convertible to <code>T*</code>. <code>D</code>
            shall be <code>CopyConstructible</code>. The copy constructor and destructor of
            <code>D</code> shall not throw exceptions. The expression <code>d(p)</code> shall
            be well-formed, shall have well defined behavior, and shall not throw exceptions.</p>
        <p>
            <em>Effects:</em> Constructs a <code>shared_ptr</code> object that <em>owns</em>
            the pointer <code>p</code> and the deleter <code>d</code>.</p>
    </blockquote>
    <p>
        in [util.smartptr.shared.const] to:</p>
    <blockquote>
        <pre>template&lt;class Y, class D&gt; shared_ptr( Y * p, D d );
template&lt;class Y, class D, class A&gt; shared_ptr( Y * p, D d, A a );</pre>
        <p>
            <em>Requires:</em> <code>p</code> shall be convertible to <code>T*</code>. <code>D</code>
            shall be <code>CopyConstructible</code>. The copy constructor and destructor of
            <code>D</code> shall not throw. The expression <code>d(p)</code> shall be well-formed,
            shall have well defined behavior, and shall not throw. <code>A</code> shall be an
            <em>allocator</em> [allocator.requirements]. The copy constructor and destructor
            of <code>A</code> shall not throw.</p>
        <p>
            <em>Effects:</em> Constructs a <code>shared_ptr</code> object that <em>owns</em>
            the pointer <code>p</code> and the deleter <code>d</code>. The second constructor
            shall use a copy of <code>a</code> to allocate memory for internal use.</p>
    </blockquote>
    <p>
        Add the following to [util.smartptr.shared.mod]:</p>
    <blockquote>
        <pre>template&lt;class Y, class D, class A&gt; void reset( Y * p, D d, A a );</pre>
        <p>
            <em>Effects:</em> Equivalent to <code>shared_ptr( p, d, a ).swap( *this )</code>.</p>
    </blockquote>
    <h3>
        Implementability:</h3>
    <p>
        This feature has been added to <code>boost::shared_ptr</code> and will be part of
        Boost 1.35. See:</p>
    <p>
        <a href="http://boost.cvs.sourceforge.net/*checkout*/boost/boost/boost/shared_ptr.hpp">
            http://boost.cvs.sourceforge.net/*checkout*/boost/boost/boost/shared_ptr.hpp</a><br />
        <a href="http://boost.cvs.sourceforge.net/*checkout*/boost/boost/libs/smart_ptr/test/shared_ptr_alloc2_test.cpp">
            http://boost.cvs.sourceforge.net/*checkout*/boost/boost/libs/smart_ptr/test/shared_ptr_alloc2_test.cpp</a></p>
    <p>
        for reference.</p>
    <h2>
        <a name="aliasing">III. Aliasing Support</a></h2>
    <p>
        Advanced users often require the ability to create a <code>shared_ptr</code> instance
        <code>p</code> that shares ownership with another (master) <code>shared_ptr</code>
        <code>q</code> but points to an object that is not a base of <code>*q</code>. <code>
            *p</code> may be a member or an element of <code>*q</code>, for example. This
        section proposes an additional constructor that can be used for this purpose.</p>
    <p>
        An interesting side effect of this increase of expressive power is that now the
        <code>*_pointer_cast</code> functions can be implemented in user code. The <code>make_shared</code>
        factory function presented later in this document can also be implemented using
        only the public interface of <code>shared_ptr</code> via the aliasing constructor.</p>
    <h3>
        Impact:</h3>
    <p>
        This feature extends the interface of <code>shared_ptr</code> in a backward-compatible
        way that increases its expressive power and is therefore strongly recommended to
        be added to the C++0x standard. It introduces no source- and binary compatibility
        issues.</p>
    <h3>
        Proposed text:</h3>
    <p>
        Add to <code>shared_ptr</code> [util.smartptr.shared] the following constructor:</p>
    <blockquote>
        <pre>template&lt;class Y&gt; shared_ptr( shared_ptr&lt;Y&gt; const &amp; r, T * p );</pre>
    </blockquote>
    <p>
        Add the following to [util.smartptr.shared.const]:</p>
    <blockquote>
        <pre>template&lt;class Y&gt; shared_ptr( shared_ptr&lt;Y&gt; const &amp; r, T * p );</pre>
        <p>
            <em>Effects:</em> Constructs a <code>shared_ptr</code> instance that stores <code>p</code>
            and <em>shares ownership</em> with <code>r</code>.</p>
        <p>
            <em>Postconditions:</em> <code>get() == p &amp;&amp; use_count() == r.use_count()</code>.</p>
        <p>
            <em>Throws:</em> nothing.</p>
        <p>
            <em>[Note:</em> To avoid the possibility of a dangling pointer, the user of this
            constructor must ensure that <code>p</code> remains valid at least until the ownership
            group of <code>r</code> is destroyed. <em>--end note.]</em></p>
        <p>
            <em>[Note:</em> This constructor allows creation of an <em>empty</em> <code>shared_ptr</code>
            instance with a non-NULL stored pointer. <em>--end note.]</em></p>
    </blockquote>
    <h3>
        Implementability:</h3>
    <p>
        This feature has been added to <code>boost::shared_ptr</code> and will be part of
        Boost 1.35. See:</p>
    <p>
        <a href="http://boost.cvs.sourceforge.net/*checkout*/boost/boost/boost/shared_ptr.hpp">
            http://boost.cvs.sourceforge.net/*checkout*/boost/boost/boost/shared_ptr.hpp</a><br />
        <a href="http://boost.cvs.sourceforge.net/*checkout*/boost/boost/libs/smart_ptr/test/shared_ptr_alias_test.cpp">
            http://boost.cvs.sourceforge.net/*checkout*/boost/boost/libs/smart_ptr/test/shared_ptr_alias_test.cpp</a></p>
    <p>
        for reference.</p>
    <h2>
        <a name="creation">IV. Object Creation</a></h2>
    <p>
        Consistent use of <code>shared_ptr</code> can eliminate the need to use an explicit
        <code>delete</code>, but it currently provides no support in avoiding explicit <code>
            new</code>. There have been repeated requests from users for a factory function
        that creates an object of a given type and returns a <code>shared_ptr</code> to
        it. Besides convenience and style, such a function is also exception safe and considerably
        faster because it can use a single allocation for the object and its corresponding
        control block, eliminating a significant portion of <code>shared_ptr</code>'s construction
        overhead. This function eliminates one of the major efficiency complaints about
        <code>shared_ptr</code>.</p>
    <p>
        This section proposes a family of overloaded function templates, <code>make_shared&lt;T&gt;</code>
        and <code>allocate_shared&lt;T&gt;</code>, to address this need. <code>make_shared</code>
        uses the global <code>operator new</code> to allocate memory, whereas <code>allocate_shared</code>
        uses an user-supplied allocator, allowing finer control consistent with <a href="#allocator">
            section II</a> of this document.</p>
    <p>
        The rationale for choosing the name <code>make_shared</code> is that the expression
        <code>make_shared&lt;Widget&gt;()</code> can be read aloud and conveys the intended
        meaning. A free function also enables a non-intrusive implementation that can be
        delivered without modifications to an existing <code>shared_ptr</code> implementation.</p>
    <h3>
        Impact:</h3>
    <p>
        This feature does not affect the interface of <code>shared_ptr</code>. It is possible
        to implement in a non-intrusive way using only the public interface, <strong>as long
            as aliasing support is present</strong>. Access to implementation details can
        eliminate between 5 and 8 bytes of storage overhead on a typical 32 bit platform.</p>
    <p>
        The addition is a strong candidate for the C++0x standard, but can be relegated
        to a technical report.</p>
    <h3>
        Proposed text:</h3>
    <h4>
        Synopsis:</h4>
    <blockquote>
        <pre>namespace std {
  template&lt;class T, class... Args&gt; shared_ptr&lt;T&gt; make_shared( Args &amp;&amp; ... args );
  template&lt;class T, class... Args&gt; shared_ptr&lt;T&gt; allocate_shared( A const &amp; a, Args &amp;&amp; ... args );
}
</pre>
    </blockquote>
    <h4>
        Description:</h4>
    <blockquote>
        <pre>
template&lt;class T, class... Args&gt; shared_ptr&lt;T&gt; make_shared( Args &amp;&amp; ... args );
template&lt;class T, class... Args&gt; shared_ptr&lt;T&gt; allocate_shared( A const &amp; a, Args &amp;&amp; ... args );
</pre>
        <p>
            <em>Requires:</em> The expression <code>new( pv ) T( std::forward&lt;Args&gt;(args)...
                )</code>, where <code>pv</code> is a <code>void*</code> pointing to storage
            suitable to hold an object of type <code>T</code>, shall be well-formed. <code>A</code> shall be an <em>allocator</em> [allocator.requirements].
            The copy constructor and destructor of <code>A</code> shall not throw.</p>
        <p>
            <em>Effects:</em> Allocates memory suitable for an object of type <code>T</code>
            and constructs an object in it via the placement new expression <code>new( pv ) T()</code>
            or <code>new( pv ) T( std::forward&lt;Args&gt;(args)... )</code>.
            <code>allocate_shared</code> uses a copy of <code>a</code> to allocate memory.
            If an exception is thrown, has no effect.</p>
        <p>
            <em>Returns:</em> A <code>shared_ptr</code> instance that stores and owns the address
            of the newly constructed object of type <code>T</code>.</p>
        <p>
            <em>Postconditions:</em> <code>get() != 0 &amp;&amp; use_count() == 1</code>.</p>
        <p>
            <em>Throws:</em> <code>bad_alloc</code>, or an exception thrown from <code>A::allocate</code>
            or the constructor of <code>T</code>.</p>
        <p>
            <em>Remarks:</em> Implementations are encouraged, but not required, to perform no more than one memory allocation. <em>[Note:</em> This provides efficiency equivalent to an intrusive smart pointer. <em>--end note]</em></p>
        <p>
            <em>[Note:</em> These functions will typically allocate more memory than <code>sizeof(T)</code>
            to allow for internal bookkeeping structures such as the reference counts. <em>--end
                note]</em></p>
    </blockquote>
    <h3>
        Implementability:</h3>
    <p>
        A proof of concept non-intrusive implementation is available at:</p>
    <p>
        <a href="http://www.pdimov.com/cpp/make_shared.cpp">http://www.pdimov.com/cpp/make_shared.cpp</a></p>
    <p>
        This implementation uses variadic templates and rvalue references. When these features
        are not available, it falls back on a family of overloaded function templates taking
        arguments by const reference.</p>
    <h2>
        <a name="move">V. Move Support</a></h2>
    <p>
        Users often express concerns over the cost of copying a <code>shared_ptr</code>
        in situations where the source of the copy is no longer needed. To address this
        use case, <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1851.pdf">
            N1851</a> proposes a separate smart pointer, <code>managed_ptr</code>, that
        is convertible from and to <code>shared_ptr</code> and enforces unique ownership.</p>
    <p>
        The current proposal does not take this approach. Instead, we propose that move
        constructors and move assignment operators be added to <code>shared_ptr</code>.
        This allows a <code>shared_ptr</code> to be as efficient as an <code>auto_ptr</code>
        or the proposed <code>unique_ptr</code> when the source of the copy or assignment
        is a temporary or no longer needed. Move-aware standard containers will automatically
        take advantage of this optimization. As an example of the consequences, reallocating
        a <code>vector&lt; shared_ptr&lt;T&gt; &gt;</code> will no longer entail any reference
        count updates.</p>
    <h3>
        Impact:</h3>
    <p>
        This feature affects the interface of <code>shared_ptr</code> in a way that reduces
        its copy overhead and is in line with the rvalue recommendations for the standard
        library presented in <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1856.html">
            N1859</a>-<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1862.html">N1862</a>.
        We believe that it is a strong candidate for addition to the C++0x standard.</p>
    <h3>
        Proposed text:</h3>
    <p>
        Add to <code>shared_ptr</code> [util.smartptr.shared] the following:</p>
    <blockquote>
        <pre>shared_ptr( shared_ptr &amp;&amp; r );
template&lt;class Y&gt; shared_ptr( shared_ptr&lt;Y&gt; &amp;&amp; r );

shared_ptr&amp; operator=( shared_ptr &amp;&amp; r );
template&lt;class Y&gt; shared_ptr&amp; operator=( shared_ptr&lt;Y&gt; &amp;&amp; r );
</pre>
    </blockquote>
    <p>
        Add the following to [util.smartptr.shared.const]:</p>
    <blockquote>
        <pre>shared_ptr( shared_ptr &amp;&amp; r );
template&lt;class Y&gt; shared_ptr( shared_ptr&lt;Y&gt; &amp;&amp; r );</pre>
        <p>
            <em>Requires:</em> For the second constructor <code>Y*</code> shall be convertible
            to <code>T*</code>.</p>
        <p>
            <em>Effects:</em> Move-constructs a <code>shared_ptr</code> instance from <code>r</code>.</p>
        <p>
            <em>Postconditions:</em> <code>*this</code> contains the old value of <code>r</code>.
            <code>r</code> is <em>empty</em>.</p>
        <p>
            <em>Throws:</em> nothing.</p>
    </blockquote>
    <p>
        Add the following to [util.smartptr.shared.assign]:</p>
    <blockquote>
        <pre>shared_ptr&amp; operator=( shared_ptr &amp;&amp; r );
template&lt;class Y&gt; shared_ptr&amp; operator=( shared_ptr&lt;Y&gt; &amp;&amp; r );</pre>
        <p>
            <em>Effects:</em> Equivalent to <code>shared_ptr( move( r ) ).swap( *this )</code>.</p>
        <p>
            <em>Returns:</em> <code>*this</code>.</p>
    </blockquote>
    <h3>
        Implementability:</h3>
    <p>
        This feature has been added to <code>boost::shared_ptr</code> and will be part of
        Boost 1.35.</p>
    <hr />
    <p>
        <em>Thanks to Joe Gottman for his comments on the move support.</em></p>
    <p>
        <em>Thanks to Jens Maurer for his comments on the <code>make_shared</code> wording.</em></p>
    <p>
        <em>--end</em></p>
</body>
</html>
