<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
<HEAD>
	<META HTTP-EQUIV="CONTENT-TYPE" CONTENT="text/html;charset=iso-8859-1">
	<TITLE>Dynamic Shared Objects: Survey and Issues</TITLE>
	<META NAME="CREATED" CONTENT="20051021;15043800">
	<META NAME="CHANGED" CONTENT="16010101;0">
	<STYLE>
	<!--
		@page { size: 8.5in 11in }
	-->
	</STYLE>
</HEAD>

<P ALIGN=LEFT>
Document number: J16/06-0046 = WG21 N1976<BR>
Date: 2006-04-20<BR>
Author: Benjamin Kosnik &lt;bkoz@redhat.com&gt; <BR>
Evolution Working Group, Modules and Linkage<BR>

<BODY LANG="en-US" DIR="LTR">
<H1 ALIGN=CENTER>Dynamic Shared Objects: Survey and Issues</H1>

<P><BR><BR>
</P>

<H2>Introduction</H2>

<P> Hosted implementations of C++ have long had the ability to collect
object files from individual translation units into a single
entity. Often, these collections of object files are called
<B>libraries</B>, and simplify software creation and maintenance by
clearly separating out dependencies, and providing interfaces between
components.
</P>

<P>For the purposes of this paper, there are two main types of
libraries.  The first, a static library, copies used elements from the
library directly to the created executable. Thus, when an executable
is created by statically linking against a library, the result is a
system with two copies of the library. (The original library, and the
newly-formed executable.)
</P>

<P> The second type of library is one without duplication, and is a
dynamic library. Instead of duplicating the data, when an executable
is created by dynamically linking against a library, the
implementation performs a magic step that allows the end result to
reference the library directly, without a copy.
</P>

<P> This paper will be concerned solely with the second type of
library, and with dynamic shared objects in particular. The use of
this kind of library in the C++ development community is widespread,
and has been in existence for over ten years. Years of use have
pointed out some of the pitfalls with parts of the C++ language and
current implementations: it is the goal of this paper to provide a
survey of current dynamic linking capabilities, techniques, and to
explicitly quantify known issues.
</P>
<P><BR><BR>
</P>

<H2>Terminology</H2>
<P>The C++ standard defines three distinct linkage types: internal,
external, none. (See 3.5 - Program and linkage [basic.link])
</P>

<P>Adopt the notion of <B>load unit</B> from Austern indicating a
binding of individual translation units together to form a single
group. (Via some undefined mechanism.) 
</P>
<P>Define <B>symbol</B> as a definition for a specific entity in a load unit.
</P>
<P>Adopt <B>visibility</B> to determine if a symbol can be used
outside of the load unit. In addition, adopt the following refinements:
</P>
<P>An entity defined within a load unit with <B>external visibility</B>
implies that other load units are able to use the defined symbol.
</P>
<P>An entity defined within a load unit with <B>internal visibility</B>
implies that other load units are not able to use the defined symbol.
</P>
<P>Adopt the notion of <B>load set</B> from Austern indicating the
closed set of all individual load units. 
</P>
<P><BR><BR>
</P>


<H2>High Level Survey of Current Techniques</H2>
<P><B>Windows</B> 
</P>
<P>Dynamic linking meta-picture: 
</P>
<P>.exe -&gt; .lib -&gt; .dll 
</P>
<P>Where .dll is the shared object, with symbol definitions. The .lib
is a stubs library, which contains a list of symbols to be resolved
in the .dll at runtime. The .exe is the final executable, and it's
dependencies are resolved against the .lib at link time. 
</P>

<p> Default visibility is internal. Annotations required both for
marking a symbol with external visibility and for importing an
external symbol.
</p>

<P>Visibility control techniques include: 
</P>
<P>One: decoration via __declspec(dllexport) and
__declspec(dllimport) on a class and member function basis, but not
both. Also allowed template specializations. 
</P>
<P>Two: by a text file containing a list of symbols to export or by
ordinal.
</P>

<P>Dynamic loading via LoadLibrary, where you can set resolution to
immediate or delayed. 
</P>
<P>Other notes: On class-scope visiblity decorations, all member
functions, static data members, virtual functions, typeinfo, etc are
visible along with the same for any base classes. Operator new
inter-position is known not to work. Versioning capability on .dll
include major, minor, build date.
</P>

<P><B>SVR4 (Sun/Linux)</B> 
</P>
<P>Dynamic linking meta-picture: 
</P>
<P>.exe -&gt; .so 
</P>
<P>Where .so is the shared object, with symbol definitions.The .exe
is the final executable, and it's dependencies are resolved against
the .lib at runtime. 
</P>

<p> Default visibility is external. Annotation or other method
required for marking a symbol with internal visibility.

</p>

<P>Visibility control techniques include:
</P>
<P>One: decoration via __attribute__((visibility(option))) where
option is one of: hidden, default, internal, protected. 
</P>
<P>Two: #pragma interface/#pragma implementation 
</P>
<P>Three: extern template, -fno-implicit-templates, and inlining
declarations 
</P>
<P>Four: #pragma GCC visibility push(hidden) /#pragma GCC visibility
pop in combination with -fvisibility,-fvisibility-inlines-hidden 
</P>
<P>Five: a text file containing a list of symbols to export which
is used as an input file to the linker, with optional minor version
refinement. Symbols with C++ linkage (ie mangled) can be exported in
namespace globs in an un-mangled syntax.
</P>

<P>Version control techniques include: 
</P>
<P>One:Versioning of libraries happens exclusive using the SONAME, a
simple string that is part of the object file format (ELF).  There can
be exactly one library for a given SONAME.
<P>Two: Versioning capability on a library include major, minor, minor
with refinements, aliasing and renaming. These versioning details are
explicitly specified in a text file that the linker uses during
library creation.
</P>
<P>Three: Explicit version number mangling on nested namespaces, and
then injecting these versioned names into the enclosing namespace via
the GNU compiler extensions described by namespace associations.
</P>
<P>Dynamic loading via dlopen/dlmopen/dlsym/dlclose, where you can
set resolution to immediate or delayed. 
</P>
<P>Other notes: Operator new
inter-position can be made to work. 
The -fvisibility and #pragma visibility options are known to not work
and or have serious flaws. 
</P>
<P><BR><BR>
</P>

<H2>Common functionality, by example</H2>
<P>
<A HREF="#example_01">1: Sharing load unit by dynamic linking</A><BR>
<A HREF="#example_02">2: Sharing load unit by dynamic loading</A><BR>
<A HREF="#example_03">3: Load unit with restricted external visibility</A><BR>
<A HREF="#example_04">4: Load unit with versioned external visibility</A><BR>
</P>

<p> In the following examples, the following color key is used:
annotations and sources in light blue are for <PRE STYLE="background:
#ccccff"> SVR4 (Sun/Linux) </PRE>
</p>

<p>sources and annotations in light red are for
<PRE STYLE="background: #ffccff"> 
Windows
</PRE>
</p>

<p>and platform-independent code will be in gray
<PRE STYLE="background: #c0c0c0"> 
as in this neutral color
</PRE>
</p>

<BR><BR>

<H4><A NAME="example_01">One: Sharing load unit by dynamic linking.</A>
</H4>

<P>The fundamental example for dynamic linking. A load unit (libfoo)
that contains a function (get_city) is then shared by an executables
(one.exe) at runtime by compile-time linking.
</P>
<P>
<I>foo.h</I>
</P>
<PRE STYLE="background: #ccccff"> 
extern const char* get_city();
</PRE>
<PRE STYLE="background: #ffccff"> 
extern const char* __declspec(dllimport) get_city();
</PRE>
<P>
<I>foo.cc </I>
</P>
<PRE STYLE="background: #ccccff"> 
static const char* city = &quot;mont tremblant&quot;;

const char* get_city()
{ return city; }</PRE>
<PRE STYLE="background: #ffccff"> 
static const char* city = &quot;mont tremblant&quot;;

const char* __declspec(dllexport) get_city()
{ return city; }</PRE>
<P>
<I>one.cc </I>
</P>
<PRE STYLE="background: #c0c0c0"> 
#include &lt;cstring&gt;
#include &quot;foo.h&quot;

int check_city_one()
{
  return std::strcmp(get_city(), &quot;chicago&quot;);
}

int main() 
{ return check_city_one(); }</PRE><P>
On linux, the situation outlined above is constructed as follows: 
</P>
<P><CODE>g++ -shared -fPIC -O2 -g foo.cc -o libfoo.so <BR>
g++ -g -O2 -L. one.cc -lfoo -o one.exe <BR></CODE><BR><BR>
</P>
<P><BR><BR>
</P>

<H4><A NAME="example_02">Two: Sharing load unit by dynamic loading. </A>
</H4>

<P>The fundamental example for dynamic loading. A load unit (libfoo)
that contains a function (get_city) is then shared by an executables
(one.exe) by loading at runtime.
</P>
<P>
<I>foo.h</I>
</P>
<PRE STYLE="background: #ccccff"> 
extern const char* get_city();
</PRE>
<PRE STYLE="background: #ffccff"> 
extern const char* __declspec(dllimport) get_city();
</PRE>
<P>
<I>foo.cc </I>
</P>
<PRE STYLE="background: #ccccff"> 
static const char* city = &quot;mont tremblant&quot;;

const char* get_city()
{ return city; }</PRE>
<PRE STYLE="background: #ffccff"> 
static const char* city = &quot;mont tremblant&quot;;

const char* __declspec(dllexport) get_city()
{ return city; }</PRE>
<P>
<I>two.cc </I>
</P>
<PRE STYLE="background: #ccccff"> 
#include &lt;dlfcn.h&gt;
#include &lt;cstring&gt;
#include &lt;stdexcept&gt;
#include &quot;foo.h&quot;

const char*
mangle(const char* unmangled)
{
  // GNU
  const char* mangled = &quot;_Z8get_cityv&quot;;
  return mangled;
}

void
dynamic_open(void*&amp; h)
{
  dlerror();
  void* tmp = dlopen(&quot;./libfoo.so&quot;, RTLD_LAZY);
  if (!tmp) 
    {
      try 
        {
          // Throws std::logic_error on NULL string.
          std::string error(dlerror());
          throw std::runtime_error(error);
        }
      catch (const std::logic_error&amp;)
        { }
    }
  h = tmp;
}

void
get_and_execute_dynamic_symbol(void*&amp; h)
{
  dlerror();

  typedef void (*function_type) (void);
  function_type fn;
  fn = reinterpret_cast&lt;function_type&gt;(dlsym(h, mangle(&quot;get_city&quot;)));

  try 
    {
      std::string error(dlerror());
      throw std::runtime_error(error);
    }
  catch (const std::logic_error&amp;)
    { }

  fn();
}

void
dynamic_close(void*&amp; h)
{
  if (dlclose(h) != 0)
    {
      try 
        {
          std::string error(dlerror());
          throw std::runtime_error(error);
        }
      catch (const std::logic_error&amp;)
        { }
    }
}

int main() 
{ 
  void* h;
  dynamic_open(h);
  get_and_execute_dynamic_symbol(h);
  dynamic_close(h);
  return 0;
}</PRE>

<PRE STYLE="background: #ffccff"> 
#include &lt;windows.h&gt;
#include &lt;cstring&gt;
#include &lt;stdexcept&gt;
#include &quot;foo.h&quot;

const char*
mangle(const char* unmangled)
{
  // Microsoft
  const char* mangled = &quot;?get_city@@YAXXZ&quot;;
  return mangled;
}

void
dynamic_open(HINSTANCE&amp; h)
{
  HINSTANCE tmp;
  tmp = LoadLibrary(&quot;./libfoo.so&quot;);
  if (!tmp) 
    {
      throw std::runtime_error(error);
    }
  h = tmp;
}

void
get_and_execute_dynamic_symbol(HINSTANCE&amp; h)
{
  typedef void (*function_type) (void);
  function_type fn;
  fn = reinterpret_cast&lt;function_type&gt;(GetProcAddress(h, mangle(&quot;get_city&quot;)));

  if (!fn) 
    {
      throw std::runtime_error(error);
    }

  (fn)();
}

void
dynamic_close(HINSTANCE&amp; h)
{
  if (FreeLibrary(h) != 0)
    {
      throw std::runtime_error(error);
    }
}

int main() 
{ 
  HINSTANCE h;
  dynamic_open(h);
  get_and_execute_dynamic_symbol(h);
  dynamic_close(h);
  return 0;
}</PRE>

<P>
On linux, the situation outlined above is constructed as follows: 
</P>
<P><CODE>g++ -shared -fPIC -O2 -g foo.cc -o libfoo.so <BR>
g++ -g -O2 two.cc -ldl -o two.exe <BR></CODE><BR><BR>
</P>
<P><BR><BR>
</P>

<H4><A NAME="example_03">Three: Load unit with restricted external visibility. </A>
</H4>

<p> There are at least three common techniques available for
specifying visibility of individual entities in a load unit. Three
will be detailed here: the use of compiler-specific pragmas, the use
of vendor-specific decorations on types and declarations, and finally
the use of vendor-specific link maps.
</p>

<H6><A NAME="example_03a">3a: Pragmas. </A>
</H6>

<P><I>container.cc</I>
</P>
<PRE STYLE="background: #ccccff"> 
#pragma GCC visibility push(hidden)
//#pragma GCC visibility push(default)

typedef int value_type;

class foo
{
  value_type v;

 public:
  foo();

  virtual ~foo() { }

  value_type&
  get_vector();
};

foo::foo()
{
  value_type apple;
  v = 1;
}

value_type&
foo::get_vector()
{ return v; }

void
swap_foo()
{
  value_type empty;
  foo f;
}

#pragma GCC visibility pop
</PRE>

<P>
On linux, the situation outlined above is constructed as follows: 
</P>
<P><CODE>g++ -g -c container.cc<BR>
</P>
<P>
Regardless of any other compiler flags (ie
<code>-fvisibility=hidden</code>) the defined symbols will have the
visibility as noted in the pragma.
</P>
<P><BR><BR>
</P>

<H6><A NAME="example_03b">3b: Decoration. </A>
</H6>

<P><I>container.cc</I>
</P>
<PRE STYLE="background: #ccccff"> 
#ifdef VIS_EXTERNAL
#define VIS __attribute__ ((visibility("default")))
#else
#define VIS __attribute__ ((visibility("hidden")))
#endif

typedef int value_type;

class VIS foo
{
  value_type v;

 public:
  foo();

  virtual ~foo() { }

  value_type&
  get_vector();
};

foo::foo()
{
  value_type apple;
  v = 1;
}

value_type&
foo::get_vector()
{ return v; }


void VIS
swap_foo()
{
  value_type empty;
  foo f;
}
</PRE>

<P>
On linux, the situation outlined above is constructed as follows: 
</P>
<P><CODE>g++ -g -c container.cc<BR>
</P>
<P>
Regardless of any other compiler flags (ie
<code>-fvisibility=hidden</code>) the defined symbols will have the
visibility as noted in the pragma.
</P>
<P><BR><BR>
</P>


<H6><A NAME="example_03c">3b: Lists. </A>
</H6>

<P><I>container.cc</I>
</P>
<PRE STYLE="background: #ccccff"> 
typedef int value_type;

class foo
{
  value_type v;

 public:
  foo();

  virtual ~foo() { }

  value_type&
  get_vector();
};

foo::foo()
{
  value_type apple;
  v = 1;
}

value_type&
foo::get_vector()
{ return v; }


void
swap_foo()
{
  value_type empty;
  foo f;
}
</PRE>

<P><I>container.ver</I>
</P>
<PRE STYLE="background: #ccccff"> 
{
  global:
        _Z8swap_foov;
  local: *;
};
</PRE>

<P>
On linux, the situation outlined above is constructed as follows: 
</P>
<P><CODE>g++ -g -Wl,--version-script=container.ver -shared container.cc -o container.so<BR>
</P>
<P>
Other compiler flags (ie <code>-fvisibility=hidden</code>) will
prevail over the visibility as noted in the link map. 
</P>
<P><BR><BR>
</P>

<H4>
<A NAME="example_04">Four: Load unit with versioned external visibility.</A>
</H4>

<p> On occasion, it is possible to safely extend class declarations
and other entities over time. In order to have this work, there must
be a way to attach a version to symbols in a load unit with external
visibility.
</p>

<p> For instance, this class:
</p>

<P><I>container.h</I>
</P>
<PRE STYLE="background: #ccccff"> 
struct foo
{
  foo();

  int
  get_value();

  private:
  int v;
};
</PRE>

<P><I>container.cc</I>
</P>
<PRE STYLE="background: #ccccff"> 
#include "container.h"

foo::foo()
{ v = 1; }

int
foo::get_value()
{ return v; }
</PRE>

<P><I>container.ver</I>
</P>
<PRE STYLE="background: #ccccff"> 
VERSION_1.0
{
  global:
        _ZN3foo9get_valueEv;
  local: *;
};
</PRE>

<P>
On linux, the situation outlined above is constructed as follows: 
</P>
<P><CODE>g++ -g -Wl,--version-script=container.ver -shared container.cc -o container.so<BR>
</P>

<p> At this point, the only externally-visible symbol is
<code>foo::get_value()</code>, and this symbol is versioned with the
tag <code>VERSION_1.0</code>.
</p>

<p> After a period of use, a new feature is added and as part of this,
a new member function is added to struct foo, say
<code>foo::get_second_value()</code>, and this symbol is versioned
with the tag <code>VERSION_1.1</code>. A new version of container.so
is generated, with both the member functions defined with the
corresponding version tag. This allows newer code to use the new
member function, but allows a graceful system response if this newer
code is run in an environment without the newer container.so.
</p>

<P><I>container.ver</I>
</P>
<PRE STYLE="background: #ccccff"> 
VERSION_1.0
{
  global:
        _ZN3foo9get_valueEv;
  local: *;
};

VERSION_1.1
{
  global:
        _ZN3foo16get_second_valueEv;
  local: *;
} VERSION_1.0;
</PRE>
<P>
This is constructed in the same manner as the first container.so file.
</P>
<P><BR><BR>
</P>


<H2>Known Problems, by example</H2>
<P>
<A HREF="#problem_01">1: Overriding global operator new</A> <BR>
<A HREF="#problem_02">2: Order of initialization</A> <BR>
<A HREF="#problem_03">3: Exceptions across load units</A> <BR>
<A HREF="#problem_04">4: Vague linkage and duplicate symbol resolution</A> <BR>
</P>
<BR><BR>

<H4><A NAME="problem_01"></A>Problem One: Overriding global operator
new. 
</H4>
<P>Standard defined behavior for global scope operator new signatures
allow users to provide custom definitions and override the default.
What happens if there are two user-defined operator new definitions:
which one is picked? This has been referred to as the operator new
&quot;inter-position&quot; issue, but could be generalized to
load-unit allocation/deallocation problems. It is important that
allocation and deallocation mechanisms match across load unit
boundaries. 
</P>
<p> There are other issues with memory management and multiple load
units. A big question is how to keep the allocator equality
requirements (ie, an instance of an allocator is equal to another
allocator iff one can free the other's allocation.)
</p>

<P><I>foo.cc </I></P>
<PRE STYLE="background: #ccccff"> 
#include &lt;cstdio&gt;
#include &lt;new&gt;
#include &lt;tr1/array&gt;

std::string*
get_string()
{ return new std::string("olive street beach"); }

void
dispose_string(std::string* s)
{ delete s; }

// Fixed external storage.
typedef std::tr1::array&lt;char, 256&gt; array_type;
static array_type _M_array;

void* operator new(std::size_t __n) throw (std::bad_alloc)
{
  puts("operator new");
  static std::size_t __array_used;
  if (__array_used + __n &gt; _M_array.size())
    std::bad_alloc();
  void* __ret = _M_array.begin() + __array_used;
  __array_used += __n;
  return __ret;
}

void operator delete(void*) throw() 
{
  // Does nothing.
  puts("operator delete");
}
</PRE>

<P><I>foo.ver </I></P>
<PRE STYLE="background: #ccccff"> 
VERSION_1.0
{
  global:
	_Z10get_stringv;
	_Z14dispose_stringPSs;
  local: *;
};
</PRE>

<P><I>test.cc </I></P>
<PRE STYLE="background: #ccccff"> 
#include &lt;new&gt;
#include &lt;vector&gt;
#include &lt;string&gt;

extern std::string* get_string();
extern void dispose_string(std::string*);

int main()
{
  typedef std::vector&lt;int&gt; vector_type;
  vector_type* v;
  try
    {
      v = new vector_type(100);
    }
  catch (const std::exception& e)
    {
      puts(e.what());
      throw;
    }
  catch (...)
    { throw; }
  delete v;

  std::string* s = get_string();
  dispose_string(s);

  return 0;
}
</PRE>

<P>
Construct the example as follows: 
</P>
<P>
<CODE>
g++ -shared -fPIC -O0 -g foo.cc -o libfoo.so
<br>
g++ -g -O0 -L. test.cc -lfoo -o problem1.exe
</CODE>
</P>
<P>And then run the resulting executable: 
</P>
<PRE>
%./problem1.exe
operator new
operator new
operator delete
operator delete
operator new
operator new
operator delete
operator delete
</PRE>

<P> As suspected, the operator new definitions in libfoo are used in
problem1.exe as well, overriding the default definitions in the
standard library. Although a trivial example, the problem is real: any
load unit that redefines operator new could, when added to a new load
set, change underlying allocations.
</P>

<P> One way around this is to limit the visibility of the operator new
and operator delete definitions to within the libfoo.so load unit. 
</P>

<CODE>
g++ -shared -fPIC -O0 -g foo.cc -Wl,--version-script=foo.ver -o libfoo.so
<br>
g++ -g -O0 -L. test.cc -lfoo -o problem1.exe
</CODE>

<P>And then run the resulting executable: 
</P>

<PRE>
%./problem1.exe
operator new
operator delete
</PRE>

<p> By limiting the visibility, the operator new and delete
definitions can be bound to a specific load unit.
</p>

<P><BR><BR>
</P>

<H4><A NAME="problem_02"></A>Problem Two: Order of initialization. 
</H4>
<P>How are global objects supposed to be initialized and finalized in
scenarios with multiple accesses and accesses that can be opened and
closed at will? In addition, using static local objects may run into
issues with initialization.
</P>

<P>
<I>foo.cc </I>
</P>
<PRE STYLE="background: #ccccff"> 
#include &lt;cstdio&gt;

struct A
{
   A()
   { puts("A ctor"); }

   ~A()
   { puts("A dtor"); }
};

void f()
{ static A obj; } 
</PRE>

<P>
<I>test.cc </I>
</P>
<PRE STYLE="background: #ccccff"> 
#include &lt;cstdio&gt;

extern void f();

struct B
{
   B()
   { puts("B ctor"); }

   ~B()
   { puts("B dtor"); }
};

static B foo;

int main()
{
   f();
   return 0;
}
</PRE>

On linux, the situation outlined above is constructed as follows: 
</P>
<P><CODE>g++ -shared -fPIC -O2 -g foo.cc -o libfoo.so <BR>
g++ -g -O2 -L. test.cc -lfoo -o two.exe <BR></CODE>
</P>
<P>And then run the resulting executable: 
</P>
<PRE>
%./one.exe
B ctor
A ctor
A dtor
B dtor
</PRE>

<P> This ordering is correct. However, on windows:</P>

<PRE>
B ctor
A ctor
B dtor
A dtor
</PRE>

<p>... which demonstrates the issue of ordering objects across
different load units.
</p>

<BR><BR>

<H4><A NAME="problem_03"></A>Problem Three: Exceptions across load units. 
</H4>
<P>Compiler-generated information for virtual functions and typeinfo
has vague linkage that is difficult for the programmer to control
given the language facilities available in standard C++. Because of this,
default visibility and the order of binding of symbols all impact the
ability to throw and catch exceptions across load units. 
</P>
<P>There is an underspecification of this necessary compiler-generated
magic with respect to multiple load units.  Symptomatic of this
include use of typeid (typeinfo) and throwing exceptions across load
units, inlining template member functions defined in multiple load
units (each with unique addresses), and the use of inheritance in
multiple load units (do base class definitions and
implicitly-generated data have to be visible across load units?)
</P>
<P><I>error_handling.h </I></P>
<PRE STYLE="background: #ccccff"> 
#include &lt;stdexcept&gt;

struct insert_error : public std::runtime_error
{
  insert_error(const std::string&amp;);
};

void check_insert();
</PRE>
<P>
<I>error_handling.cc </I>
</P>
<PRE STYLE="background: #ccccff"> 
#include &quot;error_handling.h&quot;

insert_error::insert_error(const std::string&amp; s) : std::runtime_error(s) { };

void check_insert()
{
  // Do something, assume it's wrong.
  throw insert_error(&quot;check_insert: something happened&quot;);
}</PRE><P>
<I>error_handling.ver </I>
</P>
<PRE STYLE="background: #ccccff"> 
VERSION_1.0
{
  global:
        _ZN12insert_errorC*;
#        _ZTS12insert_error;
        _Z12check_insertv;
  local: *;
};</PRE><P>
<I>test.cc </I>
</P>
<PRE STYLE="background: #ccccff"> 
#include &quot;error_handling.h&quot;
#include &lt;iostream&gt;

int main()
{
  try
    {
      check_insert();
    }
  catch (const insert_error&amp; e)
    {
      // 1: Expect catch here.
    }
  catch (const std::exception&amp; e)
    {
      // 2: Visibility issues may lead to catch here.
      std::cout &lt;&lt; &quot;caught object of type: &quot; &lt;&lt; typeid(e).name() &lt;&lt; std::endl;
    }
  catch (...)
    {
      // 3: Catch all.
      throw;
    }

  return 0;
}</PRE>

<P>
Construct the example as follows: 
</P>
<P>
<CODE>g++ -shared -fPIC -O2 -g error_handling.cc
-Wl,--version-script=error_handling.ver -o libfoo.so <BR>
g++ -g -O2 -L. test.cc -lfoo -o problem3.exe 
</CODE>
</P>
<P>And then run the resulting executable: 
</P>
<PRE>%./problem3.exe
caught object of type: 12insert_error</PRE>

<P>
Without typeinfo name sharing between libfoo and
problem3.exe, the execution path is non-intuitive and in error. The
typeinfo information is generated with vague linkage in both libfoo
and problem3.exe. By allowing the export of _ZTS12insert_error from
error_handling.ver, both will end up using the vague typeinfo name
stored in the executable, and exception handling will work as
expected. 
</P>
<P><BR><BR>
</P>

<H4><A NAME="problem_04"></A>Problem Four: Vague linkage and
duplicate symbol resolution. 
</H4>
<P>Template classes with member functions can have instantiations in
multiple files. To prevent duplicate symbols, many compilers have
implemented vague linkage semantics that coalesce multiple,
equivalent symbol names across translations units into one
definition. Picking one version across multiple dynamic load units is
tricky. Depending on the order in which different load units are
initialized, the definition that is picked for all the other load
units in a given load set may change, and may end up being different
than expected or planned when the individual load units were
constructed. Symptomatic of this problem are multi-ABI binaries,
where different compilers define the same symbol in a given load set.
Also implicated are template designs that depend on macro defines to
change behavior. 
</P>
<P>The end result is similar, in both cases: load units that have
porous boundaries, that end up using symbols defined elsewhere in the
load set to resolve symbols within the original load unit. 
</P>
<P><I>container.h </I>
</P>
<PRE STYLE="background: #ccccff"> 
template&lt;typename T&gt;
  class container
  {
    T data;

    void
    do_private();
    
  public:

    container(T value = T()) : data(value) { }

    void 
    do_public() 
    { return do_private(); }

    T
    get_data() { return data; }
  };


template&lt;typename T&gt;
void container&lt;T&gt;::do_private()
{
#ifdef OLD_VERSION
  // Clear.
  data = T();
#else
  // Multiply.
  data *= 2;
#endif
}

typedef container&lt;int&gt; container_type;</PRE><P>
<I>foo.cc </I>
</P>
<PRE STYLE="background: #ccccff"> 
#include &quot;container.h&quot;
#include &lt;iostream&gt;

void foo()
{
  container_type obj(4);
  obj.do_public();
  std::cout &lt;&lt; obj.get_data() &lt;&lt; std::endl;
}</PRE><P>
<I>test.cc </I>
</P>
<PRE STYLE="background: #ccccff"> 
#include &quot;container.h&quot;

extern void foo();

int main()
{
  container_type obj(2);
  
  // Use do_public, get weak definition.
  obj.do_public();

  // Call external function on the rest.
  foo();

  return 0;
}</PRE><P>
Construct the example as follows: 
</P>
<P><CODE>g++ -g -O2 -fPIC -shared foo.cc -o libfoo.so <BR>g++
-DOLD_VERSION -g -O2 -L. test.cc -lfoo -o problem4.exe </CODE>
</P>
<P>And then run the resulting executable: 
</P>
<PRE>%./problem4.exe 
0</PRE>

<P> Both libfoo.so and problem4.exe have vague linkage for
container&lt;int&gt;::do_private(). (Other options include no
definition, leading to undefined symbols or both having definitions,
and duplicate symbol errors.)As a result, it is system-defined and
order dependent which definition will be picked for both uses. On
linux, when problem4.exe is loaded, its symbol is used for all other
uses, including the one in libfoo.so. This is probably not what was
intended by the author of libfoo.so. In addition, the behavior of
libfoo.so now depends on optimization options: without optimization,
the un-inlined function will pick up the symbol from the executable,
and with optimization that results in inlining (try -O3) the behavior
of libfoo will change.
</P>
<P><BR><BR>
</P>

<H2>Impact on Existing Standard by Chapter</H2>
<UL>
	<LI><P STYLE="margin-bottom: 0in">01. Runtime/execution model needs
	to be modified to address load unit. 
	</P>
	<LI><P STYLE="margin-bottom: 0in">02. Are new keywords needed to
	express the idea of controlling visibility of specific entities in a
	given load unit. Ie, visible or invisible, public or private, hidden
	or &quot;exported&quot;? Multiple layers of visibility (ie
	versioning)? 
	</P>
	<LI><P STYLE="margin-bottom: 0in">03. ODR scope object lifetime
	storage duration startup/termination -- what order? linkage 
	</P>
	<LI><P STYLE="margin-bottom: 0in">05. Operators new/delete, typeid,
	dynamic_cast 
	</P>
	<LI><P STYLE="margin-bottom: 0in">07. Static, extern vs. load units
	(linkage). Namespace-scope visibility? 
	</P>
	<LI><P STYLE="margin-bottom: 0in">09. Class linkage changes. What if
	not all members of a class are visible. Differing visibility between
	nested and enclosing classes, or local and enclosing classes. Will
	nested classes and nested namespaces have the same semantics? 
	</P>
	<LI><P STYLE="margin-bottom: 0in">10. Issues with vtable, typeinfo
	visibility across multiple load units. Do base class vtables have to
	be visible in order to use a derived class in a different load unit?
		</P>
	<LI><P STYLE="margin-bottom: 0in">12. Tons of stuff, including
	constructors, destructors, operator new and delete. 
	</P>
	<LI><P STYLE="margin-bottom: 0in">15. Detail throw/catch exceptions
	across multiple load units.
	</P>
	<LI><P STYLE="margin-bottom: 0in">18. Most everything. 
	</P>
	<LI><P>19-27. How does this impact C++ standard library? 
	</P>
</UL>
<P><BR><BR>
</P>

<H2>Solution Space</H2>
<P>Going from least to most ambitious. 
</P>
<UL>
	<LI><P STYLE="margin-bottom: 0in">Nothing, go with the status quo. 
	</P>
	<LI><P STYLE="margin-bottom: 0in">Attempt a TR, or &quot;best
	practices&quot; document with suggestions for clients and vendors. 
	</P>
	<LI><P STYLE="margin-bottom: 0in">Come up with standard terminology,
	select a common subset of what's possible and figure out syntax for
	expressing it portably. Suspect that many C++ features will fall by
	the wayside. 
	</P>
	<LI><P STYLE="margin-bottom: 0in">Come up with standard terminology,
	and figure out syntax for expressing it. Includes specification for
	C++-specific requirements like throwing exceptions, templates,
	template specializations, and vague linkage. 
	</P>
	<LI><P>Come up with new C++ constructs, for example modules,
	annotation rules for namespaces, etc. 
	</P>
</UL>
<P>
</P>
<BR><BR>

<H2>Acknowledgements</H2>
<p>Contributors to the Modules and Linkage discussions at Mont
Treblant (2005) and Berlin(2006), not limited to: David Vandevoorde,
Mat Marcus, Judy Ward, PremAnand Rao, Doug Harrison, Bronek Kozicki,
Eugene Gershnik, and Thomas Witt.
</p>
<BR><BR>

<H2>References</H2>
<P>Matthew Austern. Toward standardization of dynamic libraries.
Technical Report N1400=02-0058, Sep 25, 2002. 
</P>
<P>Pete Becker. Draft Proposal for Dynamic Libraries in C++.
Technical Report N1428=03-0010, March 3, 2003. 
</P>
<P>Daveed Vandevoorde. Modules in C++ (Revision 2). Technical Report
N1778=05-0038, January 2005. 
</P>
<P>John R. Levine. Linkers and Loaders. Morgan Kaufmann, January 15,
2000. 
</P>

<P>Ulrich Drepper. <A HREF="http://people.redhat.com/drepper/dsohowto.pdf">How
To Write Shared Libraries.</A> January, 2005. 
</P>

<P>Ulrich Drepper provided feedback and corrections on previous drafts
of this document. </P>


<P>Doug Harrison. DllHelper_0.90.
</P>

<P>Jonathan H. Lundquist. 270. Order of initialization of static data
members of class templates. CWG Defects.
</P>

<P>Mark Mitchell. 362. Order of initialization in instantiation units. CWG Defects.
</P>

<P>Sun Microsystems. <A HREF="http://docs.sun.com/app/docs/doc/817-1983?q=linker">Linker
and Library Guide.</A> 2002. 
</P>
<P>Microsoft. <A HREF="http://msdn2.microsoft.com/en-us/library/1ez7dh12">DLLs.</A>
</P>
<P>Microsoft. <A HREF="http://msdn2.microsoft.com/en-us/library/ms141260">Walkthrough:
Creating and Using a Dynamic Link Library.</A> 
</P>
<P>Apple. <A HREF="http://developer.apple.com/documentation/DeveloperTools/Conceptual/CppRuntimeEnv/index.html">Overview
of the C++ Runtime Environment.</A> 
</P>

<p>ACE shared object wrappers, ie. ACE_Shared_Object.
</p>

<p>Mat Marcus. Typeinfo comparison code easily breaks shared libs.
 http://gcc.gnu.org/PR23628
</p>

<P>Ralf W. Grosse-Kunstleve, David Abrahams, Jason Merrill. Minimal
GCC/Linux shared lib + EH bug example.
http://mail.python.org/pipermail/python-dev/2002-May/023988.html<BR>http://gcc.gnu.org/ml/gcc/2002-05/msg00882.html
<BR>http://sources.redhat.com/ml/libc-alpha/2002-05/msg00222.html 
</P>

<P> Doug Harrison. Order in court! 
http://groups.google.com/group/microsoft.public.vc.mfc/msg/438bdfa1dc683ce1?hl=en&
</P>

<P> Doug Harrison. com_ptr_t as static object in a DLL.
http://groups.google.com/group/microsoft.public.vc.mfc/msg/21cfdeb16358e755?hl=en&
</P>

<p>
Static variable in template function across compilation units.
http://groups.google.com/group/comp.lang.c++.moderated/msg/50091d17e98a36a0?hl=en
</p>


</BODY>
</HTML>
