<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.14: http://docutils.sourceforge.net/" />
<title>Let's Talk About Package Specification</title>
<meta name="date" content="2018-10-07" />
<meta name="author" content="Matthew Woehlke (mwoehlke.floss&#64;gmail.com)" />
<style type="text/css">

/*
:Author: David Goodger (goodger@python.org)
:Id: $Id: html4css1.css 7952 2016-07-26 18:15:59Z milde $
:Copyright: This stylesheet has been placed in the public domain.

Default cascading style sheet for the HTML output of Docutils.

See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/

/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
  border: 0 }

table.borderless td, table.borderless th {
  /* Override padding for "table.docutils td" with "! important".
     The right padding separates the table cells. */
  padding: 0 0.5em 0 0 ! important }

.first {
  /* Override more specific margin styles with "! important". */
  margin-top: 0 ! important }

.last, .with-subtitle {
  margin-bottom: 0 ! important }

.hidden {
  display: none }

.subscript {
  vertical-align: sub;
  font-size: smaller }

.superscript {
  vertical-align: super;
  font-size: smaller }

a.toc-backref {
  text-decoration: none ;
  color: black }

blockquote.epigraph {
  margin: 2em 5em ; }

dl.docutils dd {
  margin-bottom: 0.5em }

object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] {
  overflow: hidden;
}

/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
  font-weight: bold }
*/

div.abstract {
  margin: 2em 5em }

div.abstract p.topic-title {
  font-weight: bold ;
  text-align: center }

div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
  margin: 2em ;
  border: medium outset ;
  padding: 1em }

div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
  font-weight: bold ;
  font-family: sans-serif }

div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title, .code .error {
  color: red ;
  font-weight: bold ;
  font-family: sans-serif }

/* Uncomment (and remove this text!) to get reduced vertical space in
   compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
  margin-bottom: 0.5em }

div.compound .compound-last, div.compound .compound-middle {
  margin-top: 0.5em }
*/

div.dedication {
  margin: 2em 5em ;
  text-align: center ;
  font-style: italic }

div.dedication p.topic-title {
  font-weight: bold ;
  font-style: normal }

div.figure {
  margin-left: 2em ;
  margin-right: 2em }

div.footer, div.header {
  clear: both;
  font-size: smaller }

div.line-block {
  display: block ;
  margin-top: 1em ;
  margin-bottom: 1em }

div.line-block div.line-block {
  margin-top: 0 ;
  margin-bottom: 0 ;
  margin-left: 1.5em }

div.sidebar {
  margin: 0 0 0.5em 1em ;
  border: medium outset ;
  padding: 1em ;
  background-color: #ffffee ;
  width: 40% ;
  float: right ;
  clear: right }

div.sidebar p.rubric {
  font-family: sans-serif ;
  font-size: medium }

div.system-messages {
  margin: 5em }

div.system-messages h1 {
  color: red }

div.system-message {
  border: medium outset ;
  padding: 1em }

div.system-message p.system-message-title {
  color: red ;
  font-weight: bold }

div.topic {
  margin: 2em }

h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
  margin-top: 0.4em }

h1.title {
  text-align: center }

h2.subtitle {
  text-align: center }

hr.docutils {
  width: 75% }

img.align-left, .figure.align-left, object.align-left, table.align-left {
  clear: left ;
  float: left ;
  margin-right: 1em }

img.align-right, .figure.align-right, object.align-right, table.align-right {
  clear: right ;
  float: right ;
  margin-left: 1em }

img.align-center, .figure.align-center, object.align-center {
  display: block;
  margin-left: auto;
  margin-right: auto;
}

table.align-center {
  margin-left: auto;
  margin-right: auto;
}

.align-left {
  text-align: left }

.align-center {
  clear: both ;
  text-align: center }

.align-right {
  text-align: right }

/* reset inner alignment in figures */
div.align-right {
  text-align: inherit }

/* div.align-center * { */
/*   text-align: left } */

.align-top    {
  vertical-align: top }

.align-middle {
  vertical-align: middle }

.align-bottom {
  vertical-align: bottom }

ol.simple, ul.simple {
  margin-bottom: 1em }

ol.arabic {
  list-style: decimal }

ol.loweralpha {
  list-style: lower-alpha }

ol.upperalpha {
  list-style: upper-alpha }

ol.lowerroman {
  list-style: lower-roman }

ol.upperroman {
  list-style: upper-roman }

p.attribution {
  text-align: right ;
  margin-left: 50% }

p.caption {
  font-style: italic }

p.credits {
  font-style: italic ;
  font-size: smaller }

p.label {
  white-space: nowrap }

p.rubric {
  font-weight: bold ;
  font-size: larger ;
  color: maroon ;
  text-align: center }

p.sidebar-title {
  font-family: sans-serif ;
  font-weight: bold ;
  font-size: larger }

p.sidebar-subtitle {
  font-family: sans-serif ;
  font-weight: bold }

p.topic-title {
  font-weight: bold }

pre.address {
  margin-bottom: 0 ;
  margin-top: 0 ;
  font: inherit }

pre.literal-block, pre.doctest-block, pre.math, pre.code {
  margin-left: 2em ;
  margin-right: 2em }

pre.code .ln { color: grey; } /* line numbers */
pre.code, code { background-color: #eeeeee }
pre.code .comment, code .comment { color: #5C6576 }
pre.code .keyword, code .keyword { color: #3B0D06; font-weight: bold }
pre.code .literal.string, code .literal.string { color: #0C5404 }
pre.code .name.builtin, code .name.builtin { color: #352B84 }
pre.code .deleted, code .deleted { background-color: #DEB0A1}
pre.code .inserted, code .inserted { background-color: #A3D289}

span.classifier {
  font-family: sans-serif ;
  font-style: oblique }

span.classifier-delimiter {
  font-family: sans-serif ;
  font-weight: bold }

span.interpreted {
  font-family: sans-serif }

span.option {
  white-space: nowrap }

span.pre {
  white-space: pre }

span.problematic {
  color: red }

span.section-subtitle {
  /* font-size relative to parent (h1..h6 element) */
  font-size: 80% }

table.citation {
  border-left: solid 1px gray;
  margin-left: 1px }

table.docinfo {
  margin: 2em 4em }

table.docutils {
  margin-top: 0.5em ;
  margin-bottom: 0.5em }

table.footnote {
  border-left: solid 1px black;
  margin-left: 1px }

table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
  padding-left: 0.5em ;
  padding-right: 0.5em ;
  vertical-align: top }

table.docutils th.field-name, table.docinfo th.docinfo-name {
  font-weight: bold ;
  text-align: left ;
  white-space: nowrap ;
  padding-left: 0 }

/* "booktabs" style (no vertical lines) */
table.docutils.booktabs {
  border: 0px;
  border-top: 2px solid;
  border-bottom: 2px solid;
  border-collapse: collapse;
}
table.docutils.booktabs * {
  border: 0px;
}
table.docutils.booktabs th {
  border-bottom: thin solid;
  text-align: left;
}

h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
  font-size: 100% }

ul.auto-toc {
  list-style-type: none }

</style>
</head>
<body>
<div class="document" id="let-s-talk-about-package-specification">
<h1 class="title">Let's Talk About Package Specification</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr class="document field"><th class="docinfo-name">Document:</th><td class="field-body">P1313R0</td>
</tr>
<tr><th class="docinfo-name">Date:</th>
<td>2018-10-07</td></tr>
<tr class="project field"><th class="docinfo-name">Project:</th><td class="field-body">ISO/IEC JTC1 SC22 WG21 Programming Language C++</td>
</tr>
<tr class="audience field"><th class="docinfo-name">Audience:</th><td class="field-body">Study Group 15 (Tooling)</td>
</tr>
<tr><th class="docinfo-name">Author:</th>
<td>Matthew Woehlke (<a class="reference external" href="mailto:mwoehlke.floss&#64;gmail.com">mwoehlke.floss&#64;gmail.com</a>)</td></tr>
</tbody>
</table>
<style>
  html { color: black; background: white; }
  table.docinfo { margin: 2em 0; }
  p, li { text-align: justify; }
</style><div class="section" id="abstract">
<h1><a class="toc-backref" href="#id1">Abstract</a></h1>
<p>This paper explores the concept of a package specification — an important
aspect of interaction between distinct software components — and recommends
a possible direction for improvements in this area.</p>
<div class="contents topic" id="contents">
<p class="topic-title first">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#abstract" id="id1">Abstract</a></li>
<li><a class="reference internal" href="#background" id="id2">Background</a></li>
<li><a class="reference internal" href="#details-of-the-problem" id="id3">Details of the Problem</a></li>
<li><a class="reference internal" href="#objective" id="id4">Objective</a><ul>
<li><a class="reference internal" href="#location-location-location" id="id5">Location, Location, Location</a></li>
<li><a class="reference internal" href="#what-can-you-do-for-me" id="id6">What Can You Do for Me?</a></li>
<li><a class="reference internal" href="#are-we-compatible" id="id7">Are We Compatible?</a></li>
</ul>
</li>
<li><a class="reference internal" href="#historic-approaches" id="id8">Historic Approaches</a></li>
<li><a class="reference internal" href="#a-modest-proposal" id="id9">A Modest Proposal</a></li>
<li><a class="reference internal" href="#acknowledgments" id="id10">Acknowledgments</a></li>
</ul>
</div>
</div>
<div class="section" id="background">
<h1><a class="toc-backref" href="#id2">Background</a></h1>
<p>SG15 has recently been studying C++ package management.
While we commend this effort, we are concerned that it may be overly focused
on only one aspect — albeit an important one — of the overall problem
of building a software project against external dependencies.</p>
<p>Given the prevalence of existing package managers with Linux-based operating
system distributions, we do not feel that any single C++-oriented package
manager, even should a cross-platform tool emerge, is a panacea as far as
solving for external dependencies.</p>
<p>There are three aspects to a software project consuming an external dependency.
Firstly, the dependency must be present on the system. A package manager is
primarily targeted at solving this problem, and indeed this is an important
problem that is worth consideration.</p>
<p>Secondly, the compiler and/or linker must be able to consume the dependency.
This is typically accomplished by passing the name and/or location of the
dependency or components thereof to the command line of the tool in question.
This is largely a function of how the compiler and/or linker is invoked, which
is traditionally governed by a &quot;build tool&quot; or &quot;build system&quot;.
Many such tools exist, with a variety of trade-offs,
and given the complexity of such tools
and the frequent need to support multiple languages with a single tool,
we do not feel that convergence upon a single tool is a practical goal.</p>
<p>Lastly, there is the need to integrate the previous two points.
That is, simply installing a package is insufficient;
the build tool needs to know how to <em>locate</em> an installed package,
and how to <em>use</em> the components which are provided by that package.</p>
</div>
<div class="section" id="details-of-the-problem">
<h1><a class="toc-backref" href="#id3">Details of the Problem</a></h1>
<p>There are a number of ways to make a software package available on a given
system. These include, but are not necessarily limited to:</p>
<ul class="simple">
<li>Installing the package to a system location using a dedicated installer.</li>
<li>Installing the package to a location (system-wide or otherwise) using a
mechanism which is integrated with (and usually, to some degree, provided by)
the package's build system.</li>
<li>Installing the package via a dedicated general purpose package manager which
is integrated with the operating system.</li>
<li>Installing the package via a (possibly cross-platform) C++ package manager.</li>
<li>Building the package in a user directory <em>without</em> an explicit
&quot;installation&quot;.</li>
</ul>
<p>Because a package can come from a number of sources, we feel that relying on
the package manager to integrate package usage information is a decidedly
sub-optimal approach. Nor is there any need to do so; it is not difficult for
a software package to include a &quot;specification&quot; — a file (or collection of
files) describing the contents of the package and providing the necessary
information for a consumer to make use of the components that the package
provides.</p>
<p>The mechanism by which this &quot;package specification&quot; is generated is
unimportant; it could be hand-crafted, generated by the build system, or
generated by the packing tool. The key features are, first, that there exists
a mechanism by which the specification may be located by a consumer. Second,
that the specification can be parsed by the consumer and provides information
sufficient to the consumer's needs. Finally, whether a particular package meets
a consumer's needs.</p>
</div>
<div class="section" id="objective">
<h1><a class="toc-backref" href="#id4">Objective</a></h1>
<p>By now, we hope we have identified the problem of package <em>specification</em>, and
shown how, although closely related, it differs from the problem of package
<em>distribution</em>. In particular, we hope that by showing how there are, and
likely always will be, multiple channels for <em>distribution</em>, we have made a
case for considering the problems of specification and distribution
independently.</p>
<p>While we will eventually consider a potential solution, our main objective is
to begin a dialog on the subject of specification, and to consider how this
problem might — by avoiding tight coupling with an particular build tool or
package distribution mechanism — be approached in a manner that will benefit
the largest number of consumers.</p>
<p>We will now examine the three objectives of package specification in more
detail.</p>
<div class="section" id="location-location-location">
<h2><a class="toc-backref" href="#id5">Location, Location, Location</a></h2>
<p>In order for a specification to be useful, a consumer must be able to <em>find</em>
it. Package management (distribution) will play a crucial role here, in that a
package manager should ensure that specifications are made available in a well
known location. On Linux systems, this likely means taking direction from the
<a class="reference external" href="https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard">Filesystem Hierarchy Standard</a>, while other platforms may have other existing standards, or greater
latitude for creating new standards. In addition, some packages may necessarily
live outside of &quot;standard&quot; locations. (Packages built in a user directory that
have not been &quot;installed&quot; are a prime example.) We expect that consumers will
have a mechanism by which the user may specify additional search locations or
the location of a specific package.</p>
</div>
<div class="section" id="what-can-you-do-for-me">
<h2><a class="toc-backref" href="#id6">What Can You Do for Me?</a></h2>
<p>A specification must also inform the consumer what components are available.
Consider a large product such as Qt. This product consists of a number of
components (Core, GUI, Widgets, XML, etc.) which may or may not have a
one-to-one correspondence with distributed packages. Thus, it is not only
necessary that a package specification enumerate what components and features
are available, but we can see how it would also be useful if a specification
for a single &quot;package name&quot; can be subdivided into multiple files, so that
available components may be packaged separately.</p>
<p>Additionally, a component may be available in different &quot;flavors&quot;. This can
span multiple axes, such as static versus shared libraries, with or without
debugging utilities, or different threading support. Thus, a mechanism to allow
a user to select between such alternatives is desired.</p>
<p>Lastly, a component needs to communicate its usage requirements to a user. This
could include the locations of headers (or modules), names and locations of
libraries to be linked, and other compile or link flags that must be used.
However, even here we run into complications. Usage requirements are best
communicated at an abstract level, such as &quot;I require C++14&quot; or &quot;I require
linking with pthread support&quot;. Such abstract descriptions help to avoid issues
due to compiler differences, and help consumers to make more appropriate
choices in the face of &quot;conflicting&quot; requirements.</p>
</div>
<div class="section" id="are-we-compatible">
<h2><a class="toc-backref" href="#id7">Are We Compatible?</a></h2>
<p>Although part of this answer is implicit in a package's component set, we
certainly also need to record the package version, as consumers very often
depend on a particular version of a package. It is also desirable that a
package specification is able to record compatibility information, so that a
consumer &quot;validated&quot; against an older version knows if it can safely use a
newer version. (Obviously, this relies upon the package providing accurate
information in this respect.)</p>
<p>Finally, we should consider that a system may have multiple instances of a
package installed. This could take the form of different releases of the same
package, or code compiled for different machine architectures. Thus, our
package specification should also communicate such information so that, if
multiple instances of a package are available, a consumer is able to identify
and select which (if any) matches its requirements.</p>
</div>
</div>
<div class="section" id="historic-approaches">
<h1><a class="toc-backref" href="#id8">Historic Approaches</a></h1>
<p>Many developers will be familiar with the <a class="reference external" href="https://www.freedesktop.org/wiki/Software/pkg-config/">pkg-config</a> tool, which is one of the
earliest tools created to help solve the problem we are discussing. While the
package specification format used by pkg-config is more or less adequate for
its original audience (GNU autotools on GNU/Linux systems), it has a number of
limitations. First, a <tt class="docutils literal">.pc</tt> file can only describe a single component, which
results in much duplication across multiple such files belonging to a single
conceptual product. Second, it is limited to providing tool flags, which can be
problematic for build tools (such as <a class="reference external" href="https://cmake.org/">CMake</a>) which desire the canonical
location of libraries, and makes it poorly suited for describing non-library
components (such as a code generating executable). Third, its ability to
specify architecture information or provide component variations is limited.
Fourth, it is not well suited for non-POSIX platforms where libraries use
different artifacts at link-time and run-time (such as Windows).</p>
<p>It is because of these issues that CMake discourages reliance on pkg-config, and
eventually developed its own system of package specification. However, CMake's
exported targets, while far superior in many ways (at least for CMake-using
consumers), are not without their own flaws. Architecture specification is
theoretically possible, but there exists no standard for doing so. Similarly,
while there is some concept of &quot;build configuration&quot;, there is no standard,
well defined mechanism for providing component variations, especially on
multiple axes. Most critically, however, is that CMake uses its own (Turing
complete!) language for package specification, making CMake's exported targets
effectively inaccessible to any tool other than CMake itself.</p>
<p>This list is not exhaustive. Other build systems may provide their own
tool-specific mechanisms for package specification. However, pkg-config, with all its
limitations, remains the only de-facto &quot;portable&quot; standard of package
specification.</p>
</div>
<div class="section" id="a-modest-proposal">
<h1><a class="toc-backref" href="#id9">A Modest Proposal</a></h1>
<p>It is not the primary objective of this paper to solve these problems at this
time. Rather, we desire to open discussion on these issues with a long term
goal of producing a new mechanism of package specification that can achieve the
goals that were outlined above.</p>
<p>That being said, following a discussion at the 2016 Jacksonville meeting, work
began on a <a class="reference external" href="https://mwoehlke.github.io/cps/">Common Package Specification</a> with the aim of providing a satisfactory alternative to
existing mechanisms. The Common Package Specification was carefully developed based on lessons learned
by CMake and has met with some positive reception already. We feel that CPS may
be a viable solution, but it needs wider exposure and a functional
implementation. In particular, it would benefit greatly from sponsorship
willing to contribute to its further development.</p>
</div>
<div class="section" id="acknowledgments">
<h1><a class="toc-backref" href="#id10">Acknowledgments</a></h1>
<p>We wish to thank everyone at Kitware who has supported work on CPS and made
suggestions that have furthered its development. We also wish to thank everyone
who has participated in the WG21 meetings where these issues have been, and
will be, discussed.</p>
<!-- .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. -->
<!-- kate: hl reStructuredText -->
</div>
</div>
</body>
</html>
