[Ns-developers] build system proposal

[Tom Henderson]

Here is an outline of a suggestion to modify the build system-- the result of 
a discussion between myself, Craig, and Mathieu last Friday.  Comments?

Background:

We have begun to build optional components for ns-3
including python bindings and nsc, and we have had some discussion
about aspects of the download behavior of this part of the script; e.g.
http://mailman.isi.edu/pipermail/ns-developers/2008-September/004712.html
http://mailman.isi.edu/pipermail/ns-developers/2008-September/004763.html

Proposal:

We make a new repository called "ns-3-allinone".  This repo has at least two
scripts:  "download.py" and "install.py" (maybe a third also.. see below)

A developer starts by checking out ns-3-allinone, and running 
download.py.  This script will download copies of the development
repos for ns-3 and related packages.  Prior to downloading, the
directory would look like:

ns-3-allinone/
             download.py
             install.py

After invoking download.py, it would look something like this:

ns-3-allinone/
             ns-3-dev/
             pybindgen-dev/
             nsc-dev/
             ns-3-dev-ref-traces/
             download.py
             install.py

At any time when invoked later, download.py will synchronize all repos, 
or if passed an argument or set of arguments, will synchronize all of the 
named repos only.  Note:  Craig has suggested that maybe a separate script called "sync.py" or something like that could be used to do the later synchronization, and not reuse "download.py".

To update ns-3-dev, a developer in the ns-3-dev directory may either:
i) hg pull (as usual)
ii) ../sync.py ns-3-dev  (same outcome as i)
iii) ../sync.py in which case everything is synchronized

"install.py" will build everything in the allinone directory in proper
order.

The wscript in ns-3-dev/ can use arguments to direct waf to
use another component other than the default ../ component.  e.g.
./waf configure --with-pybindgen=/path/to/pybindgen
./waf configure without arguments will search for pybindgen in the ../ directory.

Users who want to share copies of e.g., pybindgen or ns-3-traces across ns-3 
repos can simply download other ns-3 repos in parallel to ns-3-dev.  If they
want specialized versions of e.g., ns-3-ref-traces, they can
put a local copy in their own directory, or elsewhere, and tell waf where to 
find the proper directory; e.g.
./waf configure --with-traces=...

Finally, a common scenario is synchronizing ns-3-ref-traces.  When running
./waf --regression, the script should print out a warning that the
version of ns-3-ref-traces being used is stale, if it has fallen out of
sync.  However, it should not automatically fetch anything. 
This prevents the user from having to do ../sync.py in cases
where there are no recent updates to the reference traces.  It also 
facilitates offline usage since only a warning is printed out if the
traces are stale.   This requires ns-3-dev regression to keep track of
some magic number in the ref-traces repo.

Release tarballs:

These tarballs will create allinone tarballs.  For simplicity, in the future,
we just distribue self-contained releases as allinone tarballs requiring 
no further downloads. The "install.py" can still be used for building 
everything, but the download.py or sync.py wouldn't be used.

Example usage to create a allinone tarball:

hg clone ns-3-allinone
cd ns-3-allinone
./download.py
./waf dist (or makedist.py or some other script?)

One difference in the release tarball is that the resultant directories should 
not have the "-dev" postfix but should have version numbers, and the tarball
is named by the version number; e.g. 

tar xf ns-3.2-allinone.tar would yield something like

ns-3.2-allinone/
             ns-3.2/
             pybindgen-1.5/
             nsc-4.0/
             ns-3.2-ref-traces/
             install.py

The makedist script could be intelligent enough to go into each repo and pick 
out the version number and rename these directories.