[Ns-developers] ns3 distributed simulation code

[George Riley]

Tom suggested we post some information regarding the MPI Distributed
simulation code, in light of the comments below.  First an overview,
then some details.

Overview.  The distributed simulation code is designed to use allow
a single topology to be split into "n" sub-topologies, and executed
on "n" linux boxes in a distributed memory environment.  The current
implementation only allows packets sent scross point-to-point links
to be sent across simulator boundaries; the next release will support
wireless we hope.  We tried to change as little as possible of the
existing code base and isolate our changes in separate modules where
possible.  There appears to be no collision or interactions with the
multi-threaded patch posted recently.

Details.
point-to-point-channel.[h,cc]  - Made TransmitStart virtual so the
new subclass (see below) can override.

point-to-point-remote-channel.[h,cc] - new subclass of p2p channel;
used to connect simulated nodes on one simulator to nodes on a
separate simulator.  Overrides the TransmitStart function and
uses MPI to send the packet across simulator processes.

point-to-point-helper.[hcc].  Checks the system id on the two nodes
being connected, and uses a p2p-remote-channel rather than a p2p-channel
if the system id's do not match.

node.[h,cc] - Added a "SetSystemId" function, to allow the system id to
be set after the node is constructed, rather than passing parameter
to constructor; this exists because there was no way to use
non-default constructors on object creation.  I believe this might have
been fixed since then, so perhaps this might not be neded.

global-route-manager.cc - Do not compute routes for any node whose  
system-id
does not match the simulator's system id.  Clearly, we don't need to
compute routes in simulator A for nodes modeled in simulator B.

default-simulator-impl.[h.cc] = Added virtual "GetSystemId" method,
which returns the system id (MPI rank) of this process.   For
the default-simulator object it always returns zero.

distributed-simulator-impl.[h.cc] - new subclass of default-simulator- 
impl.
THe meat of this is the overridden "run" method that handles all of
the time management needed, using MPI.

mpi-interface.[h,cc] - All MPI related processing is here in a single
spot, rather than across several modules.

packetr.[h,cc] - THis still needs work (Josh is working on it now).
The existing seriallization code does not meet our needs; it adds
zero bytes for the dummy payload, making the MPI messages larger, and
it looses the unique id as packets cross simulator boundaries.  While I
agree that we no longer truly have "unique" id's because each simulator
presently starts it's unique id's a zero;  We will change this to
have a 64-bit UID, with the upper 32 bits being the simulator id and the
lower being the process-specific UID.

Josh is cleaning up the patch today and will post no later than tomorrow
for review, excepting the packet serialization that is still being
worked.

George and Josh