[Ns-developers] merging multithreaded simulation core incrementally

[Gustavo Carneiro]

2009/7/15 Mathieu Lacage <mathieu.lacage at sophia.inria.fr>

> hi,
>
> Guillaume announced recently his effort to implement a multithreaded
> simulation core to run simulations on multicore systems:
> http://mailman.isi.edu/pipermail/ns-developers/2009-July/006197.html
>
> Given the current status of the project, I don't think the simulator
> implementation will be mergeable before next release (we still need
> serious time to try various algorithms, profile, and make the
> implementation really robust) but there are some things which do make
> sense as standalone changes to ns-3 and which could be merged
> separately: the idea would be to prepare at least a single patch for
> each of the items below.
>
> 1) keep track of a 'context' (a 32bit node id) on a per-event basis. The
> user-visible feature here is that there is a new function: uint32_t
> Simulator::GetContext (void); and this function is used by the logging
> code in src/core/log.h/cc to print automatically with each log line the
> 'current' context of a running event. This feature is not completely
> deterministic: that is, in some cases, it is possible that the current
> context is '-1' (when it happens, it's easy for users to slightly change
> their model code to make it not happen. By default, it should not happen
> with the default set of models in ns-3) which means, no context/nodeid
> is associated to an event. This is why this is really a debugging
> feature but a really useful one.


I am certainly not opposed to this, having previously proposed something
very similar myself over a year ago:

http://mailman.isi.edu/pipermail/ns-developers/2008-May/004155.html



>
>
> 2) introduce an AsciiWriter which mirrors PcapWriter. It encapsulates
> the details of our ascii output file format. In the multithreaded code,
> it is used to ensure that each line in an ascii file is written
> atomically but we could merge a simple version of that code which merely
> centralizes knowledge of the ascii file format.
>
> 3) make simulator.h threadsafe: this requires the addition of assembly
> atomic operations (stolen from glib) to implement RefCountBaseThreadSafe
> and use it from EventImpl, as well as the use of a mutex lock in
> DefaultSimulatorImpl, similarly to RealtimeSimulatorImpl. A nice
> side-effect of this new feature is that it would allow getting rid of
> some ugly code in the emu and tab-bridge devices. The performance cost
> of that change is around 10% in micro-benchmarks for Scheduling
> operations, most of which is coming from the lock mutex in
> DefaultSimulatorImpl. I have not measured any impact on non-micro
> benchmarks.


10% in micro-benchmarks is OK; it will probably mean less than 1% in real
world benchmarks.

However, we should point out that the penalty will be huge for non-GCC or
non-x86 builds, thereby greatly reducing the portability of NS-3.  Should we
be sacrificing portability of NS-3 for the sake of parallel execution?

Or, put another way, how much do you think parallel execution will really
gain us, in terms of speed, and at what cost in terms of complexity?
Personally, I already have been doing parallel simulations in multicore
systems for ages, it is called multiple processes.  The only downside of
multiple processes is that you require more RAM to run multiple simulations
in parallel than you would need to run a single simulation faster.

Isn't it possible to just create an alternative scheduler which is thread
safe and leave the default scheduler alone, just like the realtime scheduler
is thread safe but does not interfere with the simple default one?


>
> 4) various changes to the simulator.h API to make it nicer to the
> non-default SimulatorImpl subclasses:
>  - make Simulator::SetScheduler take an ObjectFactory rather than a
> Ptr<Scheduler>
>  - kill Simulator::RunOneEvent, IsFinished, and, Next
>  - introduce Simulator::SetMonitoringCallback(Callback<void> cb);
>
> 5) changes to various APIs in src/core to make them threadsafe or make
> it easier to do threadsafe things. I have not yet investigated the cpu
> runtime cost of these features (they have no memory cost). This is
> really about:
>  - use RefCountBase where users use their own refcounting
> implementation.
>  - replace RefCountBase by RefCountBaseTs where needed
>  - make Object::Ref/Unref threadsafe.
>  - make AsciiWriter threadsafe
>  - remove Packet::PeekData and use Packet::CopyData instead where
> needed. This will make it easier to use a non-COW implementation of
> Packet if needed in the multithreaded case.


Again, if you add thread safety to all these classes you are sacrificing
simplicity and performance of non-threaded simulations for the sake of
parallel scheduler.  I do not know whether there is an alternative, but we
should think carefully before going down this path.

Regards,

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert