[Ns-developers] [ns3] Statistical framework (draft)

[Vincent Gauthier]

Hi Joseph, everybody,

I read the draft that Mathieu sent us last week, and I started  some  
rough implementation of the P^2 algorithm. The first step could be the  
definition of a set of methods for the statistical container that we  
could be enhance over the time (the ones that have been defined before  
mean, median (P^2 algo), counter, and confidences intervals). Once  
this step is over the next one will be to efficiently link theses  
containers to some variables inside the code (callbacks) and define  
how to update stats containers efficiently. Maybe adding a method to  
sample over time each update in the stats container instead of keeping  
track of each change of one variable. That would certainly improve the  
simulator performances (but also lead to errors if the sample time is  
over estimate).

I haven't a lot of things to add to what it has been said over the  
thread about the stats framework, It seems that the main concern about  
the stats that Tom and Mathieu have highlighted (in mailing list and  
in the draft) is to provide confidence interval, CDF and PDF to the  
users but also for simulations matters like stopping the simulation or  
starting the measurements (when the simulation has reach a steady  
state). The second point that have been highlighting is to provide an  
efficient way to gathering and storing the statistical result in a  
database like container, that enable us to give an enhanced output to  
the users through xml file and so on (that is your point Joseph). The  
last point is to provide a set of statistical container available  
through callback methods directly accessible anywhere inside the  
simulator. Theses container should handle methods to perform the  
calculation of the stats. It would also be nice if we let the user  
under certain condition the ability to have access to a raw data on  
demand (sample of somethings over the time) because certain kind of  
variables could only be analysed this way.

My major concern is what should be the policy for linking a variable  
into the code to a stats container. For me less time your are updating  
the stats container better will be the performance (even with P^2 algo  
if you want to get the median for example of somethings in Mac layer  
of wireless network for all the node it would probability slow down  
the simulation, where should be the threshold ?).

I'll send you an e-mail about how we will proceed,

Thanks,
Vincent

Le 14 mai 08 à 07:16, Joseph Kopena a écrit :

> Hi Vincent, everybody,
>
> Sorry for some delay getting back to this, I've been traveling the
> past few days.  On the upside, that meant a good chunk of airport time
> for coding.
>
> I've posted a repo at [1] that drafts some of my conception of the
> stat framework.  Here's the basic ideas:
>
> - A stat framework object provides functionality to create stat
> objects and specifies an output callback.
> - Stat objects track different generic statistics and are updated by
> events, trace firings, etc.
> - Each stat object uses the callback to give its output to the
> specific framework being used, which in turn renders that data (e.g.,
> as plain text or XML).
>
> Note that unlike most ns-3 WIPs, this is currently setup externally,
> not as a branch of ns-3-dev.  This was mostly done to reduce compile
> (make faster than waf?) and link time, as well as ease eventual
> merging.  To compile, just run make in the root directory.  It'll
> expect the env variable NS3_LIB_DIR to be set.  This is usually the
> build/debug/ subdirectory of your ns-3 repository, e.g. for me it's
> usually export NS3_LIB_DIR=/opt/ns-3-dev/build/debug/.
>
> I'll be working on the basics of this the next couple days as I need
> some of this functionality.  In addition to cleaning it up a bit, I'll
> be adding some simple statistics and demo'ing connections to trace
> sources.  I most likely won't get to more sophisticated measures
> anytime soon.
>
> Vincent, have you drafted code or more ideas?  I am not super-tied to
> the draft code in [1], but wanted to get something out there as well
> as demonstrate the kind of generic statistics I'm envisioning.  Your
> input on the approach as well as your help making this a generally
> usable component would be greatly appreciated.  Mathieu's writeup also
> provides some solid motivation, discussion, and references that should
> be followed up on.
>
> Thx!
>
> [1] http://code.nsnam.org/tjkopena/ns-3-stat/
>
> -- 
> - joe kopena
> right here and now