[Ns-developers] [ns3] Statistical framework (draft)

[Mathieu Lacage]

On Wed, 2008-05-14 at 12:56 -0400, Joseph Kopena wrote:
> For the most part I have trouble justifying the runtime hit & extra
> code associated w/ private simple members that subclasses will make
> frequent use of and will need accessors, but don't have strong
> feelings on this.

I would rather fall in the trap of early pessimization than early
optimization :)


> > If I understand the code correctly, you could presumably implement a
> > DataStatCalculator which would not perform and statistics calculation
> > and would just dump in the framework the raw data, right ?
> 
> Agreed, though I think the emphasis is on providing a simple mechanism
> to collect data, rather than focusing on supporting multiple output
> formats.

ok.

> 
> > It seems to me that it is also conceivable to implement something like a
> > GnuplotStatFramework, right ?
> 
> Yes, though I think that's a slightly different issue.  Most plots
> will be based on data from multiple runs, which the code there is

I usually run the simulation longer instead of running multiple
simulations :)

> currently not explicitly supporting.  Doing that well is something we
> have to think about a bit.

ok. What would be the issues ? Would that require integrating the
management of multiple simulation runs in the StatFramework class ?

> >  - a better name for StatFramework might be DataStorage or something
> > similar to outline its relationship to data storage format.
> 
> No objection.
> 
> The more interesting question here then is what is the relationship of
> this component in this module to the existing trace logging, e.g. pcap
> and ascii?  Is there any?  Both are doing data storage...

indeed. One could conceive of a StatCalculator subclass which merely
accumulates packets and dumps them into the StatFramework. The issue
with pcap output though is that the generated byte stream needs a magic
header to indicate the type of packet encapsulation used in the packets.

> >  - I would tend to move out Counter creation out of the StatFramework
> > base class and make the StatCalculator constructor take a pointer to its
> > associated StatFramework instead: allows you to add new subclasses to
> > StatCalculator without patching StatFramework
> 
> This I think is not so straightforward.  The rationale for the factory
> pattern is to be able to easily switch between stat framework
> implementations.  For example, to go from plain text to DB I just
> switch which framework is created, and all of the actual
> instrumentation code remains the same.  It's conceivable to me that
> there could be StatCalculators which depend on the type of the
> underlying framework, e.g. a DataStatCalculator you mentioned earlier
> could look very different in plain text vs DB form.  If sim writers

What would be the difference ? Maybe we could try to define precisely
the kind of final output we expect out of this for each of the storage
backends we would like.

> create those calculator objects directly, they need to change several
> things in order to switch output formats.  I have no opposition to
> adding functionality to StatFramework to enable manual addition of
> directly created StatCalculator objects, which would allow you to use
> new StatCalculator subclasses without patching all StatFramework
> implementations.  However, I think those changes once the module is
> mature are less likely than people switching between output formats,
> so I'm hesitant to push that hassle onto the sim writers.

I see. 

I just went through your comments above again and I think that the
biggest issue (beyond the small stylistic issues we discussed) we
should try to deal with is a small set of use-cases for the
StatFramework/StatCalculator class. I would be fine with restricting it
to a single simulation run or a small set of storage backends (or just
one) but it is hard to really review your proposal without a clear idea
of what you intend to be able to do with this code.

For example, if the goal is to eventually have an sql-based storage
backend, then, we should carefully think about what this means for our
API and try to define the output we want to get first. The same holds
true for the xml and ascii backends.

regards,
Mathieu