[Ns-developers] Tracing Rework

craigdo@ee.washington.edu craigdo at ee.washington.edu
Sun Mar 1 13:28:53 PST 2009 

I really prefer to have discussions of any length over email, so allow me to
copy your comments from bugzilla to here.

> Sorry for not sounding too enthusiastic.

Well, I'm not very enthusiastic about this one either.  I've thrown away a
couple of iterations already.

> 1) You can't just add a tx completed event in MacLow. The whole codebase 
> was designed to not use a tx completed event to 'optimize' that event. 
> If we find out that we really need to have a tx completed event for 
> tracing purposes, then, we have to restructure entirely the codebase 
> because a lot of code is there just to work without a tx completed 
> event so, if there is one, a lot of code must disappear.

This was an attempt to try and make the sources across all of the devices
return something reasonably consistent and similarly meaningful.  I don't
really think this is necessary, BTW.  It's obvious that your code doesn't
like the idea of a tx complete event, but I wanted to see how bad it would
look to plumb in all of the traces in the worst case (wifi being the most
complicated device we have).  There are actually similar situations in other
devices with solutions involving varying amounts of pain.

My concern, which led me to trying to implement a tx complete event in the
wifi code was the amount of user confusion that could be generated trying to
figure out which trace sources are meaningful in what way in which driver.
Another source of confusion is that different devices can quite easily pass
back packets at different stages of assembly/disassembly when the sources in
question are hit (you hit on this tangentially below).

I really wanted to prove it; but I'm afraid that trying to realize a real
consistency WRT generic traces in specialized devices may just be way more
trouble than its worth -- as you are also observing.  I believe similar
statements have been made here before, BTW.

I want to see some kind of structure, but I'm fine with having a set of
"standard" trace sources defined -- that are defined to fire at given
logical times; and leaving it up to each net device author to decide what
subset of these source he or she wants to implement.  This standard set
would at least give a user a basis to find and use what is available (i.e.,
I know that I'll most likely find at least one of these standard sources and
I know what they do).  I'm perfectly fine with removing the tx completed
hook from the wifi.  I stuck it in, pretty much as a prototype to get a feel
for how bad it would be.  There is currently no rx start event in csma or
point-to-point either, for example.

> 2) In wifi-mac.h: please, don't make the traces protected: if you need
> to trigger them from subclasses, use a protected Notify method. The same 
> goes for wifi-phy.h.

Do you mean that you want the sources wrapped by a function -- as in, for
example, NotifyRxDropTrace ()?  That seems like A Good Thing (TM).

> 3) WifiNetDevice::SniffPacket ?? You are making an extra copy of the
packet
> just for tracing, even if there are no connected trace sinks ? This really
does
> not look like a good idea because it is emulating something really
stupid/bad
> from linux (which, incidentely, I believe got fixed in recent wifi stacks)
and
> because its performance cost is a bit horrid.

This actually happens all over our code.  It is so that various trace
sources return some relatively sane number of variations on packet contents.
Even in the simplest cases, CSMA, for example could trace a complete packet,
a packet without an ethernet trailer, a packet without an ethernet header, a
packet with or without an LLC/SNAP header, etc., depending on when a trace
is hit.  Making a copy of the original packet that came in over the channel
allows for only one main variation -- DIX or LLC/SNAP.  If this is horrid
and should be optimized, it should be done, well, everywhere.

Somewhere in this mix we can reach a foolish consistency, but at some level
we have to make this usable by mere mortals.

> To summarize, 2) appears easily fixable, the part of 3) which makes use of
an
> ethernet header for tracing looks like a really, _really_ bad idea, and 1)
> worries me because it would be a lot of work to adjust the current
codebase to
> the presence of a tx completed event.

I think that the one area that we really want some kind of real consistency
is with respect to the pcap traces.  The current idea is that there are two
main flavors, promiscuous and non-promiscuous.  Promiscuous sniffer trace
hooks let you do pcap tracing as if you did a tcpdump -i on a particular net
device on a particular channel.  This seems like a very useful thing and
could reduce the number of pcap trace files that are generated all over our
system.

The other version, the non-promiscuous sniffer is a bit more problematic.
Currently, pcap traces written by most of our devices look this way and so I
think current users will expect to be able to see it.  It can be convenient
to see a filtered subset of the frames flowing through the device.  This is
mostly easy to do in simple net devices where everything is available in one
place.  Not so much for wifi where complexity issues keep things
modularized, and modularity is the enemy of this particular trace source.
This is especially visible in the wifi code.  It seems very complicated, way
more trouble than its worth, to make this work ... correctly.  It breaks all
kinds of modularity and spans layers.  Really ugly.

I do need a complete packet that will end up being written to a pcap file,
though.  The idea here was that this is a very high level thing in the
device and that the user really doesn't want to see lots of underlying
802.11 stuff.  In fact, the user probably doesn't care about Ethernet
frames, really.  I just needed some kind of framing to write a pcap file.  I
did what many implementations of raw packets do and cooked up an ethernet
packet out of whole cloth.  If you read raw packets from a packet socket on
Linux, I believe you will get Ethernet frames, for example.

Now, this may be another one of those things that is ugly and more trouble
than its worth as well.  One alternative is that we can just not support
Sniffer tracing on wifi devices as part of a choice of subset of standard
trace hooks.  I would support this.  It does mean that we would move toward
making the PromiscSniffer trace the one official standard always-there trace
option.  This would align us with tcpdump -i, for example.  I would resist
removing Sniffer from all devices, at least initially, just because of the
amount of work in the code and in the documentation to change all references
to pcap trace contents.

So, Perhaps we could make some policy statements:

1) There is a set of suggested trace sources for net devices.  These are
currently found in the repo I mentioned and should be documented in the
manual.

2) Implementing the complete set of suggested trace sources in a net device
is not a requirement.  A device author may pick a subset of these sources to
implement.

3) The net device author should document what trace sources are implemented
and the state the packets are in when they come out of the trace source.
For example, raw IP, ethernet header, llc/snap, dix, p2p, wifi, etc.

4) A net device author should implement a PromiscSniffer trace source to
allow tcpdump-like functionality in helpers to be called a "standard" net
device; but you are always allowed to do whatever you think is right on your
own time.

I can then 

1) Remove the objectionable Sniffer (Ethernet frame) and tx complete trace
sources from the wifi device;

2) Remove the unused rx start trace sources from the csma and point-to-point
devices;

3) Remove the non-promiscuous sniffer helper code from the wifi-helper and
leave yans-wifi-phy hooked to the promisc sniffer;

4) Write some words for the manual on this stuff.

This would mean that devices are different with respect to tracing, and
initially inconsistent with respect to promiscuous sniffing; so this isn't
really a complete resolution of the bug in question for ns-3.4 (sigh).

Future work on this subject would involve

1) Adding PromiscSniffer (tcpdump) functionality to the various net devices.

2) Documenting packet contents at the various hooks.

3) Changing the various examples and regressions to use promiscuous sniffers
as if someone were doing tcpdump on a real net.

4) Optimizing the packet trace hooks to not copy packets whenever possible.

-- Craig

> -----Original Message-----
> From: Mathieu Lacage [mailto:mathieu.lacage at sophia.inria.fr] 
> Sent: Sunday, March 01, 2009 12:43 AM
> To: craigdo at u.washington.edu
> Cc: ns-developers at ISI.EDU
> Subject: Re: [Ns-developers] Tracing Rework
> 
> One thing I forgot in my comments on the bug report: 'RxStart' + 'Rx'
> seem to be missing the obvious name symmetry: 'RxStart'+'RxEnd' (same
> goes for Tx events, of course).
> 
> Mathieu
> 
> On Sat, 2009-02-28 at 17:14 -0800, craigdo at ee.washington.edu wrote:
> > I just stuck an attachment in bugzilla showing what I've 
> done for the
> > tracing rework to kill bug 443.
> > 
> > See below
> > 
> > Comments?
> > 
> > -- Craig
> > 
> > > -----Original Message-----
> > > From: ns-bugs-bounces at ISI.EDU 
> > > [mailto:ns-bugs-bounces at ISI.EDU] On Behalf Of 
> > > code at nsnam.ece.gatech.edu
> > > Sent: Saturday, February 28, 2009 4:37 PM
> > > To: ns-bugs at ISI.EDU
> > > Subject: [Ns-bugs] [Bug 443] Net device pcap traces arent 
> > > consistent with what tcpdump would display
> > > 
> > > http://www.nsnam.org/bugzilla/show_bug.cgi?id=443
> > > 
> > > --- Comment #2 from Craig Dowell <craigdo at ee.washington.edu>  
> > > 2009-02-28 19:37:06 EDT ---
> > > Created an attachment (id=393)
> > >  --> (http://www.nsnam.org/bugzilla/attachment.cgi?id=393)
> > > Proposed Changes to Tracing in NetDevices
> > > 
> > > Significant changes to tracing in all non-bridge net devices.
> > > 
> > > 1) Splits tracing into MAC-level and PHY-level traces for 
> all devices;
> > > 2) Adds missing drop traces;
> > > 3) Adds "Sniffer" and "PromiscSniffer" trace hooks to do 
> > > tcpdump-like traces;
> > > 4) Sniffer hooks now fire at the correct times;
> > > 4) Updates helpers to hook new sources.
> > > 
> > > Note:  Most devices are simple enough that everything is 
> > > available in one
> > > place.  Wifi is different.  The Sniffer hook in the wifi 
> > > device provides
> > > Ethernet frames for simple tracing and lives in WifiMac with 
> > > the rest of the
> > > MAC-level trace sources.  Sniffer is available using Pcap 
> calls in the
> > > WifiHelper.  The PromiscSniffer hook in the wifi device 
> > > provides all of the
> > > low-level wifi packets and lives in WifiPhy with all of the 
> > > PHY-level trace
> > > sources.  PromiscSniffer is available using Pcap calls in the
> > > YansWifiPhyHelper.
> > > 
> > > 
> > > -- 
> > > Configure bugmail: 
> > > http://www.nsnam.org/bugzilla/userprefs.cgi?tab=email
> > > ------- You are receiving this mail because: -------
> > > You are on the CC list for the bug.
> > > _______________________________________________
> > > 
> > 
> > 
> 
>

More information about the Ns-developers mailing list