[Ns-developers] Summer of Code nsc/ns-3 integration status, week 05

[Tom Henderson]

Mathieu Lacage wrote:
> On Mon, 2008-05-26 at 20:59 +0200, Florian Westphal wrote:
>>  
>>> It would help if you could outline quickly for the non-nsc experts
>> like
>>> me how nsc _currently_ handles incoming and outgoing packets to/from
>> the
>>> application and layer 2.
>> Sorry for omitting all that information. I'll try to outline how ns-3 and nsc
>> are currently plumbed together.
> 
> There is no need to be sorry: it is not always obvious what is missing
> in an email when you are deep into hacking mode :)
> 
>>> Let's say my application generates some data
>>> payload and passes that down to an nsc tcp socket. What happens then ?
>> NscTcpSocketImpl::Send() calls m_nscTcpSocket->send_data(buf, size).
>> (this is in internet-node/nsc-tcp-socket-impl.cc, line 381 for those
>> that would like to look at the code).
>> There are few more details  (such as when data is sent before
>> the TCP handshake is completed, tcp setup etc.), but those are not needed to
>> get a rough understanding of how ns-3 and nsc interact.
>>
>> send_data is basically the equivalent of "write(sockfd, ...".
>> This hands the data over to NSC, no Ptr<Packet> is involved here.
>> Now, when NSC wants to send that packet out to the network, the
>> nsc ISendCallback is used to re-inject the packet back into ns-3:
>>
>> void NscTcpL4Protocol::send_callback(const void* data, int datalen)
> 
> What does this data * buffer contain ? Does it contain the application
> payload ? Or is there application payload + tcp header ? or application
> payload + tcp header + ip header ?
> 
> I suspect that it contains the application payload + tcp header + ip
> header so ...
> 
>> (this is basically 'layer 2' from NSCs point of view).
>> send_callback() creates a Ptr<Packet>, then the ns-3 TCP header
> 
> Why do you add another TCP header here ?
> 
>> is added and the packet is passed on to the ipv3 layer for transmit.
> 
> ... I think I understand why you get two ip headers in the end: the nsc
> code adds one because it is the simplest way to make the linux tcp stack
> working and the ns-3 code adds one because it does the real job of doing
> the outgoing ip routing. I suspect that the simplest way to make the
> integration work would be to just remove the linux ip header before
> sending the packet down to the ns-3 ip layer. i.e., if you make sure you
> disable the metadata tracking code, you can create a Ptr<Packet> from
> the raw bytes of the data buffer from send_callback and, then, remove
> the ip header with p->RemoveHeader (ipv4); where ipv4 is an instance of
> type Ipv4Header. 
> 

Presumably when NSC is done with the packet, in the outbound direction, 
there is a buffer with pseudoheader.  What you could consider to do is 
implement a new TcpL4Protocol variant that converted the buffer to a 
Ptr<Packet> including TcpHeader, except for the IP header portion.  The 
portion of the pseudoheader with IP header information could form 
arguments to the ipv4->Send() call.  Alternatively, you could implement 
some kind of a Ipv4PseudoHeader tag field for the packet and modify the 
IPv4 code slightly to look for this tag (and avoid another IP header 
source address selection step).


> You need to disable the metadata tracking code (Packet::DisableMetadata
> ()) because this code does some sanity checking to check that you do not
> remove a header which has not been added with AddHeader (which,
> obviously, is the case here).
> 
>> This is the main problem: packets always get two tcp/ip headers:
>> One from NSC, one from ns-3.
>>
>> So, what i'd like to do is to push that Packet to layer two without
>> going through the ns-3 tcp/ip layer. Perhaps the code that is in
> 
> You could also conceivably try to do this but doing so would require
> that you make sure that the nsc code implements correctly the IP routing
> and ARP lookups which would be clearly pretty cool but which might be
> tricky (at least, for now). If you were to do this, what you would be
> doing would be effectively replacing all the functionality provided by
> the InternetStack which means that you would be providing your own full
> ip implementation: it would make more sense to create another
> src/nsc-stack directory than to try to hack the InternetStack code to do
> this.

I agree; bypassing the layer-3 will bypass any of the layer-3 hooks for 
tracing, future netfilter hooks, etc.

OTOH, it would be nice to have the whole IP stack globalized and 
supported, but for now, I also suggest to try to fit it into the TCP 
framework.

> 
> To summarize, either you try to replace only the TCP stack, make sure
> that the linux IPv4/ARP stack is just a convenient passthru, and reuse
> the ns-3 IPv4/ARP stack in which case, hacking the internet-node
> directory code is the right thing to do, or, you try to replace both the
> TCP and the ns-3 IPv4/ARP stacks in which case, it would be better to
> just accept the fact that you are implementing your own complete stack
> and move all your code to another directory src/nsc-stack.
> 
>> I agree that 'elegant' is something else. But i'd really like
>> to get the nsc <-> ns-3 plumbing right before thinking
>> about how nsc-tcp-socket and nsc-tcp-l4 should interact
>> properly (or if e.g. nsc-tcp-l4 is even required in the first place).
> 
> I think that it is needed if you want to reuse the ns-3 IPv4/ARP stack.

Agree; I would suggest to explore whether you can create some kind of an 
abstract base class for TcpL4Protocol and implement an NSC variant of 
it, and move the existing one to an Impl subclass.  The main function of 
the ABC would be to be able to register a Receive() function in the 
demux.  Then, it seems to me you would be able to get away from the 
double header issues.

- Tom
> 
>>> the skb ? When is the skb converted back to a Ptr<Packet> ? When is the
>>> outgoing Ptr<Packet> given back to the Ipv4L3Protocol class ?
>> I hope I was able to shed some light on this. If I should try to explain
>> something in more detail please let me know, i'll also try to add this
>> to the ns-3-nsc wiki entry at
>> http://www.nsnam.org/wiki/index.php/Network_Simulation_Cradle_Integration .
> 
> What would be really helpful is a description of the content of the data
> buffer whenever it is manipulated by an important function/callback.
> 
> Mathieu
>