[Ns-developers] [ns3] socket API

[Tom Henderson]

Mathieu,
I've been working on implementing your suggestions and they've led
to some more questions.

I think this is one of the main remaining "hot spots" in our API and I'd
like to try to resolve the design this week so we have a chance to
implement it by mid-month.

To recap:

- we have a base class Socket defined in src/node/socket.{h,cc}.  This
models an asynchronous socket API that kind of looks like the typical
C-based sockets API, but asynchronous, and use of Packet as a byte
buffer.  It also has raw byte buffer variants for Send()/SendTo().

- Mathieu started the ns-3-simu branch with a posix sockets API

- I've been thinking about another mid-level helper API that might be a
way to partition the API of what we have now in ns3::Socket.

I started a tomh/ns-3-dev-socket branch for the below.  Comments inline
below.

Mathieu Lacage wrote:
> hi,
> 
> Trying to implement a synchronous socket API on top of the ns3 API
> turned out to be pretty useful, if only it showed some issues with the
> underlying async API. Tom and I talked about some of these issues on
> friday and I spent a while pondering over them and others so, here is a
> summary of the parts of the socket API which are missing to implement a
> full posix API on top. Some of the key issues I tried to address:
> 
>   - ability to implement a fully-conformant posix API on top
> 
>   - should make it possible to implement efficient versions of these
> APIs. i.e., implementations which do not copy packet data from the
> application to the socket layer.
> 
>   - allow the posix socket API to be extended with a version of
> functions which take Ptr<Packet>, to allow _really_ efficient
> synchronous applications which do not copy data from application to
> socket.
> 
> There is quite a bit of content below but it would be nice to get
> feedback on the basic approach and, then, I can try to prototype it in
> the ns-3-simu branch to validate it.
> 
> 
> 1) need to implement a "pull" API in ns3::Socket. The assumption is that
> the subclass of the ns3::Socket class we want to use will implement
> internally a tx and an rx buffer: the synchronous wrapper on top of it
> needs to access that buffer to implement the normal recv calls.
> 
> Proposal:
> 
> /**
>  * return a _single_ packet by removing it from the internal rx 
>  * buffer if it is smaller than maxSize. If the size of
>  * the next packet to return is bigger than maxSize, return 0 and
>  * set ENOMEM.
>  * If you want to get the "from" address, it is stored as a tag
>  * in the packet returned.
>  * _Must_ be possible to call this method from the Recv callback.
>  */
> Ptr<Packet> Recv (uint32_t maxSize, uint32_t flags);
> 
> /**
>  * Return number of bytes which can be returned from one or 
>  * multiple calls to Recv.
>  * Must be possible to call this method from the Recv callback.
>  */
> uint32_t GetRxAvailable (void) const;

I implemented these in the socket-helper branch.  Gustavo suggested also
the overloaded:  Ptr<Packet> Recv (); and I implemented that too.

> 
> The above two methods should be enough to implement a synchronous recv
> function. If we implement them, we should remove the packet and address
> arguments from the RecvCallback which would then be of type
> Callback<void>.
> 
> 2) the current Socket::Send method is quite painful to use and
> implement. Proposal:
> 
> /**
>  * return -1 if packet does not fit in tx buffer (current behavior is 
>  * to send the part of the packet which fits in the tx buffer).

The above aspect is part of the TCP code, which I have left for Raj to
work on.  The UDP and Packet Sockets APIs always succeed on Send(), despite
bug 141 request for modeling this as finite.

>  * If the packet does not fit, the caller can split the packet to send
>  * (based on output by GetTxAvailable) and re-attempt to send the
>  * packet.
>  * If you want to provide a per-packet "to" address, store it as a
>  * special tag in the outgoing packet.

I ran into a small problem in implementing the tagged address:  tags are
size limited to 16 bytes and class Address requires 32.  In the repo, I
bumped this size constant up to 32 bytes.  If, in general, we want
Addresses to be taggable data, we ought to allow for 32 byte tags.

>  */
> int Send (Ptr<Packet> p, uint32_t flags);
> 
> /**
>  * returns the number of bytes which can be sent in a single call
>  * to Send. 
>  */
> uint32_t GetTxAvailable (void) const;

I implemented this in the base API, and supported returning "maxint" for
the PacketSocket and UdpSocket classes.  Again, Raj will support this
correctly in the TCP code, which is where it really matters for now.

> 
> If we implement this, we should remove the SendTo method. Problem: if we
> do this, we cannot specify the "to" address for
> int Socket::SendTo (uint8_t *buffer, uint32_t size);
> because there is no Ptr<Packet> to attach the "to" address to.

I don't know why removing SendTo is related to the above.
> 
> Solution: get rid of the Socket::Send* methods which take raw buffers
> and implement them exclusively at the "synchronous" layer.

There are several options here:
- implement raw buffers only at the synchronous layer
- implement them also at the current Socket layer (what we have done for
now)
- implement them instead at the synchronous layer and a new helper layer
- do not implement raw buffers below the synchronous layer, but use the
Buffer class in place of the raw buffers (Gustavo proposed this on the list)

For now, I haven't removed them.

> 
> 3) How should socket options and ioctls be supported ?
> Proposal: 
> 
> in src/node:
> 
> class Ipv4SocketOptions
> {
>   /* Implement the linux IP_PKTINFO socket option. */
>   virtual void EnablePktInfo (void) = 0;
>   virtual void DisablePktInfo (void) = 0;
>   virtual bool IsPktInfoEnabled (void) = 0;
> 
>   /* Implement the linux IP_OPTIONS socket option. */
>   virtual void SetOptions (const uint8_t [40] buffer, uint32_t size);
>   // return size of options data.
>   virtual uint32_t GetOptions (uint8_t [40] buffer);
> 
>   /* implement IP_RECVTTL */
>   virtual void EnableRecvTtl (void) = 0;
>   virtual void DisableRecvTtl (void) = 0;
>   virtual uint32_t IsRecvTtlEnabled (void) = 0;
> 
>   // all others...
> };
> 
> class UdpSocketOptions : public Ipv4SocketOptions
> {
> public:
>   // udp-specific options.
> };
> 
> class TcpSocketOptions : public Ipv4SocketOptions {...};
> 
> All the above would be implemented in src/internet-node/ as aggregated
> classes to each Socket. i.e., user code:
> 
> Ptr<Socket> sock = ...;
> Ptr<Ipv4SocketOptions> ipv4 = sock->GetObject<Ipv4SocketOptions> ();
> NS_ASSERT (ipv4 != 0);
> ipv4->EnableRecvTtl ();
> 
> Another option here would be to just add all of the above methods to the
> existing src/node/udp.h and src/node/tcp.h classes with an extra
> Ptr<Socket> argument:
> 
> class Udp  ...
> {
> public:
>   /* Implement the linux IP_PKTINFO socket option. */
>   virtual void EnablePktInfo (Ptr<Socket> socket) = 0;
>   virtual void DisablePktInfo (Ptr<Socket> socket) = 0;
>   virtual bool IsPktInfoEnabled (Ptr<Socket> socket) = 0;
> };
> 
> The former approach feels slightly more "right".

There are also two more approaches possible, I think:

3) implement what you proposed for class Udp instead for class
UdpSocket (which is what I did for now)

4) support these through the attribute system on the sockets (i.e.,
PktInfo would be an attribute of UdpSocket)

> 
> 4) How is ancillary information supported ? It is attached as a tag to
> each packet whether it is sent or received.

I haven't tried to support ancillary information.

> 
> 5) posix sockets are extended with:
> 
> int simu_recv (int fd, Ptr<Packet> packet, int flags);
> int simu_send (int fd, Ptr<Packet> packet, int flags);
> ssize_t simu_write (int fd, Ptr<Packet> packet);
> ssize_t simu_read (int fd, Ptr<Packet> packet);
> 

no change here.

The tomh/ns-3-dev-socket branch supports the above additions to the socket API and underlying implementation for UDP and PacketSocket, and placeholders for TCP.  The ns-3-dev-socket-helper branch doesn't yet have different code but the idea there is to prototype a variant in which the API on current ns3::Socket is reduced and a new helper-like class is built on top of that with more of a traditional socket look-and-feel, albeit asynchronous.

Tom