[Ns-developers] Status ns-3 quagga porting

[Mathieu Lacage]

hi liu,



On Tue, 2008-06-03 at 22:59 +0800, Liu Jian wrote:
> Hello NS-3 community,
> 
> last week i read through code of libnl for better understanding netlink socket.
> After the NetlinkSocket class was defined, the main work for me is to implemente
> the NetlinkSocket::send()/recv(). 

ok.

> 
> the first step is to define the types of routing objects(rt_attr, rt_msg, ifinfo_msg,etc, 
> which quagga used in netlink route/addr/link).fisrtly i planned to define these as XXXHeader
> class like NetlinkMsgHeader, but i found it was not convenient when parsing the payload
> of netlink msg(eg. when fetching next route msg or netlink attribute one by one), 
> i define them in c style as structure and enum types in file netlink-route-types.h, so
> that the packet parsing would be easily to be done, after removing the NetlinkMsgHeader,
> get the payload start address pointer and parse it with the c style way used by libnl, then
> add different operations for different route msg types.
> 
> now for quagga only, there only handle 3 types of operations add_route/del_route, add_addr
> /del_route, set_link.(next step to add ipv4 APIs to fullfill them and  socket testing).

I tried to look at the details of these aspects of the netlink protocol
today and, I have to confess that it is a bit hairy so, I went back and
tried to first figure out how the netlink protocol works and how it can
be used for some simple things.

First, I noticed that we have a nice rfc for this at
ftp://ftp.rfc-editor.org/in-notes/rfc3549.txt which will, undoubtedly,
contain errors because the kernel code is the only authoritative source,
but, it is a good start. Another nice link is this:
http://people.redhat.com/nhorman/papers/netlink.pdf

The first type of messages I looked at are the messages which you can
use to get a list of network interfaces (devices) in a system. These are
defined in section 2.3.3.1. As florian and you suggested, looking over
the libnl sources helped a lot to figure out how this works. So, here is
a small summary of how these messages work.

1) construct a message header with type RTM_GETLINK and flags NLM_F_ROOT
and NLM_F_MATCH. message body contains a single 8 bit family field set
to AF_UNSPEC. send message.

2) read reply from kernel: a set of concatenated RTM_NEWLINK messages
with flag NLM_F_MULTI terminated by a message of type NLMSG_DONE. Each
RTM_NEWLINK message contains first an "Interface service message
template" (as defined in 2.3.3.1) followed by a set of TLV-encoded
attributes: IFLA_IFNAME, IFLA_TXQLEN, IFLA_OPERSTATE, IFLA_LINKMODE,
IFLA_MTU, IFLA_MAP, and, optionally, any of the IFLA_LINK, IFLA_MASTER,
IFLA_QDISC, IFLA_ADDRESS, IFLA_BROADCAST, IFLA_STATS. (I am reading
rtnl_fill_ifinfo in linux-2.6.25/net/code/rtnetlink.c). Each NEWLINK
message is aligned to the nearest 4 byte boundary (as per NLMSG_ALIGNTO)
while each attribute is stored as a (type,length,data) triplet with type
and length stored at 16 bit integers and data a variable length data
field aligned to the nearest 4 byte boundary (see NLA_HDRLEN,
linux-2.6.25/include/linux/netlink.h)

Although the rfc seems to imply a network byte order, the implementation
looks like it uses the host native byte order.

Now, as an exercise, let's look at how we could implement this without
having to copy/paste all of the linux kernel netlink implementation.

The first thing to observe is that the messages exchanged are really
made of:
1) a set of consecutive netlink message headers+body. If there is more
than one header, all but the last are flagged with NLM_F_MULTI and the
last one is of type NLMSG_DONE.
2) each body always starts with a message-specific "template" followed
by a set of message-specific TLV-encoded attributes.

I looked a bit at the current API in src/node/netlink-msg-header.h and I
would like to suggest a couple of modifications which will, hopefully,
make this whole parsing thing much easier to deal with:

1) Conceptually, I think that it makes sense to have only one Header
subclass and implement _all_ parsing of _all_ messages in this single
class.
2) With that said, I would suggest something along the lines of:

class NetlinkAttributeValue
{
public:
  NetlinkAttributeValue ();

  void SetString (std::string value);
  void SetU32 (uint16_t type, uint32_t value);
  void SetU16 (uint16_t type, uint16_t value);
  void SetU8 (uint16_t type, uint8_t value);
  void SetAddress (Address address);

  enum Type_e {
   UNSPEC, // invalid initial value.
   STRING,
   U32,
   U16,
   U8,
   ADDRESS
  };
  enum Type_e GetType (void) const;
  std::string GetString (void) const;
  uint32_t GetU32 (void) const;
  uint16_t GetU16 (void) const;
  uint8_t GetU8 (void) const;
  Address GetAddress (void) const;
private:
  enum Type_e m_type;
  uint32_t m_u32;
  uint16_t m_u16;
  uint8_t m_u8;
  std::string m_string;
};

struct NetlinkAttribute 
{
  uint16_t type;
  NetlinkAttributeValue value;
};

class InterfaceTemplate : public Header
{
public:
  InterfaceTemplate ();
  // all the boring serialization/deserialization.
  static TypeId GetTypeId (void);
  virtual TypeId GetInstanceTypeId (void) const;
  virtual void Print (std::ostream &os) const;
  virtual uint32_t GetSerializedSize (void) const;
  virtual void Serialize (Buffer::Iterator start) const;
  virtual uint32_t Deserialize (Buffer::Iterator start);

  enum {
    UP = (1<<0),
    BROADCAST = (1<<1),
    DEBUG = (1<<2),
    ...
  };

  void SetDeviceType (uint16_t type);
  void SetInterfaceIndex (uint32_t index);
  void SetDeviceFlags (uint32_t index);
  void SetChangeMask (uint32_t mask);

  uint16_t GetDeviceType (void) const;
  uint32_t GetInterfaceIndex (void) const;
  uint32_t GetDeviceFlags (void) const;
  uint32_t GetChangeMask (void) const;
private:
  uint16_t m_deviceType;
  uint32_t m_interfaceIndex;
  uint32_t m_deviceFlags;
  uint32_t m_changeMask;
};

class AddressTemplate : public Header
{
public:
  // all the boring serialization/deserialization.
  static TypeId GetTypeId (void);
  virtual TypeId GetInstanceTypeId (void) const;
  virtual void Print (std::ostream &os) const;
  virtual uint32_t GetSerializedSize (void) const;
  virtual void Serialize (Buffer::Iterator start) const;
  virtual uint32_t Deserialize (Buffer::Iterator start);

  enum {
    F_SECONDARY = 0x01,
    F_PERMANENT = 0x80,
    F_DEPRECATED = 0x20,
    F_TENTATIVE = 0x40
  };
  enum {
    SCOPE_UNIVERSE = 0,
    SCOPE_SITE = 200,
    SCOPE_LINK = 253,
    SCOPE_HOST = 254
  };

  uint8_t GetFamily (void) const;
  uint8_t GetLength (void) const;
  uint8_t GetFlags (void) const;
  uint8_t GetScope (void) const;
  uint32_t GetInterfaceIndex (void) const;

  void SetFamily (uint8_t family);
  void SetLength (uint8_t length);
  void SetFlags (uint8_t flags);
  void SetScope (uint8_t scope);
  void SetInterfaceIndex (uint32_t index);
private:
  uint8_t m_family;
  uint8_t m_length;
  uint8_t m_flags;
  uint8_t m_scope;
  uint32_t m_index;
};

class NetlinkMessage : public Header
{
public:
  NetlinkMessage ();
  // all the boring serialization/deserialization.
  static TypeId GetTypeId (void);
  virtual TypeId GetInstanceTypeId (void) const;
  virtual void Print (std::ostream &os) const;
  virtual uint32_t GetSerializedSize (void) const;
  virtual void Serialize (Buffer::Iterator start) const;
  virtual uint32_t Deserialize (Buffer::Iterator start);

  enum {
    F_REQUEST = 1,
    F_MULTI = 2,
    F_ACK = 4,
    // ...
  };

  void SetFlags (uint16_t v);
  void SetSeq (uint32_t v);
  void SetPid (uint32_t v);

  // the type argument is one of RTM_NEWLINK,RTM_DELLINK,RTM_GETLINK
  void SetInterfaceTemplate (uint16_t type, InterfaceTemplate template);
  // assert if type != RTM_NEWADDR,RTM_DELADDR,RTM_GETADDR
  void SetAddressTemplate (uint16_t type, AddressTemplate template);
  
  void AddAttribute (NetlinkAttribute attr);

  uint16_t GetFlags (void) const;
  uint16_t GetType (void) const;
  uint32_t GetSeq (void) const;
  uint32_t GetPid (void) const;

  // assert if m_type != one of RTM_NEWLINK,RTM_DELLINK,RTM_GETLINK
  InterfaceTemplate InterfaceTemplate (void) const;
  // assert if type != RTM_NEWADDR,RTM_DELADDR,RTM_GETADDR
  AddressTemplate SetAddressTemplate (void) const;

  uint32_t GetNAttributes (void) const;
  NetlinkAttribute GetAttribute (uint32_t index) const;


private:
  uint32_t m_type;
  uint16_t m_flags;
  uint32_t m_seq;
  uint32_t m_pid;
  InterfaceTemplate m_interfaceTemplate;
  AddressTemplate m_addressTemplate;
  std::vector<NetlinkAttribute> m_attributes;
};


A sample application for early testing could do something like this:

// send request for list of all interfaces.
Ptr<Socket> sock = ...;
Ptr<Packet> p = Create<Packet> ();
NetlinkMessage msg;
msg.SetSeq (0);
msg.SetPid (0);
msg.SetFlags (NetlinkMessage::F_ROOT | NetlinkMessage::F_MATCH);
msg.SetInterfaceTemplate (RTM_GETLINK, InterfaceTemplate ());
p->AddHeader (msg);
sock->Send (p);

// invoked when data is received from socket.
static void MyDataRecv (Ptr<Packet> p)
{
  NetlinkMessage msg;
  p->RemoveHeader (msg);
  bool first = true;
  while (first || msg.GetType () != NLMSG_DONE) {
    NS_ASSERT (msg.GetType () == RTM_NEWLINK);
    InterfaceTemplate interface = msg.GetInterfaceTemplate ();
    // print interface content
    // print attributes in msg.GetAttribute ()
    p->RemoveHeader (msg);
    first = false;
  }
}

The above classes are just a sketch but it should be not too hard to
extend the idea to a RouteTemplate class as per section 3.1.1. Before
getting there, it would, however, be nice to implement the
InterfaceTemplate outlined above, as a proof of concept to make sure
that I did not miss anything.

best regards,
Mathieu