[Ns-developers] [ns-3] binding Packet/Tag/TraceContext

[Gustavo Carneiro]

On 08/01/2008, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
>
> On 08/01/2008, Mathieu Lacage <mathieu.lacage at sophia.inria.fr> wrote:
> >
> > On Tue, 2008-01-08 at 12:00 +0000, Gustavo Carneiro wrote:
> >
> > >         2) non-virtual methods are going to be made virtual:
> > >           Header::Serialize, etc.
> > >
> > > Do you mean to go back to the original Header class, which contained
> > > virtual methods, before you made it an empty class and switched to
> > > using templates?  That would sound great to me.  Not only for Python
> >
> > yes.
> >
> > >  but also for pure C++ point of view...
> >
> > I personally hate the idea of doing this if only because it means that I
> > get to rewrite all this code, once more which does not make me very
> > happy. I want to know what the alternatives are first.
>
>
> The alternative is to define a PythonHeader class containing the virtual
> methods, and use it only for python.  Then the C++ API remains unchanged.
>
> >         3) Tags and TraceContextElement are going to require an
> > >         explicit
> > >         serialization/deserialization API. i.e., something a bit like
> > >         Header::Serialize/Deserialize
> > >
> > > For tags, I think they are already being serialized under the hood,
> > > right?  Just a matter of exposing an _additional_ API with it.
> >
> > They are serialized through memcpy which is not going to work if we use
> > c++ classes with virtual methods, hence, the need for an extra pair of
> > explicit Serialize/Deserialize methods which write and read the tag
> > data
>
>
> That sounds fine for the Python bindings, but awful for the C++ API.  If I
> have to implement Serialize/Deserialize I will think very hard before
> implementing a new tag.  Please don't do this.
>

Correction: Serialize/Deserialize is actually _needed_ for supporting Python
tag objects.  But it is not _wanted_ for C++ ones.  The obvious solution
would be to optionally support Serialize/Deserialize, fallback to memcpy
when not available.

Although I am concerned that serialization of a python object is going to be
inneficient; python tag objects will likely serialize (I am thinking of
using pickle) to a rather large byte array.  Since tags have a small fixed
maximum size, either the size will have to be increased or else we won't
support python tag objects.

That's why it would be much more interesting if tags were instead reference
counted objects that would be passed around and which would have a Clone ()
method for when packets are copied.

>         4) Another thing which all these C++ bits need is a way to get
> > >         access to
> > >         a pointer to an instance of the right
> > >         Header/Tag/TraceContextElement
> > >         type based on a mapping from integer id to type instance.
> > >
> > > It would be much easier for Python bindings if instead we used the
> > > 'automatic type narrowing' feature of PyBindGen, which uses the typeid
> > > operator.  For that to work we need tags/elements/headers to have at
> > > least one virtual method ( e.g. the destructor), and we need APIs that
> >
> > > return to the caller a new copy of the tag/header/element, with
> > > caller-owns-return policy.
> >
> > I am not going to store the typeid string name in the Tags or
> > TraceContext container classes, hence, the need for a unique integer id
> > which identifies what we store.
> >
> > >
> > > However I am not sure how it turns out to mix virtual methods and
> > > objects created with memcpy.  What happens to the vtable of these
> >
> > It somewhat works (at least, on Linux-x86 ) but is so evil, that I won't
> >
> > even bother try to do it.
> >
> > >  objects?  Ideally I would have these objects behaving like normal
> > > objects, but I understand if fixing it would have too much impact on
> > > the existing API.
> > >
> > >
> > >         Then, these c
> > >         ++ bits need to be able to call deserialize on the type
> > >         instance and
> > >         then, call Print on the resulting deserialized instance to
> > >         print the
> > >         content of the element.
> > >
> > > Bad requirement IMHO.  We do not want everything mapped into a string
> > > in Python.  We want a structured TraceContextElement in C++ to be
> > > mapped into an appropriate object in Python, right?
> >
> > I do not really understand what you mean here. What I am describing is
> > that in the C++ container, I need to store a unique integer id next to
> > each data blurb saved with Serialize. The unique integer id is then used
> >
> > to get access to an instance of the underlying C++ type which supports
> > Deserialize. This allows you to iterate over the set of types stored in
> > the container and invoke Print if you need to print the content of the
> > instance serialized into the container. This is needed if you want to be
> > able to store a python subclass of TraceContextElement into the
> > TraceContext.
> >
> > Now, if the only thing you need to support from python is the ability of
> >
> > reading a C++ TraceContextElement from a TraceContext, you don't need
> > anything more than what we have today:
>
>
>
> def MyPythonFunction(context, ...):
> >   index = new NodeListIndex ()
> >   context.GetElement (index)
>
>
> I guess that could work.
>
> All you have to do is to not wrap the C++ NodeListIndex directly.
> > Instead, you can wrap this:
> >
> > class PythonTraceContextElement
> > {
> > public:
> >   virtual bool ReadFrom (const TraceContext &context) = 0;
> > };
> >
> > class PythonTraceContextElementNodeListIndex : public
> > PythonTraceContextElement
> > {
> > public:
> >   uint32_t GetNodeId (void) const {
> >     return m_elem.Get ();
> >   }
> > private:
> >   bool ReadFrom (const TraceContext &context) {
> >     bool found = context.GetElement (m_elem);
> >     return found;
> >   }
> >   NodeListIndex m_elem;
> > };
> >
> > class PythonTraceContext
> > {
> > public:
> >   bool GetElement (PythonTraceContextElement *elem) {
> >     return elem->ReadFrom (m_context);
> >   }
> > private:
> >   TraceContext &m_context;
> > };
> >
> > And, then, when the user creates a python object of type NodeListIndex,
> > you create a C++ object of type PythonTraceContextElementNodeListIndex
> > and wrap _that_ object.
>
>
> I don't think it needs something so complicated.   All PyBindGen needs is
> to map each instance of the GetElement<T> method into the same method name.
> Then C++ overloading will kick in and ensure the right thing happens.  Needs
> only minor PyBindGen modifications.  But I think it's a good idea.
>
> >
> > >
> > >         So, the big thing the C++ code needs is a map
> > >         uid <-> type instance. If we allow python to create new
> > >         tags/TraceContextElements/Headers, then, the python code needs
> > >         to be
> > >         able to register new entries in this map. Gustavo: could you
> > >         recommend a
> > >         simple way to do this ?
> > >
> > >
> > > I would rather use standard C++ RTTI.
> >
> > The C++ code, on the other hand, is not going to use C++ RTTI.
>
>
> OK, so for TraceContextElements I think we have a possible solution based
> on C++ overloading.
> For tags a similar solution can be used.
> For headers probably as well.
>
> Hm.. maybe NS-3 does not need API changes after all :-)
>
> This needs to be prototyped to confirm.
>
> One thing that I'm sure I'll need is to have all possible tags inherit
> from a single common Tag base class.  Same for Header and
> TraceContextElement.  This is so that I can use gccxml introspection to
> discover all the possible classes for tags, tracing, and headers.
>
> --
> Gustavo J. A. M. Carneiro
> INESC Porto, Telecommunications and Multimedia Unit
> "The universe is always one step beyond logic." -- Frank Herbert
>



-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert