[Ns-developers] NS-3 PyBindGen again

[Gustavo Carneiro]

On 07/01/2008, Mathieu Lacage <mathieu.lacage at sophia.inria.fr> wrote:
>
> On Sun, 2008-01-06 at 23:51 +0000, Gustavo Carneiro wrote:
> >   Hello again.
> >
> >   Good news: I have finally wrapped NS-3 Simulator::Schedule* and most
> > concrete callbacks (Callback<T1,T2, ...>).  It is not uploaded yet
> because
>
> A really cool extension to this would be something like what I
> implemented in yans: a way to avoid going c++ -> python -> c++ when c++
> calls a callback which happens to be a c++ callback but wrapped in
> python. The way I implemented this was through the following function:
>
> def make_callback(function, *args):
>     try:
>         if inspect.ismethod (function):
>             name = "_yans." + function.im_class.__name__ + "_" +
> function.__name__ + "_callback_tag"
>             command = name + '(args[0])';
>         else:
>             name = "_yans." + method.__name__ + "_callback_tag"
>             command = name + '()'
>         holder = eval (command)
>     except:
>         if len(args) == 0:
>             holder = MyPyFunctionHolder (function)
>         elif len(args) > 1:
>             holder = MyPyFunctionHolder (function, *args)
>         else:
>             holder = _yans.FunctionHolder (function, args[0])
>     return holder
>
> which attempted to delegate the creation of the callback first to a c++
> function whose name was based on the name of the function to create a
> callback for. If the delegation failed, it would use a simpler
> python-based functor.
>
> The result was c++ -> c++ rather than the more expensive c++ -> python
> -> c++.


I won't comment on you specific implementation, but I have to question
whether this is really useful or not.  How often do you connect callbacks
from Python to C++ functions?

> of transient  mercurial server problems, but if anyone is impatient just
> > checkout the normal gjc/ns-3-pybindgen branch and apply this bundle:
> >  http://telecom.inescporto.pt/~gjc/ns-3-pybindgen.bundle
> >
> >   Notably missing is the support for parameters of type
> CallbackBase.  This
> > is used mainly for tracing.  But I do not find it that much useful for
> > Python (tracing with Python callbacks will surely slow down simulations
> > considerably), so I'd rather not work on it for now.  I'm sure there are
> > many APIs still missing; they will require more manual work, but I
> belive
> > most methods are being supported, at least for scritping kind of
> work.  For
> > protocol implementation I am not so sure.
>
> Do you have an idea on how this could be implemented ? If so, could you
> outline briefly how you think it could be done ?


This is where we have to provide a MakeCallback API to Python, the types
mangled into the name. Example:

In C++:

static void
CourseChange (const TraceContext &context, Ptr<const MobilityModel>
position)
{
[...]
}

[...]
      notifier->TraceConnect ("/course-change", MakeCallback
(&CourseChange));
[...]

In Python would be:

def CourseChange (context, position):
    [...]

[...]
      notifier.TraceConnect("/course-change",
MakeCallback__lt__const_TraceContext_amp_Ptr__lt__const_MobilityModel__gt____gt__(CourseChange));

Yes, it's very ugly.  But I don't see any way around it with the current
API.

It would be nice, though, if the API could be enhanced to provide
introspection of the required tracing callbacks.  An API like this would be
enough:

const std::type_info GetTraceCallbackType(const std::string path) const;

Should I open a bug report?

>   The Python API is very similar to the C++ one.  For example, the script
> > below is a port of samples/main-simple.cc to Python.
> >
> >   The generated source file is currently about 2.4MB, 68k
> lines.  Although I
> > would love to be able to split this single source file into several
> smaller
> > files, some day.
> >
> >   So what are the next steps?  First I think the NS-3 team needs to
> decide
> > whether to adopt this solution.  I certainly hope so, or else I wasted a
> ton
> > of effort :P
> >
> >   Next, how would it be best integrated with NS-3?  I can see several
> > degrees of integration:
> >
> >    1. Make python bindings generation mandatory, require gccxml
> (currently
> > from CVS only, unfortunately), pygccxml, and pybindgen, from everybody;
>
> how about detecting automatically if these tools are available and
> attempt to build the bindings if they are. If the tools are not there,
> do nothing ?


That is what is being done right now.

>    2. Ship with NS-3 a pregenerated ns3module.cc file;
>
> This makes sense for release tarballs so, it would be nice to integrate
> this in the distcheck target.


I guess.

>
> >       a) And add a --pybindgen waf option to re-generate the file (just
> in
> > case someone decides to modify the API and make the bindings stop
> compiling)
> > ?
>
> let's make this automatic and rebuild the bindings if the tools to do so
> are available.
>
> >
> >       b) Add the ns3module.cc to the repository also, for developers?  I
> > would adivse against it, though, as the file is rather large...
>
> I don't care very much about the size of the file but I think that if
> you are an ns-3 developer, and you need to use the bindings, it is
> pretty reasonable to ask you to install the tools to re-generate the
> ns3module.cc file. i.e., don't push the ns3module.cc file into the repo.
>
> >   If no major problems are found, I would envision start of python
> bindings
> > since the NS-3 release in mid February.
>
> I think that this would be pretty cool.
>
> regards,
> Mathieu
>

Thanks for the comments.

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert