[Ns-developers] ns-3 python bindings (was: Re: NS-3 core API freeze?)

[Tom Henderson]

>-----Original Message-----
>From: Gustavo Carneiro [mailto:gjcarneiro at gmail.com]
>Sent: Wednesday, April 23, 2008 09:37 AM
>To: 'Mathieu Lacage'
>Cc: 'ns-developers'
>Subject: Re: [Ns-developers] NS-3 core API freeze?
>
>2008/4/23 Mathieu Lacage <mathieu.lacage at sophia.inria.fr>:
>
>>
>> On Tue, 2008-04-22 at 20:06 +0100, Gustavo Carneiro wrote:
>>
>> > 2. Any python binding technology will require the user to install the
>> > respective tools, so changing technology will not magically fix 5.1;
>>
>> Although I hate that perspective, I consider it much more reasonable to
>> ask a user to install a released version of swig (or boost::python, or
>> anything else already released) than a cvs version of gccxml. The later
>> is _not_ ok for anyone but you.
>>
>> > 3. I have manpower issues and am not sure I could implement this for
>> > 15th May even if API was already frozen.   I only work a few hours a
>> > week on this, mostly just on weekends
>>
>> understood.
>>
>> >   3.1 One way to work around this is to have someone else help with
>> > this task
>> >
>> >      3.1.1 The software engineering rule "adding more manpower to a
>> > late project makes it later" may apply here, though I hope I'm
>> > wrong :-(
>>
>> you are probably right though.
>>
>> >
>> >   3.2 In addition, in your roadmap you are still proposing some
>> > core/common API changes, which in my experience tend to disrupt or
>> > destroy some work done on Python bindings.  Personally I am reluctant
>> > to do more Python work on top of a unstable API, as it feels like
>> > wasted work :|
>>
>> I was under the impression that you were not wrapping the Packet API so,
>> the tag stuff should not impact you. The socket stuff might though.
>>
>> > 4. Also there are some other things left to do in the Python bindings;
>> > do you think it is better to have a way to let users extend Python
>> > bindings, or is it better to fix Python API issues sooner to avoid
>> > breaking Python code?  This is why I keep saying fixing 5.1 is _not_
>> > my priority.
>>
>> As I said in my previous email, an unfinished binding which is easy
>> hacked on by users (and other developers like me) is vastly more useful
>> than a finished binding with a single developer. i.e., I personally
>> would be perfectly fine with using your python binding library by hand
>> if it was decently documented: the load of writing the bindings can be
>> distributed while the load of building the underlying infrastructure
>> cannot.
>>
>> So, using gccxml to save on the load of writing the bindings is not
>> solving the critical problem: it is solving a problem we can solve by
>> other means through offloading work from you to others. On the other
>> hand, I have a hard time seeing how we could offload work on your python
>> binding library from you: that is why I encouraged you a couple of times
>> to work on documenting its API and try to polish and improve it with
>> your users in mind.
>
>
>Lack of documentation is not the main problem here.  Usage of gccxml cvs
>seems to be the main problem.
>
>
>>
>> > 5. Also, I think fixing 5.1 should be included in the "polishing of
>> > python bindings" stage of the roadmap you propose.
>>
>> 5.1 is a must-have feature. So, we need to be sure that we can solve
>> this problem in time for the release. If that means that the bindings
>> themselves do not cover 100% of the API for the release, I think that it
>> is ok because we can fix it later.
>>
>
>If usage of gccxml cvs is such a burden (no one told me so when I started
>using it), then I see one possible course of action.  One developer
>regulardly uses gccxml and saves the output xml file in the ns-3
>repository.  Then normal users and other developers don't need gccxml, only
>pybindgen and pygccxml.  Adding API can be done using pybindgen API.
>
>Another course of development would be for pybindgen to serialize its own
>data structures to an xml format.  This would make also pygccxml optional,
>but implementing this would be a lot more work.

Gustavo,

I had a question about a statement on the wiki page:

"Python bindings for NS-3 are being developed using a new tool called PyBindGen. This tool uses gccxml to scan header files and extract API definitions in a semi-automatic way."

What do you mean by semi-automatic?  What portions are automatic and what not?  The example you posted on the wiki looks like it is completely automatic.

The use case I am concerned about, for the moment, is the developer who writes a new ns-3 model in C++ (for instance, a new routing protocol) but wants to script in Python.  Let's suppose that the new model introduces several new classes and APIs using these classes in the C++ system.

Assuming that the bindings toolchain can be installed, what is the workflow likely to look like for this developer?  
- at what point in the build are these new bindings generated?  
- how likely is this automated process to succeed, and if not, what does the developer have to do when it breaks down?  How will the developer know what to do (is the error message helpful)?

Another question is, if we were to work with the development versions of the various tools you are using, is there a reasonable possibility of bundling known-good versions of these tools and distributing with ns-3, such as what you do with waf?  What are the dependencies of these tools (gccxml, pygccxml, pybindgen)?  

Thanks,
Tom