[Ns-developers] Status of Python bindings

Wed May 7 10:54:01 PDT 2008

2008/5/6 Mathieu Lacage <mathieu.lacage at sophia.inria.fr>:

>
> On Tue, 2008-05-06 at 00:56 +0100, Gustavo Carneiro wrote:
>
> >         > 2. Once in a while, a developer does something like ./waf
> >         --scan-bindings,
> >         > which calls pybindgen in gccxml mode, and outputs a
> >         ns3module_generated.py
> >         > file (or several files, one per module, if the generated
> >         file ends up being
> >         > large)
> >
> >
> >         +1 to the latter option (one file per ns3 module)
> >
> > It is non-trivial work, but I can do it.
>
> +1 beer !
>
>
> >         I would be fine with re-running --scan-bindings (2) regularly
> >         to attempt
> >         to generate the bindings for our codebase provided we can
> >         agree that if
> >         it fails and if you cannot fix it (because you are dead or
> >         just away for
> >         an extended period of time :), we can just use its output as a
> >         starting
> >         point and then, write the missing python code by hand. To
> >         summarize, if
> >         2) works sufficiently well that the generated code is readable
> >         and
> >         editable and that we almost never have to hack its output, I
> >         am all for
> >         it because I know that, at worst, I can hack it without
> >         relying on you
> >         being alive.
> >
> > But of course the code will be perfectly readable and hackable.
>
> +2 beer !
>
> >
> > Only the customizations for supporting the core module, as well as
> > simulador schedule, will be hard to read, but that can't be helped.
> > If you want a nice Python API you need custom code for certain low
> > level stuff.  That would be true for _any_ python binding technology
> > you decided to use.
>
> yes.
>
> >
> > As for me disappearing, Python bindings for NS-3 would be in no worse
> > shape than NS-3 core module would be if you disappeared and took all
> > that C++ template black magic with you... :-)
>
> yep.
>
>
> >         Which clearly outlines the underlying C++ function and its
> >         python
> >         equivalent.
> >
> > If the python function name is different from the C++ one you are
> > probably doing something wrong.  Different <whatever> names in Python
>
> There are some cases where you want to do that to wrap some non-trivial
> C++ APIs to python: For example, the Simulator::Schedule methods cannot
> be easily wrapped and I see, as expected, that you are using renaming
> tricks to wrap them.
>
> >  and C++ will only cause confusion for NS-3 programmers, who often
> > need to switch back and forth between Python and C++ and do not want
> > to learn two similar but subtly different APIs.  So I really don't
> > agree making the python name mandatory will be helpful.
>
> ok.
>
> >
> >
> >
> >         Now, I do understand why you took the simpler path of avoiding
> >         the
> >         redundant information everywhere because the number of cases
> >         where
> >         CppFunctionName will be different from PyFunctionName will be
> >         very small
> >         but, that shortcut really does muddy the conceptual model of
> >         how things
> >         work.
> >
> >         So, I can see a couple of ways to deal with this 'muddying':
> >          a) become pedantic as suggested above at the cost of extra
> >         verbosity
> >         and pain
> >          b) structure in the API in a different way which moves all
> >         the
> >         assumptions about the C++ -> python names to a single clear
> >         location.
> >
> >         I think that a) is really too painful so, what could b) look
> >         like ?
> >
> > Bah.. you are just using the old trick of offering a worse alternative
> > as option so that the one you prefer does not seem so bad by
> > comparison... :P
>
> My boss gave me hands-on courses in rhetoric :)
>
> > But I am willing to give it a fair chance..
>
> thank you :)
>
> > .
> >
> >         Here
> >         is a proposal:
> >
> >         # describe the C++ code we want to wrap
> >         header = CppHeader ('a.h')
> >         header.add_function ('CppFunctionName', return_type, args)
> >         A = header.add_class ('CppClassName, ...)
> >         A.add_method (...)
> >         ns3 = header.add_namespace ('ns3')
> >         ns3.add_function (...)
> >         ns3.add_class (...)
> >
> > Since I really want PyBindGen to be applicable to pure C code as well
> > as C++, this would have to be just class Header, not CppHeader.
>
> I thought that we could have a CHeader class to mirror CppHeader for
> that case.
>
> >
> >
> >
> >         # describe how to wrap the above C++ code.
> >         mod = PyModule ('a')
> >         mod.wrap_header (header)
> >         mod.generate_output (out_file)
> >
> > Personally, I don't grok this model very well.  Types and functions
> > belong to namespaces, not headers.  When you are programming in C++
> > you don't think in terms of which header file each type lives in, you
> > think about namespaces.  Or at least I do.
>
> I suspect that my main concern is that I have not yet been able to
> really understand the model implemented by your API which is why I have
> tried to come up with a model which made sense to me. So, I wrote a
> couple more examples attached to this email to try to get a better
> handle on the way your API is structured: feel free to add them to your
> examples directory.
>
> Thinking some more about your API, I wrote the following summary: would
> you mind point out what I misunderstood ?
>
> The Module python class represents a set of python bindings to generate
> within a top-level python module. If you write:
> mod = Module ('a')
> mod.add_ (...)
> you effectively create a python module 'a' which contains:
>  - a set of python functions to wrap c++ functions
>  - a set of python classes to wrap c++ classes
>  - a set of python modules to wrap c++ namespaces
>  - a set of python inner classes to wrap c++ inner classes
>

>
>
> Is the above paragraph correct ?

Basically is correct.

>
>
> Part of my confusion with this API might simply be that this 'Module'
> python class does not necessarily represent a single python module: it
> _can_ represent one but it can also represent a python package.

Erm.. more or less.

One toplevel Module can contain other inner submodules, which are also
Module objects.  Each Module can have a parent Module or None.  So what you
have in the end is a tree of Modules.

All the Modules in a tree are generated to the _same_ Python module shared
library.  Although it looks like a package, in the sense that you can do
from <module> import <submodule>, they are all in the same file and
<submodule> is always present.  In normal Python packages, you need to do
"import foo.bar" before you can use foo.bar; in these
modules-containing-other-modules you can "import foo" and then access
foo.bar (like, with pygtk, you can access gtk.gdk with just importing gtk).
So, there are similarities, but also some subtle differences to a real
Python package.

What should be retained is that, in terms of wrapping C++ code, you can have
a tree of Module objects.  Of course these Modules are a Python centric
representation :-)  But what I envision and try to support directly in
PyBindGen, because I think makes sense, is that Modules can represent C++
namespaces.  A tree of C++ namespaces is mapped to a tree of modules.

Of course Modules are more generic, and you can make Python submodules for
specific groups of functions or types, if that's what you want.

> So,
> maybe the simple solution to my complaints would be to find a better
> name for that class such as 'Binding' and use it as follows:
>
> binding = Binding ('TopLevelModuleName')
> binding.add_ (...)

This is pending more discussion.

I am not sure I understand the concept of this Binding...  In C++ we have a
tree of namespaces, each containing functions and types.  This is a C++
model centric view.  Earlier you were proposing a source header file centric
view, where you modeled the API based on which header file it is contained.
Can you help me understand this Binding class?  I don't quite understand
what it tries to model...

Now, while writing the attached examples, it occured to me that a small
> set of minor syntactical changes would also improve the API:
>  - add_include ('"a.h"') -> add_include ('a.h')

Agreed to add_local_include('a.h') and add_system_include('a.h') (because of
"a.h" vs <a.h>)

>
>  - C = CppClass (...), add_class (C) -> C = add_cpp_class (...)

OK.

>
>  - add_method (CppMethod (...)) -> add_cpp_method (...)

Agreed , but add_method(name, returntype, params) instead of add_cpp_method.

>
>  - Function (return_type, name, args) -> Function (name, return_type,
> args). The same would hold for add_cpp_method, of course.

Fine.

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert