[Ns-developers] NS-3 core API freeze?

[Gustavo Carneiro]

2008/4/22 Mathieu Lacage <mathieu.lacage at sophia.inria.fr>:

> hi gustavo,
>
> On Mon, 2008-04-21 at 18:22 +0100, Gustavo Carneiro wrote:
> > Latest ns-3-dev code merge broke the python bindings again, in a
> non-trivial
> > way that requires me to spend some hours figuring out how to  adapt.
>  This
> > highlights the issue with python bindings and API changes.  If we want
> > usable python bindings during May we need stable NS-3 core API right
> about
> > now (say, until friday).  That means no touching of src/core,
> src/common, or
> > src/simulator unless it's some minor tweak to help the python bindings.
> >
> > Is the project ready to commit to this?  Will it officially commit to
> this
> > freeze?  If it's not possible then we need to review the roadmap for
> Python
> > bindings inclusion.
>
> I think that we should be ready to freeze the core API now, but, you
> raise rightfully the question of what our roadmap is for the release.
> So, I have tried to outline what still needs to be done below to reach
> "gold" status for the release:
>
> 1) documentation: need work to convert tutorial to new helper API.
>
> 2) socket implementation bugs:
>  2.1 udp needs a finite rx packet buffer
>  2.2 tcp needs a finite tx and rx byte buffer
>  2.3 tcp needs to implement its tx and rx buffers as lists of packet
> pointers, and not raw buffers to handle correctly application-level
> tagged packets
>  2.4 packet tagging implementation and API needs to be rewritten to
> track tags on a byte-basis to allow application-level tagged packets to
> work with tcp sockets
>  2.5 tcp sockets need to implement the "send" callback as was discussed
> many times, that is, it needs to invoke that callback when there is new
> room in the tx buffer, not when a packet is sent down the stack.
>
> 3) socket API additions: need to add the API I suggested in the socket
> thread to allow future implementations of posix C-style sockets
>
> 4) socket API changes: tom suggested to move to a SocketHelper class the
> raw buffer management.
>
> 5) python bindings: there are a number of key issues:
>  5.1 need to verify that gustavo's proposed implementation addresses
> our key use-cases: a) allow users to write new C++ models and bind them
> to python without having to install gccxml CVS and pygccxml, b) allow
> users to modify and hack a released version of the ns3 c++ API and have
> the python binding still work without having to install gccxml CVS and
> pygccxml.
>  5.2 document the overall structure of the python bindings: we need to
> know how they work at least from a high-level perspective. basically, a
> 1-page a4 document about how the code is generated and how the generated
> code works. Assuming that the reader knows the details of the python
> low-level binding API works is ok.
>
> I raised 5.1 a couple of times privately with you, gustavo and you seem
> to have other higher-priority issues which is perfectly fine but which
> might become a serious blocker now. I consider 5.1 to be the top
> priority and to be a showstopper for merging: if 5.1 is not resolved, we
> will have to look at other python binding technology alternatives: we
> _cannot_ request users to install gccxml CVS to build our python
> bindings. 5.2 is also a very high priority: it is not ok to merge the
> whole bindings without any kind of documentation about the way they
> work. I would be much more comfortable with unfinished documented
> bindings which solve our important use-cases than fully finished
> black-box bindings which might not solve our important use-cases.
>
> Now, to go back to the bigger question of release schedule: I think that
> the biggest issues we need to tackle are 5) and 2). I believe that craig
> agreed to start looking at 5) this week and to try to assess the status
> of your branch with regard to our use-cases. Depending on that initial
> assessment, he might be able to help you work on the branch or go in
> another direction. I will work on 2.4 this week before going away until
> the 8th and will work on general bug-fixing after, together with
> documentation (1). Tom will probably work on 1), 3), and, 4), and, will
> try to oversee work on the other items in 2). Raj will probably want to
> work on items in 2) except for 2.4 for which I promised a working
> solution :)
>
> If we are still aiming for a mid-june release, we need to complete all
> items above by the end of may to have at least 2 weeks of spare time for
> review and serious bug-fixing. So, if you ignore the relatively minor
> API changes outlined above, we can freeze our APIs now and focus on the
> issues described in this email until may the 15th at which point we can
> re-evaluate our status.
>
> to summarize, I would suggest the following schedule:
>  - 22-04-2008 -> 15-05-2008: 3), 4), 2.5), 2.3), 2.4), 5). Ideally, we
> would be ready to merge the python binding on the 15-05-2008
>
>  - 15-05-2008 -> 31-05-2008: 1), 2.2), 2.1), polishing of python
> bindings.
>
>  - 31-05-2008: release release-candidate-1
>
>  - 01-06-2008 -> 15-06-2008: bug-fix only stage, documentation review,
> polishing
>
>  - 15-06-2008: release final version.
>
> I am fairly certain that the above proposal is a bit too optimistic,
> especially, the first 3 weeks but I would hope that these first 3 weeks
> can slip by one week without really impacting the final schedule. At
> worst, we can still slip by 2 weeks the final release date to 30-06-2008
> but we cannot slip more.


OK, some comments about the whole email.

1. To resolve 5.1 (allowing users to extend the python bindings) significant
work is needed;

2. Any python binding technology will require the user to install the
respective tools, so changing technology will not magically fix 5.1;

3. I have manpower issues and am not sure I could implement this for 15th
May even if API was already frozen.   I only work a few hours a week on
this, mostly just on weekends

  3.1 One way to work around this is to have someone else help with this
task

     3.1.1 The software engineering rule "adding more manpower to a late
project makes it later" may apply here, though I hope I'm wrong :-(

  3.2 In addition, in your roadmap you are still proposing some core/common
API changes, which in my experience tend to disrupt or destroy some work
done on Python bindings.  Personally I am reluctant to do more Python work
on top of a unstable API, as it feels like wasted work :|

4. Also there are some other things left to do in the Python bindings; do
you think it is better to have a way to let users extend Python bindings, or
is it better to fix Python API issues sooner to avoid breaking Python code?
This is why I keep saying fixing 5.1 is _not_ my priority.

5. Also, I think fixing 5.1 should be included in the "polishing of python
bindings" stage of the roadmap you propose.

So, as things stand right now, these are the variables.  In my view, not
freezing the core APIs now means Python bindings delayed.  That is my offer;
take it or leave it... :-)

I'm sorry it has to be this way, but I have serious time limitations :-(
Any help will be most welcome...

Regards,
-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert