[Ns-developers] NS-3 core API freeze?

[Mathieu Lacage]

hi gustavo,

On Mon, 2008-04-21 at 18:22 +0100, Gustavo Carneiro wrote:
> Latest ns-3-dev code merge broke the python bindings again, in a non-trivial
> way that requires me to spend some hours figuring out how to  adapt.  This
> highlights the issue with python bindings and API changes.  If we want
> usable python bindings during May we need stable NS-3 core API right about
> now (say, until friday).  That means no touching of src/core, src/common, or
> src/simulator unless it's some minor tweak to help the python bindings.
> 
> Is the project ready to commit to this?  Will it officially commit to this
> freeze?  If it's not possible then we need to review the roadmap for Python
> bindings inclusion.

I think that we should be ready to freeze the core API now, but, you
raise rightfully the question of what our roadmap is for the release.
So, I have tried to outline what still needs to be done below to reach
"gold" status for the release:

1) documentation: need work to convert tutorial to new helper API.

2) socket implementation bugs:
  2.1 udp needs a finite rx packet buffer
  2.2 tcp needs a finite tx and rx byte buffer
  2.3 tcp needs to implement its tx and rx buffers as lists of packet
pointers, and not raw buffers to handle correctly application-level
tagged packets
  2.4 packet tagging implementation and API needs to be rewritten to
track tags on a byte-basis to allow application-level tagged packets to
work with tcp sockets
  2.5 tcp sockets need to implement the "send" callback as was discussed
many times, that is, it needs to invoke that callback when there is new
room in the tx buffer, not when a packet is sent down the stack.

3) socket API additions: need to add the API I suggested in the socket
thread to allow future implementations of posix C-style sockets

4) socket API changes: tom suggested to move to a SocketHelper class the
raw buffer management. 

5) python bindings: there are a number of key issues:
  5.1 need to verify that gustavo's proposed implementation addresses
our key use-cases: a) allow users to write new C++ models and bind them
to python without having to install gccxml CVS and pygccxml, b) allow
users to modify and hack a released version of the ns3 c++ API and have
the python binding still work without having to install gccxml CVS and
pygccxml. 
  5.2 document the overall structure of the python bindings: we need to
know how they work at least from a high-level perspective. basically, a
1-page a4 document about how the code is generated and how the generated
code works. Assuming that the reader knows the details of the python
low-level binding API works is ok.

I raised 5.1 a couple of times privately with you, gustavo and you seem
to have other higher-priority issues which is perfectly fine but which
might become a serious blocker now. I consider 5.1 to be the top
priority and to be a showstopper for merging: if 5.1 is not resolved, we
will have to look at other python binding technology alternatives: we
_cannot_ request users to install gccxml CVS to build our python
bindings. 5.2 is also a very high priority: it is not ok to merge the
whole bindings without any kind of documentation about the way they
work. I would be much more comfortable with unfinished documented
bindings which solve our important use-cases than fully finished
black-box bindings which might not solve our important use-cases.

Now, to go back to the bigger question of release schedule: I think that
the biggest issues we need to tackle are 5) and 2). I believe that craig
agreed to start looking at 5) this week and to try to assess the status
of your branch with regard to our use-cases. Depending on that initial
assessment, he might be able to help you work on the branch or go in
another direction. I will work on 2.4 this week before going away until
the 8th and will work on general bug-fixing after, together with
documentation (1). Tom will probably work on 1), 3), and, 4), and, will
try to oversee work on the other items in 2). Raj will probably want to
work on items in 2) except for 2.4 for which I promised a working
solution :)

If we are still aiming for a mid-june release, we need to complete all
items above by the end of may to have at least 2 weeks of spare time for
review and serious bug-fixing. So, if you ignore the relatively minor
API changes outlined above, we can freeze our APIs now and focus on the
issues described in this email until may the 15th at which point we can
re-evaluate our status.

to summarize, I would suggest the following schedule:
 - 22-04-2008 -> 15-05-2008: 3), 4), 2.5), 2.3), 2.4), 5). Ideally, we
would be ready to merge the python binding on the 15-05-2008

 - 15-05-2008 -> 31-05-2008: 1), 2.2), 2.1), polishing of python
bindings. 

 - 31-05-2008: release release-candidate-1

 - 01-06-2008 -> 15-06-2008: bug-fix only stage, documentation review,
polishing

 - 15-06-2008: release final version.

I am fairly certain that the above proposal is a bit too optimistic,
especially, the first 3 weeks but I would hope that these first 3 weeks
can slip by one week without really impacting the final schedule. At
worst, we can still slip by 2 weeks the final release date to 30-06-2008
but we cannot slip more.

regards,
Mathieu