[Ns-developers] Cooperative ns-3 simulation with third party software via JSON-RPC
mathieu.lacage at sophia.inria.fr
Wed Jan 21 07:02:57 PST 2009
Comments inline below: I hope that I answered your comments and that I
did not misrepresent your email.
On Tue, 2009-01-20 at 20:04 +0100, Alexander Pelov wrote:
> The question about BuildSimulation was worrying us a lot - we wanted
> to make it as flexible as possible, yet Keep It Simple.
> The general idea of the whole solution from the beginning was, that
> the user would have to manually create the topology in ns-3, and the
> external software would be able to modify the behavior of the
> different nodes afterwards. Thus, the BuildSimulation is provided only
> as a convenience. I do agree that it should be more flexible, and I
> think that we may have a lot of useful discussions on this subject
> (for example - how to manage topologies in a non-IP environment, etc).
> Could you please sketch how can we make it as general as possible,
> while KISSing it?
The easiest/simplest way to make the simulation topology setup and
control flexible is to export all of the ns-3 API and not try to define
a subset of the ns-3 API.
> > Now, I understand that you might be reluctant to use RPC-XML for
> > various
> > reasons but:
> > - it is totally free because we have a complete python binding so, you
> > can trivially call RPC-XML services _today_ and remotely construct
> > arbitrary ns3 topologies
> > - it is orders of magnitude more extendable and flexible
> > - it will solve your problem (among others): your mobility simulator
> > is written in python so, you don't even have the excuse that you don't
> > have an XML-RPC layer in your target language
> > I believe that you mentioned performance as a potential issue with
> > XML-RPC (other than sheer dislike of XML): do you have any hard
> > numbers
> > which show that this would be problematic for your use-case ? I mean,
> > sure, json is theoretically faster than an xml-based serialization,
> > but,
> > does it actually matter for your use-case ?
> The following part is discussing JSON-RPC vs XML-RPC - a subject I
> consider not worth getting much into, so if you're in a hurry you can
> skip it.
I have to confess that I am not too interested in the details of the
on-the-wire protocol so, I did skip it :)
> Why I see the question on XML-RPC vs JSON-RPC as a non-issue - once
> the functionality is there, it should be (mostly) trivial to change
> the specific information encoding. XML, JSON, binary - as you wish.
ok. I agree.
> Additionally, I would like to point out, that this is just the
> beginning of a more complete solution, and adding Python bindings is
> the next logical step. This way both the C++ and the Python parts of
> the ns-3 "world" would be able to use the same functionality with ease.
> > To summarize, I see very little added benefit in merging this code
> > and a
> > lot of downsides. I would much rather point users who wish to
> > inter-operate with ns-3 to XML-RPC than this very partial, complex,
> > and
> > hard to maintain solution.
> If I have understood correctly (correct me if I'm wrong), the major
> issue which is bothering you is - why bother writing marshalling/
> unmarshalling code at all? Why not "just" allow the third party to
> execute whatever function she likes and forget about all that crap?
Yes, that is exactly my question/point, and, indeed, is the main reason
why I reacted so negatively to the code I reviewed: it brings me 20
years back in time when there was no other way to build distributed
applications. (no, I was not writing code at that time :)
> I should say that Python is a great language, so it would probably be
> feasible to write a single XML-RPC entry point that would go through
> all existing object in the Python namespace and execute that method on
> the fly. That's great - it would save us the need to write code for
> each RPC function we expose. However, I think that this is going to
> create a very convoluted way of calling the functions.
Ok, well you forced me to investigate seriously these issues so, here
are a couple of important points:
1) I thought that the native python XML-RPC code could use the python
introspection facilities to automatically export and import functions
_and_ objects to make remote function/method calls transparent without
having to write _any_ marshalling/demarshalling code. It turns out that
I was wrong: the native python XML-RPC code does this for functions, but
not for objects. This sucks :/
2) I spent a while investigating other technologies: JSON-RPC as well as
SOAP and none of these provide the fully-transparent solution I know is
technically possible but which none of these guys seem very interested
in. This also sucks :/
3) However, there is a solution: it is called RPyC:
http://rpyc.wikidot.com/. It is probably the coolest piece of code I
have stumbled upon recently. Here is a sample remote session I just run
on my system:
in one terminal:
[mlacage at diese ns-3-dev]$ ../rpyc-3.02/rpyc/servers/classic_server.py
in another terminal:
[mlacage at diese code]$ python
Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
[GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpyc
>>> conn = rpyc.classic.connect("localhost")
>>> n0 = conn.modules.ns3.Node ()
>>> print n0.GetId ()
>>> n1 = conn.modules.ns3.Node ()
>>> print n1.GetId ()
I did not show it but this thing actually supports callbacks so you can
trivially schedule events _in the server_ from the client and be
notified _in the client_.
> But this will not solve the problems, as Python bindings are only what
> they are - bindings - the return value of a method invocation is a
> proxy to the underlying C++ object. You cannot automatically marshal/
> unmarshal that (unless you force ALL methods to provide the necessary
> methods). This is going to create a true nightmare for all third party
No: a c++-implemented object is just like a python object. What is
problematic in all the technologies I looked at is that they don't
really support the handling of server-side object references. Whether
these server-side objects are implemented in python or c++ does not
[snip other related comments]
> Lets look at the way mobility is implemented in the moment. There are
> two methods acting on node movement direction - GetDirection and
> SetDirection, which return/receive a Vector. The Vector does support
> an internal serialization format, but it does not have anything to do
> with XML. So, in the best case, the caller will obtain something she
> has to parse (after the whole XML-RPC over HTTP processing), change,
> then generate the fixed string format and send it over the net. And
> remember - all this is that easy just because there is a killer-all-in-
> So, we are going to have to write several lines of code for each
> function exposed via RPC (which is what everyone does) - exactly the
> approach we have taken. I do agree that the code would be more
> succinct in Python than in C++, but I think that with the right helper
> functions the difference will be minimal, and the user will be able to
> harvest the full potential of the C++ roots of ns-3.
I agree that this would not really be an improvement over what you and
sebastien did but I did not suggest doing this: I believe that a O(1)
solution is possible (as opposed to a O(n) solution where n is the
number of objects/methods to export).
> I think that a better approach is to define interfaces - specify a set
> of universally accepted methods, along with their parameters and
> return values in a language-independent way. Doing so would have
> several benefits:
> 1) Implementing it in several network simulators would allow third
> party software to communicate interchangeably with all of them. It
> would become trivial to write an application in Java and see how it
> would behave in ns-3 and omnet++ for example.
> 2) Be implementation independent. Changing an implementation detail
> would be hidden from the external user.
> 3) Be easier to use.
I agree that the above would be cool but I see this as an orthogonal
problem which you don't necessarily have to solve to be able to remotely
setup and control an ns-3 simulation. The original email sebastien sent
stated that this was the goal so, if you define new goals, then, yes,
there might be new things to consider :)
> I'm sure this list could go on, but I think that these reasons are
> > I am really very sorry to be so negative after the review :/
> > Mathieu
> In conclusion I have to say, that maybe we didn't prepare the
> announcement to the full extent, and we have taken note from your
> remarks. We are going to prepare an additional document explaining the
> proposed RPC functions, so that we could have a fruitful discussion on
> concrete use-cases.
I agree that this will be useful: if you can come up with a use-case
which cannot be fulfilled with the rpyc-based solution I outlined above,
that would be a strong incentive to consider another option.
thanks for taking time to answer my initial email,
More information about the Ns-developers