[Ns-developers] Cooperative ns-3 simulation with third party software via JSON-RPC

Mathieu Lacage mathieu.lacage at sophia.inria.fr
Wed Jan 21 07:02:57 PST 2009


hi,

Comments inline below: I hope that I answered your comments and that I
did not misrepresent your email.

On Tue, 2009-01-20 at 20:04 +0100, Alexander Pelov wrote:

> The question about BuildSimulation was worrying us a lot - we wanted  
> to make it as flexible as possible, yet Keep It Simple.
> 
> The general idea of the whole solution from the beginning was, that  
> the user would have to manually create the topology in ns-3, and the  
> external software would be able to modify the behavior of the  
> different nodes afterwards. Thus, the BuildSimulation is provided only  
> as a convenience. I do agree that it should be more flexible, and I  
> think that we may have a lot of useful discussions on this subject  
> (for example - how to manage topologies in a non-IP environment, etc).  
> Could you please sketch how can we make it as general as possible,  
> while KISSing it?

The easiest/simplest way to make the simulation topology setup and
control flexible is to export all of the ns-3 API and not try to define
a subset of the ns-3 API.

> > Now, I understand that you might be reluctant to use RPC-XML for  
> > various
> > reasons but:
> > - it is totally free because we have a complete python binding so, you
> > can trivially call RPC-XML services _today_ and remotely construct
> > arbitrary ns3 topologies
> > - it is orders of magnitude more extendable and flexible
> > - it will solve your problem (among others): your mobility simulator
> > is written in python so, you don't even have the excuse that you don't
> > have an XML-RPC layer in your target language
> >
> > I believe that you mentioned performance as a potential issue with
> > XML-RPC (other than sheer dislike of XML): do you have any hard  
> > numbers
> > which show that this would be problematic for your use-case ? I mean,
> > sure, json is theoretically faster than an xml-based serialization,  
> > but,
> > does it actually matter for your use-case ?
> 
> 
> The following part is discussing JSON-RPC vs XML-RPC - a subject I  
> consider not worth getting much into, so if you're in a hurry you can  
> skip it.

[snip]

I have to confess that I am not too interested in the details of the
on-the-wire protocol so, I did skip it :)

> Why I see the question on XML-RPC vs JSON-RPC as a non-issue - once  
> the functionality is there, it should be (mostly) trivial to change  
> the specific information encoding. XML, JSON, binary - as you wish.

ok. I agree.

> Additionally, I would like to point out, that this is just the  
> beginning of a more complete solution,  and adding Python bindings is  
> the next logical step. This way both the C++ and the Python parts of  
> the ns-3 "world" would be able to use the same functionality with ease.

ok.

> > To summarize, I see very little added benefit in merging this code  
> > and a
> > lot of downsides. I would much rather point users who wish to
> > inter-operate with ns-3 to XML-RPC than this very partial, complex,  
> > and
> > hard to maintain solution.
> >
> 
> If I have understood correctly (correct me if I'm wrong), the major  
> issue which is bothering you is - why bother writing marshalling/ 
> unmarshalling code at all? Why not "just" allow the third party to  
> execute whatever function she likes and forget about all that crap?

Yes, that is exactly my question/point, and, indeed, is the main reason
why I reacted so negatively to the code I reviewed: it brings me 20
years back in time when there was no other way to build distributed
applications. (no, I was not writing code at that time :)

> I should say that Python is a great language, so it would probably be  
> feasible to write a single XML-RPC entry point that would go through  
> all existing object in the Python namespace and execute that method on  
> the fly. That's great - it would save us the need to write code for  
> each RPC function we expose. However, I think that this is going to  
> create a very convoluted way of calling the functions.

Ok, well you forced me to investigate seriously these issues so, here
are a couple of important points:

1) I thought that the native python XML-RPC code could use the python
introspection facilities to automatically export and import functions
_and_ objects to make remote function/method calls transparent without
having to write _any_ marshalling/demarshalling code. It turns out that
I was wrong: the native python XML-RPC code does this for functions, but
not for objects. This sucks :/

2) I spent a while investigating other technologies: JSON-RPC as well as
SOAP and none of these provide the fully-transparent solution I know is
technically possible but which none of these guys seem very interested
in. This also sucks :/

3) However, there is a solution: it is called RPyC:
http://rpyc.wikidot.com/. It is probably the coolest piece of code I
have stumbled upon recently. Here is a sample remote session I just run
on my system:

in one terminal:
[mlacage at diese ns-3-dev]$ ../rpyc-3.02/rpyc/servers/classic_server.py 

in another terminal:
[mlacage at diese code]$ python
Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38) 
[GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpyc
>>> conn = rpyc.classic.connect("localhost")
>>> n0 = conn.modules.ns3.Node ()
>>> print n0.GetId ()
0
>>> n1 = conn.modules.ns3.Node ()
>>> print n1.GetId ()
1
>>> 

I did not show it but this thing actually supports callbacks so you can
trivially schedule events _in the server_ from the client and be
notified _in the client_.

> But this will not solve the problems, as Python bindings are only what  
> they are - bindings - the return value of a method invocation is a  
> proxy to the underlying C++ object. You cannot automatically marshal/ 
> unmarshal that (unless you force ALL methods to provide the necessary  
> methods). This is going to create a true nightmare for all third party  

No: a c++-implemented object is just like a python object. What is
problematic in all the technologies I looked at is that they don't
really support the handling of server-side object references. Whether
these server-side objects are implemented in python or c++ does not
matter.

[snip other related comments]

> Lets look at the way mobility is implemented in the moment. There are  
> two methods acting on node movement direction - GetDirection and  
> SetDirection, which return/receive a Vector. The  Vector does support  
> an internal serialization format, but it does not have anything to do  
> with XML. So, in the best case, the caller will obtain something she  
> has to parse (after the whole XML-RPC over HTTP processing), change,  
> then generate the fixed string format and send it over the net. And  
> remember - all this is that easy just because there is a killer-all-in- 
> one-state-of-the-art-reflection-using-python-based-single-entry-XML  
> function.
> 
> So, we are going to have to write several lines of code for each  
> function exposed via RPC (which is what everyone does) - exactly the  
> approach we have taken. I do agree that the code would be more  
> succinct in Python than in C++, but I think that with the right helper  
> functions the difference will be minimal, and the user will be able to  
> harvest the full potential of the C++ roots of ns-3.

I agree that this would not really be an improvement over what you and
sebastien did but I did not suggest doing this: I believe that a O(1)
solution is possible (as opposed to a O(n) solution where n is the
number of objects/methods to export). 

> I think that a better approach is to define interfaces - specify a set  
> of universally accepted methods, along with their parameters and  
> return values in a language-independent way. Doing so would have  
> several benefits:
> 
> 1) Implementing it in several network simulators would allow third  
> party software to communicate interchangeably with all of them. It  
> would become trivial to write an application in Java and see how it  
> would behave in ns-3 and omnet++ for example.
> 
> 2) Be implementation independent. Changing an implementation detail  
> would be hidden from the external user.
> 
> 3) Be easier to use.

I agree that the above would be cool but I see this as an orthogonal
problem which you don't necessarily have to solve to be able to remotely
setup and control an ns-3 simulation. The original email sebastien sent
stated that this was the goal so, if you define new goals, then, yes,
there might be new things to consider :)

> I'm sure this list could go on, but I think that these reasons are  
> sufficient.
> 
> 
> > I am really very sorry to be so negative after the review :/
> >
> > Mathieu
> >
> 
> 
> In conclusion I have to say, that maybe we didn't prepare the  
> announcement to the full extent, and we have taken note from your  
> remarks. We are going to prepare an additional document explaining the  
> proposed RPC functions, so that we could have a fruitful discussion on  
> concrete use-cases.

I agree that this will be useful: if you can come up with a use-case
which cannot be fulfilled with the rpyc-based solution I outlined above,
that would be a strong incentive to consider another option.

thanks for taking time to answer my initial email,
Mathieu



More information about the Ns-developers mailing list