[Ns-developers] Draft project proposal of "ns-3 in the cloud" -- GSoC2012

Ruhao Yao yaoruhao at gmail.com
Sat Mar 24 19:32:47 PDT 2012


Professor Henderson, Lalith,

I'm Ruhao Yao from Peking University. I have finished my draft project
proposal of "ns-3 in the cloud". Would you please give me some advice?

Regards.

The following is my draft proposal. If you have any confusion, please
let me know. Thank you so much!

Regards.


About Me

*Identity Information*

¡ì/Name/. Ruhao Yao

¡ì/Email/. yaoruhao at gmail.com <mailto:yaoruhao at gmail.com>

¡ì/Country/. China

¡ì/Institution/. Beijing Key Laboratory of Internet Security Technology,
School of Electronics Engineering & Computer Science, Peking University

¡ì/Advisor/. Jianyu Zhang

*Background*

I am a first year Master Student in Computer Science. My main research
is about P2P network simulator. I am experienced in using several
network simulators (for example OMNeT++ and OverSim), and have written
technical reports about these two simulators.

Furthermore, my Bachelor Thesis is ¡°/Cloud-based simulator of P2P
network/¡±. I designed and developed a distributed parallel P2P network
simulator, which can run on five virtual machines of UEC (Ubuntu
Enterprise Cloud), scale of 500,000 P2P nodes.

*Education*

I¡¯m now a first year Master student in Computer Science (2011.9 ¨C
2014.6). My main study area is about P2P network simulator.

My undergraduate study is also in Computer Science, Peking University
(2007.9 ¨C 2011.6).

I have 3 years coding experience with C++. My Bachelor Thesis is
designed in C++, with a 100,000 code size (including modified aMule
client as the overlay node).

I learnt Python 3 years ago, and it has become my daily scripting
language. I also used Django to develop an online CD store.

*Work*

Hulu R&D Center Beijing (http://www.hulu.com) -- Software QA Intern,
2012.1 ¨C 2012.4

lAutomation tests

lWiki and documents maintenance

lInternal tools, script programming (python)

Youdao Search Engine (http://www.youdao.com), Netease ¨C Software Dev
Intern, 2011.3 ¨C 2012.1

lYoudao Game Search: http://game.youdao.com(Chinese)

lYoudao Game Strategy Search:
http://www.youdao.com/gs/search?q=a&gn=mhxy
<http://www.youdao.com/gs/search?q=a&gn=mhxy> (Chinese)

lOperation Smart Result

Doodle Mobile (http://doodlemobile.com/), Innovation Works ¨C Software
Dev Intern, 2010.10 ¨C 2011.1

lAndroid Game Development

lInternal tools, script programming (python)

lGame Center (Amazon EC2, S3)

My experience in open source projects:

lSimilar to open source projects, my undergraduate Linux coursework
project. The project is about a campus info notification tool. We used
python to develop a mobile phone SMS Subscription Service. We used svn
and trac to manage our project.

**

**

*Research*

P2P network parallel simulation is my main study for my master period.
My motivation is to simulate a million scale p2p nodes network to deploy
and test new p2p techniques for evaluation and observation. As we know,
the current p2p network like BT/eMule is great larger than expect. Some
of the new techniques cannot test directly on real overlay network. So a
large scale P2P network simulator is really in need.

I first focus our research on parallel simulation. I think this is a
good way to simulate large network by several usual hosts. It can help
us get rid of the limitation of a single host¡¯s CPU and memory. From
that point, I studied some main stream P2P network simulator framework.
Among them, OverSim is the best one I think.

¡°/OverSim is an open-source overlay and peer-to-peer network simulation
framework for theOMNeT++ simulation environment. The simulator contains
several models for structured (e.g. Chord, Kademlia, Pastry) and
unstructured (e.g. GIA) P2P systems and overlay protocols./¡±

Since OMNeT++ is an extensible, modular, component-based C++ simulation
library and framework, primarily for building network simulators.
OverSim extends its powerful modular framework to simulate underlay
network by 3 architectures (simpleunderlay, InetUnderlay and
singlehostunderlay). For large scale simulation, simpleunderlay fits
best. Due to the limitation of global reference of routing table,
OverSim using simpleunderlay cannot parallel simulate. OverSim can run
100,000 chord nodes simulation, it cannot reach our expectation.

I borrow some good ideas from OverSim, and then define a new deploying
model of simulation framework. I use two types of server, CCS (Center
Control Server) and NSS (Node Simulating Server). The system needs one
CCS and several NSS. They use socket to transfer message, so I can
deploy the system on a cluster.

CCS is used to allocating node id range and node ip address (fake, not
the real network) to each NSS. So NSS knows the number of nodes (each
id, ip) simulating on itself. Also it knows other nodes info on other NSS.

The message transfer process:

1.A node creates a packate, including srcip, srcid, desip, desid, and
then send to its own NSS.

2.NSS check its routing table, get the destination NSS real ip address
of desnode.

3.NSS assemble this message and transfer it to the destination NSS.

4.The destination NSS get the message, pass it to desnode.

I use above ways to implement underlay network simulation. I also
modified aMule code to simulate overlay network. I can now simulate
500,000 nodes with 5 virtual machines (Ubuntu), each is deployed one
NSS. But I have some bugs now, with the increase of node scale, NSS
cannot handle events well. The system may abort after 10-15 minutes due
to memory leaks. I am now fixing this issue.

This job is my Bachelor¡¯s thesis research. My mentor helps me with
overlay works. I used his aMule modified code. I am still working on
this as my Master period research. He gives me a lot of guide to my
research. I really appreciate him so much.

*Personal Interests*

I love travel, sports, movies and photography. I play a point guard in
my class basketball team. I like making friends with people.

About The Project

*Project Proposal*

I¡¯m interested in ¡°ns-3 in the cloud¡± project. My proposal about this
project consists of 3 main parts.

*Documentation*

lMotivation: The existing documentation lacks the support of HOW-TO
parallelize simulation on a cloud or a cluster. This is much more
important for the new users who want to distribute a large scale of
simulation.

lComposition:

nHOW-TO deploy ns-3 distributed parallel environment on AWS, UEC (Ubuntu
Enterprise Cloud) and other clusters.

nHOW-TO run a parallel simulation example in AWS, UEC and other clusters.

nHOW-TO write an MPI-based parallel simulation example on AWS, UEC and
other clusters. (Improve the current document
http://www.nsnam.org/docs/release/3.13/models/singlehtml/index.html#document-distributed)

nBenchmarking documentation of ns-3 parallel simulation on the cloud.

lFeatures:

nSupport for AWS and private cloud.

nHelp users know how to use ns-3 in cloud.

nBenchmarks of ns-3 parallel simulation in the cloud.

*New Parallel Example Code*

lMotivation: Existing 3 MPI examples are too small for large scale ns-3
simulation. I¡¯m willing to offer a configurable example which can fully
utilize the advantages of ns-3 parallel simulation. Also it can explore
the benchmarks of ns-3 parallel simulation in the cloud.

lComposition:

nOne configurable example can use parameters to set node number per-host
and the number of machines to run parallel simulation.

lFeatures:

nFlexible. Without hard-coding the simulation parameters, we can adjust
the simulation scale by changing parameters passed to the program
instead of re-compiling the example source code.

nBenchmarks. As we don¡¯t have the exact benchmarks of how ns-3 parallel
performance in the cloud, this example will fill this gap. With these
benchmarks, we can focus on improving ns-3 and solving the efficiency
limitations in the future.

*Tests*

lMotivation: The HOW-TO documents and new parallel example should be
tested before commitment.

lComposition:

nHOW-TO documents should be carefully examined.

nA new Test Suite for the new parallel example. Including Build
Verification Test, Unit Test, System Test, and Performance Test.

nThe benchmarks should be examined on various platforms.

lFeatures:

nFully tested: Build Verification Test, Unit Test, System Test, and
Performance Test.

nConvincing benchmarks.

*Interest*

I love this project because it is relative to my research area. Also I
want to make a contribution to ns-3 community.

I think the following reasons make me the best candidate:

lRelative background: My main research is about P2P network simulator. I
have learnt and used some simulators (OMNeT++ and OverSim) in my
research. I also design and develop a simple ¡°/Cloud-based simulator of
P2P network/¡± for my Bachelor Thesis. I think my relative background
will help me contribute an effort to ns-3 community.

lRich internship experience: I¡¯m now a Software QA intern at Hulu
(http://www.hulu.com). I know the importance of documentation and
testing. I can offer high quality documentations and source codes.

lFuture research plans with ns-3: My whole Master period research is
about network simulator. I want to write a P2P network simulation with
ns-3. Also, I¡¯m preparing a paper for this.

lSupports from my lab: I¡¯m from Beijing Key Laboratory of Internet
Security Technology, Peking University. My lab advisor will offer me
some workstations to build UEC for ns-3 parallel study and exploring the
benchmarks.

lEnough time: My whole summer is at school, with plenty of time for this
project. My lab advisor will support me to do this.

**

**

*Future Plans*

I¡¯m interested in ns-3 parallel simulating, and my future plans are
related to it.

My future plans are:

lPreparing a paper about cloud capacity benchmarks measured by ns-3
parallel simulations.

nMotivation: Cloud computing is becoming more and more popular, but we
can only get some advertising benchmarks about capacity. Ns-3 parallel
simulation can play an important role in measuring the capacity benchmarks.

lAdding P2P network simulation to ns-3.

nMotivation: My research is mainly about p2p network simulation,
especially parallel simulation for large scale. I¡¯m interested in adding
p2p network simulation to ns-3.

lBe a contributor and maintainer in ns-3 community.

nMotivation: I love open source project, I want to make my contributions
to ns-3 community.

*Approach*

According to my project design, the approach will be discussed in 3
parts (Documentation, the new parallel example and tests). I will use
the Waterfall Model as my development methodology. The process will be:

1.Requirements. (Set up the project goal.)

2.Design. (Write the basic HOW-TO documents, design the new parallel
simulation example.)

3.Implementation. (Implement the new parallel simulation example.)

4.Verification. (Testing, explore the benchmarks.)

5.Maintenance. (Commit documentations and source code, help to merge
code to trunk)

*Documentation:*

lHOW-TO deploy ns-3 distributed parallel environment on AWS, UEC(Ubuntu
Enterprise Cloud) and other clusters.

nIntroduce how to setup MPI, rsh and write a hostfile for parallel
simulation environment. Use Ubuntu Server as an example.

lHOW-TO run a parallel simulation example in AWS, UEC and other clusters.

nIntroduce how to run the new parallel simulation example. (Explain the
configurable node topology, number of simulating machines.)

lHOW-TO write an MPI-based parallel simulation example on AWS, UEC and
other clusters. (Improve the current document
http://www.nsnam.org/docs/release/3.13/models/singlehtml/index.html#document-distributed)

lBenchmarking documentation of ns-3 parallel simulation in the cloud.

nEvaluation for the scale of nodes simulated on one machine.

nEvaluation for the packet transfer rate. (In one machine and across
machines.)

nEvaluation for the new parallel example performance on AWS and UEC.

*The new parallel example:*

Figure 1. The new parallel example topology

Figure 1 shows the topology of the new parallel example.

This can run on one or several machines. The simulation nodes and host
machines can be configured by parameters. For example, if node number
per-host is 2 and the number of host machine is 3, then node0 ¨C node1 is
on host1, node2 ¨C node3 is on host2 and node4 ¨C node5 is on host3. Nodes
are connected like a cycle.

The packet transfer schema is:

1.Node0 generates a packet, and then passes it to node1 (within host
message transfer).

2.When node1 receives the packet, log some information and pass it to
node2. (across host message transfer)

3.Packet transfers from one node to the other within the cycle.

4.The simulation process will terminate when node0 receive the packet
from node5.

This example can be used to explore the benchmarks of ns-3 parallel
simulation and the cloud capacity. The following experiments will be
conducted to achieve the goal.

lThe max number of nodes which can be simulated on one host. (Especially
the Amazon Instance)

lThe packet transfer rate within host and across hosts.

lAdvice for user to decide the scale of nodes per-host and simulation
host number. (For example, if the user wants to run a 100,000 nodes
parallel simulation, we can advise him to divide the 100,000 nodes to 5
hosts or to 10 hosts.)

*Testing:*

I will implement several test suites for the new parallel example. These
test suites will live in src/test/parallelexample directory. The new
test suites consist of the following parts:

lBuild Verification Test

nCheck whether MPI is enabled and the example is successfully built.

lUnit Test

nFor the new parallel example. Check whether every unit of the example
work correctly.

lSystem Test

nRun the example on one host and several hosts; check the packet
transfer process correctness.

lPerformance Test

nPerformance test is used for the benchmarks exploring experiments.

*Deliverables*

The project is planned to submit documentations, new parallel example
source code and corresponding tests. It can be achieved through the
following 4 milestones:

1.HOW-TO documentation milestone. (Approximately 2 weeks.)

2.New parallel example milestone. (Approximately 3.5 weeks.)

3.Testing, performance benchmark experiments milestone. (Approximately
3.5 weeks.)

4.Improving the documentation, source code and tests for final
submission milestone. (Approximately 1 week)

*Plan*

What is your task schedule to implement your technical plan and develop
your deliverables?
Try to break down the deliverables into small tasks and assign to each
of them the time you think it is going to take you.
Make sure you consider the time you will need to test and fix your code
after every deliverable is finished.
Order all the tasks chronologically and assign a start and an end date
to each of them. Verify that the resulting schedule fits into the 10
weeks you will have.
It is important to leave some time at the end of the schedule for
integrating, testing and fixing the complete project.

The project plan is divided into 4 milestones. Following is my planning
timeline:

1.HOW-TO documentation milestone. (From May 21 to June 4.)

a)HOW-TO deploy ns-3 distributed parallel environment on AWS, UEC. (1 week)

b)HOW-TO run a parallel simulation example in AWS, UEC and other
clusters. (0.5 week)

c)HOW-TO write an MPI-based parallel simulation example on AWS, UEC and
other clusters. (0.5 week)

2.New parallel example milestone. Mid-term evaluation submission (From
June 5 to July 5.)

a)Design and implementation. (2 week)

b)Unit test. (1 week)

c)Prepare for midterm evaluation. (1 week)

3.Testing, performance benchmark experiments milestone. (From July 10 to
July 31)

a)Testing. (1.5 weeks)

b)Experiments. (1.5 weeks)

4.Improving the documentation, source code and tests for final
submission milestone. (From August 1 to August 20)

a)Improving documentations, source code and tests. (1.5 weeks)

b)Preparing for the final submission. (1.5 weeks)

*Timezone*

My expected work hours (in UTC) is: 1:00 ¨C 4:00 AM and 6:00 ¨C 9:00 AM

The work hours can adjust if needed.

I will stay in Beijing (UTC+8) this whole summer.

*Commitments*

My whole summer will be available for GSoC schedule.


-- 
Ruhao Yao
Beijing Key Laboratory of Internet Security Technology
School of Electronics Engineering & Computer Science
Peking University
Beijing 100871
China




More information about the Ns-developers mailing list