[Ns-bugs] [Bug 480] New: Run regression tests as WAF build tasks, for parallel execution

[code@nsnam.ece.gatech.edu]

http://www.nsnam.org/bugzilla/show_bug.cgi?id=480

           Summary: Run regression tests as WAF build tasks, for parallel
                    execution
           Product: ns-3
           Version: ns-3-dev
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: build system
        AssignedTo: ns-bugs at isi.edu
        ReportedBy: gjcarneiro at gmail.com
   Estimated Hours: 0.0


Created an attachment (id=357)
 --> (http://www.nsnam.org/bugzilla/attachment.cgi?id=357)
regression system refactoring

I have a patch that refactors the whole regression testing framework to make it
run as WAF tasks.  The main advantage is that those tasks will run in parallel
on multicore systems.

In the process I made some simplifications.  Before, this happened:

WAF:
  insert fake module in sys.modules
  mod = import test script
  mod.run(bunch of parameters)
mod.run:
  import fake module
  call fake module with bunch of parameters

So we have a lot of indirection, waf calls script, script calls back some code
that runs in waf space.  Extensibility is painful.

I got rid of this system.  Instead, now the modules are only used for
consulting.

WAF:
  import test script 
  get test dir name, or infer from the test script name if not defined
  call may_run(env) to check if the test can run, or assume it can if not
defined
  get program name, guess program name if not defined
  get program arguments, assume no arguments if not defined

Bottom line is that now most test scripts are empty, as everything is guessed. 
Some scripts override the defaults, for instance test-csma-bridge.py:

---------------
  import os.path

  def may_run(env):
      """Returns 0 when it can run, return non-zero or string (reason) when it
cannot run"""
      if env['ENABLE_PYTHON_BINDINGS']:
          return 0
      else:
          return "Python bindings not available."

  pyscript = os.path.join('examples', 'csma-bridge.py')
------------------

The only problem I am having with this patch is that the test ns-2-mob is
always failing for me.  It works if I regenerate traces.  I am not sure whether
it is the fault of the refactoring or not.

---
/home/gjc/projects/ns/ns-3-allinone/ns-3-dev-ref-traces/main-ns2-mob.ref/out.tr
    2009-01-22 15:46:45.000000000 +0000
+++
/home/gjc/projects/ns/ns-3-allinone/ns-3-dev/build/debug/regression/traces/main-ns2-mob.ref/out.tr
 2009-01-22 15:56:09.000000000 +0000
@@ -1,3 +0,0 @@
-3000000000ns POS: x=0, y=25, z=0; VEL:25, y=0, z=0
-4800000000ns POS: x=45, y=25, z=0; VEL:0, y=0, z=0
-5000000000ns POS: x=45, y=25, z=0; VEL:25, y=0, z=0


-- 
Configure bugmail: http://www.nsnam.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.