[Ns-developers] Waf and Testing: Request for Opinions

[craigdo@ee.washington.edu]

Dear All,

I'm working on a way to rationally combine the unit tests (waf check), the
regression tests (waf --regression) and the new validation and verification
tests.  I haven't found a reason why all of our tests should not use the
same framework.

Right now, waf check is implemented as a waf command and runs a single
C++-based program that acts as a runner that in turn runs all of the unit
tests defined in our modules sequentially.

Right now waf --regression is implemented as new task generators that spawn
off a number of tasks that are run in parallel and don't have anything to do
with the test framework.  This is the same as waf build --regression.

It would be nice to have all of our testing use the same run model (and
framework).  I mean change to one of either waf --check or waf regression.
It would also be nice to be able to run all tests using one command,
something like waf --test or waf test.

As the system increases in size it seems like a good plan to parallelize as
much as possible which would mean running waf --check and waf --regression
(and running them both with waf --test).  

Does anyone have a strong reason to keep waf check around (versus waf
--check)?  I'd like to change it.

I am also considering how to clean up the output of the tests which is
currently bordering on noise.  I'm considering a way to build an html page
as output, making something evenually somewhat like
http://tungwaiyip.info/software/sample_test_report.html which would require
a unified output model for all of our tests.

Any comments on the kind of output you'd like to see?

-- Craig