[Ns-developers] First Stab at Validation and Verification

[craigdo@ee.washington.edu]

> > One terminology thing that's always bugged me is that, while ns has
> > had
> > validation tests seemingly forever, talk of regression testing was
> > never
> > around for ns2. And validating the models vs testing for regressions
> > in
> > behaviour strike me as slightly different in aim.
> > 
> > validation - is this model behaviour useful and realistic?
> > regression - is this model behaviour consistently unchanged by code
> > changes not intended to change the model behaviour?
> > 
> > The scope of the tests for these seems different. But they've
> > both been lumped under 'validation' to date.
> 
> There are about 20 different adjectives you can use to qualify tests:
> unit tests, system tests, integration tests, regression tests,
> validation tests, whitebox tests, blackbox tests, etc. You 
> could try to
> tag all the blackbox tests and the regression tests with a special
> directory, but, then, the problem is that you could also arbitrarily
> decide to tag only the blackbox tests in a special directory, etc. My
> point is that I don't see any useful obvious single-level 
> classification
> system to sort tests so, it seems to me that it's just simpler to dump
> them all in the same location (say, randomly, 'tests' :) and, document
> in each test what you are testing and how.

Or you could have 20 different directories representing a detailed taxonomic
decomposition agreed to by everyone ;-)  Seriously, though, I see in another
email that you found one of the places where I said that I don't want to
arrange it so there are multiple directories with tests for a given module
spread across them.

Right now there the statistical tests and the non-statistical tests are in
two different places since I don't have a handle on how similar the
environments will end up being -- I want to address one at a time for
simplicity.  I expect that it will all condense down into something like
"tests."