[Ns-developers] First Stab at Validation and Verification

[craigdo@ee.washington.edu]

[ ... ]

> Sentences such as "Prescribed conditions for which the model has been
> tested, compared against reality to the extent possible, and judged
> suitable for use;" or "Substantiation that a model, within 
> its domain of
> applicability, possesses a satisfactory range of accuracy consistent
> with the intended application of the model;" make my brain hurt in
> french, and explode in english. I had to read/process them multiple
> times.

Heh, heh.  It does sound like a lawyer got hold of it, doesn't it?
Unfortunately, these are pretty standard definitions that I've found across
the literature.  You may be surprised if you copy the string,

  prescribed conditions for which the model has been tested compared against
reality to the extent possible

into google and see what comes back -- 152,000 results :-)

It all probably traces back to one paper written in 1835.  I'll attempt to
translate that kind of stuff into something more easily parsed by
non-lawyers.

> I understand that what you are trying to do here is provide
> documentation about regression tests to allow third parties 
> to evaluate
> the quality of these tests: the kind of website you showed is 
> awesome in
> that respect.
> 
> So, what I am really asking is whether it's a good idea to invest
> resources in this rather than just write tests that you, as a test
> writer, feel good about. i.e., I am worried that generating graphs
> brings nothing to the writing of good tests which is what I 
> feel really
> concerned about.

Okay, you have a meta-issue.  Our Fearless Leader is very concerned about
how to present to the rest of the world that all of that work writing good
tests has been done.  He wants to see the graphs and web sites.