On Thu, Nov 03, 2011 at 03:04:17PM +0400, Dmitry Monakhov wrote:
> On Thu, 3 Nov 2011 06:54:16 -0400, Theodore Tso <tytso@xxxxxxx> wrote:
> > On Nov 2, 2011, at 3:55 PM, Christoph Hellwig wrote:
> > > Alex, Eric, Dave - should we add new tests with the new operations
> > > Dmitry added, or is adding new ops to the existing tests fine?
> > One argument for adding new ops to existing tests is that it makes
> > the run time of the entire test suite take longer. A QA pass is
> > already taking quite a while, and it would be nice if we could
> > keep xfstests as efficient as possible in terms of the maximum
> > testing coverage per time spent running the test suite….
> Yes, but regression test with explicit seed option should be
> preserved. Number of such test is not too big, so it is reasonable to
> hardcode set of operations in such tests and let all others use new features.
That's not what I was talking about. Of course there should be a way
to run a regression test with an explicit seed option (although in
general I think a specific test in xfstests should by default use a
random seed, and have a way to easily specify an explicit seed without
having to reverse engineer the test and running fsstress manually).
What I was talking about was the fact we already have several (half a
dozen or so, if memory serves correctly) xfstests that use fsstress
with a different set of fsstress options. In some cases it makes to
add a new numbered xfstest subtest, but I'd rather not find that we've
doubled the number of tests using fsstress in the future, and with it,
doubled the run-time of the auto or quick xfstests group....