Dirk, Jonathan,
I think that such reverse assertion logic may be the cleanest way to do
what I intended. Thanks for the idea.
Of course this can only be used with appropriate logging, like appenders
suggested by Jonathan. Otherwise there will be no way of telling which test
cases are "reversed".
Will try it out...
Andrus
At 07:06 AM 12/23/2002 +0100, Dirk Olmes wrote:
>Andrus wrote:
>>Well I was thinking more about adding a 3rd failure state:
>>1. Error - something that failed unexpectedly (no assertions in the tests)
>>2. Failure - something that failed in the assertion code
>>3. "Soft Failure" - a special assertion *for bug tracking only*.
>
>4. Expected Failure - test test case will fail but has an assertion that a
>certain error/exception will occur.
>
>To reuse Holger's example:
>
>void testSelectObjectForIdNullArg()
>{
> boolean didCatchNPE = false;
>
> try
> {
> SelectQuery q = QueryHelper.selectQueryForObjectId(null);
> Assert.fail(); // bail out here if no error occurred
> }
> catch (NullPointerException npe)
> {
> didCatchNPE = true;
> }
> Assert.assertTrue(didCatchNPE);
>}
>
>This way, if all the tests are green, we know that we still have all of
>the nonresolved bugs in the codebase. If the testcase turns red, we know
>that an error "went away" and we have to dig deeper into this issue.
>
>It's a little bit reverting the logic of JUnit but then again we don't
>have to jump around in loops. I'd put all the tests for known bugs in
>their own testsuite so we can clearly separate the good testcases from the
>bad ones.
>
>>(2) and (3) are similar, but (2) will generate a JVM exit status of "1"
>>so that the nightly build Perl script would fail, while (3) will generate an
>exit status of "0".
>
>We had a similar nuisance with ant exiting via System.exit(1) in one of
>our Daedalos projects. The fix for that situation was to carry around a
>patched version of the specific class in our project. But I wouldn't ask
>for that kind of trouble ....
>
>-dirk
>
This archive was generated by hypermail 2.0.0 : Mon Dec 23 2002 - 14:09:39 EST