Bug 95391 – LayoutTest: editing/delete/br-004(5)(6) failing on Windows ORWT

• Status: NEW
• Component: Tools / Tests

[Roger Fong]

Looks like NRWT expects different results for these tests than ORWT, as per http://trac.webkit.org/changeset/90490.
Adding these three failing tests to skip list to get bots greener.

[Roger Fong]

added to skip list here:
http://trac.webkit.org/changeset/127060

[Ryosuke Niwa]

(In reply to comment #0)
> Looks like NRWT expects different results for these tests than ORWT, as per http://trac.webkit.org/changeset/90490.
> Adding these three failing tests to skip list to get bots greener.

That doesn't make much sense. NRWT an OWRT should get the exactly same result as they both use DumpRenderTree and WebKitTestRunner.

Could you show us the difference? If there is any, we need to fix NRWT or ORWT to make sure they get the same result.

[Roger Fong]

(In reply to comment #2)
> (In reply to comment #0)
> > Looks like NRWT expects different results for these tests than ORWT, as per http://trac.webkit.org/changeset/90490.
> > Adding these three failing tests to skip list to get bots greener.
> 
> That doesn't make much sense. NRWT an OWRT should get the exactly same result as they both use DumpRenderTree and WebKitTestRunner.
> 
> Could you show us the difference? If there is any, we need to fix NRWT or ORWT to make sure they get the same result.

When I said NRWT I think I meant to say webkit-test-runner, and when I said ORWT I think i meant to say DRT...

Sorry, the difference between what Webkit2 uses for testing and Webkit is still a bit hazy for me because I never touch NRWT.

I was just trying to say that it looks like in the patch for the trac I mentioned, the test results were rebaselined to match something that webkittestrunner does, but drt does not, which causes drt to say the tests have failed.

[Ryosuke Niwa]

(In reply to comment #3)
> When I said NRWT I think I meant to say webkit-test-runner, and when I said ORWT I think i meant to say DRT...
> 
> Sorry, the difference between what Webkit2 uses for testing and Webkit is still a bit hazy for me because I never touch NRWT.
> 
> I was just trying to say that it looks like in the patch for the trac I mentioned, the test results were rebaselined to match something that webkittestrunner does, but drt does not, which causes drt to say the tests have failed.

That's strange. Windows port's DRT should know to ignore editing delegates. Do you get test failures when you run DRT from ORWT? If so, what kind of diff. are you getting?

[Roger Fong]

> That's strange. Windows port's DRT should know to ignore editing delegates. Do you get test failures when you run DRT from ORWT? If so, what kind of diff. are you getting?

Yeah.

So I'm confused. 
The delegates are indeed being ignored but my understanding was that the tests were rebaselined to match what webkit test runner was outputting, which means it no longer matches what drt is outputting.

Although in reponse to what you asid about DRT ignoring editing delegates,
I see some editing tests that are actually passing on but the expected test results include editing delegates, which makes no sense to me either.

None of this really makes sense to me so really what I should've done was added all the delete-br tests to the skip list to get the bots greener even though some of them are strangely passing.