Bug 23772 – Should LayoutTests/http/tests/history/redirect-js-form-submit-before-load-expected.txt be passing or failing?

• Status: RESOLVED
• Resolution: INVALID
• Component: WebKit Misc.

[Scott Violet]

This was a recently added test by ggaren, yet the output says it's failing:

	FAIL: History item count should be 1 but instead is 2.

Which is right, the test is wrong and it should pass with 2, or is the history code wrong and it should return 1?

[Geoffrey Garen]

The test is correct; it's an expected failure. 

[Scott Violet]

I didn't realize tests have expected output saying FAIL, yet they pass.
Thanks.

[Scott Violet]

Actually, could you clarify that. Are you saying it should say PASS, but is failing because of other bugs? Or should the output always say FAIL?

Thanks.

[Geoffrey Garen]

Happy to clarify.

A layout test passes if you run it and your results match the expected results .txt (or .png) file. That tells you that you haven't inadvertently changed anything.

Almost all of the time, the expected results will say "PASS" -- as you'd expect.

Sometimes, though, it's useful to write a layout test that documents current behavior, even though you think the current behavior is not so good. It's useful because the test can still catch inadvertent changes in behavior. We call such a test an "expected failure."

I've been doing a bit of work lately in the history code, so I wrote some layout tests to make sure that I didn't inadvertently change behavior. For many of the tests, I thought that WebKit's current behavior was not so good, so I checked in a result that said "FAIL." These tests are expected failures.

Ideally, we'll fix these expected failures in the future -- probably after some broader discussion of what behavior we want.  Then, the tests will start saying "PASS," and we'll change the expected test results to match.

Sorry if that confused you. Maybe there's a better way to do things -- honestly, I didn't think about it all that much.