Bug 23651 – REGRESSION: Acid Test 69 "Not perfect"

• Status: RESOLVED
• Resolution: WORKSFORME
• Component: Page Loading

[Michael Zedler]

Between 40239 and 40288 the Acid test 69 regressed, now one always gets the "not perfect" result. Happens on both Tiger and XP.

[Mark Rowe (bdash)]

The only change in that range that looks like it may be relevant is <http://trac.webkit.org/changeset/40278>.

[Mark Rowe (bdash)]

<rdar://problem/6544296>

[Sam Weinig]

I believe what is happening here is that starts with test 65.  It starts 7 async loads which on completion incrementally set up  kungFuDeathGrip.title to be 7 characters long.  When we get to test 69 now, all 7 loads have not completed, so a retry is necessary. 

[Sam Weinig]

Michael, I believe the issue may be that the you are not doing a fully cached run, but are instead going down a path that requires resource revalidation.  To do a fully cached run, please try running the test once, closing the window, and then selecting the first history item from the History menu and see if this reproduces. 

[Michael Zedler]

(In reply to comment #4)
> Michael, I believe the issue may be that the you are not doing a fully cached
> run, but are instead going down a path that requires resource revalidation.  To
> do a fully cached run, please try running the test once, closing the window,
> and then selecting the first history item from the History menu and see if this
> reproduces. 

I did this and indeed the 'error' is gone. But before 40239 I could simply hit re-load and everything was ok.

[Michael Zedler]

(In reply to comment #5)
> I did this and indeed the 'error' is gone. But before 40239 I could simply hit
> re-load and everything was ok.

correction: with 40239 and before it worked by simply hitting reload.

[Sam Weinig]

That is because before that before that we had a race condition and were not correctly loading a file at the right time.  Closing as WORKSFORME.

[Michael Zedler]

(In reply to comment #7)
> That is because before that before that we had a race condition and were not
> correctly loading a file at the right time.  Closing as WORKSFORME.

With Opera 10 and Chromium 2 one can simply hit reload and Test 69 passes without the 'non perfect' notification. 

[Mark Rowe (bdash)]

(In reply to comment #8)
> (In reply to comment #7)
> > That is because before that before that we had a race condition and were not
> > correctly loading a file at the right time.  Closing as WORKSFORME.
> 
> With Opera 10 and Chromium 2 one can simply hit reload and Test 69 passes
> without the 'non perfect' notification. 

Chromium has almost certainly not picked up the fix for the race condition mentioned in bug 18141 at this time.  Without that fix they will not see this behaviour, but will also fail two tests some of the time on slower connections.

Reload behavior isn't defined by any specification.  I would speculate that Opera has different revalidation logic on reloads which is leading to it succeeding in this situation.

Given that the performance aspects of the Acid 3 test have been stated to only be testable on a cached load of the test, I agree with Sam that our behaviour appears to be correct in this case.

[Michael Zedler]

(In reply to comment #9)
> (In reply to comment #8)
> > (In reply to comment #7)
> > > That is because before that before that we had a race condition and were not
> > > correctly loading a file at the right time.  Closing as WORKSFORME.
> > 
> > With Opera 10 and Chromium 2 one can simply hit reload and Test 69 passes
> > without the 'non perfect' notification. 
> 
> Chromium has almost certainly not picked up the fix for the race condition
> mentioned in bug 18141 at this time.  Without that fix they will not see this
> behaviour, but will also fail two tests some of the time on slower connections.
> 
> Reload behavior isn't defined by any specification.  I would speculate that
> Opera has different revalidation logic on reloads which is leading to it
> succeeding in this situation.
> 
> Given that the performance aspects of the Acid 3 test have been stated to only
> be testable on a cached load of the test, I agree with Sam that our behaviour
> appears to be correct in this case.

just checked Firefox beta, too. It also has the behaviour that hitting the reload button yields a completely passing Test 69. 
I just suspect that many people will expect that hitting reload should do, they'll get the 'error' and soon it'll be spread that webkit does not fully pass the Acid3 test.

[Mark Rowe (bdash)]

On a fresh load of Acid3 with an empty cache in the latest Firefox nightly build, I frequently see:

Test 69 passed, but took 30 attempts (less than perfect).

I don't think this is behaviour specific to WebKit.

[Michael Zedler]

(In reply to comment #11)
> On a fresh load of Acid3 with an empty cache in the latest Firefox nightly
> build, I frequently see:
> 
> Test 69 passed, but took 30 attempts (less than perfect).
> 
> I don't think this is behaviour specific to WebKit.

I re-checked with the current beta of Firefox and the current Minefield nighly. Upon some 10 reloads each (hitting the reload button) I didn't get a single time the result "less than perfect". Tests performed on Tiger, if it matters.