Acid3 test 69 fails when Adobe's SVG plug-in is installed

• Component: WebKit Misc.
• Status: RESOLVED WONTFIX

Hi, I'm on a PowerBook G4 and I cannot ever reproduce the 91 passed tests. I get 87 passed tests always, whether I empty the cache or not. This is also independent of whether I'm logged in as Admin, as a new user (to test default permissions) or my default User. Safari reports its version as Version 3.0.4 (5523.15). I have the debug menu enabled, but use default settings. The OS is 10.5.2, which I upgraded (upgrade-install) from Tiger, which was upgraded from Panther a few years ago.

Is there anything else I can provide?

Here are the results:

Failed 13 tests.
Test 02 failed: reached expectation 13 when expecting expectation 12
Test 11 failed: when inserting <a> into Document with another child: wrong exception raised; code = 1
Test 23 failed: no exception for createElementNS('null', ':div')
Test 25 failed: failed to raise exception
Test 26 passed, but took 313ms 
Test 40 passed, but took 18ms 
Test 46 passed, but took 20ms 
Test 65 passed, but took 28ms 
Test 69 failed: Undefined value
Test 69 passed, but took 57 attempts.
Test 70 failed: UTF-8 encoded XML document with invalid character did not have a well-formedness error
Test 73 failed: expected '10' but got '1' - click event handler called the wrong number of times
Test 74 failed: getSVGDocument failed for <iframe> referencing an svg document.
Test 75 failed: Value undefined (result of expression anim.beginElement) is not object.
Test 76 failed: expected '0' but got '100' - Incorrect animVal value after svg animation.
Test 77 failed: expected '3' but got '0' - getNumberOfChars returned incorrect string length.
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed.
Test 79 failed: expected '34' but got '33' - SVGSVGTextElement.getNumberOfChars() incorrect
Total elapsed time: 4.17s

Confirmed for ppc_32 (eMac G4 here). Will take a look at ppc_64 or maybe someone else can do that.
I hope someone can provide the results from a system that does pass 91/100 tests, so the bug that causes this 4 extra failures could be tackled quickly.

With a Mac Pro, I do get 91 tests passed, with the following failures:

Failed 9 tests.
Test 00 passed, but took 25ms 
Test 02 failed: reached expectation 13 when expecting expectation 12
Test 11 failed: when inserting <a> into Document with another child: wrong exception raised; code = 1
Test 23 failed: no exception for createElementNS('null', ':div')
Test 25 failed: failed to raise exception
Test 26 passed, but took 84ms 
Test 29 passed, but took 198ms 
Test 70 failed: UTF-8 encoded XML document with invalid character did not have a well-formedness error
Test 73 failed: expected '10' but got '1' - click event handler called the wrong number of times
Test 75 failed: Value undefined (result of expression anim.beginElement) is not object.
Test 76 failed: expected '0' but got '100' - Incorrect animVal value after svg animation.
Test 79 failed: expected '34' but got '33' - SVGSVGTextElement.getNumberOfChars() incorrect
Total elapsed time: 0.73s

The four differences that failed on the G4 Powerbook, but do not fail on my machine are:
Test 69 failed: Undefined value
Test 74 failed: getSVGDocument failed for <iframe> referencing an svg document.
Test 77 failed: expected '3' but got '0' - getNumberOfChars returned incorrect string length.
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed.

tested with r31084 under 10.4.11

I should note that when I test on a PowerBook G4, also with r31084 under 10.4.11, I get 91 passes, consistent with my MacPro, and at odds with the initial bug report.  Maybe it's Leopard specific on a G4, but I have no other data to support that.

jgrybows in #webkit is also seeing 88 passing tests rather than the correct 92.  He is also using Leopard on a PowerPC machine.

<rdar://problem/5803692>

Whatever number you see is always the "correct" number... It's not like the script is miscounting the number of passes. :-)

Seems that something is going wrong with test 69 -- if test 69 fails, then the tests after it are all likely to fail in odd ways since test 69 is the one that ensures the support files are correctly loaded from the network. I have no idea what test 69 might fail, though. A GC problem maybe? That's where I would focus attention, anyway.

Safari reporting version  3.0.4 (523.12.2), nightly 31114 (latest available at the point of writing on 3/18/2008). Powerbook G4 with 10.4.11.

I'm getting 87 or 88 successes on consecutive test loads, with the failing tests changing. Expecting 93.

Failed 11 tests.
Test 02 failed: reached expectation 13 when expecting expectation 12
Test 23 failed: no exception for createElementNS('null', ':div')
Test 25 failed: failed to raise exception
Test 26 passed, but took 281ms 
Test 65 passed, but took 22ms 
Test 69 failed: Undefined value
Test 70 failed: UTF-8 encoded XML document with invalid character did not have a well-formedness error
Test 74 failed: getSVGDocument failed for <iframe> referencing an svg document.
Test 75 failed: Value undefined (result of expression anim.beginElement) is not object.
Test 76 failed: expected '0' but got '100' - Incorrect animVal value after svg animation.
Test 77 failed: expected '3' but got '0' - getNumberOfChars returned incorrect string length.
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed.
Test 79 failed: expected '34' but got '33' - SVGSVGTextElement.getNumberOfChars() incorrect
Total elapsed time: 2.07s

Failed 12 tests.
Test 00 passed, but took 18ms 
Test 02 failed: reached expectation 13 when expecting expectation 12
Test 10 passed, but took 26ms 
Test 23 failed: no exception for createElementNS('null', ':div')
Test 25 failed: failed to raise exception
Test 26 passed, but took 278ms 
Test 46 passed, but took 17ms 
Test 65 passed, but took 29ms 
Test 69 failed: Undefined value
Test 71 failed: Value undefined (result of expression doc.open) is not object.
Test 72 failed: prerequisite failed: no image
Test 74 failed: getSVGDocument failed for <iframe> referencing an svg document.
Test 75 failed: Value undefined (result of expression anim.beginElement) is not object.
Test 76 failed: expected '0' but got '100' - Incorrect animVal value after svg animation.
Test 77 failed: expected '3' but got '0' - getNumberOfChars returned incorrect string length.
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed.
Test 79 failed: expected '34' but got '33' - SVGSVGTextElement.getNumberOfChars() incorrect
Total elapsed time: 3.18s

I can also confirm this here on a PowerPC G5 running Leopard. 
Using latest nightly (31370) I still see these failures on the first pass:

Test 69 failed: Undefined value
Test 74 failed: getSVGDocument failed for <iframe> referencing an svg document.
Test 75 failed: Value undefined (result of expression svg.setCurrentTime) is not object.
Test 76 failed: expected '0' but got '100' - Incorrect animVal value after svg animation.
Test 77 failed: expected '3' but got '0' - getNumberOfChars returned incorrect string length.
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed.
Test 79 failed: character position 6, which is between a normal character, which has some kerning after it and the first character of a two-character glyph, which has some kerning before it, is 21600 but should be 21800.

and these failures on the "reload" pass:

Test 69 failed: Undefined value
Test 74 failed: getSVGDocument failed for <iframe> referencing an svg document.
Test 75 failed: Value undefined (result of expression svg.setCurrentTime) is not object.
Test 76 failed: expected '0' but got '100' - Incorrect animVal value after svg animation.
Test 77 failed: expected '3' but got '0' - getNumberOfChars returned incorrect string length.
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed.
Test 79 passed, but took 89ms (less than 30fps)


In a completely unrelated note, on my Intel Mac Mini in Tiger, I see these failures: 

Test 77 failed: expected '4776' but got '5550' - getComputedTextLength failed. 
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed. 

Interestingly, upon retest I see a different failure: 

Test 80 failed: kungFuDeathGrip was null

All these tests in a new, standard account. 
Safari 3.1 is installed. 
Any help/regression I could do ... just ask. 

This seems to be the same bug as bug 18033.

Using a debug build and a nightly of the r31376, I consistently get the following failures:

Test 69 failed: Undefined value
Test 74 failed: getSVGDocument failed for <iframe> referencing an svg document.
Test 75 failed: Value undefined (result of expression svg.setCurrentTime) is not object.
Test 76 failed: expected '0' but got '100' - Incorrect animVal value after svg animation.
Test 77 failed: expected '3' but got '0' - getNumberOfChars returned incorrect string length.
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed.

Reloading doesn't affect the failures for me, unlike what other people have reported. It seems like different people are getting slightly different failures, although they are all in the SVG bucket. It also seems like we are all on PPC as well.

I've added more debug calls in test 69 to help with this. Instead of just having "Undefined value" it should now tell you what is undefined.

Thanks, Hixie!

I now get the following report:

Failed 7 tests.
Test 26 passed, but took 274ms (less than 30fps)
Test 69 failed: t was null
Test 74 failed: getSVGDocument failed for <iframe> referencing an svg document.
Test 75 failed: Value undefined (result of expression svg.setCurrentTime) is not object.
Test 76 failed: expected '0' but got '100' - Incorrect animVal value after svg animation.
Test 77 failed: expected '3' but got '0' - getNumberOfChars returned incorrect string length.
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed.
Test 79 failed: character position 6, which is between a normal character, which has some kerning after it and the first character of a two-character glyph, which has some kerning before it, is 21600 but should be 21800.
Total elapsed time: 2.88s

I hope this helps a bit?

Created an attachment (id=20145) [details]
tcpflow log for one acid3-run with the nighlty based on r31381

Acid3 reports the following:

Failed 6 tests.
Test 26 passed, but took 280ms (less than 30fps)
Test 69 failed: t was null
Test 74 failed: getSVGDocument failed for <iframe> referencing an svg document.
Test 75 failed: Value undefined (result of expression svg.setCurrentTime) is not object.
Test 76 failed: expected '0' but got '100' - Incorrect animVal value after svg animation.
Test 77 failed: expected '3' but got '0' - getNumberOfChars returned incorrect string length.
Test 78 failed: expected '90' but got '0' - getRotationOfChar(0) failed.
Test 79 passed, but took 74ms (less than 30fps)
Total elapsed time: 2.87s

Created an attachment (id=20146) [details]
tcpflow log for one acid3-run with the nighlty based on r31381, cache cleared

t being null suggests the SVG file isn't loading fully before firing the 'onload' handler.

*** Bug 18033 has been marked as a duplicate of this bug. ***

This bug is caused by the ancient NPSVG3Carbon-Plugin from Adobe. It sits in /Library/Internet Plugins/

removing it and then restarting Webkit gets me to 100/100

Bah! I can confirm it.

The plug-in explains the load time when moving to test 69 for the first time. I think Intel users don't have this problem since they don't have this plug-in, I could be wrong however.

I suggest WebKit disables this plug-in since the SVG support is now about equal to the adobe SVG plug-in and it isn't used everywhere anymore.

*** Bug 18131 has been marked as a duplicate of this bug. ***

*** Bug 19548 has been marked as a duplicate of this bug. ***

ASV is dead, as well as non-intel Macs. Can we close this bug? Darin?

*** Bug 18152 has been marked as a duplicate of this bug. ***

One can still use ASV in shipping Safari (SnowLeopard, 32 bit/PowerPC checked in Get Info), but it seems very clear that this issue isn't something worth looking into.