Bug 185965 – [iOS] TestWebKitAPI.PasteImage tests are flaky failures

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Ryan Haddad]

The following API test is a flaky failure on iOS Simulator bots:

    TestWebKitAPI.PasteImage.PastePNGImage
        
        /Volumes/Data/slave/ios-simulator-11-release/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/PasteImage.mm:115
        Value of: [webView stringByEvaluatingJavaScript:@"imageElement.width"]
          Actual: "0"
        Expected: "200"

https://build.webkit.org/builders/Apple%20iOS%2011%20Simulator%20Release%20WK2%20(Tests)/builds/5205/steps/run-api-tests/logs/stdio

[Ryan Haddad]

I see a similar failure with TestWebKitAPI.PasteImage.PasteGIFImage @ r231863
        
        /Volumes/Data/slave/ios-simulator-11-release/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/PasteImage.mm:71
        Value of: [webView stringByEvaluatingJavaScript:@"imageElement.width"]
          Actual: "0"
        Expected: "400"

https://build.webkit.org/builders/Apple%20iOS%2011%20Simulator%20Release%20WK2%20%28Tests%29/builds/5029

[Dawei Fenton (:realdawei)]

This is still happening:

https://build.webkit.org/builders/Apple%20iOS%2011%20Simulator%20Release%20WK2%20(Tests)/builds/5383/steps/run-api-tests/logs/stdio

[Ryan Haddad]

I can easily reproduce these failures by running the tests in a loop:
while true; do run-api-tests --ios-simulator PasteImage; done

[Radar WebKit Bug Importer]

<rdar://problem/41727126>

[Radar WebKit Bug Importer]

<rdar://problem/41727134>

[Wenson Hsieh]

I suspect we just need to wait for the image elements to become their expected sizes.

[Ryan Haddad]

(In reply to Wenson Hsieh from comment #6)
> I suspect we just need to wait for the image elements to become their
> expected sizes.
Is this a quick fix? These failures are frequently making the bots red.

[Wenson Hsieh]

(In reply to Ryan Haddad from comment #7)
> (In reply to Wenson Hsieh from comment #6)
> > I suspect we just need to wait for the image elements to become their
> > expected sizes.
> Is this a quick fix? These failures are frequently making the bots red.

If my speculation is correct, the fix should be trivial. I'll take a look at this by the end of the week and see (we may just need to sprinkle `[webView waitForMessage:@"loaded"];` in some places).

[Wenson Hsieh]

Created attachment 348478 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 348478 [details]
Patch

Clearing flags on attachment: 348478

Committed r235504: <https://trac.webkit.org/changeset/235504>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Ryan Haddad]

It looks like these tests time out on macOS now :(

TestWebKitAPI.PasteImage.PastePNGImage
https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK2%20(Tests)/builds/7823/steps/run-api-tests/logs/stdio

TestWebKitAPI.PasteImage.PasteGIFImage
https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK1%20%28Tests%29/builds/9324/steps/run-api-tests/logs/stdio

[Truitt Savell]

(In reply to Ryan Haddad from comment #12)
> It looks like these tests time out on macOS now :(
> 
> TestWebKitAPI.PasteImage.PastePNGImage
> https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK2%20(Tests)/
> builds/7823/steps/run-api-tests/logs/stdio
> 
> TestWebKitAPI.PasteImage.PasteGIFImage
> https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK1%20%28Tests%29/
> builds/9324/steps/run-api-tests/logs/stdio

Also 

TestWebKitAPI.PasteImage.PasteJPEGImage
https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK1%20%28Tests%29/builds/9326/steps/run-api-tests/logs/stdio

[Wenson Hsieh]

(In reply to Ryan Haddad from comment #12)
> It looks like these tests time out on macOS now :(
> 
> TestWebKitAPI.PasteImage.PastePNGImage
> https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK2%20(Tests)/
> builds/7823/steps/run-api-tests/logs/stdio
> 
> TestWebKitAPI.PasteImage.PasteGIFImage
> https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK1%20%28Tests%29/
> builds/9324/steps/run-api-tests/logs/stdio

Weird!

I'll look into this later today. I suppose it's possible that images can load /too/ quickly on macOS, and we dispatch the "load" event before the UI process even gets a chance to register a message handler.

I can test this by registering prior to calling insertFileAsImage.

[Wenson Hsieh]

Reopening to attach new patch.

[Wenson Hsieh]

Created attachment 348545 [details]
Patch

[Wenson Hsieh]

(In reply to Wenson Hsieh from comment #16)
> Created attachment 348545 [details]
> Patch

Luckily, I'm able to reproduce the timeouts on macOS this time. This should fix the timeouts that started after my previous attempt...

[WebKit Commit Bot]

Comment on attachment 348545 [details]
Patch

Clearing flags on attachment: 348545

Committed r235528: <https://trac.webkit.org/changeset/235528>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.