Bug 203199 – editing/pasteboard/paste-and-sanitize.html is a flaky failure

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Wenson Hsieh]

<rdar://problem/53551736>

[Wenson Hsieh]

Created attachment 381428 [details]
Attempt #1 to reproduce the failure

[Wenson Hsieh]

Created attachment 381475 [details]
Attempt #2 to reproduce the failure

[Wenson Hsieh]

From analyzing EWS results, in the pathological case, the test failure results in output like this after a single paste operation (of which there are a total of 11 in the test):

Attempting to paste (paste-and-sanitize-2.html)
The selection is: { OffsetInAnchor("hello1", 0), OffsetInAnchor(" hello2", 7) }
Pasting as rich text.

…whereas it should normally look like this:

Attempting to paste (paste-and-sanitize-2.html)
The selection is: { OffsetInAnchor("hello", 0), OffsetInAnchor("hello", 5) }
Pasting as rich text.
Replacing selection with fragment!
Applying ReplaceSelectionCommand...
Performing complex replacement.
Preparing whitespace at insertion position for split.
Clearing typing style.
Removing nodes.
Inserting nodes.
Sanitizing fragment? yes
DONE.

This means we are reaching Editor::paste, but then fail to call into Editor::replaceSelectionWithFragment.

[Wenson Hsieh]

Created attachment 381502 [details]
Attempt #3 to reproduce the failure

[Wenson Hsieh]

Created attachment 381504 [details]
Attempt #3 to reproduce the failure

[Wenson Hsieh]

In the normal (passing case):

  Attempting to paste (paste-and-sanitize-10.html)
  The selection is: { OffsetInAnchor("Hello ", 0), OffsetInAnchor("world", 5) }
  Pasting as rich text.
  Trying types: (
      "Apple Web Archive pasteboard type",
      "com.apple.webarchive",
      "com.apple.flat-rtfd",
      "public.rtf",
      "public.html",
      "public.png",
      "public.tiff",
      "public.jpeg",
      "com.compuserve.gif",
      "public.url",
      "public.text"
  )
  Successfully read type: Apple Web Archive pasteboard type
  Replacing selection with fragment!
  Sanitizing fragment? yes

...and in the failure case:

  Attempting to paste (paste-and-sanitize-10.html)
  The selection is: { OffsetInAnchor("Hello ", 0), OffsetInAnchor("world", 5) }
  2019-10-21 22:40:43.244 WebKitTestRunnerApp[78562:96340672] PBItemCollectionServicer connection disconnected.
  Pasting as rich text.
  Trying types: (
      "Apple Web Archive pasteboard type",
      "com.apple.webarchive",
      "com.apple.flat-rtfd",
      "public.rtf",
      "public.html",
      "public.png",
      "public.tiff",
      "public.jpeg",
      "com.compuserve.gif",
      "public.url",
      "public.text"
  )
  *** CHANGECOUNT CHANGED EXTERNALLY ***

So for some reason, the changeCount on the pasteboard is incrementing from underneath us...

[Wenson Hsieh]

Created attachment 381545 [details]
Test run #4

[Wenson Hsieh]

(In reply to Wenson Hsieh from comment #7)
> Created attachment 381545 [details]
> Test run #4

I think (maybe?) that I’ve found the smoking gun.

From the latest run, it looks like https://ews-build.webkit.org/results/iOS-13-Simulator-WK2-Tests-EWS/r381545-2234/editing/pasteboard/paste-and-sanitize-17-stderr.txt failed 3/11 of the test cases. I cleaned up the output and added some annotations below:

1.  Hello
    should become:
    Hello

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    2019-10-22 09:02:00.853 WebKitTestRunnerApp[2665:49792] PBItemCollectionServicer connection disconnected.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:00.874 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Trying 11 type(s)
    2019-10-22 09:02:00.886 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Successfully read type: Apple Web Archive pasteboard type

    => PASSED

2.  <b><i>Hello</i></b>
    should become:
    <b><i>Hello</i></b>

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:00.937 WebKitTestRunnerApp[2665:49792] PBItemCollectionServicer connection disconnected.

    => PASSED

3.  <div><b><i><span style="font-weight: normal"><b><i>Hello</i></b></span></i></b></div>
    should become:
    <b><i>Hello</i></b>

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    2019-10-22 09:02:01.031 WebKitTestRunnerApp[2665:48811] PBItemCollectionServicer connection disconnected.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:01.051 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Trying 11 type(s)
    2019-10-22 09:02:01.064 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Successfully read type: Apple Web Archive pasteboard type

    => PASSED

4.  <div><div><div>Hello</div></div></div>
    should become:
    Hello

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    2019-10-22 09:02:01.140 WebKitTestRunnerApp[2665:49832] PBItemCollectionServicer connection disconnected.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:01.177 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Trying 11 type(s)
    2019-10-22 09:02:01.189 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Successfully read type: Apple Web Archive pasteboard type

    => PASSED

5.  <div><b><div><i>Hello</i></div></b></div>
    should become:
    <i style="font-weight: bold;">Hello</i>

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:01.259 WebKitTestRunnerApp[2665:48811] PBItemCollectionServicer connection disconnected.

    => FAILED

6.  <div><div style="text-align: center;"><b>Hello</b></div></div>
    should become:
    <div style="text-align: center;"><b>Hello</b></div>

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    2019-10-22 09:02:01.325 WebKitTestRunnerApp[2665:49792] PBItemCollectionServicer connection disconnected.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:01.347 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Trying 11 type(s)
    2019-10-22 09:02:01.361 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Successfully read type: Apple Web Archive pasteboard type

    => PASSED

7.  <div><b><i><span style="font-weight: normal"><b><i>hello</i></b></span></i></b></div><div><b><i><span style="font-weight: normal"><b><i>world</i></b></span></i></b></div>
    should become:
    <div><b><i>hello</i></b></div><div><b><i>world</i></b></div>

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:01.425 WebKitTestRunnerApp[2665:49832] PBItemCollectionServicer connection disconnected.

    => FAILED

8.  <div><b><i><span style="font-weight: normal;"><b><i>hello1</i></b><b><i> hello2</i></b></span></i></b></div>
    should become:
    <b><i><span style="font-weight: normal;"><b><i>hello1</i></b><b><i>&nbsp;hello2</i></b></span></i></b>

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    2019-10-22 09:02:01.502 WebKitTestRunnerApp[2665:49832] PBItemCollectionServicer connection disconnected.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:01.539 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Trying 11 type(s)
    2019-10-22 09:02:01.560 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Successfully read type: Apple Web Archive pasteboard type

    => PASSED

9.  <i style="margin: 10px;"><b><i style="margin: 10px;">hello</i></b></i>
    should become:
    <i style="margin: 10px;">hello</i></b></i>

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    2019-10-22 09:02:01.643 WebKitTestRunnerApp[2665:49832] PBItemCollectionServicer connection disconnected.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:01.669 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Trying 11 type(s)
    2019-10-22 09:02:01.685 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Successfully read type: Apple Web Archive pasteboard type

    => PASSED

10. <div><b><i><span style="font-weight: normal"><b><i>Hello <!-- comment -->world</i></b></span></i></b></div>
    should become:
    <b><i>Hello&nbsp;world</i></b>

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    2019-10-22 09:02:01.766 WebKitTestRunnerApp[2665:48811] PBItemCollectionServicer connection disconnected.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:01.799 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Trying 11 type(s)
    2019-10-22 09:02:01.821 com.apple.WebKit.WebContent.Development[2927:48371] <WK>: Successfully read type: Apple Web Archive pasteboard type

    => PASSED

11. <div><b><i><span style="font-weight: normal">plain text<b><i>bold italic text</i></b></span></i></b></div>
    should become:
    <b><i><span style="font-weight: normal;">plain text<b><i>bold italic text</i></b></span></i></b>

    <WK>: >>> Writing selection to pasteboard!
    <WK>: <<< done writing selection to pasteboard.
    <WK>: Attempting to paste (paste-and-sanitize-17.html)
    2019-10-22 09:02:01.923 WebKitTestRunnerApp[2665:49792] PBItemCollectionServicer connection disconnected.

    => FAILED

[Wenson Hsieh]

In the cases where the "PBItemCollectionServicer connection disconnected.” message comes after “Attempting to paste (paste-and-sanitize-X.html)”, we will end up failing the test case unless the copied markup in the test case is identical to the final markup, in which case we’ll pass regardless of whether the paste went through (this exception applies to only the first two test cases, “Hello” and “<b><i>Hello</i></b>”).

[Wenson Hsieh]

Created attachment 381562 [details]
Let's see if this fixes the test?

[Wenson Hsieh]

Created attachment 381565 [details]
Let's (actually) see if this fixes the test?

[Wenson Hsieh]

(In reply to Wenson Hsieh from comment #11)
> Created attachment 381565 [details]
> Let's (actually) see if this fixes the test?

50/50 passed; looks like this mitigation ought to be sufficient.

[Wenson Hsieh]

Created attachment 381612 [details]
Mitigate the flaky test

[WebKit Commit Bot]

Comment on attachment 381612 [details]
Mitigate the flaky test

Clearing flags on attachment: 381612

Committed r251465: <https://trac.webkit.org/changeset/251465>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.