Bug 144247 – [EFL] Allow to run the tests on the native X display

• Component: WebKit EFL
• Status: RESOLVED FIXED

[Hunseop Jeong]

Now xvfb driver has a issue related to GL drawing on EFL port.
So added the native X display to run the tests without GL issue.
Use the X display until GL issue is fixed on EFL port.

[Hunseop Jeong]

Created attachment 251722 [details]
Patch

[Gyuyoung Kim]

Comment on attachment 251722 [details]
Patch

Hunseop, did you run whole EFL layout test ? I wonder if there is any regression when using USE_NATIVE_XDISPLAY option.

[Hunseop Jeong]

(In reply to comment #2)
> Comment on attachment 251722 [details]
> Patch
> 
> Hunseop, did you run whole EFL layout test ? I wonder if there is any
> regression when using USE_NATIVE_XDISPLAY option.

I will check whole layout test.
I will upload the result after testing the layout test.

[Hunseop Jeong]

I watch the error message "ServerError raised: Port 8000 is already in use." 
when I retry the test after finish the layout test.
It looks like port issue. I will watch this issue.

[Gyuyoung Kim]

Comment on attachment 251722 [details]
Patch

We have to land this patch after checking whole layout test.

[Csaba Osztrogonác]

Comment on attachment 251722 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=251722&action=review

This patch seems too intrusive for developers, flashing WTR windows in foreground
are so annoying and make any work impossible during running layout tests.

> Tools/Scripts/webkitpy/port/efl.py:55
> +        # FIXME: Use the native xdisplay instead of xvfb driver until GL issue is fixed on EFL port.
> +        os.environ['USE_NATIVE_XDISPLAY'] = '1'
> +

Can we make it optional somehow? With this change we can use only XorgDriver.
We aren't able to use nor the default port driver (DISABLE_XVFB_DRIVER case),
neither the XvfbDriver.

[Csaba Osztrogonác]

Comment on attachment 251722 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=251722&action=review

> Tools/ChangeLog:3
> +        [EFL] Allow to run the tests on the native X display 

trailing whitespace

[Hunseop Jeong]

(In reply to comment #6)
> Comment on attachment 251722 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=251722&action=review
> 
> This patch seems too intrusive for developers, flashing WTR windows in
> foreground
> are so annoying and make any work impossible during running layout tests.
> 
Yes, You are right. I feel like you, after applying this patch, many windows was flashed that make me annoying!!

> > Tools/Scripts/webkitpy/port/efl.py:55
> > +        # FIXME: Use the native xdisplay instead of xvfb driver until GL issue is fixed on EFL port.
> > +        os.environ['USE_NATIVE_XDISPLAY'] = '1'
> > +
> 
> Can we make it optional somehow? With this change we can use only XorgDriver.
> We aren't able to use nor the default port driver (DISABLE_XVFB_DRIVER case),
> neither the XvfbDriver.

Okay, I will make the option to select the driver.

[Hunseop Jeong]

(In reply to comment #5)
> Comment on attachment 251722 [details]
> Patch
> 
> We have to land this patch after checking whole layout test.

gyuyoung,
We can get the similar result if set the "DISABLE_XVFB_DRIVER".
How about use the "DISABLE_XVFB_DRIVER"?

If set the "DISABLE_XVFB_DRIVER" before starting the layout test.
It using the WebKitTestRunner instead of the xvfb.

Whole test result is :
=> Results: 38350/39533 tests passed (97.0%)

=> Tests to be fixed (5383):
      9 crashes                  ( 0.2%)
     53 timeouts                 ( 1.0%)
     49 missing results          ( 0.9%)
    316 image-only failures      ( 5.9%)

=> Tests that will only be fixed if they crash (WONTFIX) (1190):
      1 image-only failures      ( 0.1%)


Expected to crash, but passed: (12)
  compositing/clipping/border-radius-stacking-context-clip.html
  compositing/contents-opaque/opaque-with-clip-path.html
  compositing/masks/become-tiled-mask.html
  compositing/masks/cease-tiled-mask.html
  compositing/masks/compositing-clip-path-and-mask.html
  compositing/masks/compositing-clip-path.html
  compositing/masks/reference-clip-path-on-composited.html
  compositing/masks/solid-color-masked.html
  compositing/masks/tiled-mask.html
  compositing/patterns/direct-pattern-compositing-add-text.html
  compositing/patterns/direct-pattern-compositing-padding.html
  compositing/patterns/direct-pattern-compositing.html

Expected to fail, but passed: (3)
  compositing/animation/matrix-animation.html
  fast/sub-pixel/sub-pixel-composited-layers.html
  media/media-volume-slider-rendered-below.html


Unexpected flakiness: text-only failures (2)
  fast/dom/adopt-node-crash-2.html [ Failure Pass ]
  perf/rel-list-remove.html [ Failure Pass ]

Unexpected flakiness: image-only failures (1)
  css3/flexbox/csswg/css-flexbox-height-animation-stretch.html [ ImageOnlyFailure Pass ]

Unexpected flakiness: timeouts (2)
  media/track/audio/audio-track-mkv-vorbis-addtrack.html [ Timeout Pass ]
  media/track/track-cues-pause-on-exit.html [ Failure Timeout Pass ]


Regressions: Unexpected text-only failures (61)
  canvas/philip/tests/2d.drawImage.animated.poster.html [ Failure ]
  compositing/tiling/transform-origin-tiled.html [ Failure ]
  editing/execCommand/5142012-1.html [ Failure ]
  editing/execCommand/nsresponder-outdent.html [ Failure ]
  editing/inserting/insert-at-end-02.html [ Failure ]
  editing/selection/context-menu-text-selection-lookup.html [ Failure ]
  fast/canvas/canvas-ellipse-zero-lineto.html [ Failure ]
  fast/css-generated-content/hover-inline.html [ Failure ]
  fast/css/affected-by-hover-after-style-change.html [ Failure ]
  fast/css/first-letter-hover.html [ Failure ]
  fast/css/hover-active-drag.html [ Failure ]
  fast/css/hover-affects-ancestor.html [ Failure ]
  fast/css/hover-affects-child.html [ Failure ]
  fast/css/hover-display-block.html [ Failure ]
  fast/css/hover-subselector.html [ Failure ]
  fast/css/nested-layers-with-hover.html [ Failure ]
  fast/css/unknown-pseudo-element-matching.html [ Failure ]
  fast/dom/Window/window-resize-and-move-arguments.html [ Failure ]
  fast/events/event-sender-mouse-moved.html [ Failure ]
  fast/events/frame-scroll-fake-mouse-move.html [ Failure ]
  fast/events/ghostly-mousemoves-in-subframe.html [ Failure ]
  fast/events/mouse-cursor-change.html [ Failure ]
  fast/events/mouse-cursor-multiframecur.html [ Failure ]
  fast/events/mouse-cursor-no-mousemove.html [ Failure ]
  fast/events/mouseenter-mouseleave-capture.html [ Failure ]
  fast/events/mouseenter-mouseleave.html [ Failure ]
  fast/events/mouseenterleave-on-subframe.html [ Failure ]
  fast/events/mousemove-after-drag-over-scrollbar.html [ Failure ]
  fast/events/mouseout-dead-node.html [ Failure ]
  fast/events/mouseover-button.html [ Failure ]
  fast/events/mouseover-mouseout.html [ Failure ]
  fast/events/mouseover-mouseout2.html [ Failure ]
  fast/events/scroll-in-scaled-page-with-overflow-hidden.html [ Failure ]
  fast/events/touch/touch-before-pressing-spin-button.html [ Failure ]
  fast/events/wheelevent-basic.html [ Failure ]
  fast/events/wheelevent-mousewheel-interaction.html [ Failure ]
  fast/filter-image/filter-image-animation.html [ Failure ]
  fast/forms/input-step-as-double.html [ Failure ]
  fast/regions/assert-hit-test-image.html [ Failure ]
  fast/selectors/hover-strict.html [ Failure ]
  fast/text/international/cjk-segmentation.html [ Failure ]
  fast/text/midword-break-before-surrogate-pair-2.html [ Failure ]
  http/tests/cache/willsendrequest-returns-null-for-memory-cache-load.html [ Failure ]
  http/tests/misc/authentication-redirect-3/authentication-sent-to-redirect-same-origin-with-location-credentials.html [ Failure ]
  http/tests/security/cors-post-redirect-301.html [ Failure ]
  http/tests/security/cors-post-redirect-302.html [ Failure ]
  http/tests/security/cors-post-redirect-303.html [ Failure ]
  http/tests/security/cors-post-redirect-307.html [ Failure ]
  http/tests/security/cors-post-redirect-308.html [ Failure ]
  http/tests/security/cross-frame-access-put.html [ Failure ]
  http/tests/security/mixedContent/insecure-audio-video-in-main-frame.html [ Failure ]
  http/tests/xmlhttprequest/access-control-allow-lists-starting-with-comma.html [ Failure ]
  http/tests/xmlhttprequest/post-blob-content-type-async.html [ Failure ]
  http/tests/xmlhttprequest/post-blob-content-type-sync.html [ Failure ]
  js/dom/global-constructors-attributes.html [ Failure ]
  media/video-initially-hidden-volume-slider-up.html [ Failure ]
  scrollbars/overflow-custom-scrollbar-crash.html [ Failure ]
  scrollbars/scroll-rtl-or-bt-layer.html [ Failure ]
  sputnik/Unicode/Unicode_320/S7.6_A2.2_T2.html [ Failure ]
  sputnik/Unicode/Unicode_320/S7.6_A5.2_T8.html [ Failure ]
  svg/animations/animate-text-nested-transforms.html [ Failure ]

Regressions: Unexpected image-only failures (14)
  compositing/background-color/background-color-change-to-text.html [ ImageOnlyFailure ]
  compositing/masks/compositing-clip-path-mask-change.html [ ImageOnlyFailure ]
  compositing/overlap-blending/nested-overlap-overflow.html [ ImageOnlyFailure ]
  compositing/text-on-scaled-surface.html [ ImageOnlyFailure ]
  dom/xhtml/level3/core/canonicalform08.xhtml [ ImageOnlyFailure ]
  dom/xhtml/level3/core/canonicalform09.xhtml [ ImageOnlyFailure ]
  fast/backgrounds/background-opaque-clipped-gradients.html [ ImageOnlyFailure ]
  fast/borders/border-image-fill-no-border.html [ ImageOnlyFailure ]
  fast/css/pseudo-element-selector-scrollbar-hover.html [ ImageOnlyFailure ]
  fast/forms/button-line-height.html [ ImageOnlyFailure ]
  fast/frames/flattening/scrolling-in-object.html [ ImageOnlyFailure ]
  svg/animations/mozilla/animateMotion-mpath-targetChange-1.svg [ ImageOnlyFailure ]
  svg/clip-path/clip-path-line-use-before-defined.svg [ ImageOnlyFailure ]
  svg/transforms/transformed-text-fill-pattern.html [ ImageOnlyFailure ]

Regressions: Unexpected crashes (2)
  media/track/track-in-band-duplicate-tracks-when-source-changes.html [ Crash ]
  svg/clip-path/clip-path-precision-001.svg [ Crash ]

Regressions: Unexpected timeouts (8)
  fast/block/positioning/absolute-appended-to-inline.html [ Timeout ]
  fast/css/hover-update.html [ Timeout ]
  fast/events/iframe-onmousemove.html [ Timeout ]
  fast/events/mouse-moved-remove-frame-crash.html [ Timeout ]
  fast/events/mouseout-dead-subframe.html [ Timeout ]
  fast/events/wheelevent-in-text-node.html [ Timeout ]
  fast/layers/no-clipping-overflow-hidden-added-after-transform.html [ Timeout ]
  media/click-volume-bar-not-pausing.html [ Timeout ]


Many tests are failed after disable the xvfb.
Need to the many rebaseline work if using the Xorg or disabling the xvfb.

[Gyuyoung Kim]

Comment on attachment 251722 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=251722&action=review

>>> Tools/Scripts/webkitpy/port/efl.py:55
>>> +
>> 
>> Can we make it optional somehow? With this change we can use only XorgDriver.
>> We aren't able to use nor the default port driver (DISABLE_XVFB_DRIVER case),
>> neither the XvfbDriver.
> 
> Okay, I will make the option to select the driver.

Ah...I missed too many windows and hardcoded 'USE_NATIVE_XDISPLAY' definition. Please make it optional !

[Hunseop Jeong]

Created attachment 251735 [details]
Patch

[Hunseop Jeong]

(In reply to comment #11)
> Created attachment 251735 [details]
> Patch

(In reply to comment #10)
> Comment on attachment 251722 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=251722&action=review
> 
> >>> Tools/Scripts/webkitpy/port/efl.py:55
> >>> +
> >> 
> >> Can we make it optional somehow? With this change we can use only XorgDriver.
> >> We aren't able to use nor the default port driver (DISABLE_XVFB_DRIVER case),
> >> neither the XvfbDriver.
> > 
> > Okay, I will make the option to select the driver.
> 
> Ah...I missed too many windows and hardcoded 'USE_NATIVE_XDISPLAY'
> definition. Please make it optional !

Sorry, gyuyoung.

I changed the patch which can select the USE_NATIVE_XDISPLAY for WTR.

[Gyuyoung Kim]

(In reply to comment #9)
> (In reply to comment #5)
> > Comment on attachment 251722 [details]
> > Patch
> > 
> > We have to land this patch after checking whole layout test.
> 
> gyuyoung,
> We can get the similar result if set the "DISABLE_XVFB_DRIVER".
> How about use the "DISABLE_XVFB_DRIVER"?

When I set DISABLE_XVFB_DRIVER environment variable, I'm still able to see a lot of crash on webgl tests. To get fine result, should I install additional packages ?

[Csaba Osztrogonác]

(In reply to comment #13)
> (In reply to comment #9)
> > (In reply to comment #5)
> > > Comment on attachment 251722 [details]
> > > Patch
> > > 
> > > We have to land this patch after checking whole layout test.
> > 
> > gyuyoung,
> > We can get the similar result if set the "DISABLE_XVFB_DRIVER".
> > How about use the "DISABLE_XVFB_DRIVER"?
> 
> When I set DISABLE_XVFB_DRIVER environment variable, I'm still able to see a
> lot of crash on webgl tests. To get fine result, should I install additional
> packages ?

$ DISABLE_XVFB_DRIVER=1 Tools/Scripts/run-webkit-tests --efl webgl --force
80 failures, 10 timeouts

What about if we remove DISABLE_XVFB_DRIVER and have only USE_NATIVE_XDISPLAY ?

[Gyuyoung Kim]

(In reply to comment #14)
> (In reply to comment #13)
> > (In reply to comment #9)
> > > (In reply to comment #5)
> > > > Comment on attachment 251722 [details]
> > > > Patch
> > > > 
> > > > We have to land this patch after checking whole layout test.
> > > 
> > > gyuyoung,
> > > We can get the similar result if set the "DISABLE_XVFB_DRIVER".
> > > How about use the "DISABLE_XVFB_DRIVER"?
> > 
> > When I set DISABLE_XVFB_DRIVER environment variable, I'm still able to see a
> > lot of crash on webgl tests. To get fine result, should I install additional
> > packages ?
> 
> $ DISABLE_XVFB_DRIVER=1 Tools/Scripts/run-webkit-tests --efl webgl --force
> 80 failures, 10 timeouts
> 
> What about if we remove DISABLE_XVFB_DRIVER and have only
> USE_NATIVE_XDISPLAY ?

I'm not 100% sure if two options can return same values every cases though, it seems those options returns same value. However we need to check if there is any fail on perf bot when using USE_NATIVE_XDISPLAY as mentioned in https://bugs.webkit.org/show_bug.cgi?id=131036#c0.

[Hunseop Jeong]

(In reply to comment #15)
> (In reply to comment #14)
> > (In reply to comment #13)
> > > (In reply to comment #9)
> > > > (In reply to comment #5)
> > > > > Comment on attachment 251722 [details]
> > > > > Patch
> > > > > 
> > > > > We have to land this patch after checking whole layout test.
> > > > 
> > > > gyuyoung,
> > > > We can get the similar result if set the "DISABLE_XVFB_DRIVER".
> > > > How about use the "DISABLE_XVFB_DRIVER"?
> > > 
> > > When I set DISABLE_XVFB_DRIVER environment variable, I'm still able to see a
> > > lot of crash on webgl tests. To get fine result, should I install additional
> > > packages ?
> > 
> > $ DISABLE_XVFB_DRIVER=1 Tools/Scripts/run-webkit-tests --efl webgl --force
> > 80 failures, 10 timeouts
> > 
> > What about if we remove DISABLE_XVFB_DRIVER and have only
> > USE_NATIVE_XDISPLAY ?
> 
> I'm not 100% sure if two options can return same values every cases though,
> it seems those options returns same value. However we need to check if there
> is any fail on perf bot when using USE_NATIVE_XDISPLAY as mentioned in
> https://bugs.webkit.org/show_bug.cgi?id=131036#c0.

I checked perf test using USE_NATIVE_XDISPLAY and DISABLE_XVFB_DRIVER.
I got the same result. What do I do now?
1. Remove the DISABLE_XVFB_DRIVER and add the USE_NATIVE_XDISPLAY option?
2. Just add the USE_NATIVE_XDISPLAY option?

[Gyuyoung Kim]

(In reply to comment #16)
> (In reply to comment #15)
> > (In reply to comment #14)
> > > (In reply to comment #13)
> > > > (In reply to comment #9)
> > > > > (In reply to comment #5)
> > > > > > Comment on attachment 251722 [details]
> > > > > > Patch
> > > > > > 
> > > > > > We have to land this patch after checking whole layout test.
> > > > > 
> > > > > gyuyoung,
> > > > > We can get the similar result if set the "DISABLE_XVFB_DRIVER".
> > > > > How about use the "DISABLE_XVFB_DRIVER"?
> > > > 
> > > > When I set DISABLE_XVFB_DRIVER environment variable, I'm still able to see a
> > > > lot of crash on webgl tests. To get fine result, should I install additional
> > > > packages ?
> > > 
> > > $ DISABLE_XVFB_DRIVER=1 Tools/Scripts/run-webkit-tests --efl webgl --force
> > > 80 failures, 10 timeouts
> > > 
> > > What about if we remove DISABLE_XVFB_DRIVER and have only
> > > USE_NATIVE_XDISPLAY ?
> > 
> > I'm not 100% sure if two options can return same values every cases though,
> > it seems those options returns same value. However we need to check if there
> > is any fail on perf bot when using USE_NATIVE_XDISPLAY as mentioned in
> > https://bugs.webkit.org/show_bug.cgi?id=131036#c0.
> 
> I checked perf test using USE_NATIVE_XDISPLAY and DISABLE_XVFB_DRIVER.
> I got the same result. What do I do now?
> 1. Remove the DISABLE_XVFB_DRIVER and add the USE_NATIVE_XDISPLAY option?
> 2. Just add the USE_NATIVE_XDISPLAY option?

If so, I also prefer to remove DISABLE_XVFB_DRIVER and use USE_NATIVE_XDISPLAY as Ossy suggested.

[Hunseop Jeong]

Created attachment 251820 [details]
Patch

[Hunseop Jeong]

(In reply to comment #17)
> (In reply to comment #16)
> > (In reply to comment #15)
> > > (In reply to comment #14)
> > > > (In reply to comment #13)
> > > > > (In reply to comment #9)
> > > > > > (In reply to comment #5)
> > > > > > > Comment on attachment 251722 [details]
> > > > > > > Patch
> > > > > > > 
> > > > > > > We have to land this patch after checking whole layout test.
> > > > > > 
> > > > > > gyuyoung,
> > > > > > We can get the similar result if set the "DISABLE_XVFB_DRIVER".
> > > > > > How about use the "DISABLE_XVFB_DRIVER"?
> > > > > 
> > > > > When I set DISABLE_XVFB_DRIVER environment variable, I'm still able to see a
> > > > > lot of crash on webgl tests. To get fine result, should I install additional
> > > > > packages ?
> > > > 
> > > > $ DISABLE_XVFB_DRIVER=1 Tools/Scripts/run-webkit-tests --efl webgl --force
> > > > 80 failures, 10 timeouts
> > > > 
> > > > What about if we remove DISABLE_XVFB_DRIVER and have only
> > > > USE_NATIVE_XDISPLAY ?
> > > 
> > > I'm not 100% sure if two options can return same values every cases though,
> > > it seems those options returns same value. However we need to check if there
> > > is any fail on perf bot when using USE_NATIVE_XDISPLAY as mentioned in
> > > https://bugs.webkit.org/show_bug.cgi?id=131036#c0.
> > 
> > I checked perf test using USE_NATIVE_XDISPLAY and DISABLE_XVFB_DRIVER.
> > I got the same result. What do I do now?
> > 1. Remove the DISABLE_XVFB_DRIVER and add the USE_NATIVE_XDISPLAY option?
> > 2. Just add the USE_NATIVE_XDISPLAY option?
> 
> If so, I also prefer to remove DISABLE_XVFB_DRIVER and use
> USE_NATIVE_XDISPLAY as Ossy suggested.

I Removed the DISABLE_XVFB_DRIVER and added the USE_NATIVE_XDISPLAY.
We can select the XorgDriver if USE_NATIVE_XDISPLAY is set up before running the tests.

Maybe perf bot need to change the configuration after applying this patch.
Now perf bot use the "DISABLE_XVFB_DRIVER=1".
(https://build.webkit.org/builders/EFL%20Linux%2064-bit%20Release%20WK2%20%28Perf%29/builds/5464/steps/perf-test/logs/stdio)

[Gyuyoung Kim]

(In reply to comment #19)

> Maybe perf bot need to change the configuration after applying this patch.
> Now perf bot use the "DISABLE_XVFB_DRIVER=1".
> (https://build.webkit.org/builders/EFL%20Linux%2064-
> bit%20Release%20WK2%20%28Perf%29/builds/5464/steps/perf-test/logs/stdio)

EFL perf bot is maintained by Ossy. It looks it would be good if Ossy lands this patch after changing DISABLE_XVFB_DRIVER on the perf bot.

[Gyuyoung Kim]

(In reply to comment #20)
> (In reply to comment #19)
> 
> > Maybe perf bot need to change the configuration after applying this patch.
> > Now perf bot use the "DISABLE_XVFB_DRIVER=1".
> > (https://build.webkit.org/builders/EFL%20Linux%2064-
> > bit%20Release%20WK2%20%28Perf%29/builds/5464/steps/perf-test/logs/stdio)
> 
> EFL perf bot is maintained by Ossy. It looks it would be good if Ossy lands
> this patch after changing DISABLE_XVFB_DRIVER on the perf bot.

Ossy, ping ?

[Csaba Osztrogonác]

Sorry for the delay, I'll change the env on the perf bot and land this patch soon.

[Csaba Osztrogonác]

Comment on attachment 251820 [details]
Patch

Clearing flags on attachment: 251820

Committed r183540: <http://trac.webkit.org/changeset/183540>

[Csaba Osztrogonác]

All reviewed patches have been landed.  Closing bug.

[Gyuyoung Kim]

When enabling USE_NATIVE_XDISPLAY, 111 test are failed !

https://build.webkit.org/builders/EFL%20Linux%2064-bit%20Release%20WK2/builds/21557/steps/layout-test/logs/stdio

34062 tests ran as expected, 111 didn't: