The idea is: - Allow developers to run the layout tests under a wayland environment easily (not need to run Wayland as your display server) - Allow to run the tests also on a headless server (that way we can have the bots testing the tests with wayland)
Created attachment 276722 [details] Patch
Comment on attachment 276722 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=276722&action=review > Tools/gtk/jhbuild.modules:436 > + <autotools id="wayland" autogenargs="--disable-documentation"> > + <dependencies> > + <dep package="libffi"/> > + </dependencies> > + <branch module="releases/wayland-1.8.1.tar.xz" > + version="1.8.1" > + repo="wayland.freedesktop.org" > + hash="sha256:f17c938d1c24fd0a10f650a623a2775d329db3168b5732e498b08388ec776fc8" > + md5sum="6e877877c3e04cfb865cfcd0733c9ab1"> > + </branch> > + </autotools> > + > + <autotools id="weston" autogenargs="--enable-x11-compositor --disable-rpi-compositor --disable-fbdev-compositor --disable-setuid-install --disable-ivi-shell --with-cairo=gl"> > + <dependencies> > + <dep package="wayland"/> > + <dep package="libdrm"/> > + <dep package="xserver"/> > + <dep package="cairo"/> > + </dependencies> > + <branch module="releases/weston-1.8.0.tar.xz" > + version="1.8.0" > + repo="wayland.freedesktop.org" > + hash="sha256:8963e69f328e815cec42c58046c4af721476c7541bb7d9edc71740fada5ad312" > + md5sum="24cb8a7ed0535b4fc3642643988dab36"> I think we should keep these dependencies optional for now. > Tools/gtk/jhbuild.modules:456 > + <autotools id="libdrm" autogen-sh="configure"> > + <branch module="/libdrm/libdrm-2.4.65.tar.bz2" version="2.4.65" > + repo="dri.freedesktop.org" > + hash="sha256:71960ac8bde7d710992b1bc8879935e8300a870c36bd06f22412d0447e3d96c4"/> > + </autotools> Does this make sense on machines with NVidia cards?
(In reply to comment #2) > Comment on attachment 276722 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=276722&action=review > > > Tools/gtk/jhbuild.modules:436 > > + <autotools id="wayland" autogenargs="--disable-documentation"> > > + <dependencies> > > + <dep package="libffi"/> > > + </dependencies> > > + <branch module="releases/wayland-1.8.1.tar.xz" > > + version="1.8.1" > > + repo="wayland.freedesktop.org" > > + hash="sha256:f17c938d1c24fd0a10f650a623a2775d329db3168b5732e498b08388ec776fc8" > > + md5sum="6e877877c3e04cfb865cfcd0733c9ab1"> > > + </branch> > > + </autotools> > > + > > + <autotools id="weston" autogenargs="--enable-x11-compositor --disable-rpi-compositor --disable-fbdev-compositor --disable-setuid-install --disable-ivi-shell --with-cairo=gl"> > > + <dependencies> > > + <dep package="wayland"/> > > + <dep package="libdrm"/> > > + <dep package="xserver"/> > > + <dep package="cairo"/> > > + </dependencies> > > + <branch module="releases/weston-1.8.0.tar.xz" > > + version="1.8.0" > > + repo="wayland.freedesktop.org" > > + hash="sha256:8963e69f328e815cec42c58046c4af721476c7541bb7d9edc71740fada5ad312" > > + md5sum="24cb8a7ed0535b4fc3642643988dab36"> > > I think we should keep these dependencies optional for now. > Then running the tests with wayland may give errors depending on the wayland version that your distro ships. > > Tools/gtk/jhbuild.modules:456 > > + <autotools id="libdrm" autogen-sh="configure"> > > + <branch module="/libdrm/libdrm-2.4.65.tar.bz2" version="2.4.65" > > + repo="dri.freedesktop.org" > > + hash="sha256:71960ac8bde7d710992b1bc8879935e8300a870c36bd06f22412d0447e3d96c4"/> > > + </autotools> > > Does this make sense on machines with NVidia cards? Yes. I have tested this on my latop with the NVIDIA propietary drivers. The only way to get an EGL software rasterizer is to build Mesa with support for DRI/DRM and then tell it to use the dri_based software rasterizer.
Created attachment 276734 [details] Patch Disable DRI3 explicitily on the mesa configure arguments (we only need DRI2 for swrast)
(In reply to comment #3) > Then running the tests with wayland may give errors depending on the wayland > version that your distro ships. I agree these should be built by default; it shouldn't cause any trouble for people not using Wayland, except for a minor increase in compile time.
Created attachment 276774 [details] Patch Trying to get the GTK+ EWS green: Mesa needs libudev.h to build DRI support. Add libudev-dev/systemd-devel to the script install-dependencies.
Created attachment 276833 [details] Patch Try to get the GTK+ EWS boot green: Building weston requires some extra build-deps that were not listed before: libgbm libmtdev libxcb-composite0 libinput and libevdev.
Created attachment 276840 [details] Patch Try to get the GTK+ EWS green: The weston build was failing now because it required libpam-dev for building weston-launch. So I just disabled weston-launch in the configure arguments because we dont need this helper. Also fixed a bug in WestonDriver::stop() (the Xvfb process were not correctly terminated).
Comment on attachment 276840 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=276840&action=review > Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py:108 > + option_group_definitions.append(("GTK-specific Options", [ > + optparse.make_option("--wayland", action="store_true", default=False, > + help="Run the layout tests inside a (virtualized) weston compositor."), > + ])) I'm not sure we should make a group for this, and this doesn't need to be GTK specific, EFL might run tests in wayland eventually. I think this is probably a Testing option that for now is GTK only, which could be documented in the help string instead. > Tools/Scripts/webkitpy/port/gtk.py:122 > + if self._driver_class() != XorgDriver and self._should_use_jhbuild(): This is now != XorgDriver because it should be done for both XvfbDriver and WestonDriver? If that is the case I would make it explicit and maybe we don't even need the comment above. Something like if self._driver_class() in [ XvfbDriver, WestonDriver] and ... > Tools/Scripts/webkitpy/port/gtk.py:188 > + if self._wayland: > + search_paths.append(self.port_name + "-wayland") Please, explain this too in the changelog > Tools/Scripts/webkitpy/port/westondriver.py:45 > + weston_findcmd = port._jhbuild_wrapper + weston_findcmd Too much indentation here? > Tools/Scripts/webkitpy/port/westondriver.py:88 > + def _xvfb_read_display_id(self, read_fd): > + import errno > + import select > + > + fd_set = [read_fd] > + while fd_set: > + try: > + fd_list = select.select(fd_set, [], [])[0] > + except select.error, e: > + if e.args[0] == errno.EINTR: > + continue > + raise > + > + if read_fd in fd_list: > + # We only expect a number, so first read should be enough. > + display_id = os.read(read_fd, 256).strip('\n') > + fd_set = [] > + > + return int(display_id) > + > + def _xvfb_run(self, environment): > + read_fd, write_fd = os.pipe() > + run_xvfb = ["Xvfb", "-displayfd", str(write_fd), "-screen", "0", "1024x768x24", "-nolisten", "tcp"] > + if self._port._should_use_jhbuild(): > + run_xvfb = self._port._jhbuild_wrapper + run_xvfb > + with open(os.devnull, 'w') as devnull: > + self._xvfb_process = self._port.host.executive.popen(run_xvfb, stderr=devnull, env=environment) > + display_id = self._xvfb_read_display_id(read_fd) > + > + os.close(read_fd) > + os.close(write_fd) > + > + return display_id It seems to me there's too much code duplicated here, I wonder if could make the weston driver inherit from the Xvfb driver or something like that.
Created attachment 280710 [details] Patch Patch for landing addressing reviewer comments. Will land it tomorrow if everything looks ok
Comment on attachment 280710 [details] Patch It breaks update-webkitgtk-libs for me, the GTK+ configure script fails because it can't link to pango. Let me attach config.log; it fails with "undefined reference to `wl_proxy_get_version`" and "undefined reference to `wl_proxy_marshal_constructor_versioned".
Created attachment 280727 [details] gtk+-3.16.4 config.log
FYI: the reason I was testing this was to see if run-layout-tests was going to work properly when run without the --wayland flag under Wayland; I'm hoping it works by default, without the need to pass anything extra.
(In reply to comment #11) > Comment on attachment 280710 [details] > Patch > > It breaks update-webkitgtk-libs for me, the GTK+ configure script fails > because it can't link to pango. Let me attach config.log; it fails with > "undefined reference to `wl_proxy_get_version`" and "undefined reference to > `wl_proxy_marshal_constructor_versioned". Have you retried this from a clean build state (WebKitBuild directory wiped)? ? Which version of fedora are you running? 23?
(In reply to comment #14) > Have you retried this from a clean build state (WebKitBuild directory > wiped)? ? Yes > Which version of fedora are you running? 23? 24 (just to make things difficult)
(In reply to comment #15) > (In reply to comment #14) > > Have you retried this from a clean build state (WebKitBuild directory > > wiped)? ? > > Yes > > > Which version of fedora are you running? 23? > > 24 (just to make things difficult) This seems the problem: configure:24351: result: -pthread -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/pango-1.0 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/cairo -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/glib-2.0 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib/glib-2.0/include -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/pixman-1 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/freetype2 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/libxml2 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/freetype2 -I/usr/include/libdrm -I/usr/include/libpng16 -L/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib -lpangocairo-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lcairo configure:24385: gcc -o conftest -Wno-error -Wall -pthread -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/pango-1.0 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/cairo -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/glib-2.0 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib/glib-2.0/include -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/pixman-1 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/freetype2 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/libxml2 -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/freetype2 -I/usr/include/libdrm -I/usr/include/libpng16 -L/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib conftest.c -L/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib -lpangocairo-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lcairo >&5 /usr/lib64/libEGL.so.1: undefined reference to `wl_proxy_get_version' /usr/lib64/libEGL.so.1: undefined reference to `wl_proxy_marshal_constructor_versioned' Your system libEGL.so.1 wants to resolve libwayland symbols that the version of wayland we are building doesn't provide. $ grep -r wl_proxy_get_version WebKitBuild/DependenciesGTK/Source/wayland-1.8.1/ # nothing The easy fix is to upgrade the version of wayland we provide, but is not really a fix. Is just a workaround. It may cause problems for some other people using older/different versions of libegl But I'm not sure if there is a fix for this. Right now I can't come with any good idea to fix this issue. I think we are hitting the limit of what we can bundle inside the JHBuild. We need to use the system libgl libraries if we want to display something in the desktop. And if this libraries depend on specific versions of other libraries we can't do much about this. So... not sure what to do
(In reply to comment #16) > (In reply to comment #15) > > (In reply to comment #14) > > > Have you retried this from a clean build state (WebKitBuild directory > > > wiped)? ? > > > > Yes > > > > > Which version of fedora are you running? 23? > > > > 24 (just to make things difficult) > > This seems the problem: > > configure:24351: result: -pthread > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/pango- > 1.0 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/cairo > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/glib- > 2.0 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib/glib-2.0/ > include > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/ > pixman-1 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/ > freetype2 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/ > libxml2 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/ > freetype2 -I/usr/include/libdrm -I/usr/include/libpng16 > -L/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib > -lpangocairo-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lcairo > configure:24385: gcc -o conftest -Wno-error -Wall -pthread > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/pango- > 1.0 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/cairo > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/glib- > 2.0 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib/glib-2.0/ > include > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/ > pixman-1 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/ > freetype2 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/ > libxml2 > -I/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/include/ > freetype2 -I/usr/include/libdrm -I/usr/include/libpng16 > -L/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib > conftest.c > -L/home/mcatanzaro/src/WebKit/WebKitBuild/DependenciesGTK/Root/lib > -lpangocairo-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lcairo >&5 > /usr/lib64/libEGL.so.1: undefined reference to `wl_proxy_get_version' > /usr/lib64/libEGL.so.1: undefined reference to > `wl_proxy_marshal_constructor_versioned' > > Your system libEGL.so.1 wants to resolve libwayland symbols that the version > of wayland we are building doesn't provide. > > > $ grep -r wl_proxy_get_version > WebKitBuild/DependenciesGTK/Source/wayland-1.8.1/ > # nothing > > The easy fix is to upgrade the version of wayland we provide, but is not > really a fix. Is just a workaround. It may cause problems for some other > people using older/different versions of libegl > > > But I'm not sure if there is a fix for this. Right now I can't come with any > good idea to fix this issue. > > I think we are hitting the limit of what we can bundle inside the JHBuild. > We need to use the system libgl libraries if we want to display something in > the desktop. And if this libraries depend on specific versions of other > libraries we can't do much about this. So... not sure what to do Maybe an idea is to install not only mesa, but also weston wayland and libdrm inside the softegl directory. I will try with something along this lines unless someone suggests a better idea.
(In reply to comment #17) [...] > Maybe an idea is to install not only mesa, but also weston wayland and > libdrm inside the softegl directory. I will try with something along this > lines unless someone suggests a better idea. Forget this, it is not going to work because we need to build webkit against a specific wayland version and we can't switch from one version to another at runtime. We can do this with the opengl libraries because they have an stable API and we can change the one loaded with a simple LD_LIBRARY_PATH variable at runtime. But not with wayland because the API from one version to other changes. So maybe we have to forget about bundling weston and wayland and use the distro ones hoping that are good enough.
(In reply to comment #18) > > So maybe we have to forget about bundling weston and wayland and use the > distro ones hoping that are good enough. Not bundling weston and wayland causes another problem: * GTK+ 3.16 (that is what we ship on the JHBuild) requires xdg_shell version 5 https://git.gnome.org/browse/gtk+/commit/?id=5889905d1d8d6889dfa388e7d320ca10e154efc0 gtk+ (master) $ git describe 5889905d1d8d6889dfa388e7d320ca10e154efc0 3.15.6-39-g5889905 * Weston < 1.7 provides xdg_shell version 4 and Weston > 1.7 version 5 https://cgit.freedesktop.org/wayland/weston/commit/?id=5ba1e1d1 weston (master) $ git describe 5ba1e1d1 1.7.0-34-g5ba1e1d * So we want xdg_shell version 5 (Weston >1.7), *but* Debian stable only has Weston 1.6 (xdg_shell version 4) This causes that the tests with wayland are completely broken. WebKit will start crashing like mad with this errors: [...] worker/39 editing/deleting/delete-ws-fixup-001.html crashed, (stderr lines): (WebKitTestRunner:42404): Gdk-CRITICAL **: gdk_screen_get_monitor_scale_factor: assertion 'monitor_num < gdk_screen_get_n_monitors (screen)' failed (WebKitTestRunner:42404): Gdk-ERROR **: wl_display@1: error 1: invalid arguments for xdg_shell@16.get_xdg_surface [...] More info: https://bugzilla.gnome.org/show_bug.cgi?id=748108 https://bugs.freedesktop.org/show_bug.cgi?id=90579 Possible ways forward: - Lower the GTK+ version we ship on the JHBuild. - Drop support for Debian stable. - Please let me know any other idea you may have :) Opinions?
(In reply to comment #19) > Possible ways forward: > > - Lower the GTK+ version we ship on the JHBuild. > - Drop support for Debian stable. > - Please let me know any other idea you may have :) > > Opinions? I will try to play with the jhbuild sysdeps feature and try to only build weston/wayland (as also libdrm and and libinput) only if the ones provided by the system are not new enough. I think this may the best way forward.
(In reply to comment #20) > I will try to play with the jhbuild sysdeps feature and try to only build > weston/wayland (as also libdrm and and libinput) only if the ones provided > by the system are not new enough. > > I think this may the best way forward. Aha.... This is easy, just add the pkg-config tag like so: <autotools id="wayland" autogenargs="--disable-documentation"> <pkg-config>wayland-client.pc</pkg-config> <!-- new line --> <dependencies> <dep package="libffi"/> </dependencies> <branch module="releases/wayland-1.8.1.tar.xz" version="1.8.1" repo="wayland.freedesktop.org" hash="sha256:f17c938d1c24fd0a10f650a623a2775d329db3168b5732e498b08388ec776fc8" md5sum="6e877877c3e04cfb865cfcd0733c9ab1"> </branch> </autotools> That's it, then jhbuild will use pkg-config to check against the version tag (1.8.1) and build it only if not already installed.
Created attachment 282181 [details] Patch New patch for landing. Fixes the issues with Fedora. I have successfully tested this both on Debian 8 and Fedora 24. Note: is needed to re-run the new Tools/gtk/install-dependencies before building the jhbuild after applying this. Will land it by Wednesday 29 if everything looks ok
Comment on attachment 282181 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=282181&action=review OK > Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py:286 > + help="Run the layout tests inside a (virtualized) weston compositor (GTK only)."), Is this option required to run the layout tests in Wayland, or is it only required if you want to run the layout tests in weston under X11? > Tools/Scripts/webkitpy/port/gtk.py:81 > return WestonDriver Perhaps if self._wayland or os.environ.get("WAYLAND_DISPLAY")? I don't really want to have to pass --wayland to make the tests work when it's obvious that we're in Wayland.
(In reply to comment #23) > Comment on attachment 282181 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=282181&action=review > > OK > > > Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py:286 > > + help="Run the layout tests inside a (virtualized) weston compositor (GTK only)."), > > Is this option required to run the layout tests in Wayland, or is it only > required if you want to run the layout tests in weston under X11? > It is required passing --wayland to run the tests in weston inside Xvfb. It don't matters if you are running an X11 desktop, a Weston desktop or even no desktop at all (for example on headless server via ssh). > > Tools/Scripts/webkitpy/port/gtk.py:81 > > return WestonDriver > > Perhaps if self._wayland or os.environ.get("WAYLAND_DISPLAY")? I don't > really want to have to pass --wayland to make the tests work when it's > obvious that we're in Wayland. Then I guess we need to implement a --x11 switch or similar, otherwise there is no way to run the tests inside X11 (Xvfb) when you are on wayland. Right?
(In reply to comment #24) > It is required passing --wayland to run the tests in weston inside Xvfb. > > It don't matters if you are running an X11 desktop, a Weston desktop or even > no desktop at all (for example on headless server via ssh). OK, this seems fine to me. No further objection. :)
Committed r202619: <http://trac.webkit.org/changeset/202619>
Committed r202622: <http://trac.webkit.org/changeset/202622>
(In reply to comment #27) > Committed r202622: <http://trac.webkit.org/changeset/202622> Just for the record: For buiding mesa is incorrect. The changelog should be for building weston instead. We can't use our own mesa headers because we don't install mesa after bulding it.