[wptrunner] Add a wpewebkit_minibrowser product/browser.

This adds a new browser type to run tests with the WPE WebKit port.
It can be passed an optional `--headless` parameter to enable
headless testing with WPT. Otherwise wayland is required.

A pre-built nightly bundle can be also automatically downloaded and
installed by passing `--install-browser` to the WPT runner.
It shares with webkitgtk_minibrowser most of the logic to download
and install the built product.

Documentation is also added and the webkitgtk_minibrowser one updated
accordingly. Unit tests are also added.

docs/running-tests/from-local-system.md

@@ -173,6 +173,7 @@ here](command-line-arguments.html#run).
   android_webview
   safari
   webkitgtk_minibrowser
+  wpewebkit_minibrowser
 ```
 
 ### Running in parallel

docs/running-tests/webkitgtk_minibrowser.md

@@ -1,22 +1,71 @@
 # WebKitGTK MiniBrowser
 
+To be able to run tests with the [WebKitGTK](https://webkitgtk.org/)
+MiniBrowser you need the following packages installed:
 
-To be able to run tests with the WebKitGTK MiniBrowser you need the
-following packages installed:
-
-* Fedora: `webkit2gtk3-devel`
-* Debian or Ubuntu: `webkit2gtk-driver`
-
+* Fedora: `webkitgtk6.0`
+* Debian or Ubuntu: `webkitgtk-driver` or `webkit2gtk-driver`
+* Arch: `webkitgtk-6.0`
 
 The WebKitGTK MiniBrowser is not installed on the default binary path.
 The `wpt` script will try to automatically locate it, but if you need
 to run it manually you can find it on any of this paths:
 
-* Fedora: `/usr/libexec/webkit2gtk-${VERSION}/MiniBrowser`
-* Debian or Ubuntu: `/usr/lib/x86_64-linux-gnu/webkit2gtk-${VERSION}/MiniBrowser`
-* Note:
-     * `VERSION` is `4.0` or `4.1`.
-     * If not Fedora and the machine architecture is not `x86_64`, then it will
-      be located inside:
-      `/usr/lib/${TRIPLET}/webkit2gtk-${VERSION}/MiniBrowser`
-      where `TRIPLET=$(gcc -dumpmachine)`
+* Fedora: `/usr/libexec/webkitgtk-${APIVERSION}/MiniBrowser`
+* Arch: `/usr/lib/webkitgtk-${APIVERSION}/MiniBrowser`
+* Debian or Ubuntu: `/usr/lib/${TRIPLET}/webkitgtk-${APIVERSION}/MiniBrowser`
+  * Note: `${TRIPLET}` is the output of the command `gcc -dumpmachine`
+
+# Nightly universal bundle
+
+Alternatively you can pass to `wpt` the flags `--install-browser --channel=nightly`
+and then `wpt` will automatically download the last bundle and unpack it on the
+default `wpt` working directory (usually subdir `_venv3/browsers` in your `wpt` checkout)
+Then it will use the unpacked `MiniBrowser` and `WebKitWebDriver` binaries to run the tests.
+
+This universal bundles should work on any Linux distribution as they include inside
+the tarball all the system libraries and resources needed to run WebKitGTK, from libc
+up to the Mesa graphics drivers without requiring the usage of containers.
+
+If you are not using open source graphics drivers (Mesa) and you experience issues
+with this bundle then a possible workaround is to try to run the tests headless
+inside a virtualized display like `Xvfb` (see command `xvfb-run -a` on Debian/Ubuntu).
+You can do this also from inside a virtual machine or docker container.
+
+# Headless mode
+
+WebKitGTK does not have a native headless mode, but you can workaround that
+by running the tests inside a virtualized display. For example you can use
+`weston` with the headless backend for a virtualized `Wayland` display,
+or you can use `Xvfb` for a virtualized `X11` display.
+
+Example:
+   ```
+   xvfb-run -a ./wpt run [more-options] webkitgtk_minibrowser [tests-to-run]
+   ```
+
+# Using a custom WebKitGTK build
+
+If you want to test with a custom WebKitGTK build the easiest way is that you
+install this build in a temporary directory and then tell wpt to run it from there.
+
+Steps:
+
+  1. Build WebKitGTK passing the arguments `-DENABLE_MINIBROWSER=ON -DCMAKE_INSTALL_PREFIX=/home/user/testdir_install`
+  2. Install it: `ninja install` (or `make install`)
+  3. Locate the `MiniBrowser` and `WebKitWebDriver` binaries under `/home/user/testdir_install`
+  4. Run `wpt` passing this two paths like this:
+  ```
+      ./wpt run --webdriver-binary=/home/user/testdir_install/bin/WebKitWebDriver \
+                --binary=/home/user/testdir_install/libexec/MiniBrowser \
+                [more-options] webkitgtk_minibrowser [tests-to-run]
+  ```
+
+ * Note: It is important that you build WebKitGTK against the libraries of your system.
+Do not build WebKitGTK inside Flatpak or other container unless you run `wpt` also
+from inside this container.
+
+# Running tests locally
+
+Is a good idea that you increase the verbosity of `wpt` by passing to it the flag `--log-mach=-`
+Also, please check the documentation about [Running Tests from the Local System](from-local-system).

docs/running-tests/wpewebkit_minibrowser.md

@@ -0,0 +1,67 @@
+# WPE WebKit MiniBrowser
+
+To be able to run tests with the [WPE WebKit](https://wpewebkit.org)
+MiniBrowser you need the following packages installed:
+
+* Fedora: N/A (build your own or use the nighly bundle)
+* Debian or Ubuntu: `wpewebkit-driver` and `libwpewebkit`
+* Arch:  `wpewebkit`
+
+The WPE WebKit MiniBrowser is not installed on the default binary path.
+The `wpt` script will try to automatically locate it, but if you need
+to run it manually you can find it on any of this paths:
+
+* Arch: `/usr/lib/wpe-webkit-${APIVERSION}/MiniBrowser`
+* Debian or Ubuntu: `/usr/lib/${TRIPLET}/wpe-webkit-${APIVERSION}/MiniBrowser`
+  * Note: `${TRIPLET}` is the output of the command `gcc -dumpmachine`
+
+# Nightly universal bundle
+
+Alternatively you can pass to `wpt` the flags `--install-browser --channel=nightly`
+and then `wpt` will automatically download the last bundle and unpack it on the
+default `wpt` working directory (usually subdir `_venv3/browsers` in your `wpt` checkout)
+Then it will use the unpacked `MiniBrowser` and `WPEWebDriver` binaries to run the tests.
+
+This universal bundles should work on any Linux distribution as they include inside
+the tarball all the system libraries and resources needed to run WebKitGTK, from libc
+up to the Mesa graphics drivers without requiring the usage of containers.
+
+If you are not using open source graphics drivers (Mesa) and you experience issues
+with this bundle then a possible workaround is to try to run the tests in headless
+mode, for that pass the flag `--headless` to `wpt`
+
+# Headless mode
+
+The WPE MiniBrowser needs a Wayland display to run, but if you don't have one
+or you want to enable headless mode you can pass the flag `--headless` to `wpt`.
+
+Example:
+   ```
+   ./wpt run [more-options] --headless wpewebkit_minibrowser [tests-to-run]
+   ```
+
+# Using a custom WPE WebKit build
+
+If you want to test with a custom WPE WebKit build the easiest way is that you
+install this build in a temporary directory and then tell wpt to run it from there.
+
+Steps:
+
+  1. Build WPE WebKit passing the arguments `-DENABLE_MINIBROWSER=ON -DCMAKE_INSTALL_PREFIX=/home/user/testdir_install'
+  2. Install it: `ninja install` (or `make install`)
+  3. Locate the `MiniBrowser` and `WPEWebDriver` binaries under /home/user/testdir_install`
+  4. Run `wpt` passing this two paths like this:
+  ```
+      ./wpt run --webdriver-binary=/home/user/testdir_install/bin/WPEWebDriver \
+                --binary=/home/user/testdir_install/libexec/MiniBrowser \
+                [more-options] webkitgtk_minibrowser [tests-to-run]
+  ```
+
+Note: It is important that you build WPE WebKit against the libraries of your system.
+Do not build WPE WebKit inside Flatpak or other container unless you run `wpt` also
+from inside this container.
+
+# Running tests locally
+
+Is a good idea that you increase the verbosity of `wpt` by passing to it the flag `--log-mach=-`
+Also, please check the documentation about [Running Tests from the Local System](from-local-system).

tools/wpt/browser.py

@@ -2374,17 +2374,34 @@ def version(self, binary=None, webdriver_binary=None):
             return f.read().strip()
 
 
-class WebKitGTKMiniBrowser(WebKit):
+class WebKitGlibBaseMiniBrowser(WebKit):
+    """WebKitGTK and WPE MiniBrowser specific interface (base class)."""
+
+    # This class is not meant to be used directly.
+    # And the class variables below should be defined on the subclasses.
+    BASE_DOWNLOAD_URI = ""
+    PORT_PRETTY_NAME = ""
+    WEBDRIVER_BINARY_NAME = ""
+    LIBEXEC_SUBDIR_PREFIXES = [""]
+    product = ""
+
+    def __init__(self, *args, **kwargs):
+        if self.__class__.__name__ == "WebKitGlibBaseMiniBrowser":
+            raise RuntimeError("class WebKitGlibBaseMiniBrowser should not be used directly, but subclassed")
+        for required_class_var in ["BASE_DOWNLOAD_URI", "PORT_PRETTY_NAME", "WEBDRIVER_BINARY_NAME", "LIBEXEC_SUBDIR_PREFIXES", "product"]:
+            class_var_value = getattr(self, required_class_var, "")
+            if all(len(i) == 0 for i in class_var_value):
+                raise NotImplementedError('subclass "%s" should define class variable "%s"' % (self.__class__.__name__, required_class_var))
+        return super().__init__(*args, **kwargs)
 
     def download(self, dest=None, channel=None, rename=None):
-        base_dowload_uri = "https://webkitgtk.org/built-products/"
-        base_download_dir = base_dowload_uri + platform.machine() + "/release/" + channel + "/MiniBrowser/"
+        base_download_dir = self.BASE_DOWNLOAD_URI + platform.machine() + "/release/" + channel + "/MiniBrowser/"
         try:
             response = get(base_download_dir + "LAST-IS")
         except requests.exceptions.HTTPError as e:
             if e.response.status_code == 404:
-                raise RuntimeError("Can't find a WebKitGTK MiniBrowser %s bundle for %s at %s"
-                                   % (channel, platform.machine(), base_dowload_uri))
+                raise RuntimeError("Can't find a %s MiniBrowser %s bundle for %s at %s"
+                                   % (self.PORT_PRETTY_NAME, channel, platform.machine(), self.BASE_DOWNLOAD_URI))
             raise
 
         bundle_filename = response.text.strip()
@@ -2393,7 +2410,7 @@ def download(self, dest=None, channel=None, rename=None):
         dest = self._get_browser_download_dir(dest, channel)
         bundle_file_path = os.path.join(dest, bundle_filename)
 
-        self.logger.info("Downloading WebKitGTK MiniBrowser bundle from %s" % bundle_url)
+        self.logger.info("Downloading %s MiniBrowser bundle from %s" % (self.PORT_PRETTY_NAME, bundle_url))
         with open(bundle_file_path, "w+b") as f:
             get_download_to_descriptor(f, bundle_url)
 
@@ -2405,13 +2422,13 @@ def download(self, dest=None, channel=None, rename=None):
 
         if bundle_expected_hash != bundle_computed_hash:
             self.logger.error("Calculated SHA256 hash is %s but was expecting %s" % (bundle_computed_hash, bundle_expected_hash))
-            raise RuntimeError("The WebKitGTK MiniBrowser bundle at %s has incorrect SHA256 hash." % bundle_file_path)
+            raise RuntimeError("The %s MiniBrowser bundle at %s has incorrect SHA256 hash." % (self.PORT_PRETTY_NAME, bundle_file_path))
         return bundle_file_path
 
     def install(self, dest=None, channel=None, prompt=True):
         dest = self._get_browser_binary_dir(dest, channel)
         bundle_path = self.download(dest, channel)
-        bundle_uncompress_directory = os.path.join(dest, "webkitgtk_minibrowser")
+        bundle_uncompress_directory = os.path.join(dest, self.product)
 
         # Clean it from previous runs
         if os.path.exists(bundle_uncompress_directory):
@@ -2425,59 +2442,62 @@ def install(self, dest=None, channel=None, prompt=True):
             elif ".tar." in bundle_file_name:
                 untar(f, bundle_uncompress_directory)
             else:
-                raise NotImplementedError("Unable to install WebKitGTK MiniBrowser bundle from file:" % bundle_file_name)
+                raise NotImplementedError("Don't know how to install the file: %s" % bundle_file_name)
         os.remove(bundle_path)
 
-        for expected_binary in ["MiniBrowser", "WebKitWebDriver"]:
+        for expected_binary in ["MiniBrowser", self.WEBDRIVER_BINARY_NAME]:
             binary_path = os.path.join(bundle_uncompress_directory, expected_binary)
             if not (os.path.isfile(binary_path) and os.access(binary_path, os.X_OK)):
                 raise RuntimeError("Can't find a %s binary at %s" % (expected_binary, binary_path))
 
         minibrowser_path = os.path.join(bundle_uncompress_directory, "MiniBrowser")
         version_str = subprocess.check_output([minibrowser_path, "--version"]).decode("utf-8").strip()
-        self.logger.info("WebKitGTK MiniBrowser bundle for channel %s installed: %s" % (channel, version_str))
+        self.logger.info("%s MiniBrowser bundle for channel %s installed: %s" % (self.PORT_PRETTY_NAME, channel, version_str))
         install_ok_file = os.path.join(bundle_uncompress_directory, ".installation-ok")
         open(install_ok_file, "w").close()  # touch
         return minibrowser_path
 
     def _find_executable_in_channel_bundle(self, binary, venv_path=None, channel=None):
         if venv_path:
             venv_base_path = self._get_browser_binary_dir(venv_path, channel)
-            bundle_dir = os.path.join(venv_base_path, "webkitgtk_minibrowser")
+            bundle_dir = os.path.join(venv_base_path, self.product)
             install_ok_file = os.path.join(bundle_dir, ".installation-ok")
             if os.path.isfile(install_ok_file):
-                return which(binary, path=bundle_dir)
+                return shutil.which(binary, path=bundle_dir)
         return None
 
     def find_binary(self, venv_path=None, channel=None):
         minibrowser_path = self._find_executable_in_channel_bundle("MiniBrowser", venv_path, channel)
         if minibrowser_path:
+            self.logger.info("Found %s MiniBrowser %s at path: %s" % (self.PORT_PRETTY_NAME, channel, minibrowser_path))
             return minibrowser_path
 
+        # Find MiniBrowser on the system which is usually installed on the libexec dir
         triplet = "x86_64-linux-gnu"
         # Try to use GCC to detect this machine triplet
-        gcc = which("gcc")
+        gcc = shutil.which("gcc")
         if gcc:
             try:
                 triplet = call(gcc, "-dumpmachine").strip()
             except subprocess.CalledProcessError:
                 pass
-
-        versions = ["4.0", "4.1"]
-        libexecpaths = []
-
-        for version in versions:
-            # Fedora paths.
-            libexecpaths.append(f"/usr/libexec/webkit2gtk-{version}")
-            # Debian/Ubuntu paths
-            libexecpaths.append(f"/usr/lib/{triplet}/webkit2gtk-{version}")
-
-        return which("MiniBrowser", path=os.pathsep.join(libexecpaths))
+        for libexec_dir in ["/usr/libexec", f"/usr/lib/{triplet}", "/usr/lib"]:
+            if os.path.isdir(libexec_dir):
+                for libexec_entry in sorted(os.listdir(libexec_dir), reverse=True):
+                    for libexec_subdir_prefix in self.LIBEXEC_SUBDIR_PREFIXES:
+                        if libexec_entry.startswith(libexec_subdir_prefix):
+                            minibrowser_candidate_path = os.path.join(libexec_dir, libexec_entry, 'MiniBrowser')
+                            if os.path.isfile(minibrowser_candidate_path) and os.access(minibrowser_candidate_path, os.X_OK):
+                                self.logger.info("Found %s MiniBrowser at path: %s" % (self.PORT_PRETTY_NAME, minibrowser_candidate_path))
+                                return minibrowser_candidate_path
+        return None
 
     def find_webdriver(self, venv_path=None, channel=None):
-        webdriver_path = self._find_executable_in_channel_bundle("WebKitWebDriver", venv_path, channel)
+        webdriver_path = self._find_executable_in_channel_bundle(self.WEBDRIVER_BINARY_NAME, venv_path, channel)
         if not webdriver_path:
-            webdriver_path = which("WebKitWebDriver")
+            webdriver_path = shutil.which(self.WEBDRIVER_BINARY_NAME)
+        if webdriver_path:
+            self.logger.info("Found %s WebDriver at path: %s" % (self.PORT_PRETTY_NAME, webdriver_path))
         return webdriver_path
 
     def version(self, binary=None, webdriver_binary=None):
@@ -2489,14 +2509,34 @@ def version(self, binary=None, webdriver_binary=None):
             return None
         # Example output: "WebKitGTK 2.26.1"
         if output:
-            m = re.match(r"WebKitGTK (.+)", output)
+            m = re.match(r"%s (.+)" % self.PORT_PRETTY_NAME, output)
             if not m:
                 self.logger.warning("Failed to extract version from: %s" % output)
                 return None
             return m.group(1)
         return None
 
 
+class WebKitGTKMiniBrowser(WebKitGlibBaseMiniBrowser):
+    """WebKitGTK MiniBrowser specific interface."""
+
+    BASE_DOWNLOAD_URI = "https://webkitgtk.org/built-products/"
+    PORT_PRETTY_NAME = "WebKitGTK"
+    WEBDRIVER_BINARY_NAME = "WebKitWebDriver"
+    LIBEXEC_SUBDIR_PREFIXES = ["webkitgtk", "webkit2gtk"]
+    product = "webkitgtk_minibrowser"
+
+
+class WPEWebKitMiniBrowser(WebKitGlibBaseMiniBrowser):
+    """WPE WebKit MiniBrowser specific interface."""
+
+    BASE_DOWNLOAD_URI = "https://wpewebkit.org/built-products/"
+    PORT_PRETTY_NAME = "WPE WebKit"
+    WEBDRIVER_BINARY_NAME = "WPEWebDriver"
+    LIBEXEC_SUBDIR_PREFIXES = ["wpe-webkit"]
+    product = "wpewebkit_minibrowser"
+
+
 class Epiphany(Browser):
     """Epiphany-specific interface."""
 

tools/wpt/install.py

@@ -14,6 +14,7 @@
     'safari': 'preview',
     'servo': 'nightly',
     'webkitgtk_minibrowser': 'nightly',
+    'wpewebkit_minibrowser': 'nightly',
     'wktr': 'main',
 }