Bug 230705 - [GTK] Can't run performance tests due to a11y errors
Summary: [GTK] Can't run performance tests due to a11y errors
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Patrick Griffis
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-09-23 11:49 PDT by Sergio Villar Senin
Modified: 2022-02-22 01:26 PST (History)
6 users (show)

See Also:


Attachments
Patch (3.72 KB, patch)
2022-01-05 10:34 PST, Patrick Griffis
no flags Details | Formatted Diff | Diff
Patch (3.72 KB, patch)
2022-01-05 10:41 PST, Patrick Griffis
no flags Details | Formatted Diff | Diff
Disable a11y in WTR (1.07 KB, patch)
2022-02-18 01:03 PST, Carlos Garcia Campos
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Disable a11y in WTR (1.74 KB, patch)
2022-02-21 01:17 PST, Carlos Garcia Campos
svillar: review+
ews-feeder: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sergio Villar Senin 2021-09-23 11:49:45 PDT
$ Tools/Scripts/run-perf-tests PerformanceTests/Layout/flexbox-column-wrap.htmlRunning 1 testsRunning 
WebKit/PerformanceTests/Layout/flexbox-column-wrap.html (1 of 1)
error: WebKit/PerformanceTests/Layout/flexbox-column-wrap.html

(WebKitTestRunner:19): dbind-WARNING **: 11:46:41.779: Couldn't connect to accessibility bus: Failed to connect to socket /tmp/dbus-idgbDFkCBq: No such file or directory

(WebKitWebProcess:51): dbind-WARNING **: 11:46:41.934: Couldn't connect to accessibility bus: Failed to connect to socket /tmp/dbus-idgbDFkCBq: No such file or directory

FAILED
Finished: 5.210619 s
Comment 1 Sergio Villar Senin 2021-09-23 11:50:14 PDT
Might be related to the flatpak sandbox?
Comment 2 Philippe Normand 2021-09-23 12:07:02 PDT
I think the API test runner was recently patched to activate the a11y dbus stuff... Maybe something similar is needed here?
Comment 3 Carlos Garcia Campos 2021-09-24 02:00:28 PDT
No, this is a different error, it's indeed related to the sandbox, it happened to me always when apps using the sandbox where running, in my case evolution and devhelp. I debugged it, but didn't manage to find a solutiuon, however, it got fixed automagically after a sysem upgrade, so I assume the problem was in libdbus or some other related package. See this thread https://bugs.webkit.org/show_bug.cgi?id=229032#c3
Comment 4 Carlos Garcia Campos 2021-09-24 02:01:05 PDT
I don't know why that warning affects performance tests though
Comment 5 Sergio Villar Senin 2021-09-24 04:42:22 PDT
(In reply to Carlos Garcia Campos from comment #4)
> I don't know why that warning affects performance tests though

There should be some return value fooling the run-performance-tests tool
Comment 6 Carlos Garcia Campos 2021-09-27 03:53:46 PDT
(In reply to Sergio Villar Senin from comment #5)
> (In reply to Carlos Garcia Campos from comment #4)
> > I don't know why that warning affects performance tests though
> 
> There should be some return value fooling the run-performance-tests tool

or G_DEBUG=fatal_warnings in the test runner?
Comment 7 Patrick Griffis 2022-01-05 10:34:43 PST Comment hidden (obsolete)
Comment 8 Patrick Griffis 2022-01-05 10:35:46 PST
This patch doesn't fix all a11y issues but I believe this will fix the one mentioned here.

Also tested by lmoura as working but more testing would be appreciated.
Comment 9 Patrick Griffis 2022-01-05 10:41:21 PST
Created attachment 448406 [details]
Patch
Comment 10 Patrick Griffis 2022-01-05 13:05:53 PST
This fails on the runner because it uses `gdbus` but I also can't use pygobject's Gio bindings directly as that is also not installed.

I'm not familiar with the process of adding deps to the runner. We should add pygobject if possible as its generally useful.
Comment 11 Carlos Garcia Campos 2022-02-17 02:30:16 PST
Is this still a problem after the switch to ATSPI?
Comment 12 Sergio Villar Senin 2022-02-17 08:54:44 PST
(In reply to Carlos Garcia Campos from comment #11)
> Is this still a problem after the switch to ATSPI?

I've tried with r289814 and it's still failing
Comment 13 Carlos Garcia Campos 2022-02-18 01:03:32 PST
Created attachment 452495 [details]
Disable a11y in WTR

Sergio, could you try this patch? I really don't understand why it happens with perf tests and not layout tests, since both use WTR, but I think we can just disable a11y in WTR, because it's not really needed now with ATSPI.
Comment 14 Sergio Villar Senin 2022-02-18 02:29:32 PST
(In reply to Carlos Garcia Campos from comment #13)
> Created attachment 452495 [details]
> Disable a11y in WTR
> 
> Sergio, could you try this patch? I really don't understand why it happens
> with perf tests and not layout tests, since both use WTR, but I think we can
> just disable a11y in WTR, because it's not really needed now with ATSPI.

That patch worked for me, I got perf results.

NOTE: I still got 

** (WebKitWebProcess:251): WARNING **: 02:27:09.234: Can't connect to a11y bus: Could not connect: No such file or directory

but the test run fine
Comment 15 Carlos Garcia Campos 2022-02-18 03:57:37 PST
(In reply to Sergio Villar Senin from comment #14)
> (In reply to Carlos Garcia Campos from comment #13)
> > Created attachment 452495 [details]
> > Disable a11y in WTR
> > 
> > Sergio, could you try this patch? I really don't understand why it happens
> > with perf tests and not layout tests, since both use WTR, but I think we can
> > just disable a11y in WTR, because it's not really needed now with ATSPI.
> 
> That patch worked for me, I got perf results.
> 
> NOTE: I still got 
> 
> ** (WebKitWebProcess:251): WARNING **: 02:27:09.234: Can't connect to a11y
> bus: Could not connect: No such file or directory
> 
> but the test run fine

This comes from the MiniBrowser showing the results, which confirms the problem is not specific to the perf tests runner.

I think it's a good idea in any case to disable a11y in WTR, because it's not actually needed, but there's still a problem that I don't understand.
Comment 16 Carlos Garcia Campos 2022-02-21 01:17:25 PST
Created attachment 452719 [details]
Disable a11y in WTR
Comment 17 Carlos Garcia Campos 2022-02-22 01:24:35 PST
Committed r290293 (?): <https://commits.webkit.org/r290293>
Comment 18 Radar WebKit Bug Importer 2022-02-22 01:25:19 PST
<rdar://problem/89281902>
Comment 19 Carlos Garcia Campos 2022-02-22 01:26:53 PST
Comment on attachment 448406 [details]
Patch

Clearing flags. It's possible that we still need this patch, if that's the case we will handle it in a new bug.