Bug 252905

Summary: [GTK] Lots of WebGL2 tests are flaky
Product: WebKit Reporter: Carlos Alberto Lopez Perez <clopez>
Component: WebKitGTKAssignee: Nobody <webkit-unassigned>
Status: RESOLVED DUPLICATE    
Severity: Normal CC: alex, bugs-noreply
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=251106
https://bugs.webkit.org/show_bug.cgi?id=252906
https://bugs.webkit.org/show_bug.cgi?id=252878
https://bugs.webkit.org/show_bug.cgi?id=252909
Bug Depends on:    
Bug Blocks: 251107    

Carlos Alberto Lopez Perez
Reported 2023-02-24 07:47:24 PST
Since the WebGL2 tests were activated at 259348@main lot of them have started to cause unexpected errors. Some of this tests are really really flaky. To the point they are causing issues on the EWS bots. I have seen some runs where some of this WebGL2 tests fail with patch (a patch totally unrelated to this). When the bot repeats the failure 11 times the test fails 11 times Then the bot builds without patch And then the test passes 11 times Something weird is going on with this tests. It is not normal that they can give 11 times a failure on a row and then 11 times a pass on another row. It looks like if the flakiness was decided at build-time rather than a run-time (either you got a build that is going to fail always or always going to pass). So the result doesn't change until you rebuild or something like that. Just speculation on my part here. Example of runs where I saw this: https://ews-build.webkit.org/#/builders/35/builds/42754 https://ews-build.webkit.org/#/builders/35/builds/42758 https://ews-build.webkit.org/#/builders/35/builds/42761 https://ews-build.webkit.org/#/builders/35/builds/42768 https://ews-build.webkit.org/#/builders/35/builds/42780 https://ews-build.webkit.org/#/builders/35/builds/42809 https://ews-build.webkit.org/#/builders/35/builds/42818 https://ews-build.webkit.org/#/builders/35/builds/42838 https://ews-build.webkit.org/#/builders/35/builds/42876 https://ews-build.webkit.org/#/builders/35/builds/42883 https://ews-build.webkit.org/#/builders/35/builds/42907 https://ews-build.webkit.org/#/builders/35/builds/42917 https://ews-build.webkit.org/#/builders/35/builds/42933 https://ews-build.webkit.org/#/builders/35/builds/42934 https://ews-build.webkit.org/#/builders/35/builds/42953 https://ews-build.webkit.org/#/builders/35/builds/42969 https://ews-build.webkit.org/#/builders/35/builds/42977 https://ews-build.webkit.org/#/builders/35/builds/42978 https://ews-build.webkit.org/#/builders/35/builds/42993 https://ews-build.webkit.org/#/builders/35/builds/43002 https://ews-build.webkit.org/#/builders/35/builds/43010 https://ews-build.webkit.org/#/builders/35/builds/43044 https://ews-build.webkit.org/#/builders/35/builds/43050 https://ews-build.webkit.org/#/builders/35/builds/43054 https://ews-build.webkit.org/#/builders/35/builds/43066 https://ews-build.webkit.org/#/builders/35/builds/43072 https://ews-build.webkit.org/#/builders/35/builds/43083 https://ews-build.webkit.org/#/builders/35/builds/43089 https://ews-build.webkit.org/#/builders/35/builds/43121 I initially gardened some of this at 260776@main but I missed a few them. I'm going to add all of the ones that I detected are failing in an upcoming gardening patch On WPE the issue happens also but much less often than in GTK.
Attachments
Add attachment proposed patch, testcase, etc.
Carlos Alberto Lopez Perez
Comment 1 2023-02-24 07:52:30 PST
Here is a report that I generated with wktesthunter for the flakiness of this tests since they were enabled on 259348@main until 260773@main The bot gtk-release completed 235 runs on the interval [259349@main-260773@main] Frecuency of results for test: fast/canvas/webgl/webgl2-texture-upload-enums.html 231 times [98.3%] -> PASS 4 times [1.7%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/extensions/promoted-extensions-in-shaders.html 186 times [81.6%] -> PASS 42 times [18.4%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/query/occlusion-query.html 161 times [70.6%] -> TEXT 55 times [24.1%] -> TIMEOUT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/reading/format-r11f-g11f-b10f.html 71 times [31.1%] -> TEXT 157 times [68.9%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html 228 times [100%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/reading/read-pixels-pack-parameters.html 70 times [30.7%] -> TEXT 158 times [69.3%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/renderbuffers/multisample-with-full-sample-counts.html 158 times [69.3%] -> PASS 70 times [30.7%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/blitframebuffer-filter-srgb.html 171 times [75%] -> PASS 57 times [25%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/blitframebuffer-multisampled-readbuffer.html 171 times [75%] -> PASS 57 times [25%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/blitframebuffer-outside-readbuffer.html 171 times [75%] -> PASS 57 times [25%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/blitframebuffer-stencil-only.html 190 times [83.3%] -> PASS 38 times [16.7%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/blitframebuffer-test.html 190 times [83.3%] -> PASS 38 times [16.7%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/clear-srgb-color-buffer.html 171 times [75%] -> PASS 57 times [25%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/draw-buffers.html 171 times [75%] -> PASS 57 times [25%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/element-index-uint.html 170 times [74.9%] -> PASS 57 times [25.1%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/fs-color-type-mismatch-color-buffer-type.html 227 times [100%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/instanced-arrays.html 170 times [74.9%] -> PASS 57 times [25.1%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/rendering/out-of-bounds-index-buffers-after-copying.html 170 times [74.9%] -> PASS 57 times [25.1%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/state/gl-object-get-calls.html 191 times [83.8%] -> PASS 37 times [16.2%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-r11f_g11f_b10f-rgb-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-r11f_g11f_b10f-rgb-half_float.html 205 times [90.3%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-r11f_g11f_b10f-rgb-unsigned_int_10f_11f_11f_rev.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-r16f-red-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-r16f-red-half_float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-r32f-red-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-r8-red-unsigned_byte.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rg16f-rg-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rg16f-rg-half_float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rg32f-rg-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rg8-rg-unsigned_byte.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb16f-rgb-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb16f-rgb-half_float.html 206 times [90.7%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb32f-rgb-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb565-rgb-unsigned_byte.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb565-rgb-unsigned_short_5_6_5.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb5_a1-rgba-unsigned_byte.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb5_a1-rgba-unsigned_short_5_5_5_1.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb8-rgb-unsigned_byte.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb9_e5-rgb-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgb9_e5-rgb-half_float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgba16f-rgba-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgba16f-rgba-half_float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgba32f-rgba-float.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgba4-rgba-unsigned_byte.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgba4-rgba-unsigned_short_4_4_4_4.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-rgba8-rgba-unsigned_byte.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-srgb8_alpha8-rgba-unsigned_byte.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image_bitmap_from_video/tex-2d-srgb8-rgb-unsigned_byte.html 207 times [91.2%] -> TEXT 20 times [8.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/misc/tex-input-validation.html 191 times [84.1%] -> PASS 36 times [15.9%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/misc/tex-mipmap-levels.html 213 times [93.8%] -> TEXT 14 times [6.2%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/misc/tex-new-formats.html 213 times [93.8%] -> TEXT 14 times [6.2%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/misc/tex-srgb-mipmap.html 213 times [93.8%] -> TEXT 14 times [6.2%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/misc/tex-storage-2d.html 213 times [93.8%] -> TEXT 14 times [6.2%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/misc/texture-npot.html 213 times [93.8%] -> TEXT 14 times [6.2%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-r11f_g11f_b10f-rgb-float.html 30 times [13.2%] -> PASS 196 times [86.3%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-r11f_g11f_b10f-rgb-half_float.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-r11f_g11f_b10f-rgb-unsigned_int_10f_11f_11f_rev.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-r16f-red-float.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-r16f-red-half_float.html 30 times [13.2%] -> PASS 196 times [86.3%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-r32f-red-float.html 30 times [13.2%] -> PASS 196 times [86.3%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-r8-red-unsigned_byte.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rg16f-rg-float.html 30 times [13.2%] -> PASS 195 times [85.9%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rg16f-rg-half_float.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rg32f-rg-float.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rg8-rg-unsigned_byte.html 30 times [13.2%] -> PASS 196 times [86.3%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb16f-rgb-float.html 29 times [12.8%] -> PASS 196 times [86.3%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb16f-rgb-half_float.html 29 times [12.8%] -> PASS 198 times [87.2%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb32f-rgb-float.html 29 times [12.8%] -> PASS 197 times [87.2%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb565-rgb-unsigned_byte.html 29 times [12.8%] -> PASS 196 times [86.7%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb565-rgb-unsigned_short_5_6_5.html 29 times [12.8%] -> PASS 197 times [87.2%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb5_a1-rgba-unsigned_byte.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb5_a1-rgba-unsigned_short_5_5_5_1.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb8-rgb-unsigned_byte.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb9_e5-rgb-float.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgb9_e5-rgb-half_float.html 30 times [13.2%] -> PASS 197 times [86.8%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgba16f-rgba-float.html 29 times [12.8%] -> PASS 197 times [87.2%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgba16f-rgba-half_float.html 29 times [12.8%] -> PASS 197 times [87.2%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgba32f-rgba-float.html 29 times [12.8%] -> PASS 197 times [87.2%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgba4-rgba-unsigned_byte.html 29 times [12.8%] -> PASS 196 times [86.7%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgba4-rgba-unsigned_short_4_4_4_4.html 29 times [12.8%] -> PASS 197 times [87.2%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-rgba8-rgba-unsigned_byte.html 29 times [12.8%] -> PASS 197 times [87.2%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-srgb8_alpha8-rgba-unsigned_byte.html 29 times [12.8%] -> PASS 197 times [87.2%] -> TEXT -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/video/tex-2d-srgb8-rgb-unsigned_byte.html 29 times [12.8%] -> PASS 197 times [87.2%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/transform_feedback/transform_feedback.html 200 times [88.1%] -> TEXT 25 times [11.0%] -> TIMEOUT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/vertex_arrays/vertex-array-object.html 174 times [76.7%] -> PASS 53 times [23.3%] -> TEXT -- The bot gtk-release completed 231 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/buffers/buffer-uninitialized.html 162 times [70.1%] -> TEXT 69 times [29.9%] -> PASS -- The bot gtk-release completed 230 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/context/premultiplyalpha-test.html 230 times [100%] -> TEXT -- The bot gtk-release completed 230 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/oes-texture-float.html 217 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 230 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/oes-texture-float-with-video.html 217 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/oes-texture-half-float.html 215 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 230 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/oes-texture-half-float-linear.html 217 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/oes-texture-half-float-with-video.html 215 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/oes-vertex-array-object-bufferData.html 215 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/oes-vertex-array-object.html 215 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/webgl-compressed-texture-s3tc.html 215 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/webgl-compressed-texture-s3tc-srgb.html 215 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/webgl-debug-shaders.html 215 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html 214 times [94.3%] -> TIMEOUT 13 times [5.7%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/webgl-draw-buffers-max-draw-buffers.html 215 times [94.3%] -> TEXT 13 times [5.7%] -> PASS -- The bot gtk-release completed 231 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/glsl/bugs/conditional-discard-in-loop.html 231 times [100%] -> TEXT -- The bot gtk-release completed 230 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/limits/gl-min-textures.html 227 times [98.7%] -> PASS 3 times [1.3%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/rendering/gl-scissor-test.html 228 times [100%] -> TEXT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgba-rgba-unsigned_byte.html 186 times [81.6%] -> TEXT 42 times [18.4%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html 186 times [81.6%] -> TEXT 42 times [18.4%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html 186 times [81.6%] -> TEXT 42 times [18.4%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgb-rgb-unsigned_byte.html 186 times [81.6%] -> TEXT 42 times [18.4%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgb-rgb-unsigned_short_5_6_5.html 186 times [81.6%] -> TEXT 42 times [18.4%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/video/tex-2d-rgba-rgba-unsigned_byte.html 190 times [83.3%] -> TEXT 38 times [16.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/video/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html 190 times [83.3%] -> TEXT 38 times [16.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/video/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html 190 times [83.3%] -> TEXT 38 times [16.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/video/tex-2d-rgb-rgb-unsigned_byte.html 189 times [82.9%] -> TEXT 38 times [16.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/video/tex-2d-rgb-rgb-unsigned_short_5_6_5.html 190 times [83.3%] -> TEXT 38 times [16.7%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.y/conformance2/extensions/ext-texture-norm16.html 117 times [51.5%] -> PASS 110 times [48.5%] -> TEXT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.y/conformance2/transform_feedback/transform_feedback.html 195 times [85.9%] -> TEXT 28 times [12.3%] -> TIMEOUT The bot gtk-release completed 233 runs on the interval [259349@main-260773@main] Frecuency of results for test: fast/canvas/webgl/webgl2-getActiveUniforms.html 222 times [95.3%] -> TEXT PASS 2 times [0.9%] -> NOERROR 9 times [3.9%] -> TEXT -- The bot gtk-release completed 225 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/extensions/ext-color-buffer-float.html 183 times [81.3%] -> NOERROR 41 times [18.2%] -> TEXT PASS 1 times [0.4%] -> TEXT -- The bot gtk-release completed 225 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/extensions/ext-color-buffer-half-float.html 183 times [81.3%] -> NOERROR 41 times [18.2%] -> TEXT PASS 1 times [0.4%] -> TEXT The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/canvas_sub_rectangle/tex-2d-rgb565-rgb-unsigned_short_5_6_5.html 226 times [99.6%] -> PASS 1 times [0.4%] -> TIMEOUT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/canvas_sub_rectangle/tex-2d-rgb9_e5-rgb-half_float.html 226 times [99.6%] -> PASS 1 times [0.4%] -> TIMEOUT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/canvas_sub_rectangle/tex-2d-srgb8_alpha8-rgba-unsigned_byte.html 227 times [100%] -> PASS -- The bot gtk-release completed 226 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/canvas_sub_rectangle/tex-3d-rgb565-rgb-unsigned_byte.html 226 times [100%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/image/tex-2d-r8-red-unsigned_byte.html 46 times [20.2%] -> TIMEOUT 182 times [79.8%] -> PASS -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance2/textures/webgl_canvas/tex-2d-r8-red-unsigned_byte.html 141 times [62.1%] -> PASS 86 times [37.9%] -> TIMEOUT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/extensions/oes-texture-half-float-with-canvas.html 215 times [94.3%] -> TIMEOUT 13 times [5.7%] -> PASS -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/svg_image/tex-2d-rgb-rgb-unsigned_byte.html 220 times [96.5%] -> PASS 8 times [3.5%] -> TIMEOUT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/textures/webgl_canvas/tex-2d-rgb-rgb-unsigned_byte.html 220 times [96.5%] -> PASS 8 times [3.5%] -> TIMEOUT -- The bot gtk-release completed 228 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.0/conformance/uniforms/gl-uniform-arrays.html 204 times [89.5%] -> PASS 24 times [10.5%] -> TIMEOUT -- The bot gtk-release completed 227 runs on the interval [259349@main-260773@main] Frecuency of results for test: webgl/2.0.y/conformance2/state/gl-object-get-calls.html 131 times [57.7%] -> PASS 96 times [42.3%] -> TIMEOUT
Carlos Alberto Lopez Perez
Comment 2 2023-02-24 08:41:07 PST
(In reply to Carlos Alberto Lopez Perez from comment #0) > > Something weird is going on with this tests. > It is not normal that they can give 11 times a failure on a row and then 11 > times a pass on another row. > It looks like if the flakiness was decided at build-time rather than a > run-time (either you got a build that is going to fail always or always > going to pass). So the result doesn't change until you rebuild or something > like that. Just speculation on my part here. > Could be something also related to clean build issues like: the build-only bot is creating builds that fail because a bug that triggers due to a continous-build issue but the test-only bot creates build that work because it is doing continous builds from a further point. In any case the gardening done here is just based on the info from the post-commit bots (not the EWS) ones so I believe there is something else going here than just a 'clean build' issue. I also checked the info from other post-commit bots like the skip-failing tests one and they report same issue (lots of flakies).
Carlos Alberto Lopez Perez
Comment 3 2023-02-24 10:34:22 PST
Closing as duplicated. Let's use bug 251107 to track the flakies and failures for WebGL2 tests *** This bug has been marked as a duplicate of bug 251107 ***
Note You need to log in before you can comment on or make changes to this bug.