Bug 185628 – Test262-Runner: Update config list with some failing tests

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Leo Balter]

Test262-Runner: Update config list with some failing tests

[Leo Balter]

Created attachment 340360 [details]
Patch

[Leo Balter]

The config list is updated to match some slow / hanging tests that are currently failing due to the length used in Array or Array like objects.

This also adds matching bugs for known features that are still in progress and not enabled yet.

[EWS Watchlist]

Comment on attachment 340360 [details]
Patch

Attachment 340360 [details] did not pass win-ews (win):
Output: http://webkit-queues.webkit.org/results/7683760

New failing tests:
http/tests/security/contentSecurityPolicy/video-redirect-allowed2.html

[EWS Watchlist]

Created attachment 340382 [details]
Archive of layout-test-results from ews201 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews201  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit

[Michael Saboff]

Comment on attachment 340360 [details]
Patch

Is this still an accurate list of slow / hanging test?

r=me after updating the list and rebasing the patch.

[Leo Balter]

Created attachment 342750 [details]
Patch

[Leo Balter]

this list is still accurate, including for known failures with their proper tracking bugs.

My idea is to follow up, migrating tests from the expectations file to this list after we have a matching bug. When the bug is fixed, it should serve as an index to gradually remove the files from the list.

[Michael Saboff]

Comment on attachment 342750 [details]
Patch

r=me

[WebKit Commit Bot]

Comment on attachment 342750 [details]
Patch

Clearing flags on attachment: 342750

Committed r232853: <https://trac.webkit.org/changeset/232853>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/41139641>

[Dawei Fenton (:realdawei)]

(In reply to WebKit Commit Bot from comment #9)
> Comment on attachment 342750 [details]
> Patch
> 
> Clearing flags on attachment: 342750
> 
> Committed r232853: <https://trac.webkit.org/changeset/232853>


Seem we have quite a few Test262 failures after this patch

Leo, can you take a look?
https://build.webkit.org/builders/Apple%20Sierra%20Debug%20Test262%20%28Tests%29/builds/2895/steps/test262-test/logs/stdio

[Leo Balter]

Thanks for the report, David. 

I just noticed the Yaml module used for the test262-runner is bundling considering inline comments from the config list. So the skip list for the e.g. BigInt tests will capture the inline comment as well.

While I'm yet to fix that in the runner, I will send a quick patch to avoid the inline comments so the bot will be back to normal anytime soon

[Dawei Fenton (:realdawei)]

(In reply to Leo Balter from comment #13)
> Thanks for the report, David. 
> 
> I just noticed the Yaml module used for the test262-runner is bundling
> considering inline comments from the config list. So the skip list for the
> e.g. BigInt tests will capture the inline comment as well.
> 
> While I'm yet to fix that in the runner, I will send a quick patch to avoid
> the inline comments so the bot will be back to normal anytime soon

great thanks!