Bug 179221 - [WinCairo] Add default port TestExpectations for wincairo
Summary: [WinCairo] Add default port TestExpectations for wincairo
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Basuke Suzuki
URL:
Keywords: InRadar
Depends on:
Blocks: 179216
  Show dependency treegraph
 
Reported: 2017-11-02 17:52 PDT by Basuke Suzuki
Modified: 2018-01-08 12:47 PST (History)
7 users (show)

See Also:


Attachments
added (8.01 MB, patch)
2017-12-18 15:48 PST, Basuke Suzuki
no flags Details | Formatted Diff | Diff
resend (8.01 MB, patch)
2017-12-19 12:30 PST, Basuke Suzuki
ews-watchlist: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews107 for mac-elcapitan-wk2 (2.55 MB, application/zip)
2017-12-19 14:40 PST, EWS Watchlist
no flags Details
another trial (14.33 KB, patch)
2017-12-21 13:38 PST, Basuke Suzuki
achristensen: review-
achristensen: commit-queue-
Details | Formatted Diff | Diff
Fix reviewed points (26.13 KB, patch)
2018-01-05 17:04 PST, Basuke Suzuki
no flags Details | Formatted Diff | Diff
FIx line ending (25.61 KB, patch)
2018-01-05 17:20 PST, Basuke Suzuki
no flags Details | Formatted Diff | Diff
Fix encoding (12.81 KB, patch)
2018-01-05 17:24 PST, Basuke Suzuki
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Basuke Suzuki 2017-11-02 17:52:44 PDT
It should be reflect the current WinCairo build and runtime options.
Comment 1 Basuke Suzuki 2017-12-18 15:48:03 PST
Created attachment 329695 [details]
added

TestExpectation and expected result files are added.
Comment 2 Basuke Suzuki 2017-12-19 12:30:12 PST
Created attachment 329793 [details]
resend
Comment 3 EWS Watchlist 2017-12-19 14:40:22 PST
Comment on attachment 329793 [details]
resend

Attachment 329793 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/5759108

New failing tests:
fast/mediastream/MediaStream-MediaElement-setObject-null.html
Comment 4 EWS Watchlist 2017-12-19 14:40:23 PST
Created attachment 329830 [details]
Archive of layout-test-results from ews107 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6
Comment 5 Basuke Suzuki 2017-12-21 13:38:34 PST
Created attachment 330058 [details]
another trial
Comment 6 Alex Christensen 2018-01-05 15:25:36 PST
Comment on attachment 330058 [details]
another trial

View in context: https://bugs.webkit.org/attachment.cgi?id=330058&action=review

> LayoutTests/platform/wincairo-win10/TestExpectations:2
> +# Copyright (C) 2007-2015 Apple Inc. All rights reserved.

2018

> LayoutTests/platform/wincairo-win10/TestExpectations:13
> +# 3.  Neither the name of Apple Inc. ("Apple") nor the names of

We're using the 2-clause copyright header for new files.

> LayoutTests/platform/wincairo-win10/TestExpectations:29
> +# These are the layout test expectations for Apple's Windows port of WebKit.

WinCairo

> LayoutTests/platform/wincairo-win10/TestExpectations:75
> +fast\animation [ Skip ]

Let's use / instead of \

> LayoutTests/platform/wincairo-win10/TestExpectations:97
> +# Following tests are labeled "Skip" because not working well and causes timedout

timeout
Comment 7 Konstantin Tokarev 2018-01-05 15:28:45 PST
You should use [ Skip ] only when test is too slow or breaks execution of other tests. Otherwise it's better to specify failure kind - Failure, ImageOnlyFailure, Creash, or some combination if result is flaky. In this case you will notice when after some patch previously failing test becomes crashy, or vice versa, starts to pass

As for Timeout vs Skip - one may argue that Timeout is better, but when you have too many Timeouts tests execution becomes painfully slow
Comment 8 Basuke Suzuki 2018-01-05 16:34:18 PST
(In reply to Alex Christensen from comment #6)
> Comment on attachment 330058 [details]
> another trial
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=330058&action=review
> 
> > LayoutTests/platform/wincairo-win10/TestExpectations:2
> > +# Copyright (C) 2007-2015 Apple Inc. All rights reserved.
> 
> 2018
> 
> > LayoutTests/platform/wincairo-win10/TestExpectations:13
> > +# 3.  Neither the name of Apple Inc. ("Apple") nor the names of
> 
> We're using the 2-clause copyright header for new files.

Actually it seems you guys don't add copyright headers to recent TestExpectations files. I will remove them also.



> > LayoutTests/platform/wincairo-win10/TestExpectations:29
> > +# These are the layout test expectations for Apple's Windows port of WebKit.
> 
> WinCairo

Got it.
 
> > LayoutTests/platform/wincairo-win10/TestExpectations:75
> > +fast\animation [ Skip ]
> 
> Let's use / instead of \
> 
> > LayoutTests/platform/wincairo-win10/TestExpectations:97
> > +# Following tests are labeled "Skip" because not working well and causes timedout
> 
> timeout

Right.
Comment 9 Basuke Suzuki 2018-01-05 16:42:56 PST
(In reply to Konstantin Tokarev from comment #7)
> You should use [ Skip ] only when test is too slow or breaks execution of
> other tests. Otherwise it's better to specify failure kind - Failure,
> ImageOnlyFailure, Creash, or some combination if result is flaky. In this
> case you will notice when after some patch previously failing test becomes
> crashy, or vice versa, starts to pass
> 
> As for Timeout vs Skip - one may argue that Timeout is better, but when you
> have too many Timeouts tests execution becomes painfully slow

There're so many timeout tests which prevent our tasks. This is very beginning of LayoutTests trial for WinCairo. We wanna start with this and turn them into right label as you described.
Comment 10 Basuke Suzuki 2018-01-05 17:04:57 PST
Created attachment 330610 [details]
Fix reviewed points
Comment 11 Basuke Suzuki 2018-01-05 17:20:58 PST
Created attachment 330618 [details]
FIx line ending
Comment 12 Basuke Suzuki 2018-01-05 17:24:48 PST
Created attachment 330619 [details]
Fix encoding
Comment 13 WebKit Commit Bot 2018-01-08 12:45:48 PST
Comment on attachment 330619 [details]
Fix encoding

Clearing flags on attachment: 330619

Committed r226527: <https://trac.webkit.org/changeset/226527>
Comment 14 WebKit Commit Bot 2018-01-08 12:45:51 PST
All reviewed patches have been landed.  Closing bug.
Comment 15 Radar WebKit Bug Importer 2018-01-08 12:47:10 PST
<rdar://problem/36356825>