Bug 179221 – [WinCairo] Add default port TestExpectations for wincairo

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Basuke Suzuki]

It should be reflect the current WinCairo build and runtime options.

[Basuke Suzuki]

Created attachment 329695 [details]
added

TestExpectation and expected result files are added.

[Basuke Suzuki]

Created attachment 329793 [details]
resend

[EWS Watchlist]

Comment on attachment 329793 [details]
resend

Attachment 329793 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/5759108

New failing tests:
fast/mediastream/MediaStream-MediaElement-setObject-null.html

[EWS Watchlist]

Created attachment 329830 [details]
Archive of layout-test-results from ews107 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6

[Basuke Suzuki]

Created attachment 330058 [details]
another trial

[Alex Christensen]

Comment on attachment 330058 [details]
another trial

View in context: https://bugs.webkit.org/attachment.cgi?id=330058&action=review

> LayoutTests/platform/wincairo-win10/TestExpectations:2
> +# Copyright (C) 2007-2015 Apple Inc. All rights reserved.

2018

> LayoutTests/platform/wincairo-win10/TestExpectations:13
> +# 3.  Neither the name of Apple Inc. ("Apple") nor the names of

We're using the 2-clause copyright header for new files.

> LayoutTests/platform/wincairo-win10/TestExpectations:29
> +# These are the layout test expectations for Apple's Windows port of WebKit.

WinCairo

> LayoutTests/platform/wincairo-win10/TestExpectations:75
> +fast\animation [ Skip ]

Let's use / instead of \

> LayoutTests/platform/wincairo-win10/TestExpectations:97
> +# Following tests are labeled "Skip" because not working well and causes timedout

timeout

[Konstantin Tokarev]

You should use [ Skip ] only when test is too slow or breaks execution of other tests. Otherwise it's better to specify failure kind - Failure, ImageOnlyFailure, Creash, or some combination if result is flaky. In this case you will notice when after some patch previously failing test becomes crashy, or vice versa, starts to pass

As for Timeout vs Skip - one may argue that Timeout is better, but when you have too many Timeouts tests execution becomes painfully slow

[Basuke Suzuki]

(In reply to Alex Christensen from comment #6)
> Comment on attachment 330058 [details]
> another trial
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=330058&action=review
> 
> > LayoutTests/platform/wincairo-win10/TestExpectations:2
> > +# Copyright (C) 2007-2015 Apple Inc. All rights reserved.
> 
> 2018
> 
> > LayoutTests/platform/wincairo-win10/TestExpectations:13
> > +# 3.  Neither the name of Apple Inc. ("Apple") nor the names of
> 
> We're using the 2-clause copyright header for new files.

Actually it seems you guys don't add copyright headers to recent TestExpectations files. I will remove them also.



> > LayoutTests/platform/wincairo-win10/TestExpectations:29
> > +# These are the layout test expectations for Apple's Windows port of WebKit.
> 
> WinCairo

Got it.
 
> > LayoutTests/platform/wincairo-win10/TestExpectations:75
> > +fast\animation [ Skip ]
> 
> Let's use / instead of \
> 
> > LayoutTests/platform/wincairo-win10/TestExpectations:97
> > +# Following tests are labeled "Skip" because not working well and causes timedout
> 
> timeout

Right.

[Basuke Suzuki]

(In reply to Konstantin Tokarev from comment #7)
> You should use [ Skip ] only when test is too slow or breaks execution of
> other tests. Otherwise it's better to specify failure kind - Failure,
> ImageOnlyFailure, Creash, or some combination if result is flaky. In this
> case you will notice when after some patch previously failing test becomes
> crashy, or vice versa, starts to pass
> 
> As for Timeout vs Skip - one may argue that Timeout is better, but when you
> have too many Timeouts tests execution becomes painfully slow

There're so many timeout tests which prevent our tasks. This is very beginning of LayoutTests trial for WinCairo. We wanna start with this and turn them into right label as you described.

[Basuke Suzuki]

Created attachment 330610 [details]
Fix reviewed points

[Basuke Suzuki]

Created attachment 330618 [details]
FIx line ending

[Basuke Suzuki]

Created attachment 330619 [details]
Fix encoding

[WebKit Commit Bot]

Comment on attachment 330619 [details]
Fix encoding

Clearing flags on attachment: 330619

Committed r226527: <https://trac.webkit.org/changeset/226527>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/36356825>