Bug 153752

Summary: LayoutTest http/tests/plugins/visible_plugins.html failing on Yosemite WK1
Product: WebKit Reporter: Ryan Haddad <ryanhaddad>
Component: WebKit Misc.Assignee: Brent Fulgham <bfulgham>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, bfulgham, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch ap: review+

Description Ryan Haddad 2016-02-01 10:29:58 PST 
LayoutTest http/tests/plugins/visible_plugins.html failing on Yosemite WK1

Started after r195950.

<https://build.webkit.org/results/Apple%20Yosemite%20Release%20WK1%20(Tests)/r195950%20(11677)/results.html>
<http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fplugins%2Fvisible_plugins.html>

--- /Volumes/Data/slave/yosemite-release-tests-wk1/build/layout-test-results/http/tests/plugins/visible_plugins-expected.txt
+++ /Volumes/Data/slave/yosemite-release-tests-wk1/build/layout-test-results/http/tests/plugins/visible_plugins-actual.txt
@@ -1,3 +1,3 @@
 Check we only display the valid set of "public" plugins.
 
-None
+Java Applet Plug-in
Comment 1 Radar WebKit Bug Importer 2016-02-01 17:20:09 PST 
<rdar://problem/24450314>
Comment 2 Ryan Haddad 2016-02-01 17:39:55 PST 
Skipped this test on Yosemite to get the bots back to green during investigation <https://trac.webkit.org/r195996>
Comment 3 Brent Fulgham 2016-02-01 19:49:46 PST 
This result is inconsistent on our various test machines because they seem to have a different set of plugins installed:
* Some have Java installed
* Some have Quicktime installed
* Some have neither

We need to decide on a consistent set of plugins and deploy them on all the test machines.
Comment 4 Alexey Proskuryakov 2016-02-01 21:59:38 PST 
I don;t think that this would be the correct response. Tests need to pass on engineers'  machines too, so they should pass regardless of installed plug-ins.
Comment 5 Brent Fulgham 2016-02-03 10:36:26 PST 
Created attachment 270588 [details]
Patch
Comment 6 Darin Adler 2016-02-03 13:11:42 PST 
Comment on attachment 270588 [details]
Patch

Normally it’s better to do something to make it clear that the test ran and we didn’t just hit a JavaScript exception. Could just use "PASS" or "FAIL: Unexpected ..." and then we’d notice that the PASS was missing if the test didn't even run.
Comment 7 Alexey Proskuryakov 2016-02-03 14:24:59 PST 
Comment on attachment 270588 [details]
Patch

I agree with Darin's comment, please do address it. That seems straightforward enough to not require additional review, so r=me.
Comment 8 Brent Fulgham 2016-02-03 15:01:25 PST 
Committed r196084: <http://trac.webkit.org/changeset/196084>
Comment 9 Brent Fulgham 2016-02-03 15:02:20 PST 
(In reply to comment #6)
> Comment on attachment 270588 [details]
> Patch
> 
> Normally it’s better to do something to make it clear that the test ran and
> we didn’t just hit a JavaScript exception. Could just use "PASS" or "FAIL:
> Unexpected ..." and then we’d notice that the PASS was missing if the test
> didn't even run.

Understood -- I'll revise to address that.