Bug 179208 - Layout Test js/intl-datetimeformat.html is failing on High Sierra (GMT vs. UTC)
Summary: Layout Test js/intl-datetimeformat.html is failing on High Sierra (GMT vs. UTC)
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Andy VanWagoner
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-11-02 15:52 PDT by Matt Lewis
Modified: 2017-11-20 17:26 PST (History)
8 users (show)

See Also:


Attachments
Avoid testing against UTC text name since it can be ambiguous. (9.87 KB, patch)
2017-11-20 10:35 PST, Andy VanWagoner
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Lewis 2017-11-02 15:52:38 PDT
js/intl-datetimeformat.html is failing on High Sierra according to the dashboard:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=js%2Fintl-datetimeformat.html

While the test started failing on this build:
https://build.webkit.org/builders/Apple%20High%20Sierra%20Release%20WK2%20(Tests)/builds/888

it is most likely the cause. I have been able to reproduce the issue with revisions much farther back than the revisions associated with it.

Diff:
--- /Volumes/Data/slave/highsierra-release-tests-wk2/build/layout-test-results/js/intl-datetimeformat-expected.txt
+++ /Volumes/Data/slave/highsierra-release-tests-wk2/build/layout-test-results/js/intl-datetimeformat-actual.txt
@@ -375,9 +375,9 @@
 PASS Intl.DateTimeFormat('en', { timeZoneName:'name' }) threw exception RangeError: timeZoneName must be "short" or "long".
 PASS Intl.DateTimeFormat('en').resolvedOptions().timeZoneName is undefined
 PASS Intl.DateTimeFormat('en', { minute:'2-digit', hour:'numeric', timeZoneName:'short' }).resolvedOptions().timeZoneName is 'short'
-PASS Intl.DateTimeFormat('en', { minute:'2-digit', hour:'numeric', timeZoneName:'short', timeZone: 'UTC' }).format(0) is '12:00 AM GMT'
+FAIL Intl.DateTimeFormat('en', { minute:'2-digit', hour:'numeric', timeZoneName:'short', timeZone: 'UTC' }).format(0) should be 12:00 AM GMT. Was 12:00 AM UTC.
 PASS Intl.DateTimeFormat('pt-BR', { minute:'2-digit', hour:'numeric', timeZoneName:'long' }).resolvedOptions().timeZoneName is 'long'
-PASS ['00:00 GMT','00:00 Horário do Meridiano de Greenwich'].includes(Intl.DateTimeFormat('pt-BR', { minute:'2-digit', hour:'numeric', timeZoneName:'long', timeZone: 'UTC' }).format(0)) is true
+FAIL ['00:00 GMT','00:00 Horário do Meridiano de Greenwich'].includes(Intl.DateTimeFormat('pt-BR', { minute:'2-digit', hour:'numeric', timeZoneName:'long', timeZone: 'UTC' }).format(0)) should be true. Was false.
 PASS 
     var options = { weekday: "short", year: "numeric", month: "short", day: "numeric", hour: "numeric", minute: "numeric", second: "numeric" };
     var resolved = Intl.DateTimeFormat("ar", options).resolvedOptions();
@@ -2178,22 +2178,13 @@
 PASS new Intl.DateTimeFormat().formatToParts(Infinity) threw exception RangeError: date value is not finite in DateTimeFormat formatToParts().
 PASS new Intl.DateTimeFormat().formatToParts(-Infinity) threw exception RangeError: date value is not finite in DateTimeFormat formatToParts().
 PASS new Intl.DateTimeFormat().formatToParts(new Date(NaN)) threw exception RangeError: date value is not finite in DateTimeFormat formatToParts().
-PASS JSON.stringify(
+FAIL JSON.stringify(
   Intl.DateTimeFormat("pt-BR", {
     hour: "numeric", minute: "numeric", second: "numeric",
     year: "numeric", month: "numeric", day: "numeric",
     timeZoneName: "short", era: "short", timeZone: "UTC"
   }).formatToParts(0).filter((part) => (part.type !== "literal"))
-) is JSON.stringify([
-  {"type":"day","value":"1"},
-  {"type":"month","value":"1"},
-  {"type":"year","value":"1970"},
-  {"type":"era","value":"d.C."},
-  {"type":"hour","value":"00"},
-  {"type":"minute","value":"00"},
-  {"type":"second","value":"00"},
-  {"type":"timeZoneName","value":"GMT"}
-])
+) should be [{"type":"day","value":"1"},{"type":"month","value":"1"},{"type":"year","value":"1970"},{"type":"era","value":"d.C."},{"type":"hour","value":"00"},{"type":"minute","value":"00"},{"type":"second","value":"00"},{"type":"timeZoneName","value":"GMT"}]. Was [{"type":"day","value":"1"},{"type":"month","value":"1"},{"type":"year","value":"1970"},{"type":"era","value":"d.C."},{"type":"hour","value":"00"},{"type":"minute","value":"00"},{"type":"second","value":"00"},{"type":"timeZoneName","value":"UTC"}].
 PASS Intl.DateTimeFormat("ar", { weekday: "short", year: "numeric", month: "short", day: "numeric", hour: "numeric", minute: "numeric", second: "numeric" }).formatToParts() is an instance of Array
 PASS Intl.DateTimeFormat("ar", { weekday: "short", year: "numeric", month: "short", day: "numeric" }).formatToParts() is an instance of Array
 PASS Intl.DateTimeFormat("ar", { year: "numeric", month: "long", day: "numeric" }).formatToParts() is an instance of Array
@@ -2551,33 +2542,13 @@
 PASS Intl.DateTimeFormat("zh-TW", { month: "long", day: "numeric" }).formatToParts() is an instance of Array
 PASS Intl.DateTimeFormat("zh-TW", { hour: "numeric", minute: "numeric", second: "numeric" }).formatToParts() is an instance of Array
 PASS Intl.DateTimeFormat("zh-TW", { hour: "numeric", minute: "numeric" }).formatToParts() is an instance of Array
-PASS JSON.stringify(
+FAIL JSON.stringify(
   Intl.DateTimeFormat('en-US', {
     hour: "numeric", minute: "numeric", second: "numeric",
     year: "numeric", month: "long", day: "numeric", weekday: "long",
     timeZoneName: "short", era: "long", timeZone: "UTC"
   }).formatToParts(0)
-) is JSON.stringify([
-  {"type":"weekday","value":"Thursday"},
-  {"type":"literal","value":", "},
-  {"type":"month","value":"January"},
-  {"type":"literal","value":" "},
-  {"type":"day","value":"1"},
-  {"type":"literal","value":", "},
-  {"type":"year","value":"1970"},
-  {"type":"literal","value":" "},
-  {"type":"era","value":"Anno Domini"},
-  {"type":"literal","value":", "},
-  {"type":"hour","value":"12"},
-  {"type":"literal","value":":"},
-  {"type":"minute","value":"00"},
-  {"type":"literal","value":":"},
-  {"type":"second","value":"00"},
-  {"type":"literal","value":" "},
-  {"type":"dayPeriod","value":"AM"},
-  {"type":"literal","value":" "},
-  {"type":"timeZoneName","value":"GMT"}
-])
+) should be [{"type":"weekday","value":"Thursday"},{"type":"literal","value":", "},{"type":"month","value":"January"},{"type":"literal","value":" "},{"type":"day","value":"1"},{"type":"literal","value":", "},{"type":"year","value":"1970"},{"type":"literal","value":" "},{"type":"era","value":"Anno Domini"},{"type":"literal","value":", "},{"type":"hour","value":"12"},{"type":"literal","value":":"},{"type":"minute","value":"00"},{"type":"literal","value":":"},{"type":"second","value":"00"},{"type":"literal","value":" "},{"type":"dayPeriod","value":"AM"},{"type":"literal","value":" "},{"type":"timeZoneName","value":"GMT"}]. Was [{"type":"weekday","value":"Thursday"},{"type":"literal","value":", "},{"type":"month","value":"January"},{"type":"literal","value":" "},{"type":"day","value":"1"},{"type":"literal","value":", "},{"type":"year","value":"1970"},{"type":"literal","value":" "},{"type":"era","value":"Anno Domini"},{"type":"literal","value":", "},{"type":"hour","value":"12"},{"type":"literal","value":":"},{"type":"minute","value":"00"},{"type":"literal","value":":"},{"type":"second","value":"00"},{"type":"literal","value":" "},{"type":"dayPeriod","value":"AM"},{"type":"literal","value":" "},{"type":"timeZoneName","value":"UTC"}].
 PASS successfullyParsed is true
 
 TEST COMPLETE
Comment 1 Matt Lewis 2017-11-03 11:49:36 PDT
Marked as failing for now on High Sierra:
https://trac.webkit.org/changeset/224411/webkit/
Comment 2 Alexey Proskuryakov 2017-11-09 09:21:50 PST
This is the expected result going forward, updated in 2017c.1.0 time zone asset. We'll be getting this result in macOS High Sierra and in iOS 10+ whenever the update is installed (not sure if this works in simulator). Eventually, this data will be part of TZ data installed with the OS of course.

Cf. <rdar://problem/35415193>
Comment 3 Ryan Haddad 2017-11-16 10:53:27 PST
This test also runs as part of jsc-layout-tests, where I don't think we can add platform-specific expectation files.
Comment 4 Alexey Proskuryakov 2017-11-17 10:23:50 PST
A couple ideas:

- The test can be modified to accept either GMT or UTC.
- It can be skipped as a JSC test, I don't think that it provides much vakue being run twice. Normally, JSC tests are good to run as such because of all the different modes, but this one is unlikely to run differently in different modes.
Comment 5 Andy VanWagoner 2017-11-17 10:29:25 PST
Both should be considered correct. I think the test should be updated to accept either GMT or UTC.
Comment 6 Andy VanWagoner 2017-11-20 10:35:29 PST
Created attachment 327368 [details]
Avoid testing against UTC text name since it can be ambiguous.
Comment 7 Andy VanWagoner 2017-11-20 10:42:44 PST
Comment on attachment 327368 [details]
Avoid testing against UTC text name since it can be ambiguous.

When I got into updating the test it seemed like it would be harder to understand allowing both GMT and UTC variations. I thought it would be less fragile if we test against time zones that are less ambiguous.
Comment 8 Darin Adler 2017-11-20 17:05:16 PST
Comment on attachment 327368 [details]
Avoid testing against UTC text name since it can be ambiguous.

Seems OK. Too bad that the change in the operating system name for the time zone is visible as a difference in web browser behavior. Not great.
Comment 9 WebKit Commit Bot 2017-11-20 17:24:41 PST
Comment on attachment 327368 [details]
Avoid testing against UTC text name since it can be ambiguous.

Clearing flags on attachment: 327368

Committed r225059: <https://trac.webkit.org/changeset/225059>
Comment 10 WebKit Commit Bot 2017-11-20 17:24:42 PST
All reviewed patches have been landed.  Closing bug.
Comment 11 Radar WebKit Bug Importer 2017-11-20 17:26:03 PST
<rdar://problem/35650610>