Bug 179208 – Layout Test js/intl-datetimeformat.html is failing on High Sierra (GMT vs. UTC)

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Matt Lewis]

js/intl-datetimeformat.html is failing on High Sierra according to the dashboard:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=js%2Fintl-datetimeformat.html

While the test started failing on this build:
https://build.webkit.org/builders/Apple%20High%20Sierra%20Release%20WK2%20(Tests)/builds/888

it is most likely the cause. I have been able to reproduce the issue with revisions much farther back than the revisions associated with it.

Diff:
--- /Volumes/Data/slave/highsierra-release-tests-wk2/build/layout-test-results/js/intl-datetimeformat-expected.txt
+++ /Volumes/Data/slave/highsierra-release-tests-wk2/build/layout-test-results/js/intl-datetimeformat-actual.txt
@@ -375,9 +375,9 @@
 PASS Intl.DateTimeFormat('en', { timeZoneName:'name' }) threw exception RangeError: timeZoneName must be "short" or "long".
 PASS Intl.DateTimeFormat('en').resolvedOptions().timeZoneName is undefined
 PASS Intl.DateTimeFormat('en', { minute:'2-digit', hour:'numeric', timeZoneName:'short' }).resolvedOptions().timeZoneName is 'short'
-PASS Intl.DateTimeFormat('en', { minute:'2-digit', hour:'numeric', timeZoneName:'short', timeZone: 'UTC' }).format(0) is '12:00 AM GMT'
+FAIL Intl.DateTimeFormat('en', { minute:'2-digit', hour:'numeric', timeZoneName:'short', timeZone: 'UTC' }).format(0) should be 12:00 AM GMT. Was 12:00 AM UTC.
 PASS Intl.DateTimeFormat('pt-BR', { minute:'2-digit', hour:'numeric', timeZoneName:'long' }).resolvedOptions().timeZoneName is 'long'
-PASS ['00:00 GMT','00:00 Horário do Meridiano de Greenwich'].includes(Intl.DateTimeFormat('pt-BR', { minute:'2-digit', hour:'numeric', timeZoneName:'long', timeZone: 'UTC' }).format(0)) is true
+FAIL ['00:00 GMT','00:00 Horário do Meridiano de Greenwich'].includes(Intl.DateTimeFormat('pt-BR', { minute:'2-digit', hour:'numeric', timeZoneName:'long', timeZone: 'UTC' }).format(0)) should be true. Was false.
 PASS 
     var options = { weekday: "short", year: "numeric", month: "short", day: "numeric", hour: "numeric", minute: "numeric", second: "numeric" };
     var resolved = Intl.DateTimeFormat("ar", options).resolvedOptions();
@@ -2178,22 +2178,13 @@
 PASS new Intl.DateTimeFormat().formatToParts(Infinity) threw exception RangeError: date value is not finite in DateTimeFormat formatToParts().
 PASS new Intl.DateTimeFormat().formatToParts(-Infinity) threw exception RangeError: date value is not finite in DateTimeFormat formatToParts().
 PASS new Intl.DateTimeFormat().formatToParts(new Date(NaN)) threw exception RangeError: date value is not finite in DateTimeFormat formatToParts().
-PASS JSON.stringify(
+FAIL JSON.stringify(
   Intl.DateTimeFormat("pt-BR", {
     hour: "numeric", minute: "numeric", second: "numeric",
     year: "numeric", month: "numeric", day: "numeric",
     timeZoneName: "short", era: "short", timeZone: "UTC"
   }).formatToParts(0).filter((part) => (part.type !== "literal"))
-) is JSON.stringify([
-  {"type":"day","value":"1"},
-  {"type":"month","value":"1"},
-  {"type":"year","value":"1970"},
-  {"type":"era","value":"d.C."},
-  {"type":"hour","value":"00"},
-  {"type":"minute","value":"00"},
-  {"type":"second","value":"00"},
-  {"type":"timeZoneName","value":"GMT"}
-])
+) should be [{"type":"day","value":"1"},{"type":"month","value":"1"},{"type":"year","value":"1970"},{"type":"era","value":"d.C."},{"type":"hour","value":"00"},{"type":"minute","value":"00"},{"type":"second","value":"00"},{"type":"timeZoneName","value":"GMT"}]. Was [{"type":"day","value":"1"},{"type":"month","value":"1"},{"type":"year","value":"1970"},{"type":"era","value":"d.C."},{"type":"hour","value":"00"},{"type":"minute","value":"00"},{"type":"second","value":"00"},{"type":"timeZoneName","value":"UTC"}].
 PASS Intl.DateTimeFormat("ar", { weekday: "short", year: "numeric", month: "short", day: "numeric", hour: "numeric", minute: "numeric", second: "numeric" }).formatToParts() is an instance of Array
 PASS Intl.DateTimeFormat("ar", { weekday: "short", year: "numeric", month: "short", day: "numeric" }).formatToParts() is an instance of Array
 PASS Intl.DateTimeFormat("ar", { year: "numeric", month: "long", day: "numeric" }).formatToParts() is an instance of Array
@@ -2551,33 +2542,13 @@
 PASS Intl.DateTimeFormat("zh-TW", { month: "long", day: "numeric" }).formatToParts() is an instance of Array
 PASS Intl.DateTimeFormat("zh-TW", { hour: "numeric", minute: "numeric", second: "numeric" }).formatToParts() is an instance of Array
 PASS Intl.DateTimeFormat("zh-TW", { hour: "numeric", minute: "numeric" }).formatToParts() is an instance of Array
-PASS JSON.stringify(
+FAIL JSON.stringify(
   Intl.DateTimeFormat('en-US', {
     hour: "numeric", minute: "numeric", second: "numeric",
     year: "numeric", month: "long", day: "numeric", weekday: "long",
     timeZoneName: "short", era: "long", timeZone: "UTC"
   }).formatToParts(0)
-) is JSON.stringify([
-  {"type":"weekday","value":"Thursday"},
-  {"type":"literal","value":", "},
-  {"type":"month","value":"January"},
-  {"type":"literal","value":" "},
-  {"type":"day","value":"1"},
-  {"type":"literal","value":", "},
-  {"type":"year","value":"1970"},
-  {"type":"literal","value":" "},
-  {"type":"era","value":"Anno Domini"},
-  {"type":"literal","value":", "},
-  {"type":"hour","value":"12"},
-  {"type":"literal","value":":"},
-  {"type":"minute","value":"00"},
-  {"type":"literal","value":":"},
-  {"type":"second","value":"00"},
-  {"type":"literal","value":" "},
-  {"type":"dayPeriod","value":"AM"},
-  {"type":"literal","value":" "},
-  {"type":"timeZoneName","value":"GMT"}
-])
+) should be [{"type":"weekday","value":"Thursday"},{"type":"literal","value":", "},{"type":"month","value":"January"},{"type":"literal","value":" "},{"type":"day","value":"1"},{"type":"literal","value":", "},{"type":"year","value":"1970"},{"type":"literal","value":" "},{"type":"era","value":"Anno Domini"},{"type":"literal","value":", "},{"type":"hour","value":"12"},{"type":"literal","value":":"},{"type":"minute","value":"00"},{"type":"literal","value":":"},{"type":"second","value":"00"},{"type":"literal","value":" "},{"type":"dayPeriod","value":"AM"},{"type":"literal","value":" "},{"type":"timeZoneName","value":"GMT"}]. Was [{"type":"weekday","value":"Thursday"},{"type":"literal","value":", "},{"type":"month","value":"January"},{"type":"literal","value":" "},{"type":"day","value":"1"},{"type":"literal","value":", "},{"type":"year","value":"1970"},{"type":"literal","value":" "},{"type":"era","value":"Anno Domini"},{"type":"literal","value":", "},{"type":"hour","value":"12"},{"type":"literal","value":":"},{"type":"minute","value":"00"},{"type":"literal","value":":"},{"type":"second","value":"00"},{"type":"literal","value":" "},{"type":"dayPeriod","value":"AM"},{"type":"literal","value":" "},{"type":"timeZoneName","value":"UTC"}].
 PASS successfullyParsed is true
 
 TEST COMPLETE

[Matt Lewis]

Marked as failing for now on High Sierra:
https://trac.webkit.org/changeset/224411/webkit/

[Alexey Proskuryakov]

This is the expected result going forward, updated in 2017c.1.0 time zone asset. We'll be getting this result in macOS High Sierra and in iOS 10+ whenever the update is installed (not sure if this works in simulator). Eventually, this data will be part of TZ data installed with the OS of course.

Cf. <rdar://problem/35415193>

[Ryan Haddad]

This test also runs as part of jsc-layout-tests, where I don't think we can add platform-specific expectation files.

[Alexey Proskuryakov]

A couple ideas:

- The test can be modified to accept either GMT or UTC.
- It can be skipped as a JSC test, I don't think that it provides much vakue being run twice. Normally, JSC tests are good to run as such because of all the different modes, but this one is unlikely to run differently in different modes.

[Andy VanWagoner]

Both should be considered correct. I think the test should be updated to accept either GMT or UTC.

[Andy VanWagoner]

Created attachment 327368 [details]
Avoid testing against UTC text name since it can be ambiguous.

[Andy VanWagoner]

Comment on attachment 327368 [details]
Avoid testing against UTC text name since it can be ambiguous.

When I got into updating the test it seemed like it would be harder to understand allowing both GMT and UTC variations. I thought it would be less fragile if we test against time zones that are less ambiguous.

[Darin Adler]

Comment on attachment 327368 [details]
Avoid testing against UTC text name since it can be ambiguous.

Seems OK. Too bad that the change in the operating system name for the time zone is visible as a difference in web browser behavior. Not great.

[WebKit Commit Bot]

Comment on attachment 327368 [details]
Avoid testing against UTC text name since it can be ambiguous.

Clearing flags on attachment: 327368

Committed r225059: <https://trac.webkit.org/changeset/225059>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/35650610>