Bug 238908 - REBASLINE: [ Monterey wk2 ] 7 /paymentrequest/* tests are constant text failures
Summary: REBASLINE: [ Monterey wk2 ] 7 /paymentrequest/* tests are constant text failures
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P1 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2022-04-06 16:32 PDT by Dawn Morningstar
Modified: 2022-04-15 17:50 PDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dawn Morningstar 2022-04-06 16:32:27 PDT
http/tests/inspector/paymentrequest/payment-request-internal-properties.https.html
is a constant text failure on Monterey wk2 queues.

Regression point appears to be r291811

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Finspector%2Fpaymentrequest%2Fpayment-request-internal-properties.https.html

DIFF:
--- /Volumes/Data/worker/monterey-release-tests-wk2/build/layout-test-results/http/tests/inspector/paymentrequest/payment-request-internal-properties.https-expected.txt
+++ /Volumes/Data/worker/monterey-release-tests-wk2/build/layout-test-results/http/tests/inspector/paymentrequest/payment-request-internal-properties.https-actual.txt
@@ -1,4 +1,3 @@
-CONSOLE MESSAGE: WebKit currently uses the first shipping option even if other shipping options are marked as selected.
 Internal properties of a PaymentRequest.
 
 
DIFF-URL:
https://build.webkit.org/results/Apple-Monterey-Release-WK2-Tests/r291811%20(791)/http/tests/inspector/paymentrequest/payment-request-internal-properties.https-diff.txt
Comment 1 Radar WebKit Bug Importer 2022-04-06 16:33:52 PDT
<rdar://problem/91382499>
Comment 2 Dawn Morningstar 2022-04-06 16:59:46 PDT
When attempting to reproduce using run-webkit-tests:
Failure continues to happen at Monterey TOT
Comment 3 Dawn Morningstar 2022-04-06 17:08:47 PDT
Regression point may not be r291811 as history suggests. 

When attempting to reproduce with run-webkit-tests issue continues far before r291811...
Comment 4 Dawn Morningstar 2022-04-06 17:15:44 PDT

*** This bug has been marked as a duplicate of bug 238581 ***
Comment 5 Dawn Morningstar 2022-04-08 17:00:18 PDT
Reopening to manually resolve upstreaming issues.
Comment 6 Dawn Morningstar 2022-04-08 19:25:17 PDT
http/tests/paymentrequest/payment-address-attributes-and-toJSON-method.https.html
http/tests/inspector/paymentrequest/payment-request-internal-properties.https.html 
http/tests/paymentrequest/payment-request-change-shipping-option.https.html
http/tests/paymentrequest/payment-response-retry-method.https.html
http/tests/paymentrequest/updateWith-shippingOptions.https.html 

All appear to be suffering the same failure. Committing rebasline.
Comment 7 Dawn Morningstar 2022-04-08 19:29:38 PDT
Committed rebalance for 4/5 of these tests here:
r292672

updateWith-shippingOptions.https.html may need a fix and a rebasline as it appears to be an actual failing test.
Comment 8 Dawn Morningstar 2022-04-11 10:25:48 PDT
(In reply to Matteo Flores from comment #7)
> Committed rebalance for 4/5 of these tests here:
> r292672
> 
> updateWith-shippingOptions.https.html may need a fix and a rebasline as it
> appears to be an actual failing test.

Rebasline failed as baseline was correct for Big Sur, I will correct this.
Comment 9 Dawn Morningstar 2022-04-11 13:50:28 PDT
r292720 committed expectations until rebasline can be sorted.
Comment 10 Dawn Morningstar 2022-04-12 11:19:01 PDT
These two tests also appear to be related to the payment-request failures:

imported/w3c/web-platform-tests/payment-request/payment-request-constructor.https.sub.html
imported/w3c/web-platform-tests/payment-request/payment-request-shippingOption-attribute.https.html

HISTORY:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fpayment-request%2Fpayment-request-constructor.https.sub.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fpayment-request%2Fpayment-request-shippingOption-attribute.https.html
Comment 11 Dawn Morningstar 2022-04-13 14:28:56 PDT
This expectation update should resolve the console message: r292831, rebasline may still be needed afterward. Awaiting test results.
Comment 12 Dawn Morningstar 2022-04-14 17:29:17 PDT
Tests are nearly resolved. They were previously unintentionally activated on wk1. Issue has been fixed and last step is rebasline for Monterey.
Comment 13 Dawn Morningstar 2022-04-14 17:29:39 PDT
(In reply to Matteo Flores from comment #12)
> Tests are nearly resolved. They were previously unintentionally activated on
> wk1. Issue has been fixed and last step is rebasline for Monterey.

Fixed expectations: r292897
Comment 14 Dawn Morningstar 2022-04-15 17:50:17 PDT
Completed more rebaslines here: r292933