Bug 237992 – REGRESSION(r291201- r291193?): [ BigSur+ arm64 ] 6X api-tests dealing with crypto and credentials are consistently failing

• Status: RESOLVED
• Resolution: CONFIGURATION CHANGED
• Component: Tools / Tests

[Robert Jenner]

The following 6 api-tests started to consistently fail/crash/timeout on BigSur and above:

Failures:
TestWebKitAPI.SSLKeyGeneratorTest.DefaultTest
TestWebKitAPI.SerializedCryptoKeyWrapTest.DefaultWebCryptoMasterKey
TestWebKitAPI.SerializedCryptoKeyWrapTest.DeleteDefaultWebCryptoMasterKey

Timeouts:
TestWebKitAPI.WebKit.NavigationClientDefaultCrypto
TestWebKitAPI.Challenge.BasicPersistentCredential

Crash:
TestWebKitAPI.SerializedCryptoKeyWrapTest.SerializedCryptoKeyWrapUnwrap

HISTORY:
https://results.webkit.org/?suite=api-tests&suite=api-tests&suite=api-tests&suite=api-tests&suite=api-tests&suite=api-tests&test=TestWebKitAPI.Challenge.BasicPersistentCredential&test=TestWebKitAPI.SSLKeyGeneratorTest.DefaultTest&test=TestWebKitAPI.SerializedCryptoKeyWrapTest.DefaultWebCryptoMasterKey&test=TestWebKitAPI.SerializedCryptoKeyWrapTest.DeleteDefaultWebCryptoMasterKey&test=TestWebKitAPI.SerializedCryptoKeyWrapTest.SerializedCryptoKeyWrapUnwrap&test=TestWebKitAPI.WebKit.NavigationClientDefaultCrypto

According to the history they all began to consistently have problems at r291201. 

FAILURES:
    TestWebKitAPI.SSLKeyGeneratorTest.DefaultTest
        
        /Volumes/Data/worker/monterey-release/build/Tools/TestWebKitAPI/Tests/mac/SSLKeyGenerator.mm:100
        Value of: rawResult.isEmpty()
          Actual: true
        Expected: false
        
    TestWebKitAPI.SerializedCryptoKeyWrapTest.DefaultWebCryptoMasterKey
        Cannot store WebCrypto master key, error -25293
        Cannot store WebCrypto master key, error -25293
        
        /Volumes/Data/worker/monterey-release/build/Tools/TestWebKitAPI/Tests/WebCore/cocoa/SerializedCryptoKeyWrap.mm:50
        Value of: masterKey1
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/monterey-release/build/Tools/TestWebKitAPI/Tests/WebCore/cocoa/SerializedCryptoKeyWrap.mm:53
        Value of: masterKey2
          Actual: false
        Expected: true
        
    TestWebKitAPI.SerializedCryptoKeyWrapTest.DeleteDefaultWebCryptoMasterKey
        Cannot store WebCrypto master key, error -25293
        Cannot store WebCrypto master key, error -25293
        
        /Volumes/Data/worker/monterey-release/build/Tools/TestWebKitAPI/Tests/WebCore/cocoa/SerializedCryptoKeyWrap.mm:61
        Value of: masterKey1
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/monterey-release/build/Tools/TestWebKitAPI/Tests/WebCore/cocoa/SerializedCryptoKeyWrap.mm:62
        Value of: WebCore::deleteDefaultWebCryptoMasterKey()
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/monterey-release/build/Tools/TestWebKitAPI/Tests/WebCore/cocoa/SerializedCryptoKeyWrap.mm:65
        Value of: masterKey2
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/monterey-release/build/Tools/TestWebKitAPI/Tests/WebCore/cocoa/SerializedCryptoKeyWrap.mm:67
        Value of: masterKey1 != masterKey2
          Actual: false
        Expected: true
        

TIMEOUTS
    TestWebKitAPI.Challenge.BasicPersistentCredential
    TestWebKitAPI.WebKit.NavigationClientDefaultCrypto
        Cannot store WebCrypto master key, error -25293
        Cannot store WebCrypto master key, error -25293

CRASH

    TestWebKitAPI.SerializedCryptoKeyWrapTest.SerializedCryptoKeyWrapUnwrap
        Cannot store WebCrypto master key, error -25293

[Radar WebKit Bug Importer]

<rdar://problem/90401456>

[Robert Jenner]

I have not been able to reproduce any of the failures, timeouts, or crashes locally. However, I was able to reproduce them on one of the bots.

[Robert Jenner]

I have sin(In reply to Robert Jenner from comment #2)
> I have not been able to reproduce any of the failures, timeouts, or crashes
> locally. However, I was able to reproduce them on one of the bots.

I have since tried to reproduce again locally, and on the bot that I was able to reproduce on before, and I have been unsuccessful in reproducing any of the test failures, timeouts, or crashes.

[Robert Jenner]

I will also add that it only appears to be occurring on arm64 systems.

[Robert Jenner]

The history shows that these issues first occurred at r291194, but only on one bot. 

Then, they started consistently occurring on multiple queues at r291201. So I'm not certain what the cause is, and I can't reproduce it. But it appears to have started between r291193 and r2911201.

[Alexey Proskuryakov]

I suspect that keychain on the bot may have gotten into some bad state.