Bug 203281 – [ews] iOS-13-Simulator-WK2-Tests-EWS is failing with an KeyError exception

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Aakash Jain]

iOS-13-Simulator-WK2-Tests-EWS is failing with an KeyError exception. This issue seems to have started today (with https://ews-build.webkit.org/#/builders/24/builds/2281)

e.g.: https://ews-build.webkit.org/#/builders/24/builds/2298/steps/9/logs/err_text

Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib64/python2.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
--- <exception caught here> ---
  File "/usr/lib/python2.7/site-packages/buildbot/process/buildstep.py", line 578, in startStep
    self.results = yield self.run()
  File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/usr/lib/python2.7/site-packages/buildbot/process/buildstep.py", line 708, in run
    results = yield self._start_deferred
  File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python2.7/site-packages/buildbot/process/buildstep.py", line 1005, in <lambda>
    d.addCallback(lambda res: self.commandComplete(cmd))
  File "steps.py", line 1010, in commandComplete
    first_results = LayoutTestFailures.results_from_string(logTextJson)
  File "layout_test_failures.py", line 64, in results_from_string
    cls.parse_full_results_json(json_dict['tests'], get_failing_tests)
  File "layout_test_failures.py", line 72, in parse_full_results_json
    cls.parse_full_results_json(tree[key], handler, new_prefix)
  File "layout_test_failures.py", line 72, in parse_full_results_json
    cls.parse_full_results_json(tree[key], handler, new_prefix)
  File "layout_test_failures.py", line 74, in parse_full_results_json
    handler(new_prefix, tree[key])
  File "layout_test_failures.py", line 61, in get_failing_tests
    if result['report'] in ['REGRESSION', 'MISSING']:
exceptions.KeyError: 'report'

[Jonathan Bedard]

We’ve got to have a regression point for this...I won’t be able to investigate before tomorrow, but I don’t see any candidates that landed today.

[Aakash Jain]

The issue is because the layout-test-results json output contains few results without the 'report' key.

e.g. {u'expected': u'PASS FAIL', u'actual': u'PASS', u'has_stderr': True}

This is happening for following three tests:
- editing/pasteboard/smart-paste-paragraph-001.html 
- editing/pasteboard/smart-paste-paragraph-002.html
- editing/pasteboard/smart-paste-paragraph-004.html

TextExpectations were updated for these three tests in r251449.

Also the failure starting point in EWS (build https://ews-build.webkit.org/#/builders/24/builds/2281/steps/4/logs/stdio) matches precisely with this commit.

[Jonathan Bedard]

For now, we should roll that commit out....Russels code is correct, but I won’t be able to fix this until tomorrow 11 am....it seems that keeping this broken that long is a bad idea.

[Aakash Jain]

Created attachment 381650 [details]
Patch

[Aakash Jain]

(In reply to Aakash Jain from comment #4)
> Created attachment 381650 [details]
> Patch
This patch seems to work, and for now I have applied it on the EWS server to get the iOS queue back in shape.

[WebKit Commit Bot]

Comment on attachment 381650 [details]
Patch

Clearing flags on attachment: 381650

Committed r251497: <https://trac.webkit.org/changeset/251497>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/56552765>