<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugs.webkit.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.4.1"
          urlbase="https://bugs.webkit.org/"
          
          maintainer="admin@webkit.org"
>

    <bug>
          <bug_id>277654</bug_id>
          
          <creation_ts>2024-08-05 16:12:25 -0700</creation_ts>
          <short_desc>run-webkit-tests exit code shouldn&apos;t depend on number of unexpected results</short_desc>
          <delta_ts>2024-09-04 18:51:57 -0700</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>1</classification_id>
          <classification>Unclassified</classification>
          <product>WebKit</product>
          <component>Tools / Tests</component>
          <version>WebKit Nightly Build</version>
          <rep_platform>Unspecified</rep_platform>
          <op_sys>Unspecified</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>DUPLICATE</resolution>
          <dup_id>276081</dup_id>
          <see_also>https://bugs.webkit.org/show_bug.cgi?id=276081</see_also>
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords>InRadar</keywords>
          <priority>P2</priority>
          <bug_severity>Normal</bug_severity>
          <target_milestone>---</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Sam Sneddon [:gsnedders]">gsnedders</reporter>
          <assigned_to name="Nobody">webkit-unassigned</assigned_to>
          <cc>aakash_jain</cc>
    
    <cc>emw</cc>
    
    <cc>jbedard</cc>
    
    <cc>webkit-bug-importer</cc>
          

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>2050760</commentid>
    <comment_count>0</comment_count>
    <who name="Sam Sneddon [:gsnedders]">gsnedders</who>
    <bug_when>2024-08-05 16:12:25 -0700</bug_when>
    <thetext>Currently, if we get the expected output but exit with non-zero, EWS will totally ignore this: https://github.com/WebKit/WebKit/blob/e9e196408b4460466726b2ac2f42e709e559e764/Tools/CISupport/ews-build/steps.py#L4113-L4114

We practically have to do this, because the exit code is actually just the number of regressions: https://github.com/WebKit/WebKit/blob/e9e196408b4460466726b2ac2f42e709e559e764/Tools/Scripts/webkitpy/port/base.py#L961-L966

It would be probably be better to define an exit code which means &quot;we had regressions&quot;, as we then don&apos;t end up in situations where a single exit code can mean multiple things.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>2050761</commentid>
    <comment_count>1</comment_count>
    <who name="Radar WebKit Bug Importer">webkit-bug-importer</who>
    <bug_when>2024-08-05 16:12:38 -0700</bug_when>
    <thetext>&lt;rdar://problem/133250341&gt;</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>2056386</commentid>
    <comment_count>2</comment_count>
    <who name="Elliott Williams">emw</who>
    <bug_when>2024-08-29 14:50:24 -0700</bug_when>
    <thetext>I think this is fixed by the changes landed in https://bugs.webkit.org/show_bug.cgi?id=276081.

*** This bug has been marked as a duplicate of bug 276081 ***</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>2057785</commentid>
    <comment_count>3</comment_count>
    <who name="Sam Sneddon [:gsnedders]">gsnedders</who>
    <bug_when>2024-09-04 18:51:57 -0700</bug_when>
    <thetext>They got reverted in bug 277749 :(</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>