<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugs.webkit.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.4.1"
          urlbase="https://bugs.webkit.org/"
          
          maintainer="admin@webkit.org"
>

    <bug>
          <bug_id>92578</bug_id>
          
          <creation_ts>2012-07-28 10:47:01 -0700</creation_ts>
          <short_desc>Lots of &quot;error, test and reference image have different properties&quot; in pixel test output because of alpha differences</short_desc>
          <delta_ts>2017-07-18 08:30:03 -0700</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>1</classification_id>
          <classification>Unclassified</classification>
          <product>WebKit</product>
          <component>Tools / Tests</component>
          <version>528+ (Nightly build)</version>
          <rep_platform>Unspecified</rep_platform>
          <op_sys>Unspecified</op_sys>
          <bug_status>ASSIGNED</bug_status>
          <resolution></resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords>NRWT</keywords>
          <priority>P2</priority>
          <bug_severity>Normal</bug_severity>
          <target_milestone>---</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Simon Fraser (smfr)">simon.fraser</reporter>
          <assigned_to name="Simon Fraser (smfr)">simon.fraser</assigned_to>
          <cc>abarth</cc>
    
    <cc>beidson</cc>
    
    <cc>dpranke</cc>
    
    <cc>kbalazs</cc>
    
    <cc>simon.fraser</cc>
    
    <cc>thorton</cc>
    
    <cc>tony</cc>
          

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>681442</commentid>
    <comment_count>0</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-07-28 10:47:01 -0700</bug_when>
    <thetext>ImageDiff is spewing:
&quot;error, test and reference image have different properties.&quot;

It&apos;s very unclear what this means.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>683379</commentid>
    <comment_count>1</comment_count>
    <who name="Ojan Vafai">ojan</who>
    <bug_when>2012-07-31 12:24:16 -0700</bug_when>
    <thetext>Can you point to an example (i.e. on the bots)? I don&apos;t see this when I run locally.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>683403</commentid>
    <comment_count>2</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-07-31 12:55:56 -0700</bug_when>
    <thetext>I got it when running pixel tests on Mountain Lion. Perhaps it&apos;s specific to that OS.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>684897</commentid>
    <comment_count>3</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-08-01 19:26:05 -0700</bug_when>
    <thetext>I&apos;ve just filed bug 92934 to propagate errors back from ImageDiff. This won&apos;t address whatever your root problem is here, but it&apos;ll make it a whole lot less ignorable :).</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700199</commentid>
    <comment_count>4</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 17:01:18 -0700</bug_when>
    <thetext>I can still reproduce. My command is:
./Tools/Scripts/run-webkit-tests --debug --pixel-tests LayoutTests/compositing/

Sample output:

  compositing/animation/busy-indicator.html -&gt; pixel hash failed (but pixel test still passes)
[8/278] compositing/color-matching/image-color-matching.html failed unexpectedly (image diff)
  compositing/culling/clear-fixed-iframe.html : ImageDiff produced stderr output:
error, test and reference image have different properties.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700217</commentid>
    <comment_count>5</comment_count>
      <attachid>159567</attachid>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 17:13:16 -0700</bug_when>
    <thetext>Created attachment 159567
Improve error reporting</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700225</commentid>
    <comment_count>6</comment_count>
      <attachid>159567</attachid>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 17:17:21 -0700</bug_when>
    <thetext>Comment on attachment 159567
Improve error reporting

http://trac.webkit.org/changeset/126101</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700233</commentid>
    <comment_count>7</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 17:22:47 -0700</bug_when>
    <thetext>So now I get:
  compositing/culling/clear-fixed-iframe.html : ImageDiff produced stderr output:
Error: test and reference images differ in alpha. Test image does not have alpha, reference image has alpha.

 12$ $ sips -g all ./LayoutTests/compositing/culling/clear-fixed-iframe-expected.png 
/Volumes/DataSSD/Development/apple/webkit/WebKit.git/LayoutTests/compositing/culling/clear-fixed-iframe-expected.png
  pixelWidth: 800
  pixelHeight: 600
  typeIdentifier: public.png
  format: png
  formatOptions: default
  dpiWidth: 72.000
  dpiHeight: 72.000
  samplesPerPixel: 3
  bitsPerSample: 8
  hasAlpha: no
  space: RGB

So hasAlpha is NO in the PNG, but ImageDiff thinks it has alpha.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700237</commentid>
    <comment_count>8</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 17:28:59 -0700</bug_when>
    <thetext>But CGImageGetAlphaInfo() is saying that the image alpha flags are kCGImageAlphaLast, implying non-premultplied RGBA.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700309</commentid>
    <comment_count>9</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 18:30:05 -0700</bug_when>
    <thetext>createBitmapContext() creates a bitmap context with (kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Host), so I think we expect all the images to have alpha.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700327</commentid>
    <comment_count>10</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 18:53:55 -0700</bug_when>
    <thetext>There&apos;s something I don&apos;t understand. DRT is dumping to stdout a PNG created from an image with alpha flags 3 (kCGImageAlphaLast). However, when ImageDiff reads this in from stdin, it gets an image with alpha flags 5 (kCGImageAlphaNoneSkipLast).

This is assuming that ImageDiff is not swapping the images by mistake. The code seems to read the actual before the expected:

            if (imageSize &gt; 0 &amp;&amp; !actualImage)
                actualImage = createImageFromStdin(imageSize);
            else if (imageSize &gt; 0 &amp;&amp; !baselineImage)
                baselineImage = createImageFromStdin(imageSize);

I&apos;m not sure who&apos;s printing the expected image to stdin. Is it ImageDiffer in the python code?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700349</commentid>
    <comment_count>11</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-08-20 19:15:29 -0700</bug_when>
    <thetext>(In reply to comment #10)
&gt; There&apos;s something I don&apos;t understand. DRT is dumping to stdout a PNG created from an image with alpha flags 3 (kCGImageAlphaLast). However, when ImageDiff reads this in from stdin, it gets an image with alpha flags 5 (kCGImageAlphaNoneSkipLast).
&gt; 
&gt; This is assuming that ImageDiff is not swapping the images by mistake. The code seems to read the actual before the expected:
&gt; 
&gt;             if (imageSize &gt; 0 &amp;&amp; !actualImage)
&gt;                 actualImage = createImageFromStdin(imageSize);
&gt;             else if (imageSize &gt; 0 &amp;&amp; !baselineImage)
&gt;                 baselineImage = createImageFromStdin(imageSize);
&gt; 
&gt; I&apos;m not sure who&apos;s printing the expected image to stdin. Is it ImageDiffer in the python code?

Yes. See base.py:353 and port/image_diff.py:60.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700357</commentid>
    <comment_count>12</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 19:24:25 -0700</bug_when>
    <thetext>I&apos;m checksumming the data spat out by DRT (after the checksum injection), and the data ingested by ImageDiff, and they don&apos;t seem to match:

Dumped PNG has hash efb0c7c9967868bc58f1c9c989f6878f
Original image alpha flags: 2, new image alpha flags: 3
ImageDiff produced stderr output:
image in imageDiff has hash af52930d6c48db65a6e957ad80869650
image in imageDiff has hash 6f08bd8f8505e7b86567159012c459cf</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700396</commentid>
    <comment_count>13</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 20:19:18 -0700</bug_when>
    <thetext>I&apos;m pretty sure that _compare_image() is swapping the images by mistake:

            diff_result = self._port.diff_image(driver_output.image, expected_driver_output.image)

    def diff_image(self, expected_contents, actual_contents, tolerance):

Note flipped order of params.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700400</commentid>
    <comment_count>14</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 20:24:02 -0700</bug_when>
    <thetext>Re: (In reply to comment #12)
&gt; I&apos;m checksumming the data spat out by DRT (after the checksum injection), and the data ingested by ImageDiff, and they don&apos;t seem to match:
&gt; 
&gt; Dumped PNG has hash efb0c7c9967868bc58f1c9c989f6878f
&gt; Original image alpha flags: 2, new image alpha flags: 3
&gt; ImageDiff produced stderr output:
&gt; image in imageDiff has hash af52930d6c48db65a6e957ad80869650
&gt; image in imageDiff has hash 6f08bd8f8505e7b86567159012c459cf

Ah, they do match, I was just getting my output out of order..

So the reason for the pixel comparison failures is that the expected image result files which have been committed by chromium people lack alpha. For example, LayoutTests/compositing/culling/clear-fixed-iframe-expected.png, committed in http://trac.webkit.org/changeset/109514, has no alpha.

We need to make sure that expected image files generated on different platforms match.

I have not yet looked at ref test failures.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700401</commentid>
    <comment_count>15</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 20:24:35 -0700</bug_when>
    <thetext>Fix for flipped actual/expected;

diff --git a/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py b/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
index 7379d97..45b47f1 100644
--- a/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
+++ b/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
@@ -264,7 +264,7 @@ class SingleTestRunner(object):
         elif not expected_driver_output.image_hash:
             failures.append(test_failures.FailureMissingImageHash())
         elif driver_output.image_hash != expected_driver_output.image_hash:
-            diff_result = self._port.diff_image(driver_output.image, expected_driver_output.image)
+            diff_result = self._port.diff_image(expected_driver_output.image, driver_output.image)
             err_str = diff_result[2]
             # FIXME: see https://bugs.webkit.org/show_bug.cgi?id=94277 and
             # https://bugs.webkit.org/show_bug.cgi?id=81962; ImageDiff doesn&apos;t</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700403</commentid>
    <comment_count>16</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 20:26:09 -0700</bug_when>
    <thetext>Filed bug 94567 to land that patch.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700405</commentid>
    <comment_count>17</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 20:28:39 -0700</bug_when>
    <thetext>Giving this bug to Dirk to ensure that Chromium tools dump PNG images with alpha. I also note that the color profiles in the Chromium-supplied images don&apos;t match the Mac ones.

Maybe we should give up on the notion of platform-agnostic expected.png files?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700452</commentid>
    <comment_count>18</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-08-20 21:48:42 -0700</bug_when>
    <thetext>(In reply to comment #15)
&gt; Fix for flipped actual/expected;
&gt; 
&gt; diff --git a/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py b/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
&gt; index 7379d97..45b47f1 100644
&gt; --- a/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
&gt; +++ b/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
&gt; @@ -264,7 +264,7 @@ class SingleTestRunner(object):
&gt;          elif not expected_driver_output.image_hash:
&gt;              failures.append(test_failures.FailureMissingImageHash())
&gt;          elif driver_output.image_hash != expected_driver_output.image_hash:
&gt; -            diff_result = self._port.diff_image(driver_output.image, expected_driver_output.image)
&gt; +            diff_result = self._port.diff_image(expected_driver_output.image, driver_output.image)
&gt;              err_str = diff_result[2]
&gt;              # FIXME: see https://bugs.webkit.org/show_bug.cgi?id=94277 and
&gt;              # https://bugs.webkit.org/show_bug.cgi?id=81962; ImageDiff doesn&apos;t

So, single_test_runner currently passes expected, actual to base.Port.diff_image(), which is expecting actual, expected :(. However, base.Port.diff_image then flips around and calls  calls image_differ(expected, actual), which writes actual, then expected to ImageDiff, which then reads actual, then expected. For reference, the signature in ChromiumPort.diff_image is (expected, actual). So, single_test_runner is consistent with ChromiumPort, which isn&apos;t surprising, since chromium actually uses pixel diffs regularly :)

In short we should leave single_test_runner alone and fix base.py because it&apos;s the only one that isn&apos;t (expected, actual) in the python code (and I would argue that expected, actual is the order that make sense insofar as it is consistent with &quot;diff old new&quot;; I realize the non-chromium ImageDiff appears to want &quot;diff new old&quot;, but that seems wrong.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700456</commentid>
    <comment_count>19</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-08-20 21:53:27 -0700</bug_when>
    <thetext>(In reply to comment #14)
&gt; Re: (In reply to comment #12)
&gt; &gt; I&apos;m checksumming the data spat out by DRT (after the checksum injection), and the data ingested by ImageDiff, and they don&apos;t seem to match:
&gt; &gt; 
&gt; &gt; Dumped PNG has hash efb0c7c9967868bc58f1c9c989f6878f
&gt; &gt; Original image alpha flags: 2, new image alpha flags: 3
&gt; &gt; ImageDiff produced stderr output:
&gt; &gt; image in imageDiff has hash af52930d6c48db65a6e957ad80869650
&gt; &gt; image in imageDiff has hash 6f08bd8f8505e7b86567159012c459cf
&gt; 
&gt; Ah, they do match, I was just getting my output out of order..
&gt; 
&gt; So the reason for the pixel comparison failures is that the expected image result files which have been committed by chromium people lack alpha. For example, LayoutTests/compositing/culling/clear-fixed-iframe-expected.png, committed in http://trac.webkit.org/changeset/109514, has no alpha.
&gt; 
&gt; We need to make sure that expected image files generated on different platforms match.
&gt; 

Adding Tony to this, since I&apos;ve discussed this issue with him before. 

It is true that different ports have different interpretations of what the PNG format is supposed to be, and we are inconsistent. Given the aggressiveness of the baseline optimizer, this is probably bad, but I don&apos;t think this is insoluble. I think we just need to standardize on how we want to store (and compare) PNGs.

If we did that, we could probably get rid of the port-specific ImageDiff code altogether. I had looked into using pure-python png decoding but gave up after realizing we had different conventions.

IIRC, there&apos;s a reason the chromium port doesn&apos;t store alphas; it has something to do with the differences between our win and linux ports.
&gt; I have not yet looked at ref test failures.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700457</commentid>
    <comment_count>20</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2012-08-20 21:54:48 -0700</bug_when>
    <thetext>I didn&apos;t find any issues with ref tests on Mountain Lion. When running nrwt with --pixel-tests (to force ref testing) on ML, things worked fine.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700459</commentid>
    <comment_count>21</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-08-20 21:57:37 -0700</bug_when>
    <thetext>Also, I thought premultiplied alpha meant that the alpha was multiplied first? so, if you have #fff with an alpha of 0.5, do you store 0x80808080 or 0x00808080 ? I&apos;ve never really understood this premultiplied algorithm ...</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700460</commentid>
    <comment_count>22</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-08-20 21:58:15 -0700</bug_when>
    <thetext>(In reply to comment #20)
&gt; I didn&apos;t find any issues with ref tests on Mountain Lion. When running nrwt with --pixel-tests (to force ref testing) on ML, things worked fine.

How can that be? Aren&apos;t the same images being compared on Lion and ML?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700463</commentid>
    <comment_count>23</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-08-20 22:01:34 -0700</bug_when>
    <thetext>(In reply to comment #18)
&gt; (In reply to comment #15)
&gt; &gt; Fix for flipped actual/expected;
&gt; &gt; 
&gt; &gt; diff --git a/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py b/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
&gt; &gt; index 7379d97..45b47f1 100644
&gt; &gt; --- a/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
&gt; &gt; +++ b/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
&gt; &gt; @@ -264,7 +264,7 @@ class SingleTestRunner(object):
&gt; &gt;          elif not expected_driver_output.image_hash:
&gt; &gt;              failures.append(test_failures.FailureMissingImageHash())
&gt; &gt;          elif driver_output.image_hash != expected_driver_output.image_hash:
&gt; &gt; -            diff_result = self._port.diff_image(driver_output.image, expected_driver_output.image)
&gt; &gt; +            diff_result = self._port.diff_image(expected_driver_output.image, driver_output.image)
&gt; &gt;              err_str = diff_result[2]
&gt; &gt;              # FIXME: see https://bugs.webkit.org/show_bug.cgi?id=94277 and
&gt; &gt;              # https://bugs.webkit.org/show_bug.cgi?id=81962; ImageDiff doesn&apos;t
&gt; 
&gt; So, single_test_runner currently passes expected, actual to base.Port.diff_image(), which is expecting actual, expected :(. However, base.Port.diff_image then flips around and calls  calls image_differ(expected, actual), which writes actual, then expected to ImageDiff, which then reads actual, then expected. For reference, the signature in ChromiumPort.diff_image is (expected, actual). So, single_test_runner is consistent with ChromiumPort, which isn&apos;t surprising, since chromium actually uses pixel diffs regularly :)
&gt; 
&gt; In short we should leave single_test_runner alone and fix base.py because it&apos;s the only one that isn&apos;t (expected, actual) in the python code (and I would argue that expected, actual is the order that make sense insofar as it is consistent with &quot;diff old new&quot;; I realize the non-chromium ImageDiff appears to want &quot;diff new old&quot;, but that seems wrong.

Argh. the code as currently checked in calls diff_image(actual, expected), of course. Basically we need to trace through the whole path to make sure they&apos;re consistent :(.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700474</commentid>
    <comment_count>24</comment_count>
    <who name="Tim Horton">thorton</who>
    <bug_when>2012-08-20 22:11:08 -0700</bug_when>
    <thetext>(In reply to comment #23)
&gt; Argh. the code as currently checked in calls diff_image(actual, expected), of course. Basically we need to trace through the whole path to make sure they&apos;re consistent :(.

Named arguments FTW. (and this might not be a bad time to adopt them here!)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700982</commentid>
    <comment_count>25</comment_count>
    <who name="Tony Chang">tony</who>
    <bug_when>2012-08-21 12:12:05 -0700</bug_when>
    <thetext>(In reply to comment #19)
&gt; If we did that, we could probably get rid of the port-specific ImageDiff code altogether. I had looked into using pure-python png decoding but gave up after realizing we had different conventions.

I thought this was also slow. Seems like it would be.

&gt; IIRC, there&apos;s a reason the chromium port doesn&apos;t store alphas; it has something to do with the differences between our win and linux ports.

http://trac.webkit.org/browser/trunk/Tools/DumpRenderTree/chromium/TestShell.cpp#L660

I didn&apos;t write the comment.  I&apos;m not sure what about text/form control drawing causes problems (brettw may know or remember).</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>701048</commentid>
    <comment_count>26</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-08-21 13:18:45 -0700</bug_when>
    <thetext>(In reply to comment #25)
&gt; (In reply to comment #19)
&gt; &gt; If we did that, we could probably get rid of the port-specific ImageDiff code altogether. I had looked into using pure-python png decoding but gave up after realizing we had different conventions.
&gt; 
&gt; I thought this was also slow. Seems like it would be.
&gt; 

Yeah, probably. We&apos;d need to benchmark it to see if there is a real impact, since we only do the diffs when the hashes don&apos;t match.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>701093</commentid>
    <comment_count>27</comment_count>
    <who name="Ojan Vafai">ojan</who>
    <bug_when>2012-08-21 13:47:26 -0700</bug_when>
    <thetext>(In reply to comment #26)
&gt; (In reply to comment #25)
&gt; &gt; (In reply to comment #19)
&gt; &gt; &gt; If we did that, we could probably get rid of the port-specific ImageDiff code altogether. I had looked into using pure-python png decoding but gave up after realizing we had different conventions.
&gt; &gt; 
&gt; &gt; I thought this was also slow. Seems like it would be.
&gt; &gt; 
&gt; 
&gt; Yeah, probably. We&apos;d need to benchmark it to see if there is a real impact, since we only do the diffs when the hashes don&apos;t match.

The perf of ImageDiff matters a lot less in a world in which we check in expected failures. Then the only times we&apos;d be running this are when there are regressions on ToT or for flaky pixel failures. Seems unlikely to me that this would have a huge perf impact on the time of running the full test suite. But, there&apos;s never a harm in measuring.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>725978</commentid>
    <comment_count>28</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-09-21 16:53:00 -0700</bug_when>
    <thetext>I&apos;m not actually planning to fix this in the near future, so someone else should feel free to take a look.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>876206</commentid>
    <comment_count>29</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2013-04-15 18:44:32 -0700</bug_when>
    <thetext>Simon is this still an issue, or have you fixed all of the non-alpha-having PNGs now that Chromium is gone?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>876234</commentid>
    <comment_count>30</comment_count>
    <who name="Simon Fraser (smfr)">simon.fraser</who>
    <bug_when>2013-04-15 19:54:22 -0700</bug_when>
    <thetext>(In reply to comment #29)
&gt; Simon is this still an issue, or have you fixed all of the non-alpha-having PNGs now that Chromium is gone?

I have not, but I think we will eventually.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>876265</commentid>
    <comment_count>31</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2013-04-15 21:17:35 -0700</bug_when>
    <thetext>(In reply to comment #30)
&gt; (In reply to comment #29)
&gt; &gt; Simon is this still an issue, or have you fixed all of the non-alpha-having PNGs now that Chromium is gone?
&gt; 
&gt; I have not, but I think we will eventually.

Okay, I&apos;m gonna reassign this to you if that&apos;s okay, unless there&apos;s something I can do. It wouldn&apos;t be hard to go through and find all the bad PNGs and replace them, for example.</thetext>
  </long_desc>
      
          <attachment
              isobsolete="1"
              ispatch="1"
              isprivate="0"
          >
            <attachid>159567</attachid>
            <date>2012-08-20 17:13:16 -0700</date>
            <delta_ts>2012-08-20 17:17:20 -0700</delta_ts>
            <desc>Improve error reporting</desc>
            <filename>bug-92578-20120820171236.patch</filename>
            <type>text/plain</type>
            <size>2954</size>
            <attacher name="Simon Fraser (smfr)">simon.fraser</attacher>
            
              <data encoding="base64">U3VidmVyc2lvbiBSZXZpc2lvbjogMTI2MDkwCmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg
Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggNTcwOThiYzg2NDQwOGNmNTIzNDI1OWU0ZWU3OTE2MjI4
YmY1YzE3OC4uNGY4ODEwOTJjNmJjOGZlMTllNWYxN2E1ZGFjYTdlYmIwNDJjNGFkMiAxMDA2NDQK
LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDE1
IEBACisyMDEyLTA4LTIwICBTaW1vbiBGcmFzZXIgIDxzaW1vbi5mcmFzZXJAYXBwbGUuY29tPgor
CisgICAgICAgIExvdHMgb2YgImVycm9yLCB0ZXN0IGFuZCByZWZlcmVuY2UgaW1hZ2UgaGF2ZSBk
aWZmZXJlbnQgcHJvcGVydGllcyIgaW4gcGl4ZWwgdGVzdCBvdXRwdXQKKyAgICAgICAgaHR0cHM6
Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTkyNTc4CisKKyAgICAgICAgUmV2aWV3
ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgSW1wcm92ZSBJbWFnZURpZmYncyBlcnJv
ciByZXBvcnRpbmcgd2hlbiB0ZXN0IHJlc3VsdCBpbWFnZSBhbmQgZXhwZWN0ZWQgaW1hZ2UKKyAg
ICAgICAgZGlmZmVyIGluIHRoZWlyIHByb3BlcnRpZXMuCisKKyAgICAgICAgKiBEdW1wUmVuZGVy
VHJlZS9jZy9JbWFnZURpZmZDRy5jcHA6CisKIDIwMTItMDgtMjAgIE5hdGUgQ2hhcGluICA8amFw
aGV0QGNocm9taXVtLm9yZz4KIAogICAgICAgICBVbnNhZmUgdnNwcmludGYgdXNhZ2UgaW4gVGVz
dE5ldHNjYXBlUGx1Z2luCmRpZmYgLS1naXQgYS9Ub29scy9EdW1wUmVuZGVyVHJlZS9jZy9JbWFn
ZURpZmZDRy5jcHAgYi9Ub29scy9EdW1wUmVuZGVyVHJlZS9jZy9JbWFnZURpZmZDRy5jcHAKaW5k
ZXggZGIzNTQzNzMwNjFlNTUzNWI1YzU5OGM2YzliN2ZiZWI3ZDY0ODE4Mi4uN2ZhNTExMGJiZDU5
ZTdlNTEyMjFmODRiZTNkNTJjNWRiMGIwZDU0OSAxMDA2NDQKLS0tIGEvVG9vbHMvRHVtcFJlbmRl
clRyZWUvY2cvSW1hZ2VEaWZmQ0cuY3BwCisrKyBiL1Rvb2xzL0R1bXBSZW5kZXJUcmVlL2NnL0lt
YWdlRGlmZkNHLmNwcApAQCAtMjE0LDcgKzIxNCw3IEBAIGludCBtYWluKGludCBhcmdjLCBjb25z
dCBjaGFyKiBhcmd2W10pCiAgICAgICAgICAgICBlbHNlIGlmIChpbWFnZVNpemUgPiAwICYmICFi
YXNlbGluZUltYWdlKQogICAgICAgICAgICAgICAgIGJhc2VsaW5lSW1hZ2UgPSBjcmVhdGVJbWFn
ZUZyb21TdGRpbihpbWFnZVNpemUpOwogICAgICAgICAgICAgZWxzZQotICAgICAgICAgICAgICAg
IGZwdXRzKCJlcnJvciwgaW1hZ2Ugc2l6ZSBtdXN0IGJlIHNwZWNpZmllZC5cbiIsIHN0ZGVycik7
CisgICAgICAgICAgICAgICAgZnB1dHMoIkVycm9yOiBpbWFnZSBzaXplIG11c3QgYmUgc3BlY2lm
aWVkLlxuIiwgc3RkZXJyKTsKICAgICAgICAgfQogCiAgICAgICAgIGlmIChhY3R1YWxJbWFnZSAm
JiBiYXNlbGluZUltYWdlKSB7CkBAIC0yMjksOSArMjI5LDE3IEBAIGludCBtYWluKGludCBhcmdj
LCBjb25zdCBjaGFyKiBhcmd2W10pCiAgICAgICAgICAgICAgICAgICAgIGRpZmZlcmVuY2UgPSBy
b3VuZGYoZGlmZmVyZW5jZSAqIDEwMC4wZikgLyAxMDAuMGY7CiAgICAgICAgICAgICAgICAgICAg
IGRpZmZlcmVuY2UgPSBtYXgoZGlmZmVyZW5jZSwgMC4wMWYpOyAvLyByb3VuZCB0byAyIGRlY2lt
YWwgcGxhY2VzCiAgICAgICAgICAgICAgICAgfQotICAgICAgICAgICAgfSBlbHNlCi0gICAgICAg
ICAgICAgICAgZnB1dHMoImVycm9yLCB0ZXN0IGFuZCByZWZlcmVuY2UgaW1hZ2UgaGF2ZSBkaWZm
ZXJlbnQgcHJvcGVydGllcy5cbiIsIHN0ZGVycik7Ci0KKyAgICAgICAgICAgIH0gZWxzZSB7Cisg
ICAgICAgICAgICAgICAgaWYgKENHSW1hZ2VHZXRXaWR0aChhY3R1YWxJbWFnZS5nZXQoKSkgIT0g
Q0dJbWFnZUdldFdpZHRoKGJhc2VsaW5lSW1hZ2UuZ2V0KCkpIHx8IENHSW1hZ2VHZXRIZWlnaHQo
YWN0dWFsSW1hZ2UuZ2V0KCkpICE9IENHSW1hZ2VHZXRIZWlnaHQoYmFzZWxpbmVJbWFnZS5nZXQo
KSkpCisgICAgICAgICAgICAgICAgICAgIGZwcmludGYoc3RkZXJyLCAiRXJyb3I6IHRlc3QgYW5k
IHJlZmVyZW5jZSBpbWFnZXMgaGF2ZSBkaWZmZXJlbnQgc2l6ZXMuIFRlc3QgaW1hZ2UgaXMgJWx1
eCVsdSwgcmVmZXJlbmNlIGltYWdlIGlzICVsdXglbHVcbiIsCisgICAgICAgICAgICAgICAgICAg
ICAgICBDR0ltYWdlR2V0V2lkdGgoYWN0dWFsSW1hZ2UuZ2V0KCkpLCBDR0ltYWdlR2V0SGVpZ2h0
KGFjdHVhbEltYWdlLmdldCgpKSwKKyAgICAgICAgICAgICAgICAgICAgICAgIENHSW1hZ2VHZXRX
aWR0aChiYXNlbGluZUltYWdlLmdldCgpKSwgQ0dJbWFnZUdldEhlaWdodChiYXNlbGluZUltYWdl
LmdldCgpKSk7CisgICAgICAgICAgICAgICAgZWxzZSBpZiAoaW1hZ2VIYXNBbHBoYShhY3R1YWxJ
bWFnZS5nZXQoKSkgIT0gaW1hZ2VIYXNBbHBoYShiYXNlbGluZUltYWdlLmdldCgpKSkKKyAgICAg
ICAgICAgICAgICAgICAgZnByaW50ZihzdGRlcnIsICJFcnJvcjogdGVzdCBhbmQgcmVmZXJlbmNl
IGltYWdlcyBkaWZmZXIgaW4gYWxwaGEuIFRlc3QgaW1hZ2UgJXMgYWxwaGEsIHJlZmVyZW5jZSBp
bWFnZSAlcyBhbHBoYS5cbiIsCisgICAgICAgICAgICAgICAgICAgICAgICBpbWFnZUhhc0FscGhh
KGFjdHVhbEltYWdlLmdldCgpKSA/ICJoYXMiIDogImRvZXMgbm90IGhhdmUiLAorICAgICAgICAg
ICAgICAgICAgICAgICAgaW1hZ2VIYXNBbHBoYShiYXNlbGluZUltYWdlLmdldCgpKSA/ICJoYXMi
IDogImRvZXMgbm90IGhhdmUiKTsKKyAgICAgICAgICAgIH0KKyAgICAgICAgICAgIAogICAgICAg
ICAgICAgaWYgKGRpZmZlcmVuY2UgPiAwLjBmKSB7CiAgICAgICAgICAgICAgICAgaWYgKGRpZmZJ
bWFnZSkgewogICAgICAgICAgICAgICAgICAgICBSZXRhaW5QdHI8Q0ZNdXRhYmxlRGF0YVJlZj4g
aW1hZ2VEYXRhKEFkb3B0Q0YsIENGRGF0YUNyZWF0ZU11dGFibGUoMCwgMCkpOwo=
</data>

          </attachment>
      

    </bug>

</bugzilla>