<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugs.webkit.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.4.1"
          urlbase="https://bugs.webkit.org/"
          
          maintainer="admin@webkit.org"
>

    <bug>
          <bug_id>72039</bug_id>
          
          <creation_ts>2011-11-10 11:53:44 -0800</creation_ts>
          <short_desc>Tests occasionally report missing expectations</short_desc>
          <delta_ts>2013-07-24 16:29:51 -0700</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>1</classification_id>
          <classification>Unclassified</classification>
          <product>WebKit</product>
          <component>Tools / Tests</component>
          <version>528+ (Nightly build)</version>
          <rep_platform>Unspecified</rep_platform>
          <op_sys>Unspecified</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>P2</priority>
          <bug_severity>Normal</bug_severity>
          <target_milestone>---</target_milestone>
          <dependson>72109</dependson>
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Tony Gentilcore">tonyg</reporter>
          <assigned_to name="Erik Arvidsson">arv</assigned_to>
          <cc>abarth</cc>
    
    <cc>aboxhall</cc>
    
    <cc>arv</cc>
    
    <cc>dominicc</cc>
    
    <cc>dpranke</cc>
    
    <cc>eric</cc>
    
    <cc>japhet</cc>
    
    <cc>jcivelli</cc>
    
    <cc>kbr</cc>
    
    <cc>ojan</cc>
    
    <cc>rafaelw</cc>
    
    <cc>tony</cc>
    
    <cc>tonyg</cc>
    
    <cc>ukai</cc>
          

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>500084</commentid>
    <comment_count>0</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-10 11:53:44 -0800</bug_when>
    <thetext>Ojan and I both noticed this while gardening today. We have several flaky tests that typically PASS, but every so often report MISSING.

It seems like this must be a script bug.

For example:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=platform%2Fchromium-cg-mac%2Fediting%2Finput%2Fime-candidate-window-position.html</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500090</commentid>
    <comment_count>1</comment_count>
    <who name="Ojan Vafai">ojan</who>
    <bug_when>2011-11-10 11:59:04 -0800</bug_when>
    <thetext>What I observe is that it&apos;s trying to do a pixel diff on dumpAsText tests. It obviously can&apos;t find the png and reports it as MISSING. Looking at the test, it&apos;s hard for me to see how it&apos;s possible that these tests would not hit the dumpAsText line. 

Also, notably, it&apos;s getting a JS exception saying that &quot;debug&quot; is not defined, which is defined in the same file as the dumpAsText call. So, sometimes that file isn&apos;t getting loaded. But the path is correct and most of the time it gets loaded fine.

In conclusion, my best guess is that Chrome/WebKit has a bug with script loading where we continue loading the page instead of blocking until the script has loaded. Not sure who the experts on html parsing / script loading are. CCing a few likely candidates. :)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500092</commentid>
    <comment_count>2</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2011-11-10 12:03:11 -0800</bug_when>
    <thetext>I have definitely seen things in the past where the test errors out before calling dumpAsText, and we get MISSING reports as a result.

Obviously one way to debug this would be to modify the test to call dumpAsText() in the main .html file and see if that makes the problem go away.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500093</commentid>
    <comment_count>3</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-10 12:05:01 -0800</bug_when>
    <thetext>
&gt; In conclusion, my best guess is that Chrome/WebKit has a bug with script loading where we continue loading the page instead of blocking until the script has loaded.

Definitely explanative, but afaik not much has changed there lately. If we weren&apos;t blocking on script loading in certain circumstances I&apos;d also expect that failure to manifest itself in a much louder way (ie. lots of real world bugs and other test failures).</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500103</commentid>
    <comment_count>4</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-10 12:10:29 -0800</bug_when>
    <thetext>You know, another thing is that these tests used to flakily timeout. Now they flakily go missing. No timeouts have occurred since the flaky missings started and the missings seem to be at roughly the same frequency that the timeouts were at previously.

Perhaps either we are now timing out earlier (before dumpAsText()) or timeouts before dumpAsText() are now handled differently?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500108</commentid>
    <comment_count>5</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-10 12:14:58 -0800</bug_when>
    <thetext>(In reply to comment #4)
&gt; You know, another thing is that these tests used to flakily timeout. Now they flakily go missing. No timeouts have occurred since the flaky missings started and the missings seem to be at roughly the same frequency that the timeouts were at previously.

The barrier between the timeouts and the missings is this relatively tight range:
http://trac.webkit.org/log/?rev=99829&amp;stop_rev=99822&amp;verbose=on</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500115</commentid>
    <comment_count>6</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-10 12:19:51 -0800</bug_when>
    <thetext>
&gt; The barrier between the timeouts and the missings is this relatively tight range:
&gt; http://trac.webkit.org/log/?rev=99829&amp;stop_rev=99822&amp;verbose=on

https://bugs.webkit.org/show_bug.cgi?id=71859 is the only thing in that range that could be relevant (although I don&apos;t immediately see how).</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500122</commentid>
    <comment_count>7</comment_count>
    <who name="Ojan Vafai">ojan</who>
    <bug_when>2011-11-10 12:23:10 -0800</bug_when>
    <thetext>I think the TIMEOUT thing is a red herring. We *know* that the tests are being run as pixel tests and that the test code is getting executed before the js-test-pre script has successfully loaded.

Whether there&apos;s a timing issue or that the js-test-pre script isn&apos;t loading at all for some reason is the question.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500337</commentid>
    <comment_count>8</comment_count>
    <who name="Ojan Vafai">ojan</who>
    <bug_when>2011-11-10 15:12:23 -0800</bug_when>
    <thetext>Tony points out http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fmultipart%2Fpolicy-ignore-crash.php%2Chttp%2Ftests%2Fmultipart%2Fmultipart-wait-before-boundary.html.

http/tests/multipart/multipart-wait-before-boundary.html runs right before http/tests/multipart/policy-ignore-crash.php. And http/tests/multipart/policy-ignore-crash.php only passes when http/tests/multipart/multipart-wait-before-boundary.html times out!</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500625</commentid>
    <comment_count>9</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-11 03:24:33 -0800</bug_when>
    <thetext>(In reply to comment #8)
&gt; Tony points out http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fmultipart%2Fpolicy-ignore-crash.php%2Chttp%2Ftests%2Fmultipart%2Fmultipart-wait-before-boundary.html.
&gt; 
&gt; http/tests/multipart/multipart-wait-before-boundary.html runs right before http/tests/multipart/policy-ignore-crash.php. And http/tests/multipart/policy-ignore-crash.php only passes when http/tests/multipart/multipart-wait-before-boundary.html times out!

Good observation. Along the same lines, I notice that ime-candidate-window-position.html goes missing when it runs immediately after mhtml/simple_page_unmht.mht and passes when it runs after something else.

Based on what looks to me to be the regression range and the mhtml connection, I&apos;m going to try rolling out http://trac.webkit.org/changeset/99826 . If a &quot;missing&quot; happens after that, I&apos;ll roll it forward again right away.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500725</commentid>
    <comment_count>10</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-11 06:22:43 -0800</bug_when>
    <thetext>Looks like after the rollout the flaky timeouts of before are back and the missings are gone.

Closing this bug.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500773</commentid>
    <comment_count>11</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-11 08:02:25 -0800</bug_when>
    <thetext>Looks like this still happens on fast/forms/form-associated-element-crash3.html and it isn&apos;t strongly tied to which test runs immediately before it (usually is fast/forms/form-associated-element-crash2.html).

http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=fast%2Fforms%2Fform-associated-element-crash3.html

It seems to have been happening there for a long time and it looks independent of the problem that was on ime-candidate-window-position.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>500871</commentid>
    <comment_count>12</comment_count>
    <who name="Jay Civelli">jcivelli</who>
    <bug_when>2011-11-11 10:06:49 -0800</bug_when>
    <thetext>So what is the conclusion.
Was my CL responsible?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>501793</commentid>
    <comment_count>13</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-14 04:36:48 -0800</bug_when>
    <thetext>(In reply to comment #12)
&gt; So what is the conclusion.
&gt; Was my CL responsible?

Rolling out that patch fixed this issue for ime-candidate-window-position.html. You can see the &quot;missings&quot; are all exactly correlated with landing and rolling out the patch here:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=platform%2Fchromium-cg-mac%2Fediting%2Finput%2Fime-candidate-window-position.html

This is still occurring on form-associated-element-crash3.html, but it occurred there prior to your patch.
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=fast%2Fforms%2Fform-associated-element-crash3.html

I suspect there is a different underlying issue there, but this needs investigation.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>501801</commentid>
    <comment_count>14</comment_count>
      <attachid>114919</attachid>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-14 04:44:19 -0800</bug_when>
    <thetext>Created attachment 114919
Patch</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>501802</commentid>
    <comment_count>15</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-14 04:44:59 -0800</bug_when>
    <thetext>Committed r100128: &lt;http://trac.webkit.org/changeset/100128&gt;</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>501938</commentid>
    <comment_count>16</comment_count>
      <attachid>114953</attachid>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-14 08:38:10 -0800</bug_when>
    <thetext>Created attachment 114953
Patch</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>501941</commentid>
    <comment_count>17</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-14 08:41:18 -0800</bug_when>
    <thetext>Committed r100147: &lt;http://trac.webkit.org/changeset/100147&gt;</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>501946</commentid>
    <comment_count>18</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-14 08:43:39 -0800</bug_when>
    <thetext>Same symptoms on fast/forms/form-associated-element-crash.html and fast/forms/form-attribute-elements-order2.html.
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=fast%2Fforms%2Fform-associated-element-crash.html%2Cfast%2Fforms%2Fform-attribute-elements-order2.html

First occurrence there was before r99864.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>501963</commentid>
    <comment_count>19</comment_count>
      <attachid>114958</attachid>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-14 08:58:31 -0800</bug_when>
    <thetext>Created attachment 114958
Patch</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>501965</commentid>
    <comment_count>20</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-14 08:59:32 -0800</bug_when>
    <thetext>Committed r100151: &lt;http://trac.webkit.org/changeset/100151&gt;</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>501966</commentid>
    <comment_count>21</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-14 08:59:53 -0800</bug_when>
    <thetext>Another one:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fmultipart%2Fpolicy-ignore-crash.php</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>502024</commentid>
    <comment_count>22</comment_count>
    <who name="Erik Arvidsson">arv</who>
    <bug_when>2011-11-14 09:53:19 -0800</bug_when>
    <thetext>I think I have a good idea what is causing this.

My test refactoring removes the &lt;link rel=stylesheet&gt; from the html files. Instead it injects it using js. However, when the link is inserted using js it might not be ready when the first layout happens. The css file has some white-space: pre so that would explain why we are seeing missing newlines and other white spaces.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>502873</commentid>
    <comment_count>23</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-15 06:48:55 -0800</bug_when>
    <thetext>3 more with the same symptoms:
fast/forms/form-attribute-elements.html
fast/forms/form-attribute-nonexistence-form-id.html
fast/forms/form-attribute-elements-order.html</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>502877</commentid>
    <comment_count>24</comment_count>
      <attachid>115158</attachid>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-15 06:51:57 -0800</bug_when>
    <thetext>Created attachment 115158
Patch</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>502883</commentid>
    <comment_count>25</comment_count>
    <who name="Tony Gentilcore">tonyg</who>
    <bug_when>2011-11-15 06:58:31 -0800</bug_when>
    <thetext>Committed r100278: &lt;http://trac.webkit.org/changeset/100278&gt;</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>575992</commentid>
    <comment_count>26</comment_count>
    <who name="Fumitoshi Ukai">ukai</who>
    <bug_when>2012-03-12 01:31:46 -0700</bug_when>
    <thetext>I see this again.

http://test-results.appspot.com/dashboards/flakiness_dashboard.html#group=%40DEPS%20-%20chromium.org&amp;tests=fast%2Fforms%2Fform-associated-element-removal.html

fast/forms/form-associated-element-removal.html loads js-test-pre.js (so requests dumpAsText), but it emits render tree and png, so failed as MISSING.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>636418</commentid>
    <comment_count>27</comment_count>
    <who name="Ojan Vafai">ojan</who>
    <bug_when>2012-05-29 14:06:49 -0700</bug_when>
    <thetext>*** Bug 87776 has been marked as a duplicate of this bug. ***</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>647202</commentid>
    <comment_count>28</comment_count>
    <who name="Ojan Vafai">ojan</who>
    <bug_when>2012-06-12 12:58:37 -0700</bug_when>
    <thetext>An even simpler example of a test that hits this error:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&amp;tests=http%2Ftests%2Floading%2Fempty-subframe.html

The test page&apos;s contents are:
&lt;script&gt;
if (window.layoutTestController)
    layoutTestController.dumpAsText();
&lt;/script&gt;
This is a test of load callbacks. It is only useful inside the regression test tool.&lt;br&gt;
&lt;iframe name=&quot;f1&quot;&gt;&lt;/iframe&gt;

On the chromium mac bots it sometimes runs that test as a pixel test.

Up to now, I&apos;ve assumed the bug was the dumpAsText isn&apos;t getting called. I wonder if it&apos;s actually that we&apos;re overwriting the state in DRT somehow (e.g. when running tests in parallel).</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>647230</commentid>
    <comment_count>29</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-06-12 13:29:31 -0700</bug_when>
    <thetext>(In reply to comment #28)
&gt; An even simpler example of a test that hits this error:
&gt; http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&amp;tests=http%2Ftests%2Floading%2Fempty-subframe.html
&gt; 
&gt; The test page&apos;s contents are:
&gt; &lt;script&gt;
&gt; if (window.layoutTestController)
&gt;     layoutTestController.dumpAsText();
&gt; &lt;/script&gt;
&gt; This is a test of load callbacks. It is only useful inside the regression test tool.&lt;br&gt;
&gt; &lt;iframe name=&quot;f1&quot;&gt;&lt;/iframe&gt;
&gt; 
&gt; On the chromium mac bots it sometimes runs that test as a pixel test.
&gt; 
&gt; Up to now, I&apos;ve assumed the bug was the dumpAsText isn&apos;t getting called. I wonder if it&apos;s actually that we&apos;re overwriting the state in DRT somehow (e.g. when running tests in parallel).

Interesting. It seems like it would be hard for that not to be called.

I wonder if we see this issue across all the ports? I suppose on non-pixel-testing ports we&apos;d end up seeing this as a flaky text diff instead of a MISSING.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>647232</commentid>
    <comment_count>30</comment_count>
    <who name="Dirk Pranke">dpranke</who>
    <bug_when>2012-06-12 13:30:41 -0700</bug_when>
    <thetext>(In reply to comment #29)
&gt; (In reply to comment #28)
&gt; &gt; An even simpler example of a test that hits this error:
&gt; &gt; http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&amp;tests=http%2Ftests%2Floading%2Fempty-subframe.html
&gt; &gt; 
&gt; &gt; The test page&apos;s contents are:
&gt; &gt; &lt;script&gt;
&gt; &gt; if (window.layoutTestController)
&gt; &gt;     layoutTestController.dumpAsText();
&gt; &gt; &lt;/script&gt;
&gt; &gt; This is a test of load callbacks. It is only useful inside the regression test tool.&lt;br&gt;
&gt; &gt; &lt;iframe name=&quot;f1&quot;&gt;&lt;/iframe&gt;
&gt; &gt; 
&gt; &gt; On the chromium mac bots it sometimes runs that test as a pixel test.
&gt; &gt; 
&gt; &gt; Up to now, I&apos;ve assumed the bug was the dumpAsText isn&apos;t getting called. I wonder if it&apos;s actually that we&apos;re overwriting the state in DRT somehow (e.g. when running tests in parallel).
&gt; 
&gt; Interesting. It seems like it would be hard for that not to be called.
&gt; 
&gt; I wonder if we see this issue across all the ports? I suppose on non-pixel-testing ports we&apos;d end up seeing this as a flaky text diff instead of a MISSING.

This also makes me think that one thing we could do would be, if we were to get a missing pixel result, go back and look at the text we got and see if it looks like we got an actual render tree but the expected result wasn&apos;t a render tree, and treat that as a TEXT+IMAGE instead.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>700194</commentid>
    <comment_count>31</comment_count>
    <who name="Ojan Vafai">ojan</who>
    <bug_when>2012-08-20 17:00:22 -0700</bug_when>
    <thetext>*** Bug 94532 has been marked as a duplicate of this bug. ***</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>757735</commentid>
    <comment_count>32</comment_count>
    <who name="Alice Boxhall">aboxhall</who>
    <bug_when>2012-11-02 16:23:29 -0700</bug_when>
    <thetext>This is now happening on XP as well:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=fast%2Fforms%2Fformaction-attribute.html

I&apos;m going to make the test expectation general, assuming it&apos;ll start happening on other platforms as well.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>757736</commentid>
    <comment_count>33</comment_count>
    <who name="Alice Boxhall">aboxhall</who>
    <bug_when>2012-11-02 16:24:37 -0700</bug_when>
    <thetext>(In reply to comment #32)
&gt; This is now happening on XP as well:
&gt; http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=fast%2Fforms%2Fformaction-attribute.html
&gt; 
&gt; I&apos;m going to make the test expectation general, assuming it&apos;ll start happening on other platforms as well.

* for fast/forms/formaction-attribute.html only.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>911456</commentid>
    <comment_count>34</comment_count>
    <who name="Eric Seidel (no email)">eric</who>
    <bug_when>2013-07-24 16:29:51 -0700</bug_when>
    <thetext>I don&apos;t think we&apos;ve seen these failures since 2012.</thetext>
  </long_desc>
      
          <attachment
              isobsolete="1"
              ispatch="1"
              isprivate="0"
          >
            <attachid>114919</attachid>
            <date>2011-11-14 04:44:19 -0800</date>
            <delta_ts>2011-11-14 08:38:03 -0800</delta_ts>
            <desc>Patch</desc>
            <filename>bug-72039-20111114124417.patch</filename>
            <type>text/plain</type>
            <size>1473</size>
            <attacher name="Tony Gentilcore">tonyg</attacher>
            
              <data encoding="base64">U3VidmVyc2lvbiBSZXZpc2lvbjogMTAwMTI1CmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9DaGFu
Z2VMb2cgYi9MYXlvdXRUZXN0cy9DaGFuZ2VMb2cKaW5kZXggMDMyYzI5ZWNkZTZhODNiZTk4NDhh
ZmY4ZWJiN2Q3MWI5MjA1ZWUzNy4uNTNiM2RlY2JiMzE4NmI3YmRjNTA5NDBjY2M4NDdiNWIwZDQz
Zjk3MyAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3Rz
L0NoYW5nZUxvZwpAQCAtMSwzICsxLDEwIEBACisyMDExLTExLTE0ICBUb255IEdlbnRpbGNvcmUg
IDx0b255Z0BjaHJvbWl1bS5vcmc+CisKKyAgICAgICAgVGVzdHMgb2NjYXNpb25hbGx5IHJlcG9y
dCBtaXNzaW5nIGV4cGVjdGF0aW9ucworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9z
aG93X2J1Zy5jZ2k/aWQ9NzIwMzkKKworICAgICAgICAqIHBsYXRmb3JtL2Nocm9taXVtL3Rlc3Rf
ZXhwZWN0YXRpb25zLnR4dDogRXhwZWN0IGZvcm0tYXNzb2NpYXRlZC1lbGVtZW50LWNyYXNoMyB0
byBmbGFraWx5IGdvIE1JU1NJTkcuCisKIDIwMTEtMTEtMTQgIFZzZXZvbG9kIFZsYXNvdiAgPHZz
ZXZpa0BjaHJvbWl1bS5vcmc+CiAKICAgICAgICAgaHR0cC90ZXN0cy9pbnNwZWN0b3IvcmVzb3Vy
Y2UtdHJlZS9hcHBjYWNoZS1pZnJhbWUtbWFuaWZlc3RzLmh0bWwgZmFpbGluZyBvbiBzb21lIGNo
cm9taXVtIGJvdHMgYWZ0ZXIgcjk5OTg4CmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9wbGF0Zm9y
bS9jaHJvbWl1bS90ZXN0X2V4cGVjdGF0aW9ucy50eHQgYi9MYXlvdXRUZXN0cy9wbGF0Zm9ybS9j
aHJvbWl1bS90ZXN0X2V4cGVjdGF0aW9ucy50eHQKaW5kZXggMjk2NzUyZmQzZTVhYjIxYmY4ZGEw
ZmI4MzQwZDJmNjA4NWQ1M2Y0Yy4uY2Q1Y2VkYTM5MWQ0MWFlNmI5ZDNjNWM4ZTk5NjI0MmFmODE0
MzFjNCAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvcGxhdGZvcm0vY2hyb21pdW0vdGVzdF9leHBl
Y3RhdGlvbnMudHh0CisrKyBiL0xheW91dFRlc3RzL3BsYXRmb3JtL2Nocm9taXVtL3Rlc3RfZXhw
ZWN0YXRpb25zLnR4dApAQCAtMzc3Myw2ICszNzczLDcgQEAgQlVHV0s3MDc0OSBTTk9XTEVPUEFS
RCBDUFUgR1BVIDogZmFzdC9ydWJ5L2Jhc2Utc2hvcnRlci10aGFuLXRleHQuaHRtbCA9IElNQUdF
K1QKIEJVR1dLNzA4MzcgOiBjb21wb3NpdGluZy92aWRlby92aWRlby1wb3N0ZXIuaHRtbCA9IFBB
U1MgVElNRU9VVCBURVhUCiAKIEJVR1dLNzA4NjYgTElOVVggREVCVUcgOiBmYXN0L2Zvcm1zL2Zv
cm0tYXNzb2NpYXRlZC1lbGVtZW50LWNyYXNoMy5odG1sID0gVElNRU9VVAorQlVHV0s3MjAzOSBM
SU5VWCBSRUxFQVNFIDogZmFzdC9mb3Jtcy9mb3JtLWFzc29jaWF0ZWQtZWxlbWVudC1jcmFzaDMu
aHRtbCA9IE1JU1NJTkcgUEFTUwogCiBCVUdXSzcwOTcxIE1BQyA6IHN0b3JhZ2UvZG9tc3RvcmFn
ZS9ldmVudHMvYmFzaWMtc2V0YXR0cmlidXRlLmh0bWwgPSBQQVNTIENSQVNICiAK
</data>

          </attachment>
          <attachment
              isobsolete="1"
              ispatch="1"
              isprivate="0"
          >
            <attachid>114953</attachid>
            <date>2011-11-14 08:38:10 -0800</date>
            <delta_ts>2011-11-14 08:58:24 -0800</delta_ts>
            <desc>Patch</desc>
            <filename>bug-72039-20111114163808.patch</filename>
            <type>text/plain</type>
            <size>1576</size>
            <attacher name="Tony Gentilcore">tonyg</attacher>
            
              <data encoding="base64">U3VidmVyc2lvbiBSZXZpc2lvbjogMTAwMTQxCmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9DaGFu
Z2VMb2cgYi9MYXlvdXRUZXN0cy9DaGFuZ2VMb2cKaW5kZXggNTdiYjA5M2UwZmZhNmUzNmMyYjA5
OGE3YmE0ZjA3YzBmMGQ4NzhjYi4uMGEyMzdkM2EwZGUwZTgwMjRhM2ExZWYwNWUxMWVlYTVlZjll
ZjU2YiAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3Rz
L0NoYW5nZUxvZwpAQCAtMSw1ICsxLDEyIEBACiAyMDExLTExLTE0ICBUb255IEdlbnRpbGNvcmUg
IDx0b255Z0BjaHJvbWl1bS5vcmc+CiAKKyAgICAgICAgVGVzdHMgb2NjYXNpb25hbGx5IHJlcG9y
dCBtaXNzaW5nIGV4cGVjdGF0aW9ucworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9z
aG93X2J1Zy5jZ2k/aWQ9NzIwMzkKKworICAgICAgICAqIHBsYXRmb3JtL2Nocm9taXVtL3Rlc3Rf
ZXhwZWN0YXRpb25zLnR4dDogRXhwZWN0IHRlc3QgdG8gZmxha2lseSByZXBvcnQgbWlzc2luZyBl
eHBlY3RhdGlvbnMuCisKKzIwMTEtMTEtMTQgIFRvbnkgR2VudGlsY29yZSAgPHRvbnlnQGNocm9t
aXVtLm9yZz4KKwogICAgICAgICBbY2hyb21pdW1dIHNlY3VyaXR5L2NyeXB0by1yYW5kb20tdmFs
dWVzLXR5cGVzLmh0bWwgaXMgZmxha3kgb24gd2luCiAgICAgICAgIGh0dHBzOi8vYnVncy53ZWJr
aXQub3JnL3Nob3dfYnVnLmNnaT9pZD03MjI3MgogCmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9w
bGF0Zm9ybS9jaHJvbWl1bS90ZXN0X2V4cGVjdGF0aW9ucy50eHQgYi9MYXlvdXRUZXN0cy9wbGF0
Zm9ybS9jaHJvbWl1bS90ZXN0X2V4cGVjdGF0aW9ucy50eHQKaW5kZXggZGJjMzAwOWRmZjQzYTAy
NTQ2YWJjZDMzOTY3NjA1NDVmNDU4ZWFjZS4uNjA3MTA0YTRmYmY0MzhiYmFlODc3NmU1MmI5NjQw
NTQzOWRmZTFlMiAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvcGxhdGZvcm0vY2hyb21pdW0vdGVz
dF9leHBlY3RhdGlvbnMudHh0CisrKyBiL0xheW91dFRlc3RzL3BsYXRmb3JtL2Nocm9taXVtL3Rl
c3RfZXhwZWN0YXRpb25zLnR4dApAQCAtMzc3NCw2ICszNzc0LDggQEAgQlVHV0s3MDgzNyA6IGNv
bXBvc2l0aW5nL3ZpZGVvL3ZpZGVvLXBvc3Rlci5odG1sID0gUEFTUyBUSU1FT1VUIFRFWFQKIAog
QlVHV0s3MDg2NiBMSU5VWCBERUJVRyA6IGZhc3QvZm9ybXMvZm9ybS1hc3NvY2lhdGVkLWVsZW1l
bnQtY3Jhc2gzLmh0bWwgPSBUSU1FT1VUCiBCVUdXSzcyMDM5IExJTlVYIFJFTEVBU0UgOiBmYXN0
L2Zvcm1zL2Zvcm0tYXNzb2NpYXRlZC1lbGVtZW50LWNyYXNoMy5odG1sID0gTUlTU0lORyBQQVNT
CitCVUdXSzcyMDM5IExJTlVYIFJFTEVBU0UgOiBmYXN0L2Zvcm1zL2Zvcm0tYXNzb2NpYXRlZC1l
bGVtZW50LWNyYXNoLmh0bWwgPSBNSVNTSU5HIFBBU1MKK0JVR1dLNzIwMzkgTElOVVggUkVMRUFT
RSA6IGZhc3QvZm9ybXMvZm9ybS1hdHRyaWJ1dGUtZWxlbWVudHMtb3JkZXIyLmh0bWwgPSBNSVNT
SU5HIFBBU1MKIAogQlVHV0s3MDk3MSBNQUMgOiBzdG9yYWdlL2RvbXN0b3JhZ2UvZXZlbnRzL2Jh
c2ljLXNldGF0dHJpYnV0ZS5odG1sID0gUEFTUyBDUkFTSAogCg==
</data>

          </attachment>
          <attachment
              isobsolete="1"
              ispatch="1"
              isprivate="0"
          >
            <attachid>114958</attachid>
            <date>2011-11-14 08:58:31 -0800</date>
            <delta_ts>2011-11-15 06:51:49 -0800</delta_ts>
            <desc>Patch</desc>
            <filename>bug-72039-20111114165829.patch</filename>
            <type>text/plain</type>
            <size>1592</size>
            <attacher name="Tony Gentilcore">tonyg</attacher>
            
              <data encoding="base64">U3VidmVyc2lvbiBSZXZpc2lvbjogMTAwMTQ5CmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9DaGFu
Z2VMb2cgYi9MYXlvdXRUZXN0cy9DaGFuZ2VMb2cKaW5kZXggMjY0MTkwMzRjNDQzYmFkZWI1YmZm
NjMwMmQ5ZmNjM2NiY2M0ZmEwMS4uZGQ0NjU1M2Y4YzVkM2UwMWJhNTg3NDIwNjNmZWVkZWYwN2Vl
OGYyOSAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3Rz
L0NoYW5nZUxvZwpAQCAtMSw1ICsxLDEyIEBACiAyMDExLTExLTE0ICBUb255IEdlbnRpbGNvcmUg
IDx0b255Z0BjaHJvbWl1bS5vcmc+CiAKKyAgICAgICAgVGVzdHMgb2NjYXNpb25hbGx5IHJlcG9y
dCBtaXNzaW5nIGV4cGVjdGF0aW9ucworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9z
aG93X2J1Zy5jZ2k/aWQ9NzIwMzkKKworICAgICAgICAqIHBsYXRmb3JtL2Nocm9taXVtL3Rlc3Rf
ZXhwZWN0YXRpb25zLnR4dDogRXhwZWN0IHRlc3QgdG8gZmxha2lseSByZXBvcnQgbWlzc2luZyBl
eHBlY3RhdGlvbnMuCisKKzIwMTEtMTEtMTQgIFRvbnkgR2VudGlsY29yZSAgPHRvbnlnQGNocm9t
aXVtLm9yZz4KKwogICAgICAgICBtZWRpYS90cmFjay90cmFjay13ZWJ2dHQtdGMwMDQtbWFnaWMt
aGVhZGVyLmh0bWwgZmxha2lseSB0aW1lcyBvdXQKICAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtp
dC5vcmcvc2hvd19idWcuY2dpP2lkPTcyMjc5CiAKZGlmZiAtLWdpdCBhL0xheW91dFRlc3RzL3Bs
YXRmb3JtL2Nocm9taXVtL3Rlc3RfZXhwZWN0YXRpb25zLnR4dCBiL0xheW91dFRlc3RzL3BsYXRm
b3JtL2Nocm9taXVtL3Rlc3RfZXhwZWN0YXRpb25zLnR4dAppbmRleCBkNmU3OGM5NzI3ZTAwNjdi
ZmQ0ZTZkZTk5ODdjODk1NTU4OGIwNGQ4Li40YTRhZGQ5MDMxOWExNDk5MjFkNjhmYzg3NmVhYzg3
OGJkYzFmMDdjIDEwMDY0NAotLS0gYS9MYXlvdXRUZXN0cy9wbGF0Zm9ybS9jaHJvbWl1bS90ZXN0
X2V4cGVjdGF0aW9ucy50eHQKKysrIGIvTGF5b3V0VGVzdHMvcGxhdGZvcm0vY2hyb21pdW0vdGVz
dF9leHBlY3RhdGlvbnMudHh0CkBAIC0zNzYwLDYgKzM3NjAsNyBAQCBCVUdXSzcwODY2IExJTlVY
IERFQlVHIDogZmFzdC9mb3Jtcy9mb3JtLWFzc29jaWF0ZWQtZWxlbWVudC1jcmFzaDMuaHRtbCA9
IFRJTUVPVQogQlVHV0s3MjAzOSBMSU5VWCBSRUxFQVNFIDogZmFzdC9mb3Jtcy9mb3JtLWFzc29j
aWF0ZWQtZWxlbWVudC1jcmFzaDMuaHRtbCA9IE1JU1NJTkcgUEFTUwogQlVHV0s3MjAzOSBMSU5V
WCBSRUxFQVNFIDogZmFzdC9mb3Jtcy9mb3JtLWFzc29jaWF0ZWQtZWxlbWVudC1jcmFzaC5odG1s
ID0gTUlTU0lORyBQQVNTCiBCVUdXSzcyMDM5IExJTlVYIFJFTEVBU0UgOiBmYXN0L2Zvcm1zL2Zv
cm0tYXR0cmlidXRlLWVsZW1lbnRzLW9yZGVyMi5odG1sID0gTUlTU0lORyBQQVNTCitCVUdXSzcy
MDM5IExJTlVYIE1BQyBERUJVRyA6IGh0dHAvdGVzdHMvbXVsdGlwYXJ0L3BvbGljeS1pZ25vcmUt
Y3Jhc2gucGhwID0gTUlTU0lORyBQQVNTCiAKIEJVR1dLNzA5NzEgTUFDIDogc3RvcmFnZS9kb21z
dG9yYWdlL2V2ZW50cy9iYXNpYy1zZXRhdHRyaWJ1dGUuaHRtbCA9IFBBU1MgQ1JBU0gKIAo=
</data>

          </attachment>
          <attachment
              isobsolete="0"
              ispatch="1"
              isprivate="0"
          >
            <attachid>115158</attachid>
            <date>2011-11-15 06:51:57 -0800</date>
            <delta_ts>2011-11-15 06:51:57 -0800</delta_ts>
            <desc>Patch</desc>
            <filename>bug-72039-20111115145155.patch</filename>
            <type>text/plain</type>
            <size>2078</size>
            <attacher name="Tony Gentilcore">tonyg</attacher>
            
              <data encoding="base64">U3VidmVyc2lvbiBSZXZpc2lvbjogMTAwMjc1CmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9DaGFu
Z2VMb2cgYi9MYXlvdXRUZXN0cy9DaGFuZ2VMb2cKaW5kZXggYjJkZDFkZDQ3Yzg2ZmJjYmE1Njk0
NGJmZDgyNzk1ZThhMmQ4MjI4NS4uZThlMzczZTJjZmM1ZTlkNTg1OGEwNDY1ZWEzMDU4MmI5YTk4
MDkwMiAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3Rz
L0NoYW5nZUxvZwpAQCAtMSw1ICsxLDEyIEBACiAyMDExLTExLTE1ICBUb255IEdlbnRpbGNvcmUg
IDx0b255Z0BjaHJvbWl1bS5vcmc+CiAKKyAgICAgICAgVGVzdHMgb2NjYXNpb25hbGx5IHJlcG9y
dCBtaXNzaW5nIGV4cGVjdGF0aW9ucworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9z
aG93X2J1Zy5jZ2k/aWQ9NzIwMzkKKworICAgICAgICAqIHBsYXRmb3JtL2Nocm9taXVtL3Rlc3Rf
ZXhwZWN0YXRpb25zLnR4dDogQWRkIDMgbW9yZSB0ZXN0cyB3aGljaCBmbGFraWx5IHJlcG9ydCBt
aXNzaW5nLgorCisyMDExLTExLTE1ICBUb255IEdlbnRpbGNvcmUgIDx0b255Z0BjaHJvbWl1bS5v
cmc+CisKICAgICAgICAgbWVkaWEtYmxvY2tlZC1ieS13aWxsc2VuZHJlcXVlc3QuaHRtbCBpcyBm
bGFreQogICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9NzIz
ODEKIApkaWZmIC0tZ2l0IGEvTGF5b3V0VGVzdHMvcGxhdGZvcm0vY2hyb21pdW0vdGVzdF9leHBl
Y3RhdGlvbnMudHh0IGIvTGF5b3V0VGVzdHMvcGxhdGZvcm0vY2hyb21pdW0vdGVzdF9leHBlY3Rh
dGlvbnMudHh0CmluZGV4IGY3ZWRhZTVhMDRiMTIyMjA4Njg1NjJlNGQ4MzQ2MGFiNzQ0YTMwMTAu
LmFkYTY5YmI5NjFhY2MyY2E3YzI1NjA0MmMxMGM1Mzc0MmNjNGMyNmQgMTAwNjQ0Ci0tLSBhL0xh
eW91dFRlc3RzL3BsYXRmb3JtL2Nocm9taXVtL3Rlc3RfZXhwZWN0YXRpb25zLnR4dAorKysgYi9M
YXlvdXRUZXN0cy9wbGF0Zm9ybS9jaHJvbWl1bS90ZXN0X2V4cGVjdGF0aW9ucy50eHQKQEAgLTM3
NDgsOSArMzc0OCwxMiBAQCBCVUdXSzcwNzQ5IFNOT1dMRU9QQVJEIENQVSBHUFUgOiBmYXN0L3J1
YnkvYmFzZS1zaG9ydGVyLXRoYW4tdGV4dC5odG1sID0gSU1BR0UrVAogQlVHV0s3MDgzNyA6IGNv
bXBvc2l0aW5nL3ZpZGVvL3ZpZGVvLXBvc3Rlci5odG1sID0gUEFTUyBUSU1FT1VUIFRFWFQKIAog
QlVHV0s3MDg2NiBMSU5VWCBERUJVRyA6IGZhc3QvZm9ybXMvZm9ybS1hc3NvY2lhdGVkLWVsZW1l
bnQtY3Jhc2gzLmh0bWwgPSBUSU1FT1VUCi1CVUdXSzcyMDM5IExJTlVYIFJFTEVBU0UgOiBmYXN0
L2Zvcm1zL2Zvcm0tYXNzb2NpYXRlZC1lbGVtZW50LWNyYXNoMy5odG1sID0gTUlTU0lORyBQQVNT
CiBCVUdXSzcyMDM5IExJTlVYIFJFTEVBU0UgOiBmYXN0L2Zvcm1zL2Zvcm0tYXNzb2NpYXRlZC1l
bGVtZW50LWNyYXNoLmh0bWwgPSBNSVNTSU5HIFBBU1MKK0JVR1dLNzIwMzkgTElOVVggUkVMRUFT
RSA6IGZhc3QvZm9ybXMvZm9ybS1hc3NvY2lhdGVkLWVsZW1lbnQtY3Jhc2gzLmh0bWwgPSBNSVNT
SU5HIFBBU1MKK0JVR1dLNzIwMzkgTElOVVggUkVMRUFTRSA6IGZhc3QvZm9ybXMvZm9ybS1hdHRy
aWJ1dGUtZWxlbWVudHMuaHRtbCA9IE1JU1NJTkcgUEFTUworQlVHV0s3MjAzOSBMSU5VWCBSRUxF
QVNFIDogZmFzdC9mb3Jtcy9mb3JtLWF0dHJpYnV0ZS1lbGVtZW50cy1vcmRlci5odG1sID0gTUlT
U0lORyBQQVNTCiBCVUdXSzcyMDM5IExJTlVYIFJFTEVBU0UgOiBmYXN0L2Zvcm1zL2Zvcm0tYXR0
cmlidXRlLWVsZW1lbnRzLW9yZGVyMi5odG1sID0gTUlTU0lORyBQQVNTCitCVUdXSzcyMDM5IExJ
TlVYIFJFTEVBU0UgOiBmYXN0L2Zvcm1zL2Zvcm0tYXR0cmlidXRlLW5vbmV4aXN0ZW5jZS1mb3Jt
LWlkLmh0bWwgPSBNSVNTSU5HIFBBU1MKIEJVR1dLNzIwMzkgTElOVVggTUFDIERFQlVHIDogaHR0
cC90ZXN0cy9tdWx0aXBhcnQvcG9saWN5LWlnbm9yZS1jcmFzaC5waHAgPSBNSVNTSU5HIFBBU1MK
IAogQlVHV0s3MDk3MSBNQUMgOiBzdG9yYWdlL2RvbXN0b3JhZ2UvZXZlbnRzL2Jhc2ljLXNldGF0
dHJpYnV0ZS5odG1sID0gUEFTUyBDUkFTSAo=
</data>

          </attachment>
      

    </bug>

</bugzilla>