Bug 126619 – [GStreamer] http/tests/media/video-auth.html is failing

• Component: WebKitGTK
• Status: RESOLVED FIXED

[Zan Dobersek]

The http/tests/media/video-auth.html layout test is failing on GTK since introduced in r161455.
http://trac.webkit.org/changeset/161455
http://webkit-test-results.appspot.com/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fmedia%2Fvideo-auth.html

Diff:
--- /home/slave/webkitgtk/gtk-linux-64-release/build/layout-test-results/http/tests/media/video-auth-expected.txt
+++ /home/slave/webkitgtk/gtk-linux-64-release/build/layout-test-results/http/tests/media/video-auth-actual.txt
@@ -1,6 +1,5 @@
-http://127.0.0.1:8000/media/resources/video-auth.php?name=test.mp4&type=video/mp4 - didReceiveAuthenticationChallenge - Responding with username:password
  
 Tests that the media player sends authorization credentials when requesting a media file.  
-EVENT(canplay)
+EVENT(error) TEST(false) FAIL
 END OF TEST

[Michal Pakula vel Rutka]

This test is failing on EFL bot too:
http://webkit-test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fmedia%2Fvideo-auth.html

The diff is the same.

[youenn fablet]

Created attachment 231881 [details]
Patch

[Eric Carlson]

Comment on attachment 231881 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=231881&action=review

> LayoutTests/ChangeLog:35
> [phone]  Youenn Fablet  <youenn.fablet>
> +
> +        [GTK] http/tests/media/video-auth.html is failing
> +        https://bugs.webkit.org/show_bug.cgi?id=126619
> +
> +        Reviewed by NOBODY (OOPS!).
> +
> +        Enabled http/tests/media/video-auth.html for gtk and efl-wk2 ports.
> +
> +        * platform/efl-wk1/TestExpectations:
> +        * platform/efl/TestExpectations:
> +        * platform/efl/http/tests/media/video-auth-expected.txt: Added.
> +        * platform/gtk/TestExpectations:
> +        * platform/gtk/http/tests/media/video-auth-expected.txt: Added.
> +

This extra entry is not needed.

> LayoutTests/http/tests/media/resources/video-auth.php:6
> +    header("Access-Control-Allow-Origin: http://127.0.0.1:8000");
> +    header("Access-Control-Allow-Credentials: true");

Are these now necessary in video-auth.html as well? Probably worth making them conditional on a new parameter in the url.

> LayoutTests/http/tests/media/video-auth-cors.html:24
> +                        source.src = 'http://localhost:8000/media/resources/video-auth.php?name=' + movie + '&type=' + type;

"localhost"? How does this work?

> LayoutTests/http/tests/media/video-auth-cors.html:43
> +        Tests that the media player does not request authorization credentials to user when requesting a media file with CORS option.

Nit: "authorization credentials to user" -> "authorization credentials from user"

[Build Bot]

Comment on attachment 231881 [details]
Patch

Attachment 231881 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/6444025636716544

New failing tests:
http/tests/media/video-auth-cors.html
http/tests/media/video-buffered-range-contains-currentTime.html
http/tests/media/video-throttled-load-metadata.html
http/tests/media/video-served-as-text.html
http/tests/media/video-error-does-not-exist.html
http/tests/media/video-error-abort.html

[Build Bot]

Created attachment 231888 [details]
Archive of layout-test-results from webkit-ews-04 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-04  Port: mac-mountainlion  Platform: Mac OS X 10.8.5

[Build Bot]

Comment on attachment 231881 [details]
Patch

Attachment 231881 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.appspot.com/results/5050869779267584

New failing tests:
http/tests/media/video-auth-cors.html
http/tests/media/video-buffered-range-contains-currentTime.html
http/tests/media/video-throttled-load-metadata.html
http/tests/media/video-served-as-text.html
http/tests/media/video-error-does-not-exist.html
http/tests/media/video-error-abort.html

[Build Bot]

Created attachment 231890 [details]
Archive of layout-test-results from webkit-ews-11 for mac-mountainlion-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: webkit-ews-11  Port: mac-mountainlion-wk2  Platform: Mac OS X 10.8.5

[Build Bot]

Comment on attachment 231881 [details]
Patch

Attachment 231881 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/5825110648815616

New failing tests:
http/tests/media/video-auth-cors.html
http/tests/media/video-buffered-range-contains-currentTime.html
http/tests/media/video-throttled-load-metadata.html
http/tests/media/video-served-as-text.html
http/tests/media/video-error-does-not-exist.html
http/tests/media/video-error-abort.html

[Build Bot]

Created attachment 231891 [details]
Archive of layout-test-results from webkit-ews-08 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-08  Port: mac-mountainlion  Platform: Mac OS X 10.8.5

[youenn fablet]

Thanks for the review.
More comments below.

> > LayoutTests/http/tests/media/resources/video-auth.php:6
> > +    header("Access-Control-Allow-Origin: http://127.0.0.1:8000");
> > +    header("Access-Control-Allow-Credentials: true");
> 
> Are these now necessary in video-auth.html as well? Probably worth making them conditional on a new parameter in the url.

I will add a cors=true URL parameter
 
> > LayoutTests/http/tests/media/video-auth-cors.html:24
> > +                        source.src = 'http://localhost:8000/media/resources/video-auth.php?name=' + movie + '&type=' + type;
> 
> "localhost"? How does this work?

localhost is working fine in my setup, not sure what is wrong with it.
I can change it to 127.0.01:8080

 
> > LayoutTests/http/tests/media/video-auth-cors.html:43
> > +        Tests that the media player does not request authorization credentials to user when requesting a media file with CORS option.
> 
> Nit: "authorization credentials to user" -> "authorization credentials from user"

OK

[youenn fablet]

(In reply to comment #8)
> (From update of attachment 231881 [details])
> Attachment 231881 [details] did not pass mac-ews (mac):
> Output: http://webkit-queues.appspot.com/results/5825110648815616
> 
> New failing tests:
> http/tests/media/video-auth-cors.html

Test is new and could fail.

> http/tests/media/video-buffered-range-contains-currentTime.html
> http/tests/media/video-throttled-load-metadata.html
> http/tests/media/video-served-as-text.html
> http/tests/media/video-error-does-not-exist.html
> http/tests/media/video-error-abort.html

All these tests fail due to a timeout error.
I do not know why since the patch is GTK/EFL specific.
I will try again with a new patch updated according above comments.

[youenn fablet]

Created attachment 232081 [details]
Updated according first comments

[Build Bot]

Comment on attachment 232081 [details]
Updated according first comments

Attachment 232081 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/5615623350517760

New failing tests:
http/tests/media/video-auth-cors.html
http/tests/media/video-buffered-range-contains-currentTime.html
http/tests/media/video-throttled-load-metadata.html
http/tests/media/video-served-as-text.html
http/tests/media/video-error-does-not-exist.html
http/tests/media/video-error-abort.html

[Build Bot]

Created attachment 232087 [details]
Archive of layout-test-results from webkit-ews-05 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-05  Port: mac-mountainlion  Platform: Mac OS X 10.8.5

[Build Bot]

Comment on attachment 232081 [details]
Updated according first comments

Attachment 232081 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.appspot.com/results/5523401812738048

New failing tests:
http/tests/media/video-auth-cors.html
http/tests/media/video-buffered-range-contains-currentTime.html
http/tests/media/video-throttled-load-metadata.html
http/tests/media/video-served-as-text.html
http/tests/media/video-error-does-not-exist.html
http/tests/media/video-error-abort.html

[Build Bot]

Created attachment 232089 [details]
Archive of layout-test-results from webkit-ews-13 for mac-mountainlion-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: webkit-ews-13  Port: mac-mountainlion-wk2  Platform: Mac OS X 10.8.5

[Build Bot]

Comment on attachment 232081 [details]
Updated according first comments

Attachment 232081 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/6562067444137984

New failing tests:
http/tests/media/video-auth-cors.html
http/tests/media/video-buffered-range-contains-currentTime.html
http/tests/media/video-throttled-load-metadata.html
http/tests/media/video-served-as-text.html
http/tests/media/video-error-does-not-exist.html
http/tests/media/video-error-abort.html

[Build Bot]

Created attachment 232090 [details]
Archive of layout-test-results from webkit-ews-03 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-03  Port: mac-mountainlion  Platform: Mac OS X 10.8.5

[youenn fablet]

Created attachment 232612 [details]
Removing cors additional test

[Philippe Normand]

Comment on attachment 232612 [details]
Removing cors additional test

Sorry I had forgotten about this patch.

[WebKit Commit Bot]

Comment on attachment 232612 [details]
Removing cors additional test

Rejecting attachment 232612 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.appspot.com', '--bot-id=webkit-cq-02', 'apply-attachment', '--no-update', '--non-interactive', 232612, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
urce/WebCore/platform/graphics/gstreamer/WebKitWebSourceGStreamer.cpp.rej
patching file LayoutTests/ChangeLog
Hunk #1 succeeded at 1 with fuzz 3.
patching file LayoutTests/platform/gtk/TestExpectations
Hunk #1 succeeded at 1852 with fuzz 2 (offset 104 lines).
patching file LayoutTests/platform/gtk/http/tests/media/video-auth-expected.txt

Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force', '--reviewer', u'Philippe Normand']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Full output: http://webkit-queues.appspot.com/results/5587409903288320

[youenn fablet]

(In reply to comment #21)
> (From update of attachment 232612 [details])
> Rejecting attachment 232612 [details] from commit-queue.
> 
> Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.appspot.com', '--bot-id=webkit-cq-02', 'apply-attachment', '--no-update', '--non-interactive', 232612, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit
> 
> Last 500 characters of output:
> urce/WebCore/platform/graphics/gstreamer/WebKitWebSourceGStreamer.cpp.rej
> patching file LayoutTests/ChangeLog
> Hunk #1 succeeded at 1 with fuzz 3.
> patching file LayoutTests/platform/gtk/TestExpectations
> Hunk #1 succeeded at 1852 with fuzz 2 (offset 104 lines).
> patching file LayoutTests/platform/gtk/http/tests/media/video-auth-expected.txt
> 
> Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force', '--reviewer', u'Philippe Normand']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit
> 
> Full output: http://webkit-queues.appspot.com/results/5587409903288320

I will rebase the patch.

[youenn fablet]

Created attachment 238119 [details]
Rebasing and updating EFL test expectation

[Philippe Normand]

Comment on attachment 238119 [details]
Rebasing and updating EFL test expectation

Looking at the default expected result, it seems that we now only fails with the first line that should also display the URL. Can you have a look at fixing this? See LayoutTests/http/tests/media/video-auth-expected.txt

[Philippe Normand]

Should be a small patch for the gtk/efl WebKit TestRunner, I think.

[youenn fablet]

(In reply to comment #25)
> Should be a small patch for the gtk/efl WebKit TestRunner, I think.

I think I looked at it in the past.
IIRC, it was requiring to rebase some other tests.
I will look at it again.

[youenn fablet]

(In reply to comment #24)
> (From update of attachment 238119 [details])
> Looking at the default expected result, it seems that we now only fails with the first line that should also display the URL. Can you have a look at fixing this? See LayoutTests/http/tests/media/video-auth-expected.txt

Mac port is displaying the URL that triggers the auth challenge.
GTK auth challenge callback has access to the page and frame URLs but not to this URL right now.

We could extend AuthenticationChallengeProxy to get access to the whole response if that is worth it. What do you think?

[Philippe Normand]

(In reply to comment #27)
> (In reply to comment #24)
> > (From update of attachment 238119 [details] [details])
> > Looking at the default expected result, it seems that we now only fails with the first line that should also display the URL. Can you have a look at fixing this? See LayoutTests/http/tests/media/video-auth-expected.txt
> 
> Mac port is displaying the URL that triggers the auth challenge.

Only on WK1/DRT. It looks like the Mac WK2 bots don't run this test at all, it's flagged in TestExpectations.

> GTK auth challenge callback has access to the page and frame URLs but not to this URL right now.
> 
> We could extend AuthenticationChallengeProxy to get access to the whole response if that is worth it. What do you think?

I'm not sure what we're stepping into here :)
DRT prints the URL but WKR doesn't. So for now I think it's best to keep our platform expectations and open a separate bug about this... IMHO WKR should be coherent with DRT in that regard but it's not up to me to decide ;)

[youenn fablet]

(In reply to comment #28)
> (In reply to comment #27)
> > (In reply to comment #24)
> > > (From update of attachment 238119 [details] [details] [details])
> > > Looking at the default expected result, it seems that we now only fails with the first line that should also display the URL. Can you have a look at fixing this? See LayoutTests/http/tests/media/video-auth-expected.txt
> > 
> > Mac port is displaying the URL that triggers the auth challenge.
> 
> Only on WK1/DRT. It looks like the Mac WK2 bots don't run this test at all, it's flagged in TestExpectations.
> 
> > GTK auth challenge callback has access to the page and frame URLs but not to this URL right now.
> > 
> > We could extend AuthenticationChallengeProxy to get access to the whole response if that is worth it. What do you think?
> 
> I'm not sure what we're stepping into here :)

Right.

> DRT prints the URL but WKR doesn't. So for now I think it's best to keep our platform expectations and open a separate bug about this... IMHO WKR should be coherent with DRT in that regard but it's not up to me to decide ;)

I filed bug 136847

[WebKit Commit Bot]

Comment on attachment 238119 [details]
Rebasing and updating EFL test expectation

Clearing flags on attachment: 238119

Committed r173654: <http://trac.webkit.org/changeset/173654>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.