Bug 120482 – [GStreamer] - Playing 2 mpeg2ts videos from Apache Server causes lockup

• Status: RESOLVED
• Resolution: WORKSFORME
• Component: Media

[Lori Anderson]

I see playback stall when attempting to playback two videos simultaneously.  I am using GStreamer with SoupHttpSrc rather than WebkitWebSource.  I believe the issue is related to the buffering and state transitions initiated by Queue2 GStreamer element which is added by UriDecodeBin to GStreamer Playbin.  I do not see this issue when using WebKitWebSource because there is no Queue2 element present and hence no buffering and no need to pause/resume. 

It appears from the log messages that the autoaudiosink1-actual-sink-alsa element fails to complete an asynchronous state transition from PLAYING to PAUSED.  Since the state transition never complete, the video stalls because MediaPlayerPrivateGStreamer::updateStates() sees that there is an async state transition in progress.  

I sometimes see this issue when there is only one video element (one MediaPlayerPrivateGStreamer) but it happens every time when there are two players.  It appears related to timing because adjusting log message levels can affect the frequency at which the issue occurs or how soon it happens when playing back.  

I tried to create a simple GStreamer program to try and reproduce the issue. The program created two pipelines and initiated a state change from paused to playing, back and forth, every 5 seconds.  I was unable to reproduce the issue using just GStreamer.  

I have tried this scenario with another WebServer and I do not see the issue.  If I turn on buffering log messages, it appears that Queue2 is being under run, initiating all the state transitions.

I have tweaked log levels in order to produce a somewhat concise log which I think shows the problem.  If you look at fail.log (see attachment), prior to lock up, you will see this message: "child 'autoaudiosink0-actual-sink-alsa' is changing state asynchronously to PAUSED".  From additional runs that I have done and various logging combinations I have tried, I suspect that the alsasink element does not transition state for some reason. 

I will attach the *.png which shows the pipeline elements.

I will also attempt to attach the actual mpeg2 ts file.

[Lori Anderson]

Created attachment 210009 [details]
Pipeline diagram showing gstreamer elements

[Lori Anderson]

Created attachment 210011 [details]
Log file from lock up run with one video

[Lori Anderson]

Here is a link to the video stream:

https://www.dropbox.com/sc/1h5qohtw2rh96ty/ZkHjgcjib4

[Philippe Normand]

Why aren't you using the webkitsrc element? It internally uses appsrc which, IIRC, has an internal queue.

If the issue doesn't happen when webkitsrc is used I don't think I'll bother to work on this, to be honest.

[Lori Anderson]

(In reply to comment #4)
> Why aren't you using the webkitsrc element? It internally uses appsrc which, IIRC, has an internal queue.
> 
> If the issue doesn't happen when webkitsrc is used I don't think I'll bother to work on this, to be honest.

We are working on a GStreamer DLNA "manager" element which will encapsulate GStreamer souphttpsrc and provide additional capabilities by issuing a HEAD request and gathering more detailed DLNA specific information about the server.  One of the major requirements is that an additional gstreamer filter element needs to be setup if the content coming from the server is encrypted (needs a decryption filter included after the souphttpsrc).  

This GStreamer DLNA manager element in also needed for another DMP which is outside of WebKit framework.  The plan is to submit this element for inclusion in the GStreamer open source project.  This source/manager element needs to work within and outside of the WebKit framework that is why we could not use the webkitsrc element.


The WebKitSrc does use an appsrc where souphttpsrc does not.  When playbin autoplugs the pipeline, uridecodebin queries the source.  Souphttpsrc responds with "bandwidth_limited", so Queue2 gets included where WebKitSrc does not since it already has its queue. 

I can understand your reluctance to work on this issue since it does not use WebKitSrc.  I'm stuck here because I can't get a simple gstreamer pipeline to reproduce the issue, currently I can only reproduce it within the WebKit framework.  

My best guess at this point on time is this is an issue with alsasink and its asynchronous state changes within autoaudiosink.  What would you suggest?  I think I will go back to gstreamer chat and see if anyone has any insight on why alsasink would have issues with asnyc state transitions.

I do have a workaround to this issue for fixed duration content.  I can enable the download buffering on uridecodebin which buffers everything to a file and eliminates the state transitions, hence eliminates the problem with alsasink state change.  Unfortunately, this won't work for live streams.

[Philippe Normand]

Ah, perhaps the patch from bug 108088 can help you. Among other things it makes the src element handle the SCHEDULING query and set the BANDWIDTH_LIMITED flag.

[Lori Anderson]

OK - I'll give that a try - thanks

[Lori Anderson]

I have a one line fix for this issue.  If I set "async-handling" property of autoaudiosink to true, the state is no longer stuck when trying to go into paused state and I no longer see this lockup.

[Lori Anderson]

Created attachment 213040 [details]
One line fix which prevents stall due to state transistions.

[Philippe Normand]

Setting async-handling shouldn't be needed I think, looks more like a workaround to me.

Which GStreamer version are you using?

[Lori Anderson]

(In reply to comment #10)
> Setting async-handling shouldn't be needed I think, looks more like a workaround to me.

I agree it is a workaround.  The problem is this problem only occurs in WebKit using an HTTP source which is not the webkit source.  The reason the source is needed is explained above.  A simple gstreamer program with the same pipeline does not exhibit this problem.  When using the non-webkit source, a Queue2 element is included in the pipeline. A queue2 element is not included when you are using the webkit source because it is an appsrc and has an internal queue built in.  The problem with Queue2 is that it causes many state transitions between playing and paused to handle the min and max thresholds of buffering.  Some kind of deadly embrace happens within updateStates().  The "deadly embrace" is that pipeline never transitions into paused and although the buffering reaches %100 it never restarts the pipeline to playing because of the logic in updateStates().  

I believe the proper fix would require some extensive changes to management of the pipeline state, waiting for async state transitions to occur and mutex control over threads setting pipeline state.       
> 
> Which GStreamer version are you using?

Tip of GStreamer development branch.

[Lori Anderson]

One more point about the supplied workaround...  The description of the "async-handling" property description from gstbin.c

  /**
   * GstBin:async-handling:
   *
   * If set to #TRUE, the bin will handle asynchronous state changes.
   * This should be used only if the bin subclass is modifying the state
   * of its children on its own.
   */
 
If you look in gstautoaudiosink.c you can see that is modifies the state of its child on its own.  It could be that this is necessary.  

The counterpoint is this isn't needed when running the same pipeline in a gstreamer test program, it is just needed when running within WebKit.  So it still boils down to some interaction within WebKit state management.

[Philippe Normand]

Hey Lori, is this issue still happening?

[Philippe Normand]

No reply in almost 2 years. Feel free to reopen if the issue is still present!