Bug 72698 - Audio object garbage collection is incorrect (media/audio-garbage-collect.html test is flaky)
Summary: Audio object garbage collection is incorrect (media/audio-garbage-collect.htm...
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Philippe Normand
URL:
Keywords: Gtk, InRadar, LayoutTestFailure
: 117555 (view as bug list)
Depends on: 82537
Blocks:
  Show dependency treegraph
 
Reported: 2011-11-18 00:53 PST by Philippe Normand
Modified: 2013-10-11 03:16 PDT (History)
11 users (show)

See Also:


Attachments
proposed patch (2.04 KB, patch)
2011-11-18 01:00 PST, Philippe Normand
no flags Details | Formatted Diff | Diff
Patch (3.06 KB, patch)
2012-03-07 05:51 PST, Philippe Normand
no flags Details | Formatted Diff | Diff
non-working fix (2.78 KB, patch)
2013-09-23 10:48 PDT, Alexey Proskuryakov
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Normand 2011-11-18 00:53:31 PST
There are two issues:

- the test should wait for "seeked" before setting element to play
- even by fixing the first issue the test remains flaky because of the test just before, it seems, audio-delete-while-step-button-clicked.html
Comment 1 Philippe Normand 2011-11-18 01:00:20 PST
Created attachment 115760 [details]
proposed patch

Not setting r flag because of the second issue mentioned above.
Comment 2 Philippe Normand 2011-11-18 01:05:37 PST
Marked flaky in http://trac.webkit.org/changeset/100733
Comment 3 Eugene Nalimov 2011-11-21 13:49:38 PST
According to

http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.htm

it is perfectly Ok to run play() on audio object immediately after setting its currentTime. E.g. there is example on that page that contains

  function playSound(id) {
    sfx.currentTime = sounds.getCueById(id).startTime;
    sfx.play();
  }

So I don't believe flakiness is caused by calling play() after setting currentTime -- and you still see flakiness when you apply your patch.

Can the test somehow fail because of the previous test failure? Are they using the same browser instance?
Comment 4 Philippe Normand 2011-11-22 00:32:50 PST
(In reply to comment #3)
> According to
> 
> http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.htm
> 
> it is perfectly Ok to run play() on audio object immediately after setting its currentTime. E.g. there is example on that page that contains
> 
>   function playSound(id) {
>     sfx.currentTime = sounds.getCueById(id).startTime;
>     sfx.play();
>   }
> 
> So I don't believe flakiness is caused by calling play() after setting currentTime

Right, indeed!

> -- and you still see flakiness when you apply your patch.
> 
> Can the test somehow fail because of the previous test failure? Are they using the same browser instance?

Well yes, a single WebView is used per instance of DRT. We currently limit NRWT to one worker.
Comment 5 Philippe Normand 2012-03-06 06:52:19 PST
I dedicated some hours to this issue today... Here are my findings:

- test executed standalone: works
- test executed after a video test: works
- test executed after an audio test: times out

If I replace:

var a = new Audio(audioFile);

with:

setSrcByTagName("audio", findMediaFile("audio", "content/silence"));
var a = document.getElementById('audio');

(and a <audio id="audio"/> element in the body).

And:

a = null;

with:

a.parentNode.removeChild(a);

The test passes in all cases. Could this be a JSC-related issue?
Comment 6 Eugene Nalimov 2012-03-06 08:26:20 PST
When you got rid of
  a = null;
you are avoiding situation test is checking -- variable "a" still contains reference to audio object, so it would not be garbage collected regardless of its status. Test checks that audio object cannot be collected while it is playing even if there is no reference to it left.

When you are saying "JSC" do you mean JS interpreter/JIT-ter used in Safari? Or is the test flaky with Chromium as well?
Comment 7 Philippe Normand 2012-03-06 08:51:54 PST
(In reply to comment #6)
> When you got rid of
>   a = null;
> you are avoiding situation test is checking -- variable "a" still contains reference to audio object, so it would not be garbage collected regardless of its status. Test checks that audio object cannot be collected while it is playing even if there is no reference to it left.
> 

Alright.

> When you are saying "JSC" do you mean JS interpreter/JIT-ter used in Safari? Or is the test flaky with Chromium as well?

This issue happens on the GTK port of WebKit, which uses JavaScriptCore.
Comment 8 Eugene Nalimov 2012-03-06 09:09:28 PST
Then my guess would be that JSC/GTK contains bug similar to v8 one I fixed.

I had to fix 2 different bugs in V8:
* One in HTMLAudioElement, it was not "active" object (i.e. object that can have some hidden activity, meaning that it cannot be collected before it finishes that activity).
* Second bug was is v8 bindings, code handling active objects was not designed to work with active objects derived from "Node".

For details please see
https://bugs.webkit.org/show_bug.cgi?id=66878
https://bugs.webkit.org/show_bug.cgi?id=70421
Comment 9 Philippe Normand 2012-03-07 05:33:25 PST
(In reply to comment #8)
> Then my guess would be that JSC/GTK contains bug similar to v8 one I fixed.
> 
> I had to fix 2 different bugs in V8:
> * One in HTMLAudioElement, it was not "active" object (i.e. object that can have some hidden activity, meaning that it cannot be collected before it finishes that activity).
> * Second bug was is v8 bindings, code handling active objects was not designed to work with active objects derived from "Node".
> 
> For details please see
> https://bugs.webkit.org/show_bug.cgi?id=66878
> https://bugs.webkit.org/show_bug.cgi?id=70421

I think this is indeed a bug in the JSC bindings, patch incoming. Thanks a lot for the hints Eugene.
Comment 10 Philippe Normand 2012-03-07 05:51:50 PST
Created attachment 130608 [details]
Patch
Comment 11 Eugene Nalimov 2012-03-07 08:22:38 PST
Comment on attachment 130608 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=130608&action=review

> Source/WebCore/bindings/js/JSNodeCustom.cpp:119
> +            if (!static_cast<HTMLAudioElement*>(node)->hasPendingActivity())

hasPendingActivity() is definitely better than paused(), but why it matters for that particular case?
Comment 12 Philippe Normand 2012-03-07 08:37:54 PST
(In reply to comment #11)
> (From update of attachment 130608 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=130608&action=review
> 
> > Source/WebCore/bindings/js/JSNodeCustom.cpp:119
> > +            if (!static_cast<HTMLAudioElement*>(node)->hasPendingActivity())
> 
> hasPendingActivity() is definitely better than paused(), but why it matters for that particular case?

Because the media element ends up paused and thus without this patch prevents the garbage collection to succeed (I think).

HTMLAudioElement::hasPendingActivity() first checks isPlaying(), which is false.

So it seems that witch this patch the correct code path is used during the test and it doesn't time out.
Comment 13 Eugene Nalimov 2012-03-07 08:48:29 PST
LGTM, but I don't know how to make it official, and I don't have commit rights...
Comment 14 Xan Lopez 2012-03-07 09:20:28 PST
Comment on attachment 130608 [details]
Patch

r=me since this does what the image element is already doing and fixes the test.
Comment 15 Philippe Normand 2012-03-07 09:27:32 PST
Comment on attachment 130608 [details]
Patch

Clearing flags on attachment: 130608

Committed r110064: <http://trac.webkit.org/changeset/110064>
Comment 16 Philippe Normand 2012-03-07 09:27:43 PST
All reviewed patches have been landed.  Closing bug.
Comment 17 Alexey Proskuryakov 2012-03-28 11:56:20 PDT
This has caused a serious regression, bug 82421.

Bugs with [platform] prefixes should never have patches that change cross platform code.

This is because most core developers don't look at such bugs, so they effectively get less scrutiny. Please be very careful to only use the [GTK] prefix when you expect that the issue is purely port specific, and remove the prefix if it turns out to be cross platform.

Also, you should have CC'ed folks working on media elements, and on JavaScriptCore bindings. This code has been looked at many times in the past, so there may be subtle consequences to changes like this.
Comment 18 Geoffrey Garen 2012-03-28 12:52:46 PDT
> Bugs with [platform] prefixes should never have patches that change cross platform code.

Eugene, Philippe, Xan, a number of bad things happened here:

(1) As Alexey mentioned, you marked this patch as a [platform] bug, when it's not.

(2) Eugene asserted without evidence that "hasPendingActivity() is definitely better than paused()", and didn't consult any media experts or anyone in the SVN blame list for this code about his assertion.

(3) Nobody tested that, after this change, audio elements could still be garbage collected.

(4) Xan r+-ed this patch despite all the above problems.

Please use better judgement in the future.
Comment 19 Martin Robinson 2012-03-28 13:08:58 PDT
(In reply to comment #18)
> (1) As Alexey mentioned, you marked this patch as a [platform] bug, when it's not.

It is often the case that a bug will start out to be platform-specific (in this case it was some GTK+ tests showing flakiness) and later gradually turn into a platform-independent bug. People, by nature, are forgetful or busy or tired or overwhelmed with work, so often forget to remove tags like "[GTK]."

I think there are some avenues for improving this situation though:

1. People could make sure to add themsevles to a watchlist for files they care about, which will CC them automatically.
2. check-webkit-style could be taught to look out for patches that contain strings like "[GTK]"  and change files outside of the platform-specific directories.
Comment 20 Stephanie Lewis 2012-03-28 18:03:30 PDT
Rolled out in http://trac.webkit.org/projects/webkit/changeset/112483.  Reopening this bug.

Eric's comments from https://bugs.webkit.org/show_bug.cgi?id=82421

Comment #9 From Eric Carlson 2012-03-28 17:41:24 PST (-) [reply]
(In reply to comment #6)
> I think it would be even better just to revert http://trac.webkit.org/projects/webkit/changeset/110064 and consult Eric and Jer about what the best fix is. It's not clear that HTMLAudioElement::hasPendingActivity is the right test for GC.

HTMLAudioElement::hasPendingActivity is definitely NOT the correct test. The reason that we care about this is that the spec says when a media element can be collected:

    Media elements that are potentially playing while not in a Document 
    must not play any video, but should play any audio component. Media 
    elements must not stop playing just because all references to them 
    have been removed; only once a media element is in a state where 
    no further audio could ever be played by that element may the 
    element be garbage collected.

and "potentially playing" is defined as:

    A media element is said to be potentially playing when its paused 
    attribute is false, the element has not ended playback, playback has 
    not stopped due to errors, the element either has no current media 
    controller or has a current media controller but is not blocked on its 
    media controller, and the element is not a blocked media element.

As it happens, we have HTMLMediaElement::potentiallyPlaying because "potentially playing" is used in other parts of the media element spec.
Comment 21 Philippe Normand 2012-03-30 01:14:12 PDT
I'm so sorry for all the trouble caused. I'll use better judgement in the future, especially about bug title tags CC'ed people.
Comment 22 Alexey Proskuryakov 2013-09-18 14:03:32 PDT
*** Bug 117555 has been marked as a duplicate of this bug. ***
Comment 23 Alexey Proskuryakov 2013-09-23 10:48:28 PDT
Created attachment 212365 [details]
non-working fix

Attaching my unsuccessful cut at fixing this.

Changed WebCore to use potentiallyPlaying, and made the test more strict - currently it's pretty much ineffective, as it only calls gc() when a pointer is almost certain to be on the stack anyway. Looks like issues with Audio garbage collection are deeper yet.
Comment 24 Alexey Proskuryakov 2013-09-23 10:56:33 PDT
Skipped the test in <http://trac.webkit.org/r156280>.

<rdar://problem/15056308>