Bug 150449 - [MediaStream] Play MediaStream through media element and render to canvas
Summary: [MediaStream] Play MediaStream through media element and render to canvas
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Eric Carlson
URL:
Keywords: InRadar
Depends on:
Blocks: 147125 150195
  Show dependency treegraph
 
Reported: 2015-10-22 10:28 PDT by Eric Carlson
Modified: 2016-10-30 13:20 PDT (History)
5 users (show)

See Also:


Attachments
Patch for the bots. (115.63 KB, patch)
2015-10-27 21:17 PDT, Eric Carlson
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews113 for mac-yosemite (820.67 KB, application/zip)
2015-10-27 22:14 PDT, Build Bot
no flags Details
Updated patch. (125.47 KB, patch)
2015-10-28 09:09 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff
Updated patch. (125.52 KB, patch)
2015-10-28 09:25 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff
Another updated patch. (125.58 KB, patch)
2015-10-28 09:37 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff
Another updated patch. (125.64 KB, patch)
2015-10-28 09:57 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff
Another updated patch. (125.63 KB, patch)
2015-10-28 10:11 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff
Patch for landing. (118.15 KB, patch)
2015-10-28 14:35 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff
Updated path for landing. (118.19 KB, patch)
2015-10-28 16:59 PDT, Eric Carlson
commit-queue: commit-queue-
Details | Formatted Diff | Diff
Patch for landing. (118.18 KB, patch)
2015-10-28 21:36 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Carlson 2015-10-22 10:28:01 PDT
Enable a MediaStream to be played through a media element and rendered to a canvas.
Comment 1 Eric Carlson 2015-10-27 21:17:42 PDT
Created attachment 264192 [details]
Patch for the bots.
Comment 2 Build Bot 2015-10-27 22:14:29 PDT
Comment on attachment 264192 [details]
Patch for the bots.

Attachment 264192 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/347124

New failing tests:
media/video-source-error-no-candidate.html
media/W3C/video/error/error_onerror_called_on_bogus_source.html
http/tests/media/text-served-as-text.html
media/video-src-change.html
media/W3C/audio/error/error_onerror_called_on_bogus_source.html
media/invalid-media-url-crash.html
http/tests/media/pdf-served-as-pdf.html
Comment 3 Build Bot 2015-10-27 22:14:33 PDT
Created attachment 264194 [details]
Archive of layout-test-results from ews113 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 4 Eric Carlson 2015-10-28 09:09:51 PDT
Created attachment 264214 [details]
Updated patch.
Comment 5 Eric Carlson 2015-10-28 09:25:23 PDT
Created attachment 264215 [details]
Updated patch.
Comment 6 Eric Carlson 2015-10-28 09:37:27 PDT
Created attachment 264217 [details]
Another updated patch.
Comment 7 Eric Carlson 2015-10-28 09:57:33 PDT
Created attachment 264219 [details]
Another updated patch.
Comment 8 Eric Carlson 2015-10-28 10:11:52 PDT
Created attachment 264223 [details]
Another updated patch.
Comment 9 Jer Noble 2015-10-28 13:04:23 PDT
Comment on attachment 264223 [details]
Another updated patch.

View in context: https://bugs.webkit.org/attachment.cgi?id=264223&action=review

r=me, with nits.

> Source/WebCore/Modules/mediastream/MediaStream.cpp:264
> +        // FIXME: This can't be right, the event type is set from m_isActive in scheduleActiveStateChange!

I don't see a ChangeLog entry for this. Does this need a bugzilla bug?

> Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp:83
> +MediaStreamPrivate::~MediaStreamPrivate()
> +{
> +}

Don't you need to remove the observers from the tracks in m_trackSet here?

> Source/WebCore/platform/mediastream/MediaStreamPrivate.h:76
> +    MediaStream* publicStream() const { return m_publicStream; }
> +    void setPublicStream(MediaStream* stream) { m_publicStream = stream; }

This is a _little_ concerning. I realize that we're not pulling in MediaStream.h here, just its forward declaration, but it still seems like it may be a layering violation, or at least, would allow layering violations to propagate.

If the point of this method is to allow clients of MediaStreamPrivate to connect to a MediaStream, perhaps we could have a static method in MediaStream, which, when given a MediaStreamPrivate returns the MediaStream associated with that private?

> Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.mm:84
> +    // FIXME: finish this implementation

bug #?

> Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.mm:96
> -    {
> -        LockHolder lock(m_lock);
> -        m_observers.append(observer);
> -    }
> -
> +    LockHolder lock(m_lock);
> +    m_observers.append(observer);

The ChangeLog just says "hold the lock while calling observers", but not why.
Comment 10 Eric Carlson 2015-10-28 14:35:22 PDT
Comment on attachment 264223 [details]
Another updated patch.

View in context: https://bugs.webkit.org/attachment.cgi?id=264223&action=review

>> Source/WebCore/Modules/mediastream/MediaStream.cpp:264
>> +        // FIXME: This can't be right, the event type is set from m_isActive in scheduleActiveStateChange!
> 
> I don't see a ChangeLog entry for this. Does this need a bugzilla bug?

Fixed.

>> Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp:83
>> +}
> 
> Don't you need to remove the observers from the tracks in m_trackSet here?

Oops, fixed!

>> Source/WebCore/platform/mediastream/MediaStreamPrivate.h:76
>> +    void setPublicStream(MediaStream* stream) { m_publicStream = stream; }
> 
> This is a _little_ concerning. I realize that we're not pulling in MediaStream.h here, just its forward declaration, but it still seems like it may be a layering violation, or at least, would allow layering violations to propagate.
> 
> If the point of this method is to allow clients of MediaStreamPrivate to connect to a MediaStream, perhaps we could have a static method in MediaStream, which, when given a MediaStreamPrivate returns the MediaStream associated with that private?

As discussed on IRC, fixed.

>> Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.mm:84
>> +    // FIXME: finish this implementation
> 
> bug #?

Added.

>> Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.mm:96
>> +    m_observers.append(observer);
> 
> The ChangeLog just says "hold the lock while calling observers", but not why.

Fixed.
Comment 11 Eric Carlson 2015-10-28 14:35:53 PDT
Created attachment 264248 [details]
Patch for landing.
Comment 12 Eric Carlson 2015-10-28 16:59:50 PDT
Created attachment 264268 [details]
Updated path for landing.
Comment 13 WebKit Commit Bot 2015-10-28 20:14:23 PDT
Comment on attachment 264268 [details]
Updated path for landing.

Rejecting attachment 264268 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'validate-changelog', '--check-oops', '--non-interactive', 264268, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

ChangeLog entry in Source/WebCore/ChangeLog contains OOPS!.

Full output: http://webkit-queues.webkit.org/results/351339
Comment 14 Eric Carlson 2015-10-28 21:36:24 PDT
Created attachment 264297 [details]
Patch for landing.
Comment 15 WebKit Commit Bot 2015-10-28 22:31:29 PDT
Comment on attachment 264297 [details]
Patch for landing.

Clearing flags on attachment: 264297

Committed r191721: <http://trac.webkit.org/changeset/191721>
Comment 16 Csaba Osztrogonác 2015-10-29 02:13:23 PDT
(In reply to comment #15)
> Comment on attachment 264297 [details]
> Patch for landing.
> 
> Clearing flags on attachment: 264297
> 
> Committed r191721: <http://trac.webkit.org/changeset/191721>

It made 12 tests timeout on El Capitan bots. 
Could you possibly fix this regression?
Comment 17 Eric Carlson 2015-10-29 06:17:44 PDT
(In reply to comment #16)
> (In reply to comment #15)
> > Comment on attachment 264297 [details]
> > Patch for landing.
> > 
> > Clearing flags on attachment: 264297
> > 
> > Committed r191721: <http://trac.webkit.org/changeset/191721>
> 
> It made 12 tests timeout on El Capitan bots. 
> Could you possibly fix this regression?

I am looking at this.
Comment 18 Eric Carlson 2015-10-29 06:30:20 PDT
It looks like it is a not a problem with this patch per-se, but this patch has uncovered a very old bug in MediaPlayer::getSupportedTypes. Working on a fix.
Comment 19 Eric Carlson 2015-10-29 06:40:54 PDT
I will fix this in bug 150669.