WebKit Bugzilla
Attachment 342160 Details for
Bug 185787
: [GTK][WPE] Start implementing MediaStream API
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
Remember
[x]
|
Forgot Password
Login:
[x]
[patch]
Addressed comments.
GTKWPE-Implement-MediaStream-API.patch (text/plain), 146.74 KB, created by
Thibault Saunier
on 2018-06-07 07:47:43 PDT
(
hide
)
Description:
Addressed comments.
Filename:
MIME Type:
Creator:
Thibault Saunier
Created:
2018-06-07 07:47:43 PDT
Size:
146.74 KB
patch
obsolete
>From a75326f1ffd8eead7d3c6df227a7a776f720024b Mon Sep 17 00:00:00 2001 >From: "Alejandro G. Castro" <alex@igalia.com> >Date: Thu, 24 May 2018 12:55:00 +0200 >Subject: [PATCH xserver] [GTK][WPE]: Implement MediaStream API > >https://bugs.webkit.org/show_bug.cgi?id=185787 >--- > LayoutTests/ChangeLog | 11 + > LayoutTests/platform/gtk/TestExpectations | 33 +- > Source/WebCore/ChangeLog | 117 ++++ > Source/WebCore/PlatformGTK.cmake | 2 + > Source/WebCore/platform/GStreamer.cmake | 16 + > .../platform/audio/AudioStreamDescription.h | 1 + > .../platform/audio/PlatformAudioData.h | 1 + > .../graphics/gstreamer/GStreamerCommon.cpp | 47 ++ > .../graphics/gstreamer/GStreamerCommon.h | 3 + > .../gstreamer/MediaPlayerPrivateGStreamer.cpp | 63 ++- > .../gstreamer/MediaPlayerPrivateGStreamer.h | 7 +- > .../MediaPlayerPrivateGStreamerBase.cpp | 7 + > .../gstreamer/VideoTrackPrivateGStreamer.cpp | 8 + > .../mediastream/RealtimeMediaSource.h | 2 +- > .../mediastream/RealtimeMediaSourceCenter.cpp | 7 - > .../gstreamer/GStreamerAudioCaptureSource.cpp | 202 +++++++ > .../gstreamer/GStreamerAudioCaptureSource.h | 65 +++ > .../gstreamer/GStreamerAudioCapturer.cpp | 73 +++ > .../gstreamer/GStreamerAudioCapturer.h | 43 ++ > .../gstreamer/GStreamerAudioData.h | 59 ++ > .../GStreamerAudioStreamDescription.h | 105 ++++ > .../gstreamer/GStreamerCaptureDevice.h | 50 ++ > .../GStreamerCaptureDeviceManager.cpp | 134 +++++ > .../gstreamer/GStreamerCaptureDeviceManager.h | 81 +++ > .../gstreamer/GStreamerCapturer.cpp | 198 +++++++ > .../mediastream/gstreamer/GStreamerCapturer.h | 72 +++ > .../gstreamer/GStreamerMediaStreamSource.cpp | 531 ++++++++++++++++++ > .../gstreamer/GStreamerMediaStreamSource.h | 46 ++ > .../gstreamer/GStreamerVideoCaptureSource.cpp | 270 +++++++++ > .../gstreamer/GStreamerVideoCaptureSource.h | 64 +++ > .../gstreamer/GStreamerVideoCapturer.cpp | 104 ++++ > .../gstreamer/GStreamerVideoCapturer.h | 46 ++ > .../MockGStreamerAudioCaptureSource.cpp | 106 ++++ > .../MockGStreamerAudioCaptureSource.h | 51 ++ > .../MockGStreamerVideoCaptureSource.cpp | 146 +++++ > .../MockGStreamerVideoCaptureSource.h | 56 ++ > .../RealtimeMediaSourceCenterLibWebRTC.cpp | 91 +++ > .../RealtimeMediaSourceCenterLibWebRTC.h | 68 +++ > .../platform/mock/MockRealtimeAudioSource.cpp | 2 +- > .../platform/mock/MockRealtimeVideoSource.cpp | 2 +- > Tools/ChangeLog | 12 + > Tools/Scripts/webkitpy/style/checker.py | 2 + > Tools/gstreamer/jhbuild.modules | 4 +- > ...REAM_COLLECTION-on-EVENT_STREAM_COLL.patch | 33 ++ > 44 files changed, 2998 insertions(+), 43 deletions(-) > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioData.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDevice.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp > create mode 100644 Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.h > create mode 100644 Tools/gstreamer/patches/gst-plugins-base-0001-parsebin-Post-STREAM_COLLECTION-on-EVENT_STREAM_COLL.patch > >diff --git a/LayoutTests/ChangeLog b/LayoutTests/ChangeLog >index 832cf6b825d..7bcbf4f6d6f 100644 >--- a/LayoutTests/ChangeLog >+++ b/LayoutTests/ChangeLog >@@ -1,3 +1,14 @@ >+2018-05-18 Thibault Saunier <tsaunier@igalia.com> >+ >+ [GTK][WPE] Start implementing MediaStream API >+ https://bugs.webkit.org/show_bug.cgi?id=185787 >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ * platform/gtk/TestExpectations: Reactivate MediaStream tests and enable all tests >+ related to the mediaDevice.enumerateDevices and MediaStream (not RTCPeerConnection nor >+ webaudio). >+ > 2018-06-06 Youenn Fablet <youenn@apple.com> > > HTTP Header values validation is too strict >diff --git a/LayoutTests/platform/gtk/TestExpectations b/LayoutTests/platform/gtk/TestExpectations >index a7e67477d01..59752370dce 100644 >--- a/LayoutTests/platform/gtk/TestExpectations >+++ b/LayoutTests/platform/gtk/TestExpectations >@@ -572,40 +572,31 @@ webkit.org/b/99036 fast/shadow-dom/pointerlockelement-in-slot.html [ Skip ] > webkit.org/b/85211 ietestcenter/css3/flexbox/flexbox-align-stretch-001.htm [ ImageOnlyFailure ] > webkit.org/b/85212 ietestcenter/css3/flexbox/flexbox-layout-002.htm [ ImageOnlyFailure ] > >-# Skipping until backend is implemented >-fast/mediastream [ Skip ] > webrtc [ Skip ] > imported/w3c/web-platform-tests/webrtc/ [ Skip ] > http/tests/webrtc [ Skip ] > # The MediaStream implementation is also still not completed > webkit.org/b/79203 fast/mediastream/mock-media-source-webaudio.html [ Timeout ] > webkit.org/b/79203 fast/mediastream/getUserMedia-webaudio.html [ Failure ] >-webkit.org/b/79203 fast/mediastream/mock-media-source.html [ Failure ] >-webkit.org/b/79203 fast/mediastream/MediaStream-video-element-track-stop.html [ Pass Crash ] >+webkit.org/b/79203 fast/mediastream/MediaStream-video-element-track-stop.html [ Timeout ] > webkit.org/b/79203 fast/mediastream/RTCPeerConnection-dtmf.html [ Timeout ] > webkit.org/b/79203 fast/mediastream/RTCPeerConnection-icecandidate-event.html [ Failure Crash ] >+webkit.org/b/79203 fast/mediastream/RTCPeerConnection-media-setup-two-dialogs.html [ Failure Crash ] > webkit.org/b/79203 fast/mediastream/RTCRtpSender-replaceTrack.html [ Failure ] > # Crash is webkit.org/b/184292 > webkit.org/b/79203 fast/mediastream/RTCPeerConnection-addIceCandidate.html [ Failure Pass Crash ] >-webkit.org/b/79203 fast/mediastream/RTCPeerConnection-closed-state.html [ Timeout ] >-webkit.org/b/79203 fast/mediastream/RTCPeerConnection-ice.html [ Timeout ] >-webkit.org/b/79203 fast/mediastream/RTCPeerConnection-iceconnectionstatechange-event.html [ Timeout ] >+webkit.org/b/79203 fast/mediastream/RTCPeerConnection-iceconnectionstatechange-event.html [ Failure Timeout ] >+webkit.org/b/79203 fast/mediastream/RTCPeerConnection-ice.html [ Failure Timeout ] > webkit.org/b/79203 fast/mediastream/RTCPeerConnection-inspect-answer.html [ Failure ] > webkit.org/b/79203 fast/mediastream/RTCPeerConnection-inspect-offer-bundlePolicy-bundle-only.html [ Failure ] > webkit.org/b/79203 fast/mediastream/RTCPeerConnection-inspect-offer.html [ Failure ] > webkit.org/b/79203 fast/mediastream/RTCPeerConnection-legacy-stream-based-api.html [ Failure ] >+webkit.org/b/79203 fast/mediastream/RTCPeerConnection-media-setup-callbacks-single-dialog.html [ Crash Failure ] > webkit.org/b/79203 webaudio/mediastreamaudiosourcenode.html [ Failure ] >-webkit.org/b/79203 fast/mediastream/MediaStream-video-element-displays-buffer.html [ Crash ] >-webkit.org/b/79203 fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html [ Crash ] >-webkit.org/b/79203 fast/mediastream/MediaStream-video-element.html [ Crash ] >-webkit.org/b/79203 fast/mediastream/apply-constraints-advanced.html [ Crash ] >+webkit.org/b/79203 fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html [ Timeout Failure ] >+webkit.org/b/79203 fast/mediastream/MediaStream-video-element-displays-buffer.html [ Failure ] > webkit.org/b/79203 http/tests/media/media-stream/getusermedia-with-canvas.html [ Crash ] >-webkit.org/b/79203 fast/mediastream/MediaStream-MediaElement-setObject-null.html [ Timeout ] > webkit.org/b/79203 fast/mediastream/RTCPeerConnection-stats.html [ Timeout ] >-webkit.org/b/79203 fast/mediastream/captureStream/canvas2d-heavy-drawing.html [ Timeout ] >-webkit.org/b/79203 fast/mediastream/get-user-media-on-loadedmetadata.html [ Timeout ] >-webkit.org/b/79203 fast/mediastream/local-audio-playing-event.html [ Timeout ] >-webkit.org/b/79203 fast/mediastream/media-stream-track-source-failure.html [ Timeout ] > webkit.org/b/79203 imported/w3c/web-platform-tests/mediacapture-streams/MediaStream-MediaElement-preload-none.https.html [ Failure ] > webkit.org/b/79203 imported/w3c/web-platform-tests/mediacapture-streams/MediaStreamTrack-end-manual.https.html [ Failure ] > webkit.org/b/151344 fast/mediastream/MediaStream-add-ended-tracks.html [ Timeout ] >@@ -616,6 +607,9 @@ webkit.org/b/173257 fast/mediastream/getUserMedia-grant-persistency3.html [ Pass > > # Canvas captureStream support is not implemented > webkit.org/b/169811 fast/mediacapturefromelement [ Skip ] >+webkit.org/b/169811 fast/mediastream/captureStream/ [ Failure ] >+webkit.org/b/169811 fast/mediastream/captureStream/canvas2d.html [ Crash Timeout ] >+webkit.org/b/169811 fast/mediastream/captureStream/canvas3d.html [ Crash Timeout ] > > # Requires Resolution Media Query support > webkit.org/b/100137 fast/media/mq-resolution.html [ Failure ] >@@ -1241,25 +1235,18 @@ webkit.org/b/131546 media/track/track-in-band.html [ Crash Timeout Failure ] > > webkit.org/b/163782 media/video-played-ranges-1.html [ Crash Pass ] > >-webkit.org/b/169811 fast/mediastream/captureStream/canvas2d.html [ Crash Timeout ] >-webkit.org/b/169811 fast/mediastream/captureStream/canvas3d.html [ Crash Timeout ] >- > webkit.org/b/172281 accessibility/insert-children-assert.html [ Crash ] > > webkit.org/b/172955 media/video-preload.html [ Crash Pass ] > > webkit.org/b/175575 imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks.html [ Crash Pass ] > >-webkit.org/b/176801 fast/mediastream/apply-constraints-audio.html [ Crash ] > webkit.org/b/176801 fast/mediastream/argument-types.html [ Crash Pass ] > > webkit.org/b/176803 http/tests/local/blob/send-hybrid-blob-using-open-panel.html [ Crash ] > >-webkit.org/b/176856 fast/mediastream/apply-constraints-video.html [ Crash ] >- > # See also webkit.org/b/141699 > webkit.org/b/177534 fast/attachment/attachment-respects-css-size.html [ Crash ImageOnlyFailure ] >-webkit.org/b/177535 fast/mediastream/media-device-info.html [ Crash Pass ] > # See also webkit.org/b/163528 Threaded compositor failures caused by one of r203496, r203497, or r203498 > webkit.org/b/177536 media/video-object-fit.html [ Crash ImageOnlyFailure ] > >diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog >index 292985617e1..1696d344bfe 100644 >--- a/Source/WebCore/ChangeLog >+++ b/Source/WebCore/ChangeLog >@@ -1,3 +1,120 @@ >+2018-05-30 Thibault Saunier <tsaunier@igalia.com> and Alejandro G. Castro <alex@igalia.com> >+ >+ [GTK][WPE] Start implementing MediaStream API >+ https://bugs.webkit.org/show_bug.cgi?id=185787 >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ We are adding all the required classes to make the >+ MediaStream API work, that means our own RealtimeMediaSourceCenterLibWebRTC >+ for the platform, the GStreamerCaptureDeviceManager, the audio/video capturers >+ and their respective audio/video sources as well as a dedicated GStreamer Source >+ that adds support for using MediaStream stream inside playbin3. >+ We are using the GstDeviceMonitor to list devices on the devices. >+ >+ Enable mediastream tests. >+ >+ * platform/GStreamer.cmake: Added the new files to the compilation. >+ * platform/audio/AudioStreamDescription.h: Added new GStreamer type. >+ * platform/audio/PlatformAudioData.h: Added new GStreamer type for >+ the GStreamerAudioData class. >+ * platform/graphics/gstreamer/GStreamerCommon.cpp: >+ (WebCore::simpleBusMessageCallback): This function and the next >+ one help us to connect a monitoring callback to a pipeline for >+ debugging. >+ (WebCore::connectSimpleBusMessageCallback): Ditto. >+ * platform/graphics/gstreamer/GStreamerCommon.h: Ditto >+ * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: >+ (WebCore::MediaPlayerPrivateGStreamer::load): Make use of the loadFull() method. >+ (WebCore::MediaPlayerPrivateGStreamer::loadFull): Very similar to load() >+ but allows specifying what pipeline type to use (null to let the function determine >+ which one should be used). This is required as we force to always use playbin3 for the >+ mediastream source as it relies on the GstStream API. >+ (WebCore::MediaPlayerPrivateGStreamer::playbackPosition const): Style fix. >+ (WebCore::MediaPlayerPrivateGStreamer::naturalSize const): Added, use MediaStream specific information if available. >+ (WebCore::MediaPlayerPrivateGStreamer::updateTracks): Some style fixes. >+ (WebCore::MediaPlayerPrivateGStreamer::handleMessage): Enhance dotfiles names. >+ (WebCore::MediaPlayerPrivateGStreamer::processTableOfContentsEntry): Minor formatting fix. >+ (WebCore::MediaPlayerPrivateGStreamer::sourceSetup): Set MediaStream on WebKitMediaStreamSource when setting it up. >+ (WebCore::MediaPlayerPrivateGStreamer::supportsType): Advertise that we support MediaStream if support is built. >+ (WebCore::MediaPlayerPrivateGStreamer::createGSTPlayBin): Make sure playbin3 is forced when loading a MediaStream. >+ * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add a reference to the MediaStream object and declare loadFull and naturalSize override. >+ * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp: >+ (WebCore::registerWebKitGStreamerElements): Register the new MediaStreamSrc element. >+ * platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp: >+ (WebCore::VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer): Make sure that MediaStream MAIN tracks are selected by default. We have no way to do it in MediaStreamSrc now as the GstStreamCollection is recreated by parsebin. >+ * platform/mediastream/RealtimeMediaSource.h: Make CaptureFailed a virtual method as in our mocks we require need to make >+ * platform/mediastream/RealtimeMediaSourceCenter.cpp: >+ (WebCore::RealtimeMediaSourceCenter::singleton): Remove the code >+ used for compilation for the platform when we do not have a >+ RealtimeMediaSourceCenterLibWebRTC. Now we return the proper class >+ for the platform. >+ * platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp: >+ Added class representing the RealtimeMediaSource for the Audio >+ with GStreamer. >+ * platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h: >+ Ditto. >+ * platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp: Added >+ this class that represents the GStreamer pipeline that captures >+ audio from the system devices, it inherits from GStreamerCapturer. >+ * platform/mediastream/gstreamer/GStreamerAudioCapturer.h: Dito. >+ * platform/mediastream/gstreamer/GStreamerAudioData.h: Added this >+ class implementing PlatformAudioData for the GStreamer platform, >+ used to pass the samples information. >+ * platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h: >+ Added this class implementing AudioStreamDescription to export the >+ information about the audio stream to libwebrtc. >+ * platform/mediastream/gstreamer/GStreamerCaptureDevice.h: Added >+ this base class for the audio and video capturing devices, it >+ implements general WebKit CaptureDevice class. >+ * platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp: >+ Added this class that implements the system monitor to get the >+ list of available devices in the system. It uses GstDeviceMonitor >+ to handle the operation. It uses two singleton device managers one >+ for audio and another one for video, as required by the >+ RealtimeMediaSourceCenter design. >+ * platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h: Ditto. >+ * platform/mediastream/gstreamer/GStreamerCapturer.cpp: Added this >+ base class representing how GStreamer captures the media from the >+ input devices in the system. Two classes inherit from this one to >+ capture audio and video. It setups the GStreamer pipeline and adds >+ functions to control it. >+ * platform/mediastream/gstreamer/GStreamerCapturer.h: Ditto. >+ * platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp: Added. >+ Implements a subclass of GstBin as a source element that will contain several >+ GstAppSrc, basically one per MediaStreamTrackPrivate of the MediaStreamPrivate >+ passed in parameter. It adds Observers on the MediaStreamTracks and >+ pushes the data to the sources as required. The element implements the GstURIHandler >+ interface so it can be used in playbin. The MediaPlayerPrivateGStreamer is responsible >+ for passing the MediaStreamPrivate object to the source when required. >+ (WebCore::webkitMediaStreamSrcPadProbeCb): Event probe that fixes stream_start events (setting the ID etc) >+ and finally add src pads to the pipeline. >+ * platform/mediastream/gstreamer/GStreamerMediaStreamSource.h: Ditto. >+ * platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp: >+ Added this RealtimeMediaSource representing the source of the >+ video data for the GStreamer platform. It handles the settings and >+ capabilities of the source and creates the capturer used to >+ control the operation of the stream. >+ * platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h: Ditto. >+ * platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp: Added >+ this class that inherits from the GStreamerCapturer and controls >+ the GStreamer pipelines of the video streams of the system. >+ * platform/mediastream/gstreamer/GStreamerVideoCapturer.h: Ditto. >+ * platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp: Added. Implementation of a Mock capturer for audio stream. >+ Subclasses GStreamerAudioCapturer and wraps a MockRealtimeAudioSource so that the behaviour is the same a MockRealtimeAudioSource >+ but still the GStreamer implementation code paths are tested. >+ * platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h: Ditto. >+ * platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp: Added. Implementation of a Mock capturer for video stream. >+ Subclasses GStreamerVideoCapturer and wraps a MockRealtimeVideoSource so that the behaviour is the same a MockRealtimeVideoSource >+ but still the GStreamer implementation code paths are tested. >+ * platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h: Ditto. >+ * platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp: >+ Added this class that implements the key RealtimeMediaSourceCenter >+ functions to configure the base class for the platform, using the >+ other GStreamer classes. >+ * platform/mock/MockRealtimeAudioSource.cpp: Do not build ::create if GStreamer implementation is built >+ * platform/mock/MockRealtimeVideoSource.cpp: Do not build ::create if GStreamer implementation is built >+ > 2018-06-07 Thibault Saunier <tsaunier@igalia.com> > > [GStreamer] Fix the way GstStreamCollection is handled >diff --git a/Source/WebCore/PlatformGTK.cmake b/Source/WebCore/PlatformGTK.cmake >index 823c77ddb21..c2d4cae0f60 100644 >--- a/Source/WebCore/PlatformGTK.cmake >+++ b/Source/WebCore/PlatformGTK.cmake >@@ -24,12 +24,14 @@ list(APPEND WebCore_INCLUDE_DIRECTORIES > "${WEBCORE_DIR}/platform/gtk" > "${WEBCORE_DIR}/platform/graphics/egl" > "${WEBCORE_DIR}/platform/graphics/glx" >+ "${WEBCORE_DIR}/platform/graphics/gstreamer" > "${WEBCORE_DIR}/platform/graphics/gtk" > "${WEBCORE_DIR}/platform/graphics/opengl" > "${WEBCORE_DIR}/platform/graphics/opentype" > "${WEBCORE_DIR}/platform/graphics/wayland" > "${WEBCORE_DIR}/platform/graphics/x11" > "${WEBCORE_DIR}/platform/mediastream/gtk" >+ "${WEBCORE_DIR}/platform/mediastream/gstreamer" > "${WEBCORE_DIR}/platform/mock/mediasource" > "${WEBCORE_DIR}/platform/network/gtk" > "${WEBCORE_DIR}/platform/network/soup" >diff --git a/Source/WebCore/platform/GStreamer.cmake b/Source/WebCore/platform/GStreamer.cmake >index 3d5a6e0025a..333f1d65bff 100644 >--- a/Source/WebCore/platform/GStreamer.cmake >+++ b/Source/WebCore/platform/GStreamer.cmake >@@ -33,6 +33,22 @@ if (ENABLE_VIDEO OR ENABLE_WEB_AUDIO) > platform/graphics/gstreamer/mse/PlaybackPipeline.cpp > platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp > platform/graphics/gstreamer/mse/WebKitMediaSourceGStreamer.cpp >+ >+ platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp >+ platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp >+ platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp >+ platform/mediastream/gstreamer/GStreamerCapturer.cpp >+ platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp >+ platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp >+ platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp >+ platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp >+ platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp >+ platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp >+ >+ platform/mock/MockRealtimeAudioSource.cpp >+ platform/mock/MockRealtimeMediaSource.cpp >+ platform/mock/MockRealtimeMediaSourceCenter.cpp >+ platform/mock/MockRealtimeVideoSource.cpp > ) > > list(APPEND WebCore_SYSTEM_INCLUDE_DIRECTORIES >diff --git a/Source/WebCore/platform/audio/AudioStreamDescription.h b/Source/WebCore/platform/audio/AudioStreamDescription.h >index e440dc90a3e..2ba585c9c05 100644 >--- a/Source/WebCore/platform/audio/AudioStreamDescription.h >+++ b/Source/WebCore/platform/audio/AudioStreamDescription.h >@@ -35,6 +35,7 @@ struct PlatformDescription { > enum { > None, > CAAudioStreamBasicType, >+ GStreamerAudioStreamDescription, > } type; > Variant<std::nullptr_t, const AudioStreamBasicDescription*> description; > }; >diff --git a/Source/WebCore/platform/audio/PlatformAudioData.h b/Source/WebCore/platform/audio/PlatformAudioData.h >index e4544f63254..cda72e8ab3c 100644 >--- a/Source/WebCore/platform/audio/PlatformAudioData.h >+++ b/Source/WebCore/platform/audio/PlatformAudioData.h >@@ -34,6 +34,7 @@ public: > enum class Kind { > None, > WebAudioBufferList, >+ GStreamerAudioData, > }; > > virtual Kind kind() const { return Kind::None; } >diff --git a/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp b/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp >index e514fe3428d..d5475b74cdd 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp >+++ b/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp >@@ -30,6 +30,7 @@ > #include <mutex> > #include <wtf/glib/GLibUtilities.h> > #include <wtf/glib/GUniquePtr.h> >+#include <wtf/glib/RunLoopSourcePriority.h> > > #if ENABLE(VIDEO_TRACK) && USE(GSTREAMER_MPEGTS) > #define GST_USE_UNSTABLE_API >@@ -320,6 +321,52 @@ bool gstRegistryHasElementForMediaType(GList* elementFactories, const char* caps > return result; > } > >+static void simpleBusMessageCallback(GstBus*, GstMessage* message, GstBin* pipeline) >+{ >+ switch (GST_MESSAGE_TYPE(message)) { >+ case GST_MESSAGE_ERROR: >+ GST_ERROR_OBJECT(pipeline, "Got message: %" GST_PTR_FORMAT, message); >+ { >+ WTF::String dotFileName = String::format("%s_error", GST_OBJECT_NAME(pipeline)); >+ GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(pipeline, GST_DEBUG_GRAPH_SHOW_ALL, dotFileName.utf8().data()); >+ } >+ break; >+ case GST_MESSAGE_STATE_CHANGED: >+ if (GST_MESSAGE_SRC(message) == GST_OBJECT(pipeline)) { >+ GstState oldState, newState, pending; >+ gst_message_parse_state_changed(message, &oldState, &newState, &pending); >+ >+ GST_INFO_OBJECT(pipeline, "State changed (old: %s, new: %s, pending: %s)", >+ gst_element_state_get_name(oldState), >+ gst_element_state_get_name(newState), >+ gst_element_state_get_name(pending)); >+ >+ WTF::String dotFileName = String::format("%s_%s_%s", >+ GST_OBJECT_NAME(pipeline), >+ gst_element_state_get_name(oldState), >+ gst_element_state_get_name(newState)); >+ >+ GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(GST_BIN(pipeline), GST_DEBUG_GRAPH_SHOW_ALL, dotFileName.utf8().data()); >+ } >+ break; >+ default: >+ break; >+ } >+} >+ >+void disconnectSimpleBusMessageCallback(GstElement *pipeline) >+{ >+ GRefPtr<GstBus> bus = adoptGRef(gst_pipeline_get_bus(GST_PIPELINE(pipeline))); >+ g_signal_handlers_disconnect_by_func(bus.get(), reinterpret_cast<gpointer>(simpleBusMessageCallback), pipeline); >+} >+ >+void connectSimpleBusMessageCallback(GstElement *pipeline) >+{ >+ GRefPtr<GstBus> bus = adoptGRef(gst_pipeline_get_bus(GST_PIPELINE(pipeline))); >+ gst_bus_add_signal_watch_full(bus.get(), RunLoopSourcePriority::RunLoopDispatcher); >+ g_signal_connect(bus.get(), "message", G_CALLBACK(simpleBusMessageCallback), pipeline); >+} >+ > } > > #endif // USE(GSTREAMER) >diff --git a/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h b/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h >index 259255e741e..cbb49323a68 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h >+++ b/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h >@@ -82,6 +82,9 @@ inline GstClockTime toGstClockTime(const MediaTime &mediaTime) > } > > bool gstRegistryHasElementForMediaType(GList* elementFactories, const char* capsString); >+void connectSimpleBusMessageCallback(GstElement *pipeline); >+void disconnectSimpleBusMessageCallback(GstElement *pipeline); >+ > } > > #ifndef GST_BUFFER_DTS_OR_PTS >diff --git a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp >index 7d847f18d76..0ec4b13f891 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp >+++ b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp >@@ -51,6 +51,10 @@ > #include <wtf/glib/RunLoopSourcePriority.h> > #include <wtf/text/CString.h> > >+#if ENABLE(MEDIA_STREAM) && GST_CHECK_VERSION(1, 10, 0) >+#include "GStreamerMediaStreamSource.h" >+#endif >+ > #if ENABLE(VIDEO_TRACK) > #include "AudioTrackPrivateGStreamer.h" > #include "InbandMetadataTextTrackPrivateGStreamer.h" >@@ -237,6 +241,11 @@ void MediaPlayerPrivateGStreamer::setPlaybinURL(const URL& url) > } > > void MediaPlayerPrivateGStreamer::load(const String& urlString) >+{ >+ loadFull(urlString, nullptr); >+} >+ >+void MediaPlayerPrivateGStreamer::loadFull(const String& urlString, const gchar *playbinName) > { > // FIXME: This method is still called even if supportsType() returned > // IsNotSupported. This would deserve more investigation but meanwhile make >@@ -254,7 +263,7 @@ void MediaPlayerPrivateGStreamer::load(const String& urlString) > return; > > if (!m_pipeline) >- createGSTPlayBin(isMediaSource() ? "playbin" : nullptr); >+ createGSTPlayBin(isMediaSource() ? "playbin" : playbinName); > > if (m_fillTimer.isActive()) > m_fillTimer.stop(); >@@ -292,12 +301,19 @@ void MediaPlayerPrivateGStreamer::load(const String&, MediaSourcePrivateClient*) > #endif > > #if ENABLE(MEDIA_STREAM) >-void MediaPlayerPrivateGStreamer::load(MediaStreamPrivate&) >+void MediaPlayerPrivateGStreamer::load(MediaStreamPrivate& stream) > { >+#if GST_CHECK_VERSION(1, 10, 0) >+ m_streamPrivate = &stream; >+ loadFull(String("mediastream://") + stream.id(), "playbin3"); >+ ensureGLVideoSinkContext(); >+ m_player->play(); >+#else > // Properly fail so the global MediaPlayer tries to fallback to the next MediaPlayerPrivate. > m_networkState = MediaPlayer::FormatError; > m_player->networkStateChanged(); > notImplemented(); >+#endif > } > #endif > >@@ -330,7 +346,7 @@ MediaTime MediaPlayerPrivateGStreamer::playbackPosition() const > > // Position is only available if no async state change is going on and the state is either paused or playing. > gint64 position = GST_CLOCK_TIME_NONE; >- GstQuery* query= gst_query_new_position(GST_FORMAT_TIME); >+ GstQuery* query = gst_query_new_position(GST_FORMAT_TIME); > if (gst_element_query(m_pipeline.get(), query)) > gst_query_parse_position(query, 0, &position); > gst_query_unref(query); >@@ -648,6 +664,25 @@ void MediaPlayerPrivateGStreamer::clearTracks() > m_current##Type##StreamId = String(gst_stream_get_stream_id(stream.get())); \ > } \ > } >+ >+FloatSize MediaPlayerPrivateGStreamer::naturalSize() const >+{ >+#if ENABLE(MEDIA_STREAM) >+ if (!m_isLegacyPlaybin && !m_currentVideoStreamId.isEmpty()) { >+ RefPtr<VideoTrackPrivateGStreamer> videoTrack = m_videoTracks.get(m_currentVideoStreamId); >+ >+ if (videoTrack) { >+ auto tags = gst_stream_get_tags(videoTrack->stream()); >+ gint width, height; >+ >+ if (gst_tag_list_get_int(tags, WEBKIT_MEDIA_TRACK_TAG_WIDTH, &width) && gst_tag_list_get_int(tags, WEBKIT_MEDIA_TRACK_TAG_HEIGHT, &height)) >+ return FloatSize(width, height); >+ } >+ } >+#endif // ENABLE(MEDIA_STREAM) >+ >+ return MediaPlayerPrivateGStreamerBase::naturalSize(); >+} > #else > #define CREATE_TRACK(type, _id, tracks, method, stream) m_has##Type## = true; > #endif // ENABLE(VIDEO_TRACK) >@@ -1165,7 +1200,8 @@ void MediaPlayerPrivateGStreamer::handleMessage(GstMessage* message) > // Construct a filename for the graphviz dot file output. > GstState newState; > gst_message_parse_state_changed(message, ¤tState, &newState, nullptr); >- CString dotFileName = String::format("webkit-video.%s_%s", gst_element_state_get_name(currentState), gst_element_state_get_name(newState)).utf8(); >+ CString dotFileName = String::format("%s.%s_%s", GST_OBJECT_NAME(m_pipeline.get()), >+ gst_element_state_get_name(currentState), gst_element_state_get_name(newState)).utf8(); > GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(GST_BIN(m_pipeline.get()), GST_DEBUG_GRAPH_SHOW_ALL, dotFileName.data()); > > break; >@@ -1431,7 +1467,7 @@ void MediaPlayerPrivateGStreamer::processTableOfContentsEntry(GstTocEntry* entry > > GstTagList* tags = gst_toc_entry_get_tags(entry); > if (tags) { >- gchar* title = nullptr; >+ gchar* title = nullptr; > gst_tag_list_get_string(tags, GST_TAG_TITLE, &title); > if (title) { > cue->setContent(title); >@@ -1726,6 +1762,12 @@ void MediaPlayerPrivateGStreamer::sourceSetup(GstElement* sourceElement) > if (WEBKIT_IS_WEB_SRC(m_source.get())) { > webKitWebSrcSetMediaPlayer(WEBKIT_WEB_SRC(m_source.get()), m_player); > g_signal_connect(GST_ELEMENT_PARENT(m_source.get()), "element-added", G_CALLBACK(uriDecodeBinElementAddedCallback), this); >+#if ENABLE(MEDIA_STREAM) && GST_CHECK_VERSION(1, 10, 0) >+ } else if (WEBKIT_IS_MEDIA_STREAM_SRC(sourceElement)) { >+ auto stream = m_streamPrivate.get(); >+ ASSERT(stream); >+ webkitMediaStreamSrcSetStream(WEBKIT_MEDIA_STREAM_SRC(sourceElement), stream); >+#endif > } > } > >@@ -2269,8 +2311,10 @@ MediaPlayer::SupportsType MediaPlayerPrivateGStreamer::supportsType(const MediaE > return result; > #endif > >+#if !ENABLE(MEDIA_STREAM) || !GST_CHECK_VERSION(1, 10, 0) > if (parameters.isMediaStream) > return result; >+#endif > > if (parameters.type.isEmpty()) > return result; >@@ -2433,17 +2477,20 @@ void MediaPlayerPrivateGStreamer::createGSTPlayBin(const gchar* playbinName) > ASSERT(!m_pipeline); > > #if GST_CHECK_VERSION(1, 10, 0) >- m_isLegacyPlaybin = !g_getenv("USE_PLAYBIN3"); >- if (!m_isLegacyPlaybin) >+ if (g_getenv("USE_PLAYBIN3")) > playbinName = "playbin3"; >+#else >+ playbinName = "playbin"; > #endif > > if (!playbinName) > playbinName = "playbin"; > >+ m_isLegacyPlaybin = !g_strcmp0(playbinName, "playbin"); >+ > // gst_element_factory_make() returns a floating reference so > // we should not adopt. >- setPipeline(gst_element_factory_make(playbinName, "play")); >+ setPipeline(gst_element_factory_make(playbinName, String::format("play_%p", this).utf8().data())); > setStreamVolumeElement(GST_STREAM_VOLUME(m_pipeline.get())); > > GST_INFO("Using legacy playbin element: %s", boolForPrinting(m_isLegacyPlaybin)); >diff --git a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h >index 1ae4a54bae0..b2ee5e695bd 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h >+++ b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h >@@ -179,6 +179,7 @@ private: > static void downloadBufferFileCreatedCallback(MediaPlayerPrivateGStreamer*); > > void setPlaybinURL(const URL& urlString); >+ void loadFull(const String& url, const gchar *playbinName); > > #if GST_CHECK_VERSION(1, 10, 0) > void updateTracks(); >@@ -262,7 +263,11 @@ private: > bool m_isLegacyPlaybin; > #if GST_CHECK_VERSION(1, 10, 0) > GRefPtr<GstStreamCollection> m_streamCollection; >-#endif >+ FloatSize naturalSize() const final; >+#if ENABLE(MEDIA_STREAM) >+ RefPtr<MediaStreamPrivate> m_streamPrivate; >+#endif // ENABLE(MEDIA_STREAM) >+#endif // GST_CHECK_VERSION(1, 10, 0) > String m_currentAudioStreamId; > String m_currentVideoStreamId; > String m_currentTextStreamId; >diff --git a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp >index 2eb441d2753..627c4b2a27b 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp >+++ b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp >@@ -81,6 +81,10 @@ > #endif // !GST_CHECK_VERSION(1, 14, 0) > #endif // USE(LIBEPOXY) > >+#if ENABLE(MEDIA_STREAM) && GST_CHECK_VERSION(1, 10, 0) >+#include "GStreamerMediaStreamSource.h" >+#endif >+ > #define GST_USE_UNSTABLE_API > #include <gst/gl/gl.h> > #undef GST_USE_UNSTABLE_API >@@ -143,6 +147,9 @@ void registerWebKitGStreamerElements() > if (!clearKeyDecryptorFactory) > gst_element_register(nullptr, "webkitclearkey", GST_RANK_PRIMARY + 100, WEBKIT_TYPE_MEDIA_CK_DECRYPT); > #endif >+#if ENABLE(MEDIA_STREAM) && GST_CHECK_VERSION(1,10,0) >+ gst_element_register(nullptr, "mediastreamsrc", GST_RANK_PRIMARY, WEBKIT_TYPE_MEDIA_STREAM_SRC); >+#endif > } > > bool MediaPlayerPrivateGStreamerBase::initializeGStreamerAndRegisterWebKitElements() >diff --git a/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp b/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp >index c54200eae81..4d99786e012 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp >+++ b/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp >@@ -48,6 +48,14 @@ VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivat > : TrackPrivateBaseGStreamer(this, index, stream) > , m_player(player) > { >+ gint kind; >+ auto tags = gst_stream_get_tags(m_stream.get()); >+ >+ if (gst_tag_list_get_int(tags, "webkit-media-stream-kind", &kind) && kind == static_cast<int>(VideoTrackPrivate::Kind::Main)) { >+ GstStreamFlags streamFlags = gst_stream_get_stream_flags(stream.get()); >+ gst_stream_set_stream_flags(stream.get(), static_cast<GstStreamFlags>(streamFlags | GST_STREAM_FLAG_SELECT)); >+ } >+ > m_id = gst_stream_get_stream_id(stream.get()); > setActive(gst_stream_get_stream_flags(stream.get()) & GST_STREAM_FLAG_SELECT); > notifyTrackOfActiveChanged(); >diff --git a/Source/WebCore/platform/mediastream/RealtimeMediaSource.h b/Source/WebCore/platform/mediastream/RealtimeMediaSource.h >index 9b451205e5b..5411085c4a3 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeMediaSource.h >+++ b/Source/WebCore/platform/mediastream/RealtimeMediaSource.h >@@ -216,7 +216,7 @@ public: > > virtual void monitorOrientation(OrientationNotifier&) { } > >- void captureFailed(); >+ virtual void captureFailed(); > > // Testing only > virtual void delaySamples(Seconds) { }; >diff --git a/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp b/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp >index 2efb82d1e58..3886ff875ce 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp >+++ b/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp >@@ -64,14 +64,7 @@ RealtimeMediaSourceCenter& RealtimeMediaSourceCenter::singleton() > RealtimeMediaSourceCenter* override = mediaStreamCenterOverride(); > if (override) > return *override; >-#if PLATFORM(GTK) >- ASSERT(isMainThread()); >- notImplemented(); // Return MockRealtimeMediaSourceCenter to avoid crash. >- static NeverDestroyed<MockRealtimeMediaSourceCenter> center; >- return center; >-#else > return RealtimeMediaSourceCenter::platformCenter(); >-#endif > } > > void RealtimeMediaSourceCenter::setSharedStreamCenterOverride(RealtimeMediaSourceCenter* center) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp >new file mode 100644 >index 00000000000..9c5aeafffcf >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp >@@ -0,0 +1,202 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerAudioCaptureSource.h" >+ >+#include "GStreamerAudioData.h" >+#include "GStreamerAudioStreamDescription.h" >+#include "GStreamerCaptureDeviceManager.h" >+ >+#include <gst/app/gstappsink.h> >+#include <gst/gst.h> >+#include <wtf/NeverDestroyed.h> >+ >+namespace WebCore { >+ >+const static CapabilityValueOrRange defaultVolumeCapability = CapabilityValueOrRange(0.0, 1.0); >+const static RealtimeMediaSourceCapabilities::EchoCancellation defaultEchoCancellationCapability = RealtimeMediaSourceCapabilities::EchoCancellation::ReadWrite; >+ >+GST_DEBUG_CATEGORY(webkit_audio_capture_source_debug); >+#define GST_CAT_DEFAULT webkit_audio_capture_source_debug >+ >+static void initializeGStreamerDebug() >+{ >+ static std::once_flag debugRegisteredFlag; >+ std::call_once(debugRegisteredFlag, [] { >+ GST_DEBUG_CATEGORY_INIT(webkit_audio_capture_source_debug, "webkitaudiocapturesource", 0, "WebKit Audio Capture Source."); >+ }); >+} >+ >+class GStreamerAudioCaptureSourceFactory : public RealtimeMediaSource::AudioCaptureFactory { >+public: >+ CaptureSourceOrError createAudioCaptureSource(const CaptureDevice& device, const MediaConstraints* constraints) final >+ { >+ return GStreamerAudioCaptureSource::create(device.persistentId(), constraints); >+ } >+}; >+ >+static GStreamerAudioCaptureSourceFactory& libWebRTCAudioCaptureSourceFactory() >+{ >+ static NeverDestroyed<GStreamerAudioCaptureSourceFactory> factory; >+ return factory.get(); >+} >+ >+CaptureSourceOrError GStreamerAudioCaptureSource::create(const String& deviceID, const MediaConstraints* constraints) >+{ >+ auto device = GStreamerAudioCaptureDeviceManager::singleton().gstreamerDeviceWithUID(deviceID); >+ if (!device) { >+ auto errorMessage = String::format("GStreamerAudioCaptureSource::create(): GStreamer did not find the device: %s.", deviceID.utf8().data()); >+ return CaptureSourceOrError(WTFMove(errorMessage)); >+ } >+ >+ auto source = adoptRef(*new GStreamerAudioCaptureSource(device.value())); >+ >+ if (constraints) { >+ auto result = source->applyConstraints(*constraints); >+ if (result) >+ return WTFMove(result.value().first); >+ } >+ return CaptureSourceOrError(WTFMove(source)); >+} >+ >+RealtimeMediaSource::AudioCaptureFactory& GStreamerAudioCaptureSource::factory() >+{ >+ return libWebRTCAudioCaptureSourceFactory(); >+} >+ >+GStreamerAudioCaptureSource::GStreamerAudioCaptureSource(GStreamerCaptureDevice device) >+ : RealtimeMediaSource(device.persistentId(), RealtimeMediaSource::Type::Audio, device.label()) >+ , m_capturer(std::make_unique<GStreamerAudioCapturer>(device)) >+{ >+ initializeGStreamerDebug(); >+} >+ >+GStreamerAudioCaptureSource::GStreamerAudioCaptureSource(const String& deviceID, const String& name) >+ : RealtimeMediaSource(deviceID, RealtimeMediaSource::Type::Audio, name) >+ , m_capturer(std::make_unique<GStreamerAudioCapturer>()) >+{ >+ initializeGStreamerDebug(); >+} >+ >+GStreamerAudioCaptureSource::~GStreamerAudioCaptureSource() >+{ >+} >+ >+void GStreamerAudioCaptureSource::startProducingData() >+{ >+ m_capturer->setupPipeline(); >+ m_capturer->setSampleRate(sampleRate()); >+ g_signal_connect(m_capturer->sink(), "new-sample", G_CALLBACK(newSampleCallback), this); >+ m_capturer->play(); >+} >+ >+GstFlowReturn GStreamerAudioCaptureSource::newSampleCallback(GstElement* sink, GStreamerAudioCaptureSource* source) >+{ >+ auto sample = adoptGRef(gst_app_sink_pull_sample(GST_APP_SINK(sink))); >+ >+ // FIXME - figure out a way to avoid copying (on write) the data. >+ GstBuffer* buf = gst_sample_get_buffer(sample.get()); >+ auto frames(std::unique_ptr<GStreamerAudioData>(new GStreamerAudioData(WTFMove(sample)))); >+ auto streamDesc(std::unique_ptr<GStreamerAudioStreamDescription>(new GStreamerAudioStreamDescription(frames->getAudioInfo()))); >+ >+ source->audioSamplesAvailable( >+ MediaTime(GST_TIME_AS_USECONDS(GST_BUFFER_PTS(buf)), G_USEC_PER_SEC), >+ *frames, *streamDesc, gst_buffer_get_size(buf) / frames->getAudioInfo().bpf); >+ >+ return GST_FLOW_OK; >+} >+ >+void GStreamerAudioCaptureSource::stopProducingData() >+{ >+ g_signal_handlers_disconnect_by_func(m_capturer->sink(), reinterpret_cast<gpointer>(newSampleCallback), this); >+ m_capturer->stop(); >+} >+ >+const RealtimeMediaSourceCapabilities& GStreamerAudioCaptureSource::capabilities() const >+{ >+ if (m_capabilities) >+ return m_capabilities.value(); >+ >+ uint i; >+ GRefPtr<GstCaps> caps = m_capturer->caps(); >+ int minSampleRate = 0, maxSampleRate = 0; >+ for (i = 0; i < gst_caps_get_size(caps.get()); i++) { >+ int capabilityMinSampleRate = 0, capabilityMaxSampleRate = 0; >+ GstStructure* str = gst_caps_get_structure(caps.get(), i); >+ >+ // Only accept raw audio for now. >+ if (!gst_structure_has_name(str, "audio/x-raw")) >+ continue; >+ >+ gst_structure_get(str, "rate", GST_TYPE_INT_RANGE, &capabilityMinSampleRate, &capabilityMaxSampleRate, nullptr); >+ if (i > 0) { >+ minSampleRate = std::min(minSampleRate, capabilityMinSampleRate); >+ maxSampleRate = std::max(maxSampleRate, capabilityMaxSampleRate); >+ } else { >+ minSampleRate = capabilityMinSampleRate; >+ maxSampleRate = capabilityMaxSampleRate; >+ } >+ } >+ >+ RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints()); >+ capabilities.setDeviceId(id()); >+ capabilities.setEchoCancellation(defaultEchoCancellationCapability); >+ capabilities.setVolume(defaultVolumeCapability); >+ capabilities.setSampleRate(CapabilityValueOrRange(minSampleRate, maxSampleRate)); >+ m_capabilities = WTFMove(capabilities); >+ >+ return m_capabilities.value(); >+} >+ >+bool GStreamerAudioCaptureSource::applySampleRate(int sampleRate) >+{ >+ return m_capturer->setSampleRate(sampleRate); >+} >+ >+const RealtimeMediaSourceSettings& GStreamerAudioCaptureSource::settings() const >+{ >+ if (!m_currentSettings) { >+ RealtimeMediaSourceSettings settings; >+ settings.setDeviceId(id()); >+ >+ RealtimeMediaSourceSupportedConstraints supportedConstraints; >+ supportedConstraints.setSupportsDeviceId(true); >+ supportedConstraints.setSupportsEchoCancellation(true); >+ supportedConstraints.setSupportsVolume(true); >+ supportedConstraints.setSupportsSampleRate(true); >+ settings.setSupportedConstraints(supportedConstraints); >+ >+ m_currentSettings = WTFMove(settings); >+ } >+ >+ m_currentSettings->setVolume(volume()); >+ m_currentSettings->setSampleRate(sampleRate()); >+ m_currentSettings->setEchoCancellation(echoCancellation()); >+ >+ return m_currentSettings.value(); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h >new file mode 100644 >index 00000000000..3fa4a9c2e32 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h >@@ -0,0 +1,65 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerAudioCapturer.h" >+#include "GStreamerCaptureDevice.h" >+#include "RealtimeMediaSource.h" >+ >+namespace WebCore { >+ >+class GStreamerAudioCaptureSource : public RealtimeMediaSource { >+public: >+ static CaptureSourceOrError create(const String& deviceID, const MediaConstraints*); >+ WEBCORE_EXPORT static AudioCaptureFactory& factory(); >+ >+ const RealtimeMediaSourceCapabilities& capabilities() const override; >+ const RealtimeMediaSourceSettings& settings() const override; >+ >+ GstElement* pipeline() { return m_capturer->pipeline(); } >+ GStreamerCapturer* capturer() { return m_capturer.get(); } >+ >+protected: >+ GStreamerAudioCaptureSource(GStreamerCaptureDevice); >+ GStreamerAudioCaptureSource(const String& deviceID, const String& name); >+ virtual ~GStreamerAudioCaptureSource(); >+ void startProducingData() override; >+ void stopProducingData() override; >+ >+ mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >+ mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings; >+ >+private: >+ bool applySampleRate(int) final; >+ bool isCaptureSource() const final { return true; } >+ bool applyVolume(double) final { return true; } >+ >+ std::unique_ptr<GStreamerAudioCapturer> m_capturer; >+ >+ static GstFlowReturn newSampleCallback(GstElement*, GStreamerAudioCaptureSource*); >+ void triggerSampleAvailable(GstSample*); >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp >new file mode 100644 >index 00000000000..64157ef70a6 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp >@@ -0,0 +1,73 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerAudioCapturer.h" >+ >+#include "LibWebRTCAudioFormat.h" >+ >+#include <gst/app/gstappsink.h> >+ >+namespace WebCore { >+ >+GStreamerAudioCapturer::GStreamerAudioCapturer(GStreamerCaptureDevice device) >+ : GStreamerCapturer(device, adoptGRef(gst_caps_new_simple("audio/x-raw", "rate", G_TYPE_INT, LibWebRTCAudioFormat::sampleRate, nullptr))) >+{ >+} >+ >+GStreamerAudioCapturer::GStreamerAudioCapturer() >+ : GStreamerCapturer("audiotestsrc", adoptGRef(gst_caps_new_simple("audio/x-raw", "rate", G_TYPE_INT, LibWebRTCAudioFormat::sampleRate, nullptr))) >+{ >+} >+ >+GstElement* GStreamerAudioCapturer::createConverter() >+{ >+ auto converter = gst_parse_bin_from_description("audioconvert ! audioresample", TRUE, nullptr); >+ >+ ASSERT(converter); >+ >+ return converter; >+} >+ >+bool GStreamerAudioCapturer::setSampleRate(int sampleRate) >+{ >+ >+ if (sampleRate > 0) { >+ GST_INFO_OBJECT(m_pipeline.get(), "Setting SampleRate %d", sampleRate); >+ m_caps = adoptGRef(gst_caps_new_simple("audio/x-raw", "rate", G_TYPE_INT, sampleRate, nullptr)); >+ } else { >+ GST_INFO_OBJECT(m_pipeline.get(), "Not forcing sample rate"); >+ m_caps = adoptGRef(gst_caps_new_empty_simple("audio/x-raw")); >+ } >+ >+ if (!m_capsfilter.get()) >+ return false; >+ >+ g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr); >+ >+ return true; >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.h >new file mode 100644 >index 00000000000..9aaf761d6c4 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.h >@@ -0,0 +1,43 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GStreamerCapturer.h" >+ >+namespace WebCore { >+ >+class GStreamerAudioCapturer : public GStreamerCapturer { >+public: >+ GStreamerAudioCapturer(GStreamerCaptureDevice); >+ GStreamerAudioCapturer(); >+ >+ GstElement* createConverter() final; >+ const char* name() final { return "Audio"; } >+ >+ bool setSampleRate(int); >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioData.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioData.h >new file mode 100644 >index 00000000000..953f3a8a0e4 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioData.h >@@ -0,0 +1,59 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GRefPtrGStreamer.h" >+#include "PlatformAudioData.h" >+ >+#include <gst/audio/audio.h> >+ >+namespace WebCore { >+class GStreamerAudioData : public PlatformAudioData { >+public: >+ GStreamerAudioData(GRefPtr<GstSample>&& sample, GstAudioInfo info) >+ : m_sample(WTFMove(sample)) >+ , m_audioInfo(info) >+ { >+ } >+ >+ GStreamerAudioData(GRefPtr<GstSample>&& sample) >+ : m_sample(WTFMove(sample)) >+ { >+ gst_audio_info_from_caps(&m_audioInfo, gst_sample_get_caps(m_sample.get())); >+ } >+ >+ GstSample* getSample() { return m_sample.get(); } >+ GstAudioInfo getAudioInfo() {return m_audioInfo; } >+ >+private: >+ Kind kind() const { return Kind::GStreamerAudioData; } >+ GRefPtr<GstSample> m_sample; >+ GRefPtr<GstCaps> m_caps; >+ >+ GstAudioInfo m_audioInfo; >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h >new file mode 100644 >index 00000000000..1ff59b2a174 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h >@@ -0,0 +1,105 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "AudioStreamDescription.h" >+#include <gst/audio/audio.h> >+ >+namespace WebCore { >+ >+class GStreamerAudioStreamDescription final: public AudioStreamDescription { >+public: >+ GStreamerAudioStreamDescription(GstAudioInfo info) >+ : m_info(info) >+ , m_caps(adoptGRef(gst_audio_info_to_caps(&m_info))) >+ { >+ } >+ >+ GStreamerAudioStreamDescription(GstAudioInfo *info) >+ : m_info(*info) >+ , m_caps(adoptGRef(gst_audio_info_to_caps(&m_info))) >+ { >+ } >+ >+ GStreamerAudioStreamDescription() >+ { >+ gst_audio_info_init(&m_info); >+ } >+ >+ WEBCORE_EXPORT ~GStreamerAudioStreamDescription() { }; >+ >+ const PlatformDescription& platformDescription() const >+ { >+ m_platformDescription = { PlatformDescription::GStreamerAudioStreamDescription, reinterpret_cast<const AudioStreamBasicDescription*>(&m_info) }; >+ >+ return m_platformDescription; >+ } >+ >+ WEBCORE_EXPORT PCMFormat format() const final { >+ switch (GST_AUDIO_INFO_FORMAT(&m_info)) { >+ case GST_AUDIO_FORMAT_S16LE: >+ case GST_AUDIO_FORMAT_S16BE: >+ return Int16; >+ case GST_AUDIO_FORMAT_S32LE: >+ case GST_AUDIO_FORMAT_S32BE: >+ return Int32; >+ case GST_AUDIO_FORMAT_F32LE: >+ case GST_AUDIO_FORMAT_F32BE: >+ return Float32; >+ case GST_AUDIO_FORMAT_F64LE: >+ case GST_AUDIO_FORMAT_F64BE: >+ return Float64; >+ default: >+ break; >+ } >+ return None; >+ } >+ >+ double sampleRate() const final { return GST_AUDIO_INFO_RATE(&m_info); } >+ bool isPCM() const final { return format() != None; } >+ bool isInterleaved() const final { return GST_AUDIO_INFO_LAYOUT(&m_info) == GST_AUDIO_LAYOUT_INTERLEAVED; } >+ bool isSignedInteger() const final { return GST_AUDIO_INFO_IS_INTEGER(&m_info); } >+ bool isNativeEndian() const final { return GST_AUDIO_INFO_ENDIANNESS(&m_info) == G_BYTE_ORDER; } >+ bool isFloat() const final { return GST_AUDIO_INFO_IS_FLOAT(&m_info); } >+ >+ uint32_t numberOfInterleavedChannels() const final { return isInterleaved() ? GST_AUDIO_INFO_CHANNELS(&m_info) : TRUE; } >+ uint32_t numberOfChannelStreams() const final { return GST_AUDIO_INFO_CHANNELS(&m_info); } >+ uint32_t numberOfChannels() const final { return GST_AUDIO_INFO_CHANNELS(&m_info); } >+ uint32_t sampleWordSize() const final { return GST_AUDIO_INFO_BPS(&m_info); } >+ >+ bool operator==(const GStreamerAudioStreamDescription& other) { return gst_audio_info_is_equal(&m_info, &other.m_info); } >+ bool operator!=(const GStreamerAudioStreamDescription& other) { return !operator == (other); } >+ >+ GstCaps* caps() { return m_caps.get(); } >+ GstAudioInfo* getInfo() { return &m_info; } >+ >+private: >+ GstAudioInfo m_info; >+ GRefPtr<GstCaps> m_caps; >+ mutable PlatformDescription m_platformDescription; >+}; >+ >+} // WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDevice.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDevice.h >new file mode 100644 >index 00000000000..6422ddd83a3 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDevice.h >@@ -0,0 +1,50 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >+ >+#include "CaptureDevice.h" >+#include "GStreamerCommon.h" >+ >+#include <gst/gst.h> >+ >+namespace WebCore { >+ >+class GStreamerCaptureDevice : public CaptureDevice { >+public: >+ GStreamerCaptureDevice(GRefPtr<GstDevice>&& device, const String& persistentId, DeviceType type, const String& label, const String& groupId = emptyString()) >+ : CaptureDevice(persistentId, type, label, groupId) >+ , m_device(device) >+ { >+ } >+ >+ GstCaps* caps() const { return gst_device_get_caps(m_device.get()); } >+ GstDevice* device() { return m_device.get(); } >+ >+private: >+ GRefPtr<GstDevice> m_device; >+}; >+ >+} >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp >new file mode 100644 >index 00000000000..3b78287e7aa >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp >@@ -0,0 +1,134 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >+#include "GStreamerCaptureDeviceManager.h" >+ >+#include "GStreamerCommon.h" >+ >+namespace WebCore { >+ >+GStreamerAudioCaptureDeviceManager& GStreamerAudioCaptureDeviceManager::singleton() >+{ >+ static NeverDestroyed<GStreamerAudioCaptureDeviceManager> manager; >+ return manager; >+} >+ >+GStreamerVideoCaptureDeviceManager& GStreamerVideoCaptureDeviceManager::singleton() >+{ >+ static NeverDestroyed<GStreamerVideoCaptureDeviceManager> manager; >+ return manager; >+} >+ >+GStreamerDisplayCaptureDeviceManager& GStreamerDisplayCaptureDeviceManager::singleton() >+{ >+ static NeverDestroyed<GStreamerDisplayCaptureDeviceManager> manager; >+ return manager; >+} >+ >+std::optional<GStreamerCaptureDevice> GStreamerCaptureDeviceManager::gstreamerDeviceWithUID(const String& deviceID) >+{ >+ captureDevices(); >+ >+ for (auto& device : m_gstreamerDevices) { >+ if (device.persistentId() == deviceID) >+ return device; >+ } >+ return std::nullopt; >+} >+ >+const Vector<CaptureDevice>& GStreamerCaptureDeviceManager::captureDevices() >+{ >+ initializeGStreamer(); >+ if (m_devices.isEmpty()) >+ refreshCaptureDevices(); >+ >+ return m_devices; >+} >+ >+void GStreamerCaptureDeviceManager::deviceAdded(GRefPtr<GstDevice>&& device) >+{ >+ GUniquePtr<GstStructure> properties(gst_device_get_properties(device.get())); >+ const char* klass = gst_structure_get_string(properties.get(), "device.class"); >+ >+ if (klass && !g_strcmp0(klass, "monitor")) >+ return; >+ >+ CaptureDevice::DeviceType type = deviceType(); >+ GUniquePtr<char> deviceClassChar(gst_device_get_device_class(device.get())); >+ String deviceClass(String(deviceClassChar.get())); >+ if (type == CaptureDevice::DeviceType::Microphone && !deviceClass.startsWith("Audio")) >+ return; >+ if (type == CaptureDevice::DeviceType::Camera && !deviceClass.startsWith("Video")) >+ return; >+ >+ // This isn't really a UID but should be good enough (libwebrtc >+ // itself does that at least for pulseaudio devices). >+ GUniquePtr<char> deviceName(gst_device_get_display_name(device.get())); >+ String identifier = String::fromUTF8(deviceName.get()); >+ >+ auto gstCaptureDevice = GStreamerCaptureDevice(WTFMove(device), identifier, type, identifier); >+ gstCaptureDevice.setEnabled(true); >+ m_gstreamerDevices.append(WTFMove(gstCaptureDevice)); >+ // FIXME: We need a CaptureDevice copy in other vector just for captureDevices API. >+ auto captureDevice = CaptureDevice(identifier, type, identifier); >+ captureDevice.setEnabled(true); >+ m_devices.append(WTFMove(captureDevice)); >+} >+ >+void GStreamerCaptureDeviceManager::refreshCaptureDevices() >+{ >+ if (!m_deviceMonitor) { >+ m_deviceMonitor = adoptGRef(gst_device_monitor_new()); >+ >+ CaptureDevice::DeviceType type = deviceType(); >+ if (type == CaptureDevice::DeviceType::Camera) { >+ GRefPtr<GstCaps> caps = adoptGRef(gst_caps_new_empty_simple("video/x-raw")); >+ gst_device_monitor_add_filter(m_deviceMonitor.get(), "Video/Source", caps.get()); >+ } else if (type == CaptureDevice::DeviceType::Microphone) { >+ GRefPtr<GstCaps> caps = adoptGRef(gst_caps_new_empty_simple("audio/x-raw")); >+ gst_device_monitor_add_filter(m_deviceMonitor.get(), "Audio/Source", caps.get()); >+ } >+ // FIXME: Add monitor for added/removed messages on the bus. >+ } >+ >+ if (!gst_device_monitor_start(m_deviceMonitor.get())) { >+ GST_WARNING_OBJECT(m_deviceMonitor.get(), "Could not start device monitor"); >+ m_deviceMonitor = nullptr; >+ >+ return; >+ } >+ >+ GList* devices = gst_device_monitor_get_devices(m_deviceMonitor.get()); >+ while (devices) { >+ GRefPtr<GstDevice> device = adoptGRef(GST_DEVICE_CAST(devices->data)); >+ deviceAdded(WTFMove(device)); >+ devices = g_list_delete_link(devices, devices); >+ } >+ >+ gst_device_monitor_stop(m_deviceMonitor.get()); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h >new file mode 100644 >index 00000000000..86392414d01 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h >@@ -0,0 +1,81 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >+ >+#include "CaptureDeviceManager.h" >+#include "GRefPtrGStreamer.h" >+#include "GStreamerCaptureDevice.h" >+ >+namespace WebCore { >+ >+class GStreamerCaptureDeviceManager : public CaptureDeviceManager { >+public: >+ std::optional<GStreamerCaptureDevice> gstreamerDeviceWithUID(const String&); >+ >+ void refreshCaptureDevices() final; >+ const Vector<CaptureDevice>& captureDevices() final; >+ virtual CaptureDevice::DeviceType deviceType() = 0; >+ >+private: >+ void deviceAdded(GRefPtr<GstDevice>&&); >+ >+ GRefPtr<GstDeviceMonitor> m_deviceMonitor; >+ Vector<GStreamerCaptureDevice> m_gstreamerDevices; >+ Vector<CaptureDevice> m_devices; >+}; >+ >+class GStreamerAudioCaptureDeviceManager final : public GStreamerCaptureDeviceManager { >+ friend class NeverDestroyed<GStreamerAudioCaptureDeviceManager>; >+public: >+ static GStreamerAudioCaptureDeviceManager& singleton(); >+ CaptureDevice::DeviceType deviceType() final { return CaptureDevice::DeviceType::Microphone; } >+private: >+ GStreamerAudioCaptureDeviceManager() = default; >+ ~GStreamerAudioCaptureDeviceManager() = default; >+}; >+ >+class GStreamerVideoCaptureDeviceManager final : public GStreamerCaptureDeviceManager { >+ friend class NeverDestroyed<GStreamerVideoCaptureDeviceManager>; >+public: >+ static GStreamerVideoCaptureDeviceManager& singleton(); >+ static RealtimeMediaSource::VideoCaptureFactory& videoFactory(); >+ CaptureDevice::DeviceType deviceType() final { return CaptureDevice::DeviceType::Camera; } >+private: >+ GStreamerVideoCaptureDeviceManager() = default; >+ ~GStreamerVideoCaptureDeviceManager() = default; >+}; >+ >+class GStreamerDisplayCaptureDeviceManager final : public GStreamerCaptureDeviceManager { >+ friend class NeverDestroyed<GStreamerDisplayCaptureDeviceManager>; >+public: >+ static GStreamerDisplayCaptureDeviceManager& singleton(); >+ CaptureDevice::DeviceType deviceType() final { return CaptureDevice::DeviceType::Screen; } >+private: >+ GStreamerDisplayCaptureDeviceManager() = default; >+ ~GStreamerDisplayCaptureDeviceManager() = default; >+}; >+ >+} >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp >new file mode 100644 >index 00000000000..ecff0351b92 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp >@@ -0,0 +1,198 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if USE(GSTREAMER) >+#include <gst/gst.h> >+#endif >+ >+#if ENABLE(VIDEO) && ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) && GST_CHECK_VERSION(1, 10, 0) >+#include "GStreamerCapturer.h" >+ >+#include <gst/app/gstappsink.h> >+#include <gst/app/gstappsrc.h> >+#include <mutex> >+#include <webrtc/api/mediastreaminterface.h> >+ >+GST_DEBUG_CATEGORY(webkit_capturer_debug); >+#define GST_CAT_DEFAULT webkit_capturer_debug >+ >+namespace WebCore { >+ >+static void initializeGStreamerAndDebug() >+{ >+ initializeGStreamer(); >+ >+ static std::once_flag debugRegisteredFlag; >+ std::call_once(debugRegisteredFlag, [] { >+ GST_DEBUG_CATEGORY_INIT(webkit_capturer_debug, "webkitcapturer", 0, "WebKit Capturer"); >+ }); >+} >+ >+GStreamerCapturer::GStreamerCapturer(GStreamerCaptureDevice device, GRefPtr<GstCaps> caps) >+ : m_device(device.device()) >+ , m_caps(caps) >+ , m_sourceFactory(nullptr) >+{ >+ initializeGStreamerAndDebug(); >+} >+ >+GStreamerCapturer::GStreamerCapturer(const char* sourceFactory, GRefPtr<GstCaps> caps) >+ : m_device(nullptr) >+ , m_caps(caps) >+ , m_sourceFactory(sourceFactory) >+{ >+ initializeGStreamerAndDebug(); >+} >+ >+GStreamerCapturer::~GStreamerCapturer() >+{ >+ if (m_pipeline) >+ disconnectSimpleBusMessageCallback(pipeline()); >+} >+ >+GstElement* GStreamerCapturer::createSource() >+{ >+ if (m_sourceFactory) { >+ m_src = makeElement(m_sourceFactory); >+ if (GST_IS_APP_SRC(m_src.get())) >+ g_object_set(m_src.get(), "is-live", true, "format", GST_FORMAT_TIME, nullptr); >+ >+ ASSERT(m_src); >+ return m_src.get(); >+ } >+ >+ ASSERT(m_device); >+ GUniquePtr<char> sourceName(g_strdup_printf("%s_%p", name(), this)); >+ m_src = gst_device_create_element(m_device.get(), sourceName.get()); >+ ASSERT(m_src); >+ >+ return m_src.get(); >+} >+ >+GstCaps* GStreamerCapturer::caps() >+{ >+ if (m_sourceFactory) { >+ GRefPtr<GstElement> element = makeElement(m_sourceFactory); >+ auto pad = adoptGRef(gst_element_get_static_pad(element.get(), "src")); >+ >+ return gst_pad_query_caps(pad.get(), nullptr); >+ } >+ >+ ASSERT(m_device); >+ return gst_device_get_caps(m_device.get()); >+} >+ >+void GStreamerCapturer::setupPipeline() >+{ >+ if (m_pipeline) >+ disconnectSimpleBusMessageCallback(pipeline()); >+ >+ m_pipeline = makeElement("pipeline"); >+ >+ GRefPtr<GstElement> source = createSource(); >+ GRefPtr<GstElement> converter = createConverter(); >+ >+ m_capsfilter = makeElement("capsfilter"); >+ m_tee = makeElement("tee"); >+ m_sink = makeElement("appsink"); >+ >+ gst_app_sink_set_emit_signals(GST_APP_SINK(m_sink.get()), TRUE); >+ g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr); >+ >+ gst_bin_add_many(GST_BIN(m_pipeline.get()), source.get(), converter.get(), m_capsfilter.get(), m_tee.get(), nullptr); >+ gst_element_link_many(source.get(), converter.get(), m_capsfilter.get(), m_tee.get(), nullptr); >+ >+ addSink(m_sink.get()); >+ >+ connectSimpleBusMessageCallback(pipeline()); >+} >+ >+GstElement* GStreamerCapturer::makeElement(const char* factoryName) >+{ >+ auto element = gst_element_factory_make(factoryName, nullptr); >+ ASSERT(element); >+ GUniquePtr<char> capturerName(g_strdup_printf("%s_capturer_%s_%p", name(), GST_OBJECT_NAME(element), this)); >+ gst_object_set_name(GST_OBJECT(element), capturerName.get()); >+ >+ return element; >+} >+ >+void GStreamerCapturer::addSink(GstElement* newSink) >+{ >+ ASSERT(m_pipeline); >+ ASSERT(m_tee); >+ >+ auto queue = makeElement("queue"); >+ gst_bin_add_many(GST_BIN(pipeline()), queue, newSink, nullptr); >+ gst_element_sync_state_with_parent(queue); >+ gst_element_sync_state_with_parent(newSink); >+ >+ if (!gst_element_link_pads(m_tee.get(), "src_%u", queue, "sink")) { >+ ASSERT_NOT_REACHED(); >+ return; >+ } >+ >+ if (!gst_element_link(queue, newSink)) { >+ ASSERT_NOT_REACHED(); >+ return; >+ } >+ >+ if (newSink == m_sink.get()) { >+ GST_INFO_OBJECT(m_pipeline.get(), "Setting queue as leaky upstream" >+ " so that the player can set the sink to PAUSED without " >+ " setting the whole capturer to PAUSED"); >+ g_object_set(queue, "leaky", 2 /* upstream */, nullptr); >+ } >+ >+ GST_INFO_OBJECT(pipeline(), "Adding sink: %" GST_PTR_FORMAT, newSink); >+ >+ GUniquePtr<char> dumpName(g_strdup_printf("%s_sink_%s_added", GST_OBJECT_NAME(pipeline()), GST_OBJECT_NAME(newSink))); >+ GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(GST_BIN(pipeline()), GST_DEBUG_GRAPH_SHOW_ALL, dumpName.get()); >+} >+ >+void GStreamerCapturer::play() >+{ >+ ASSERT(m_pipeline); >+ >+ GST_INFO_OBJECT(pipeline(), "Going to PLAYING!"); >+ >+ gst_element_set_state(pipeline(), GST_STATE_PLAYING); >+} >+ >+void GStreamerCapturer::stop() >+{ >+ ASSERT(m_pipeline); >+ >+ GST_INFO_OBJECT(pipeline(), "Tearing down!"); >+ >+ // Make sure to remove sync handler before tearing down, avoiding >+ // possible deadlocks. >+ GRefPtr<GstBus> bus = adoptGRef(gst_pipeline_get_bus(GST_PIPELINE(pipeline()))); >+ gst_bus_set_sync_handler(bus.get(), nullptr, nullptr, nullptr); >+ >+ gst_element_set_state(pipeline(), GST_STATE_NULL); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(VIDEO) && ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) && GST_CHECK_VERSION(1, 10, 0) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h >new file mode 100644 >index 00000000000..7bfcedad801 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h >@@ -0,0 +1,72 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GStreamerCaptureDevice.h" >+#include "GStreamerCommon.h" >+#include "LibWebRTCMacros.h" >+ >+#include <gst/gst.h> >+ >+namespace WebCore { >+ >+class GStreamerCapturer { >+public: >+ GStreamerCapturer(GStreamerCaptureDevice, GRefPtr<GstCaps>); >+ GStreamerCapturer(const char* sourceFactory, GRefPtr<GstCaps>); >+ ~GStreamerCapturer(); >+ >+ void setupPipeline(); >+ virtual void play(); >+ virtual void stop(); >+ GstCaps* caps(); >+ void addSink(GstElement *newSink); >+ GstElement* makeElement(const char* factoryName); >+ GstElement* createSource(); >+ GstElement* source() { return m_src.get(); } >+ virtual const char* name() = 0; >+ >+ GstElement* sink() const { return m_sink.get(); } >+ void setSink(GstElement* sink) { m_sink = adoptGRef(sink); }; >+ >+ GstElement* pipeline() const { return m_pipeline.get(); } >+ virtual GstElement* createConverter() = 0; >+ >+protected: >+ GRefPtr<GstElement> m_sink; >+ GRefPtr<GstElement> m_src; >+ GRefPtr<GstElement> m_tee; >+ GRefPtr<GstElement> m_capsfilter; >+ GRefPtr<GstDevice> m_device; >+ GRefPtr<GstCaps> m_caps; >+ GRefPtr<GstElement> m_pipeline; >+ >+private: >+ const char* m_sourceFactory; >+ >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp >new file mode 100644 >index 00000000000..799405c29c6 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp >@@ -0,0 +1,531 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if USE(GSTREAMER) >+#include <gst/gst.h> >+#endif >+ >+#if ENABLE(VIDEO) && ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) && GST_CHECK_VERSION(1, 10, 0) >+#include "GStreamerMediaStreamSource.h" >+ >+#include "AudioTrackPrivate.h" >+#include "GStreamerCommon.h" >+#include "GStreamerVideoCaptureSource.h" >+#include "MediaSampleGStreamer.h" >+#include "VideoTrackPrivate.h" >+#include "GStreamerAudioData.h" >+ >+#include <gst/app/gstappsrc.h> >+#include <gst/base/gstflowcombiner.h> >+ >+namespace WebCore { >+ >+static void webkitMediaStreamSrcPushVideoSample(WebKitMediaStreamSrc* self, GstSample* gstsample); >+static void webkitMediaStreamSrcPushAudioSample(WebKitMediaStreamSrc* self, GstSample* gstsample); >+static void webkitMediaStreamSrcTrackEnded(WebKitMediaStreamSrc* self, MediaStreamTrackPrivate&); >+ >+static GstStaticPadTemplate videoSrcTemplate = GST_STATIC_PAD_TEMPLATE("video_src", >+ GST_PAD_SRC, >+ GST_PAD_SOMETIMES, >+ GST_STATIC_CAPS("video/x-raw")); >+ >+static GstStaticPadTemplate audioSrcTemplate = GST_STATIC_PAD_TEMPLATE("audio_src", >+ GST_PAD_SRC, >+ GST_PAD_SOMETIMES, >+ GST_STATIC_CAPS("audio/x-raw(ANY);")); >+ >+static GstTagList* mediaStreamTrackPrivateGetTags(MediaStreamTrackPrivate* track) >+{ >+ auto taglist = gst_tag_list_new_empty(); >+ >+ if (!track->label().isEmpty()) { >+ gst_tag_list_add(taglist, GST_TAG_MERGE_APPEND, >+ GST_TAG_TITLE, track->label().utf8().data(), nullptr); >+ } >+ >+ if (track->type() == RealtimeMediaSource::Type::Audio) { >+ gst_tag_list_add(taglist, GST_TAG_MERGE_APPEND, WEBKIT_MEDIA_TRACK_TAG_KIND, >+ static_cast<int>(AudioTrackPrivate::Kind::Main), nullptr); >+ } else if (track->type() == RealtimeMediaSource::Type::Video) { >+ gst_tag_list_add(taglist, GST_TAG_MERGE_APPEND, WEBKIT_MEDIA_TRACK_TAG_KIND, >+ static_cast<int>(VideoTrackPrivate::Kind::Main), nullptr); >+ >+ if (track->isCaptureTrack()) { >+ GStreamerVideoCaptureSource& source = static_cast<GStreamerVideoCaptureSource&>( >+ track->source()); >+ >+ gst_tag_list_add(taglist, GST_TAG_MERGE_APPEND, >+ WEBKIT_MEDIA_TRACK_TAG_WIDTH, source.size().width(), >+ WEBKIT_MEDIA_TRACK_TAG_HEIGHT, source.size().height(), nullptr); >+ } >+ } >+ >+ return taglist; >+} >+ >+GstStream* webkitMediaStreamNew(MediaStreamTrackPrivate* track) >+{ >+ GRefPtr<GstCaps> caps; >+ GstStreamType type; >+ >+ if (track->type() == RealtimeMediaSource::Type::Audio) { >+ caps = adoptGRef(gst_static_pad_template_get_caps(&audioSrcTemplate)); >+ type = GST_STREAM_TYPE_AUDIO; >+ } else if (track->type() == RealtimeMediaSource::Type::Video) { >+ caps = adoptGRef(gst_static_pad_template_get_caps(&videoSrcTemplate)); >+ type = GST_STREAM_TYPE_VIDEO; >+ } else { >+ GST_FIXME("Handle %d type", (gint) track->type()); >+ >+ return nullptr; >+ } >+ >+ auto gststream = (GstStream*)gst_stream_new(track->id().utf8().data(), >+ caps.get(), type, GST_STREAM_FLAG_SELECT); >+ auto tags = adoptGRef(mediaStreamTrackPrivateGetTags(track)); >+ gst_stream_set_tags(gststream, tags.get()); >+ >+ return gststream; >+} >+ >+class WebKitMediaStreamTrackObserver >+ : public MediaStreamTrackPrivate::Observer { >+public: >+ virtual ~WebKitMediaStreamTrackObserver() { }; >+ WebKitMediaStreamTrackObserver(WebKitMediaStreamSrc* src) >+ : m_mediaStreamSrc(src) { } >+ void trackStarted(MediaStreamTrackPrivate&) final { }; >+ >+ void trackEnded(MediaStreamTrackPrivate& track) final >+ { >+ webkitMediaStreamSrcTrackEnded(m_mediaStreamSrc, track); >+ } >+ >+ void trackMutedChanged(MediaStreamTrackPrivate&) final { }; >+ void trackSettingsChanged(MediaStreamTrackPrivate&) final { }; >+ void trackEnabledChanged(MediaStreamTrackPrivate&) final { }; >+ void readyStateChanged(MediaStreamTrackPrivate&) final { }; >+ >+ void sampleBufferUpdated(MediaStreamTrackPrivate&, MediaSample& sample) final >+ { >+ auto gstsample = static_cast<MediaSampleGStreamer*>(&sample)->platformSample().sample.gstSample; >+ >+ webkitMediaStreamSrcPushVideoSample(m_mediaStreamSrc, gstsample); >+ } >+ >+ void audioSamplesAvailable(MediaStreamTrackPrivate&, const MediaTime&, const PlatformAudioData& audioData, const AudioStreamDescription&, size_t) final >+ { >+ auto audiodata = static_cast<const GStreamerAudioData&>(audioData); >+ >+ webkitMediaStreamSrcPushAudioSample(m_mediaStreamSrc, audiodata.getSample()); >+ } >+ >+private: >+ WebKitMediaStreamSrc* m_mediaStreamSrc; >+}; >+ >+typedef struct _WebKitMediaStreamSrcClass WebKitMediaStreamSrcClass; >+struct _WebKitMediaStreamSrc { >+ GstBin parent_instance; >+ >+ gchar* uri; >+ >+ GstElement* audioSrc; >+ GstElement* videoSrc; >+ >+ std::unique_ptr<WebKitMediaStreamTrackObserver> observer; >+ String videoTrackID; >+ volatile gint npads; >+ gulong probeid; >+ RefPtr<MediaStreamPrivate> stream; >+ >+ GstFlowCombiner* flowCombiner; >+ GRefPtr<GstStreamCollection> streamCollection; >+}; >+ >+struct _WebKitMediaStreamSrcClass { >+ GstBinClass parent_class; >+}; >+ >+enum { >+ PROP_0, >+ PROP_IS_LIVE, >+ PROP_LAST >+}; >+ >+static GstURIType webkit_media_stream_src_uri_get_type(GType) >+{ >+ return GST_URI_SRC; >+} >+ >+static const gchar* const* webkit_media_stream_src_uri_get_protocols(GType) >+{ >+ static const gchar* protocols[] = { "mediastream", nullptr }; >+ >+ return protocols; >+} >+ >+static gchar* webkit_media_stream_src_uri_get_uri(GstURIHandler* handler) >+{ >+ WebKitMediaStreamSrc* self = WEBKIT_MEDIA_STREAM_SRC(handler); >+ >+ /* FIXME: make thread-safe */ >+ return g_strdup(self->uri); >+} >+ >+static gboolean webkitMediaStreamSrcUriSetUri(GstURIHandler* handler, const gchar* uri, >+ GError**) >+{ >+ WebKitMediaStreamSrc* self = WEBKIT_MEDIA_STREAM_SRC(handler); >+ self->uri = g_strdup(uri); >+ >+ return TRUE; >+} >+ >+static void webkitMediaStreamSrcUriHandlerInit(gpointer g_iface, gpointer) >+{ >+ GstURIHandlerInterface* iface = (GstURIHandlerInterface*)g_iface; >+ >+ iface->get_type = webkit_media_stream_src_uri_get_type; >+ iface->get_protocols = webkit_media_stream_src_uri_get_protocols; >+ iface->get_uri = webkit_media_stream_src_uri_get_uri; >+ iface->set_uri = webkitMediaStreamSrcUriSetUri; >+} >+ >+GST_DEBUG_CATEGORY_STATIC(webkitMediaStreamSrcDebug); >+#define GST_CAT_DEFAULT webkitMediaStreamSrcDebug >+ >+#define doInit \ >+ G_IMPLEMENT_INTERFACE(GST_TYPE_URI_HANDLER, webkitMediaStreamSrcUriHandlerInit); \ >+ GST_DEBUG_CATEGORY_INIT(webkitMediaStreamSrcDebug, "webkitwebmediastreamsrc", 0, "mediastreamsrc element"); \ >+ gst_tag_register_static(WEBKIT_MEDIA_TRACK_TAG_WIDTH, GST_TAG_FLAG_META, G_TYPE_INT, "Webkit MediaStream width", "Webkit MediaStream width", gst_tag_merge_use_first); \ >+ gst_tag_register_static(WEBKIT_MEDIA_TRACK_TAG_HEIGHT, GST_TAG_FLAG_META, G_TYPE_INT, "Webkit MediaStream height", "Webkit MediaStream height", gst_tag_merge_use_first); \ >+ gst_tag_register_static(WEBKIT_MEDIA_TRACK_TAG_KIND, GST_TAG_FLAG_META, G_TYPE_INT, "Webkit MediaStream Kind", "Webkit MediaStream Kind", gst_tag_merge_use_first); >+ >+G_DEFINE_TYPE_WITH_CODE(WebKitMediaStreamSrc, webkit_media_stream_src, GST_TYPE_BIN, doInit); >+ >+static void webkitMediaStreamSrcSetProperty(GObject* object, guint prop_id, >+ const GValue*, GParamSpec* pspec) >+{ >+ switch (prop_id) { >+ default: >+ G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); >+ break; >+ } >+} >+ >+static void webkitMediaStreamSrcGetProperty(GObject* object, guint prop_id, GValue* value, >+ GParamSpec* pspec) >+{ >+ switch (prop_id) { >+ case PROP_IS_LIVE: >+ g_value_set_boolean(value, TRUE); >+ break; >+ default: >+ G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); >+ break; >+ } >+} >+ >+static void webkitMediaStreamSrcDispose(GObject* object) >+{ >+ WebKitMediaStreamSrc* self = WEBKIT_MEDIA_STREAM_SRC(object); >+ >+ if (self->audioSrc) { >+ gst_bin_remove(GST_BIN(self), self->audioSrc); >+ self->audioSrc = nullptr; >+ } >+ >+ if (self->videoSrc) { >+ gst_bin_remove(GST_BIN(self), self->videoSrc); >+ self->videoSrc = nullptr; >+ } >+} >+ >+static void webkitMediaStreamSrcFinalize(GObject* object) >+{ >+ WebKitMediaStreamSrc* self = WEBKIT_MEDIA_STREAM_SRC(object); >+ >+ GST_OBJECT_LOCK(self); >+ for (auto& track : self->stream->tracks()) >+ track->removeObserver(*self->observer.get()); >+ GST_OBJECT_UNLOCK(self); >+ >+ g_clear_pointer(&self->uri, g_free); >+ gst_flow_combiner_free(self->flowCombiner); >+} >+ >+static GstStateChangeReturn webkitMediaStreamSrcChangeState(GstElement* element, GstStateChange transition) >+{ >+ GstStateChangeReturn result; >+ auto* self = WEBKIT_MEDIA_STREAM_SRC(element); >+ >+ if (transition == GST_STATE_CHANGE_PAUSED_TO_READY) { >+ >+ GST_OBJECT_LOCK(self); >+ for (auto& track : self->stream->tracks()) >+ track->removeObserver(*self->observer.get()); >+ GST_OBJECT_UNLOCK(self); >+ } >+ >+ result = GST_ELEMENT_CLASS(webkit_media_stream_src_parent_class)->change_state(element, transition); >+ >+ if (transition == GST_STATE_CHANGE_READY_TO_PAUSED) >+ result = GST_STATE_CHANGE_NO_PREROLL; >+ >+ return result; >+} >+ >+static void webkit_media_stream_src_class_init(WebKitMediaStreamSrcClass* klass) >+{ >+ GObjectClass* gobject_class = G_OBJECT_CLASS(klass); >+ GstElementClass* gstelement_klass = GST_ELEMENT_CLASS(klass); >+ >+ gobject_class->finalize = webkitMediaStreamSrcFinalize; >+ gobject_class->dispose = webkitMediaStreamSrcDispose; >+ gobject_class->get_property = webkitMediaStreamSrcGetProperty; >+ gobject_class->set_property = webkitMediaStreamSrcSetProperty; >+ >+ g_object_class_install_property(gobject_class, PROP_IS_LIVE, >+ g_param_spec_boolean("is-live", "Is Live", >+ "Let playbin3 know we are a live source.", >+ TRUE, (GParamFlags)(G_PARAM_READABLE | G_PARAM_STATIC_STRINGS))); >+ >+ gstelement_klass->change_state = webkitMediaStreamSrcChangeState; >+ gst_element_class_add_pad_template(gstelement_klass, >+ gst_static_pad_template_get(&videoSrcTemplate)); >+ gst_element_class_add_pad_template(gstelement_klass, >+ gst_static_pad_template_get(&audioSrcTemplate)); >+} >+ >+static void webkit_media_stream_src_init(WebKitMediaStreamSrc* self) >+{ >+ self->observer = std::make_unique<WebKitMediaStreamTrackObserver>(self); >+ self->flowCombiner = gst_flow_combiner_new(); >+} >+ >+typedef struct { >+ WebKitMediaStreamSrc* self; >+ RefPtr<MediaStreamTrackPrivate> track; >+ GstStaticPadTemplate* pad_template; >+} ProbeData; >+ >+static GstFlowReturn webkitMediaStreamSrcChain(GstPad* pad, GstObject* object, GstBuffer* buffer) >+{ >+ GstFlowReturn result; >+ GRefPtr<WebKitMediaStreamSrc> self = adoptGRef(WEBKIT_MEDIA_STREAM_SRC(gst_object_get_parent(object))); >+ >+ result = gst_flow_combiner_update_pad_flow(self.get()->flowCombiner, pad, >+ gst_proxy_pad_chain_default(pad, GST_OBJECT(self.get()), buffer)); >+ >+ return result; >+} >+ >+static void webkitMediaStreamSrcAddPad(WebKitMediaStreamSrc* self, GstPad* target, GstStaticPadTemplate* pad_template) >+{ >+ auto padname = String::format("src_%u", g_atomic_int_add(&(self->npads), 1)); >+ auto ghostpad = gst_ghost_pad_new_from_template(padname.utf8().data(), target, >+ gst_static_pad_template_get(pad_template)); >+ >+ GST_DEBUG_OBJECT(self, "%s Ghosting %" GST_PTR_FORMAT, >+ gst_object_get_path_string(GST_OBJECT_CAST(self)), >+ target); >+ >+ auto proxypad = adoptGRef(GST_PAD(gst_proxy_pad_get_internal(GST_PROXY_PAD(ghostpad)))); >+ gst_pad_set_chain_function(proxypad.get(), >+ static_cast<GstPadChainFunction>(webkitMediaStreamSrcChain)); >+ gst_pad_set_active(ghostpad, TRUE); >+ ASSERT(gst_element_add_pad(GST_ELEMENT(self), GST_PAD (ghostpad))); >+} >+ >+static GstPadProbeReturn webkitMediaStreamSrcPadProbeCb(GstPad* pad, GstPadProbeInfo* info, ProbeData* data) >+{ >+ GstEvent* event = GST_PAD_PROBE_INFO_EVENT(info); >+ WebKitMediaStreamSrc* self = data->self; >+ >+ switch (GST_EVENT_TYPE(event)) { >+ case GST_EVENT_STREAM_START: { >+ const gchar* stream_id; >+ GRefPtr<GstStream> stream = nullptr; >+ >+ gst_event_parse_stream_start(event, &stream_id); >+ if (!g_strcmp0(stream_id, data->track->id().utf8().data())) { >+ GST_INFO_OBJECT(pad, "Event has been sticked already"); >+ return GST_PAD_PROBE_OK; >+ } >+ >+ auto stream_start = gst_event_new_stream_start(data->track->id().utf8().data()); >+ gst_event_set_group_id(stream_start, 1); >+ gst_event_unref(event); >+ >+ gst_pad_push_event(pad, stream_start); >+ gst_pad_push_event(pad, gst_event_new_stream_collection(self->streamCollection.get())); >+ gst_pad_push_event(pad, gst_event_new_tag(mediaStreamTrackPrivateGetTags(data->track.get()))); >+ >+ webkitMediaStreamSrcAddPad(self, pad, data->pad_template); >+ >+ return GST_PAD_PROBE_HANDLED; >+ } >+ default: >+ break; >+ } >+ >+ return GST_PAD_PROBE_OK; >+} >+ >+static gboolean webkitMediaStreamSrcSetupSrc(WebKitMediaStreamSrc* self, >+ MediaStreamTrackPrivate* track, GstElement* element, >+ GstStaticPadTemplate* pad_template, gboolean observe_track) >+{ >+ auto pad = adoptGRef(gst_element_get_static_pad(element, "src")); >+ >+ gst_bin_add(GST_BIN(self), element); >+ >+ >+ ProbeData* data = new ProbeData; >+ data->self = WEBKIT_MEDIA_STREAM_SRC(self); >+ data->pad_template = pad_template; >+ data->track = track; >+ >+ self->probeid = gst_pad_add_probe(pad.get(), (GstPadProbeType)GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM, >+ (GstPadProbeCallback)webkitMediaStreamSrcPadProbeCb, data, >+ [](gpointer data) { >+ delete (ProbeData*)data; >+ }); >+ >+ if (observe_track) >+ track->addObserver(*self->observer.get()); >+ >+ gst_element_sync_state_with_parent(element); >+ return TRUE; >+} >+ >+static gboolean webkitMediaStreamSrcSetupAppSrc(WebKitMediaStreamSrc* self, >+ MediaStreamTrackPrivate* track, GstElement** element, >+ GstStaticPadTemplate* pad_template) >+{ >+ *element = gst_element_factory_make("appsrc", nullptr); >+ g_object_set(*element, "is-live", true, "format", GST_FORMAT_TIME, nullptr); >+ >+ return webkitMediaStreamSrcSetupSrc(self, track, *element, pad_template, TRUE); >+} >+ >+static void webkitMediaStreamSrcPostStreamCollection(WebKitMediaStreamSrc* self, MediaStreamPrivate* stream) >+{ >+ GST_OBJECT_LOCK(self); >+ self->streamCollection = adoptGRef(gst_stream_collection_new(stream->id().utf8().data())); >+ for (auto& track : stream->tracks()) { >+ auto gststream = webkitMediaStreamNew(track.get()); >+ >+ gst_stream_collection_add_stream(self->streamCollection.get(), gststream); >+ } >+ GST_OBJECT_UNLOCK(self); >+ >+ gst_element_post_message(GST_ELEMENT(self), >+ gst_message_new_stream_collection(GST_OBJECT(self), self->streamCollection.get())); >+} >+ >+gboolean webkitMediaStreamSrcSetStream(WebKitMediaStreamSrc* self, MediaStreamPrivate* stream) >+{ >+ g_return_val_if_fail(WEBKIT_IS_MEDIA_STREAM_SRC(self), FALSE); >+ >+ if (self->audioSrc) { >+ gst_element_set_state(self->audioSrc, GST_STATE_NULL); >+ gst_bin_remove(GST_BIN(self), self->audioSrc); >+ self->audioSrc = nullptr; >+ } >+ >+ if (self->videoSrc) { >+ gst_element_set_state(self->videoSrc, GST_STATE_NULL); >+ gst_bin_remove(GST_BIN(self), self->videoSrc); >+ self->videoSrc = nullptr; >+ } >+ >+ webkitMediaStreamSrcPostStreamCollection(self, stream); >+ >+ self->stream = stream; >+ for (auto& track : stream->tracks()) { >+ if (track->type() == RealtimeMediaSource::Type::Audio) { >+ webkitMediaStreamSrcSetupAppSrc(self, track.get(), &self->audioSrc, >+ &audioSrcTemplate); >+ } else if (track->type() == RealtimeMediaSource::Type::Video) { >+ webkitMediaStreamSrcSetupAppSrc(self, track.get(), &self->videoSrc, >+ &videoSrcTemplate); >+ } else { >+ GST_INFO("Unsuported track type: %d", (gint) track->type()); >+ continue; >+ } >+ } >+ >+ return TRUE; >+} >+ >+static void webkitMediaStreamSrcPushVideoSample(WebKitMediaStreamSrc* self, GstSample* gstsample) >+{ >+ if (self->videoSrc) >+ gst_app_src_push_sample(GST_APP_SRC(self->videoSrc), gstsample); >+} >+ >+static void webkitMediaStreamSrcPushAudioSample(WebKitMediaStreamSrc* self, GstSample* gstsample) >+{ >+ if (self->audioSrc) >+ gst_app_src_push_sample(GST_APP_SRC(self->audioSrc), gstsample); >+} >+ >+static void webkitMediaStreamSrcTrackEnded(WebKitMediaStreamSrc* self, >+ MediaStreamTrackPrivate& track) >+{ >+ GRefPtr<GstPad> pad = nullptr; >+ >+ GST_OBJECT_LOCK(self); >+ for (auto tmp = GST_ELEMENT(self)->srcpads; tmp; tmp = tmp->next) { >+ GstPad* tmppad = GST_PAD(tmp->data); >+ const gchar* stream_id; >+ >+ GstEvent* stream_start = gst_pad_get_sticky_event(tmppad, GST_EVENT_STREAM_START, 0); >+ if (!stream_start) >+ continue; >+ >+ gst_event_parse_stream_start(stream_start, &stream_id); >+ if (String(stream_id) == track.id()) { >+ pad = tmppad; >+ break; >+ } >+ } >+ GST_OBJECT_UNLOCK(self); >+ >+ if (!pad) { >+ GST_ERROR_OBJECT(self, "No pad found for %s", track.id().utf8().data()); >+ >+ return; >+ } >+ >+ // Make sure that the video.videoWidth is reset to 0 >+ webkitMediaStreamSrcPostStreamCollection(self, self->stream.get()); >+ auto tags = mediaStreamTrackPrivateGetTags(&track); >+ gst_pad_push_event(pad.get(), gst_event_new_tag(tags)); >+ gst_pad_push_event(pad.get(), gst_event_new_eos()); >+} >+ >+} // WebCore >+#endif // ENABLE(VIDEO) && ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.h >new file mode 100644 >index 00000000000..eb4112e707f >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.h >@@ -0,0 +1,46 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(VIDEO) && ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >+ >+#include "MediaStreamPrivate.h" >+#include "MediaStreamTrackPrivate.h" >+ >+#include <gst/gst.h> >+ >+#define WEBKIT_MEDIA_TRACK_TAG_WIDTH "webkit-media-stream-width" >+#define WEBKIT_MEDIA_TRACK_TAG_HEIGHT "webkit-media-stream-height" >+#define WEBKIT_MEDIA_TRACK_TAG_KIND "webkit-media-stream-kind" >+ >+namespace WebCore { >+ >+typedef struct _WebKitMediaStreamSrc WebKitMediaStreamSrc; >+ >+#define WEBKIT_MEDIA_STREAM_SRC(o) (G_TYPE_CHECK_INSTANCE_CAST((o), WEBKIT_TYPE_MEDIA_STREAM_SRC, WebKitMediaStreamSrc)) >+#define WEBKIT_IS_MEDIA_STREAM_SRC(o) (G_TYPE_CHECK_INSTANCE_TYPE((o), WEBKIT_TYPE_MEDIA_STREAM_SRC)) >+#define WEBKIT_TYPE_MEDIA_STREAM_SRC (webkit_media_stream_src_get_type()) >+GType webkit_media_stream_src_get_type(void) G_GNUC_CONST; >+gboolean webkitMediaStreamSrcSetStream(WebKitMediaStreamSrc*, MediaStreamPrivate*); >+} // WebCore >+ >+#endif // ENABLE(VIDEO) && ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp >new file mode 100644 >index 00000000000..9ae1defb0cd >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp >@@ -0,0 +1,270 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerVideoCaptureSource.h" >+ >+#include "GStreamerCaptureDeviceManager.h" >+#include "MediaSampleGStreamer.h" >+ >+#include <gst/app/gstappsink.h> >+#include <webrtc/api/mediastreaminterface.h> >+#include <webrtc/api/peerconnectioninterface.h> >+#include <webrtc/media/base/videocommon.h> >+#include <webrtc/media/engine/webrtcvideocapturer.h> >+#include <webrtc/media/engine/webrtcvideocapturerfactory.h> >+#include <webrtc/modules/video_capture/video_capture_defines.h> >+ >+namespace WebCore { >+ >+const static int defaultWidth = 640; >+const static int defaultHeight = 480; >+ >+GST_DEBUG_CATEGORY(webkit_video_capture_source_debug); >+#define GST_CAT_DEFAULT webkit_video_capture_source_debug >+ >+static void initializeGStreamerDebug() >+{ >+ static std::once_flag debugRegisteredFlag; >+ std::call_once(debugRegisteredFlag, [] { >+ GST_DEBUG_CATEGORY_INIT(webkit_video_capture_source_debug, "webkitvideocapturesource", 0, >+ "WebKit Video Capture Source."); >+ }); >+} >+ >+class GStreamerVideoCaptureSourceFactory final : public RealtimeMediaSource::VideoCaptureFactory { >+public: >+ CaptureSourceOrError createVideoCaptureSource(const CaptureDevice& device, const MediaConstraints* constraints) final >+ { >+ return GStreamerVideoCaptureSource::create(device.persistentId(), constraints); >+ } >+}; >+ >+RealtimeMediaSource::VideoCaptureFactory& libWebRTCVideoCaptureSourceFactory() >+{ >+ static NeverDestroyed<GStreamerVideoCaptureSourceFactory> factory; >+ return factory.get(); >+} >+ >+CaptureSourceOrError GStreamerVideoCaptureSource::create(const String& deviceID, const MediaConstraints* constraints) >+{ >+ auto device = GStreamerVideoCaptureDeviceManager::singleton().gstreamerDeviceWithUID(deviceID); >+ if (!device) { >+ auto errorMessage = String::format("GStreamerVideoCaptureSource::create(): GStreamer did not find the device: %s.", deviceID.utf8().data()); >+ return CaptureSourceOrError(WTFMove(errorMessage)); >+ } >+ >+ auto source = adoptRef(*new GStreamerVideoCaptureSource(device.value())); >+ >+ if (constraints) { >+ auto result = source->applyConstraints(*constraints); >+ if (result) >+ return WTFMove(result.value().first); >+ } >+ return CaptureSourceOrError(WTFMove(source)); >+} >+ >+RealtimeMediaSource::VideoCaptureFactory& GStreamerVideoCaptureSource::factory() >+{ >+ return libWebRTCVideoCaptureSourceFactory(); >+} >+ >+GStreamerVideoCaptureSource::GStreamerVideoCaptureSource(const String& deviceID, const String& name, const gchar *source_factory) >+ : RealtimeMediaSource(deviceID, RealtimeMediaSource::Type::Video, name) >+ , m_capturer(std::make_unique<GStreamerVideoCapturer>(source_factory)) >+{ >+ initializeGStreamerDebug(); >+} >+ >+GStreamerVideoCaptureSource::GStreamerVideoCaptureSource(GStreamerCaptureDevice device) >+ : RealtimeMediaSource(device.persistentId(), RealtimeMediaSource::Type::Video, device.label()) >+ , m_capturer(std::make_unique<GStreamerVideoCapturer>(device)) >+{ >+ initializeGStreamerDebug(); >+} >+ >+GStreamerVideoCaptureSource::~GStreamerVideoCaptureSource() >+{ >+} >+ >+bool GStreamerVideoCaptureSource::applySize(const IntSize &size) >+{ >+ m_capturer->setSize(size.width(), size.height()); >+ >+ return true; >+} >+ >+bool GStreamerVideoCaptureSource::applyFrameRate(double framerate) >+{ >+ m_capturer->setFrameRate(framerate); >+ >+ return true; >+} >+ >+void GStreamerVideoCaptureSource::startProducingData() >+{ >+ m_capturer->setupPipeline(); >+ m_capturer->setSize(size().width(), size().height()); >+ m_capturer->setFrameRate(frameRate()); >+ g_signal_connect(m_capturer->sink(), "new-sample", G_CALLBACK(newSampleCallback), this); >+ m_capturer->play(); >+} >+ >+GstFlowReturn GStreamerVideoCaptureSource::newSampleCallback(GstElement* sink, GStreamerVideoCaptureSource* source) >+{ >+ auto gstSample = adoptGRef(gst_app_sink_pull_sample(GST_APP_SINK(sink))); >+ auto mediaSample = MediaSampleGStreamer::create(WTFMove(gstSample), WebCore::FloatSize(), String()); >+ >+ // FIXME - Check how presentationSize is supposed to be used here. >+ callOnMainThread([protectedThis = makeRef(*source), mediaSample = WTFMove(mediaSample)] { >+ protectedThis->videoSampleAvailable(mediaSample.get()); >+ }); >+ >+ return GST_FLOW_OK; >+} >+ >+void GStreamerVideoCaptureSource::stopProducingData() >+{ >+ g_signal_handlers_disconnect_by_func(m_capturer->sink(), reinterpret_cast<gpointer>(newSampleCallback), this); >+ m_capturer->stop(); >+ >+ GST_INFO("Reset height and width after stopping source"); >+ setHeight(0); >+ setWidth(0); >+} >+ >+const RealtimeMediaSourceCapabilities& GStreamerVideoCaptureSource::capabilities() const >+{ >+ if (m_capabilities) >+ return m_capabilities.value(); >+ >+ RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints()); >+ GRefPtr<GstCaps> caps = adoptGRef(m_capturer->caps()); >+ int32_t minWidth = G_MAXINT32, maxWidth = 0, minHeight = G_MAXINT32, maxHeight = 0; >+ double minFramerate = G_MAXDOUBLE, maxFramerate = G_MINDOUBLE; >+ >+ for (guint i = 0; i < gst_caps_get_size(caps.get()); i++) { >+ GstStructure* str = gst_caps_get_structure(caps.get(), i); >+ >+ // Only accept raw video for now. >+ if (!gst_structure_has_name(str, "video/x-raw")) >+ continue; >+ >+ int32_t tmpMinWidth, tmpMinHeight, tmpMinFPSNumerator, tmpMinFPSDenominator; >+ int32_t tmpMaxWidth, tmpMaxHeight, tmpMaxFPSNumerator, tmpMaxFPSDenominator; >+ double tmpMinFramerate = G_MAXDOUBLE, tmpMaxFramerate = G_MINDOUBLE; >+ >+ if (!gst_structure_get(str, "width", GST_TYPE_INT_RANGE, &tmpMinWidth, &tmpMaxWidth, "height", GST_TYPE_INT_RANGE, &tmpMinHeight, &tmpMaxHeight, nullptr)) { >+ if (!gst_structure_get(str, "width", G_TYPE_INT, &tmpMinWidth, "height", G_TYPE_INT, &tmpMinHeight, nullptr)) >+ continue; >+ >+ tmpMaxWidth = tmpMinWidth; >+ tmpMaxHeight = tmpMinHeight; >+ } >+ >+ if (gst_structure_get(str, "framerate", GST_TYPE_FRACTION_RANGE, &tmpMinFPSNumerator, &tmpMinFPSDenominator, &tmpMaxFPSNumerator, &tmpMaxFPSDenominator, nullptr)) { >+ gst_util_fraction_to_double (tmpMinFPSNumerator, tmpMinFPSDenominator, &tmpMinFramerate); >+ gst_util_fraction_to_double (tmpMaxFPSNumerator, tmpMaxFPSDenominator, &tmpMaxFramerate); >+ } else if (gst_structure_get(str, >+ "framerate", GST_TYPE_FRACTION, &tmpMinFPSNumerator, &tmpMinFPSDenominator, nullptr)) { >+ tmpMaxFPSNumerator = tmpMinFPSNumerator; >+ tmpMaxFPSDenominator = tmpMinFPSDenominator; >+ gst_util_fraction_to_double (tmpMinFPSNumerator, tmpMinFPSDenominator, &tmpMinFramerate); >+ gst_util_fraction_to_double (tmpMaxFPSNumerator, tmpMaxFPSDenominator, &tmpMaxFramerate); >+ } else { >+ const GValue* frameRates(gst_structure_get_value(str, "framerate")); >+ tmpMinFramerate = G_MAXDOUBLE; >+ tmpMaxFramerate = 0.0; >+ >+ guint frameRatesLength = static_cast<guint>(gst_value_list_get_size(frameRates)) - 1; >+ >+ for (guint i = 0; i < frameRatesLength; i++) { >+ double tmpFrameRate; >+ const GValue* val = gst_value_list_get_value(frameRates, i); >+ >+ ASSERT(G_VALUE_TYPE(val) == GST_TYPE_FRACTION); >+ gst_util_fraction_to_double (gst_value_get_fraction_numerator(val), >+ gst_value_get_fraction_denominator(val), &tmpFrameRate); >+ >+ tmpMinFramerate = std::min(tmpMinFramerate, tmpFrameRate); >+ tmpMaxFramerate = std::max(tmpMaxFramerate, tmpFrameRate); >+ } >+ >+ if (i > 0) { >+ minWidth = std::min(tmpMinWidth, minWidth); >+ minHeight = std::min(tmpMinHeight, minHeight); >+ minFramerate = std::min(tmpMinFramerate, minFramerate); >+ >+ maxWidth = std::max(tmpMaxWidth, maxWidth); >+ maxHeight = std::max(tmpMaxHeight, maxHeight); >+ maxFramerate = std::max(tmpMaxFramerate, maxFramerate); >+ } else { >+ minWidth = tmpMinWidth; >+ minHeight = tmpMinHeight; >+ minFramerate = tmpMinFramerate; >+ >+ maxWidth = tmpMaxWidth; >+ maxHeight = tmpMaxHeight; >+ maxFramerate = tmpMaxFramerate; >+ } >+ } >+ >+ capabilities.setDeviceId(id()); >+ capabilities.setWidth(CapabilityValueOrRange(minWidth, maxWidth)); >+ capabilities.setHeight(CapabilityValueOrRange(minHeight, maxHeight)); >+ capabilities.setFrameRate(CapabilityValueOrRange(minFramerate, maxFramerate)); >+ m_capabilities = WTFMove(capabilities); >+ } >+ >+ return m_capabilities.value(); >+} >+ >+const RealtimeMediaSourceSettings& GStreamerVideoCaptureSource::settings() const >+{ >+ if (!m_currentSettings) { >+ RealtimeMediaSourceSettings settings; >+ settings.setDeviceId(id()); >+ >+ RealtimeMediaSourceSupportedConstraints supportedConstraints; >+ supportedConstraints.setSupportsDeviceId(true); >+ supportedConstraints.setSupportsFacingMode(true); >+ supportedConstraints.setSupportsWidth(true); >+ supportedConstraints.setSupportsHeight(true); >+ supportedConstraints.setSupportsAspectRatio(true); >+ supportedConstraints.setSupportsFrameRate(true); >+ settings.setSupportedConstraints(supportedConstraints); >+ >+ m_currentSettings = WTFMove(settings); >+ } >+ >+ m_currentSettings->setWidth(size().width()); >+ m_currentSettings->setHeight(size().height()); >+ m_currentSettings->setFrameRate(frameRate()); >+ m_currentSettings->setAspectRatio(aspectRatio()); >+ m_currentSettings->setFacingMode(facingMode()); >+ return m_currentSettings.value(); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h >new file mode 100644 >index 00000000000..379ba7e0fd6 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h >@@ -0,0 +1,64 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerVideoCapturer.h" >+#include "RealtimeMediaSource.h" >+ >+namespace WebCore { >+ >+class GStreamerVideoCaptureSource : public RealtimeMediaSource { >+public: >+ static CaptureSourceOrError create(const String& deviceID, const MediaConstraints*); >+ WEBCORE_EXPORT static VideoCaptureFactory& factory(); >+ >+ const RealtimeMediaSourceCapabilities& capabilities() const override; >+ const RealtimeMediaSourceSettings& settings() const override; >+ GstElement* pipeline() { return m_capturer->pipeline(); } >+ GStreamerCapturer* capturer() { return m_capturer.get(); } >+ >+ >+protected: >+ GStreamerVideoCaptureSource(const String& deviceID, const String& name, const gchar * source_factory); >+ GStreamerVideoCaptureSource(GStreamerCaptureDevice); >+ virtual ~GStreamerVideoCaptureSource(); >+ void startProducingData() override; >+ void stopProducingData() override; >+ >+ mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >+ mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings; >+ >+private: >+ static GstFlowReturn newSampleCallback(GstElement*, GStreamerVideoCaptureSource*); >+ >+ bool isCaptureSource() const final { return true; } >+ bool applySize(const IntSize&) final; >+ bool applyFrameRate(double) final; >+ bool applyAspectRatio(double) final { return true; } >+ >+ std::unique_ptr<GStreamerVideoCapturer> m_capturer; >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp >new file mode 100644 >index 00000000000..88b6050216f >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp >@@ -0,0 +1,104 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerVideoCapturer.h" >+ >+#include <gst/app/gstappsink.h> >+ >+namespace WebCore { >+ >+GStreamerVideoCapturer::GStreamerVideoCapturer(GStreamerCaptureDevice device) >+ : GStreamerCapturer(device, adoptGRef(gst_caps_new_empty_simple("video/x-raw"))) >+{ >+} >+ >+GStreamerVideoCapturer::GStreamerVideoCapturer(const char* sourceFactory) >+ : GStreamerCapturer(sourceFactory, adoptGRef(gst_caps_new_empty_simple("video/x-raw"))) >+{ >+} >+ >+GstElement* GStreamerVideoCapturer::createConverter() >+{ >+ auto converter = gst_parse_bin_from_description("videoscale ! videoconvert ! videorate", TRUE, nullptr); >+ >+ ASSERT(converter); >+ >+ return converter; >+} >+ >+GstVideoInfo GStreamerVideoCapturer::getBestFormat() >+{ >+ GRefPtr<GstCaps> caps = adoptGRef(gst_caps_fixate(gst_device_get_caps(m_device.get()))); >+ GstVideoInfo info; >+ gst_video_info_from_caps(&info, caps.get()); >+ >+ return info; >+} >+ >+bool GStreamerVideoCapturer::setSize(int width, int height) >+{ >+ if (!width || !height) >+ return false; >+ >+ m_caps = adoptGRef(gst_caps_copy(m_caps.get())); >+ gst_caps_set_simple(m_caps.get(), "width", G_TYPE_INT, width, "height", G_TYPE_INT, height, nullptr); >+ >+ if (!m_capsfilter) >+ return false; >+ >+ g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr); >+ >+ return true; >+} >+ >+bool GStreamerVideoCapturer::setFrameRate(double frameRate) >+{ >+ int numerator, denominator; >+ >+ gst_util_double_to_fraction(frameRate, &numerator, &denominator); >+ >+ if (numerator < -G_MAXINT) { >+ GST_INFO_OBJECT(m_pipeline.get(), "Framerate %f not allowed", frameRate); >+ return false; >+ } >+ >+ if (!numerator) { >+ GST_INFO_OBJECT(m_pipeline.get(), "Do not force variable framerate"); >+ return false; >+ } >+ >+ m_caps = adoptGRef(gst_caps_copy(m_caps.get())); >+ gst_caps_set_simple(m_caps.get(), "framerate", GST_TYPE_FRACTION, numerator, denominator, nullptr); >+ >+ if (!m_capsfilter) >+ return false; >+ >+ g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr); >+ >+ return true; >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h >new file mode 100644 >index 00000000000..57b0b4686d3 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h >@@ -0,0 +1,46 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GStreamerCapturer.h" >+ >+#include <gst/video/video.h> >+ >+namespace WebCore { >+ >+class GStreamerVideoCapturer : public GStreamerCapturer { >+public: >+ GStreamerVideoCapturer(GStreamerCaptureDevice); >+ GStreamerVideoCapturer(const char* source_factory); >+ >+ GstElement* createConverter() final; >+ const char* name() final { return "Video"; } >+ >+ bool setSize(int width, int height); >+ bool setFrameRate(double); >+ GstVideoInfo getBestFormat(); >+}; >+ >+} // namespace WebCore >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp >new file mode 100644 >index 00000000000..995f05720eb >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp >@@ -0,0 +1,106 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "MockGStreamerAudioCaptureSource.h" >+ >+#include "MockRealtimeAudioSource.h" >+ >+namespace WebCore { >+ >+class WrappedMockRealtimeAudioSource : public MockRealtimeAudioSource { >+public: >+ WrappedMockRealtimeAudioSource(const String& deviceID, const String& name) >+ : MockRealtimeAudioSource(deviceID, name) >+ { >+ } >+}; >+ >+CaptureSourceOrError MockRealtimeAudioSource::create(const String& deviceID, >+ const String& name, const MediaConstraints* constraints) >+{ >+ auto source = adoptRef(*new MockGStreamerAudioCaptureSource(deviceID, name)); >+ >+ if (constraints && source->applyConstraints(*constraints)) >+ return { }; >+ >+ return CaptureSourceOrError(WTFMove(source)); >+} >+ >+std::optional<std::pair<String, String>> MockGStreamerAudioCaptureSource::applyConstraints(const MediaConstraints& constraints) >+{ >+ m_wrappedSource->applyConstraints(constraints); >+ return GStreamerAudioCaptureSource::applyConstraints(constraints); >+} >+ >+void MockGStreamerAudioCaptureSource::applyConstraints(const MediaConstraints& constraints, SuccessHandler&& successHandler, FailureHandler&& failureHandler) >+{ >+ m_wrappedSource->applyConstraints(constraints, WTFMove(successHandler), WTFMove(failureHandler)); >+} >+ >+MockGStreamerAudioCaptureSource::MockGStreamerAudioCaptureSource(const String& deviceID, const String& name) >+ : GStreamerAudioCaptureSource(deviceID, name) >+ , m_wrappedSource(std::make_unique<WrappedMockRealtimeAudioSource>(deviceID, name)) >+{ >+ m_wrappedSource->addObserver(*this); >+} >+ >+MockGStreamerAudioCaptureSource::~MockGStreamerAudioCaptureSource() >+{ >+ m_wrappedSource->removeObserver(*this); >+} >+ >+void MockGStreamerAudioCaptureSource::stopProducingData() >+{ >+ m_wrappedSource->stop(); >+ >+ GStreamerAudioCaptureSource::stopProducingData(); >+} >+ >+void MockGStreamerAudioCaptureSource::startProducingData() >+{ >+ GStreamerAudioCaptureSource::startProducingData(); >+ m_wrappedSource->start(); >+} >+ >+const RealtimeMediaSourceSettings& MockGStreamerAudioCaptureSource::settings() const >+{ >+ return m_wrappedSource->settings(); >+} >+ >+const RealtimeMediaSourceCapabilities& MockGStreamerAudioCaptureSource::capabilities() const >+{ >+ m_capabilities = m_wrappedSource->capabilities(); >+ m_currentSettings = m_wrappedSource->settings(); >+ return m_wrappedSource->capabilities(); >+} >+ >+void MockGStreamerAudioCaptureSource::captureFailed() >+{ >+ stop(); >+ RealtimeMediaSource::captureFailed(); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h >new file mode 100644 >index 00000000000..bc11f179b98 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h >@@ -0,0 +1,51 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GStreamerAudioCaptureSource.h" >+ >+namespace WebCore { >+ >+class MockGStreamerAudioCaptureSource final : public GStreamerAudioCaptureSource, RealtimeMediaSource::Observer { >+public: >+ MockGStreamerAudioCaptureSource(const String& deviceID, const String& name); >+ ~MockGStreamerAudioCaptureSource(); >+ std::optional<std::pair<String, String>> applyConstraints(const MediaConstraints&); >+ void applyConstraints(const MediaConstraints&, SuccessHandler&&, FailureHandler&&) final; >+ >+private: >+ void stopProducingData() final; >+ void startProducingData() final; >+ const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() const final; >+ >+ void captureFailed(); >+ std::unique_ptr<RealtimeMediaSource> m_wrappedSource; >+ >+ void videoSampleAvailable(MediaSample&) override { }; >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp >new file mode 100644 >index 00000000000..bdfabedd46f >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp >@@ -0,0 +1,146 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "MockGStreamerVideoCaptureSource.h" >+ >+#include "MediaSampleGStreamer.h" >+#include "MockRealtimeVideoSource.h" >+ >+#include <gst/app/gstappsrc.h> >+ >+namespace WebCore { >+ >+class WrappedMockRealtimeVideoSource : public MockRealtimeVideoSource { >+public: >+ WrappedMockRealtimeVideoSource(const String& deviceID, const String& name) >+ : MockRealtimeVideoSource(deviceID, name) >+ { >+ } >+ >+ void updateSampleBuffer() >+ { >+ int fpsNumerator, fpsDenominator; >+ auto imageBuffer = this->imageBuffer(); >+ >+ if (!imageBuffer) >+ return; >+ >+ gst_util_double_to_fraction(frameRate(), &fpsNumerator, &fpsDenominator); >+ auto data = imageBuffer->toBGRAData(); >+ auto size = data.size(); >+ auto image_size = imageBuffer->internalSize(); >+ auto gstsample = gst_sample_new(gst_buffer_new_wrapped(static_cast<guint8*>(data.releaseBuffer().get()), size), >+ adoptGRef(gst_caps_new_simple("video/x-raw", >+ "format", G_TYPE_STRING, "BGRA", >+ "width", G_TYPE_INT, image_size.width(), >+ "height", G_TYPE_INT, image_size.height(), >+ "framerate", GST_TYPE_FRACTION, fpsNumerator, fpsDenominator, >+ nullptr)).get(), >+ nullptr, nullptr); >+ >+ auto sample = MediaSampleGStreamer::create(WTFMove(gstsample), >+ WebCore::FloatSize(), String()); >+ videoSampleAvailable(sample); >+ } >+}; >+ >+CaptureSourceOrError MockRealtimeVideoSource::create(const String& deviceID, >+ const String& name, const MediaConstraints* constraints) >+{ >+ auto source = adoptRef(*new MockGStreamerVideoCaptureSource(deviceID, name)); >+ >+ if (constraints && source->applyConstraints(*constraints)) >+ return { }; >+ >+ return CaptureSourceOrError(WTFMove(source)); >+} >+ >+void MockGStreamerVideoCaptureSource::startProducingData() >+{ >+ GStreamerVideoCaptureSource::startProducingData(); >+ m_wrappedSource->start(); >+} >+ >+void MockGStreamerVideoCaptureSource::stopProducingData() >+{ >+ m_wrappedSource->stop(); >+ >+ GStreamerVideoCaptureSource::stopProducingData(); >+} >+ >+void MockGStreamerVideoCaptureSource::videoSampleAvailable(MediaSample& sample) >+{ >+ auto src = capturer()->source(); >+ >+ if (src) { >+ auto gstsample = static_cast<MediaSampleGStreamer*>(&sample)->platformSample().sample.gstSample; >+ gst_app_src_push_sample(GST_APP_SRC(src), gstsample); >+ } >+} >+ >+MockGStreamerVideoCaptureSource::MockGStreamerVideoCaptureSource(const String& deviceID, const String& name) >+ : GStreamerVideoCaptureSource(deviceID, name, "appsrc") >+ , m_wrappedSource(std::make_unique<WrappedMockRealtimeVideoSource>(deviceID, name)) >+{ >+ m_wrappedSource->addObserver(*this); >+} >+ >+MockGStreamerVideoCaptureSource::~MockGStreamerVideoCaptureSource() >+{ >+ m_wrappedSource->removeObserver(*this); >+} >+ >+std::optional<std::pair<String, String>> MockGStreamerVideoCaptureSource::applyConstraints(const MediaConstraints& constraints) >+{ >+ m_wrappedSource->applyConstraints(constraints); >+ return GStreamerVideoCaptureSource::applyConstraints(constraints); >+} >+ >+void MockGStreamerVideoCaptureSource::applyConstraints(const MediaConstraints& constraints, SuccessHandler&& successHandler, FailureHandler&& failureHandler) >+{ >+ m_wrappedSource->applyConstraints(constraints, WTFMove(successHandler), WTFMove(failureHandler)); >+} >+ >+const RealtimeMediaSourceSettings& MockGStreamerVideoCaptureSource::settings() const >+{ >+ return m_wrappedSource->settings(); >+} >+ >+const RealtimeMediaSourceCapabilities& MockGStreamerVideoCaptureSource::capabilities() const >+{ >+ m_capabilities = m_wrappedSource->capabilities(); >+ m_currentSettings = m_wrappedSource->settings(); >+ return m_capabilities.value(); >+} >+ >+void MockGStreamerVideoCaptureSource::captureFailed() >+{ >+ stop(); >+ >+ RealtimeMediaSource::captureFailed(); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h >new file mode 100644 >index 00000000000..dc802768d7f >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h >@@ -0,0 +1,56 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GStreamerVideoCaptureSource.h" >+#include "MockRealtimeMediaSource.h" >+ >+namespace WebCore { >+ >+class WrappedMockRealtimeVideoSource; >+// We are subclassing GStreamerVideoCaptureSource and not MockRealtimeMediaSource >+// because the MediaPlayer expects GStreamerVideoCaptureSource to be able to properly >+// build the GStreamer pipeline. Still we make it so that it behaves as closely as possible >+// to the MockRealtimeMediaSource class by wrapping our own subclass of it. >+class MockGStreamerVideoCaptureSource final : public GStreamerVideoCaptureSource, RealtimeMediaSource::Observer { >+public: >+ MockGStreamerVideoCaptureSource(const String& deviceID, const String& name); >+ ~MockGStreamerVideoCaptureSource(); >+ std::optional<std::pair<String, String>> applyConstraints(const MediaConstraints&); >+ void applyConstraints(const MediaConstraints&, SuccessHandler&&, FailureHandler&&) final; >+ >+private: >+ void stopProducingData() final; >+ void startProducingData() final; >+ const RealtimeMediaSourceSettings& settings() const final; >+ std::unique_ptr<RealtimeMediaSource> m_wrappedSource; >+ const RealtimeMediaSourceCapabilities& capabilities() const final; >+ void captureFailed() override; >+ >+ void videoSampleAvailable(MediaSample&) override; >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp b/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp >new file mode 100644 >index 00000000000..bdce1d6e60f >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp >@@ -0,0 +1,91 @@ >+/* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >+#include "RealtimeMediaSourceCenterLibWebRTC.h" >+ >+#include "GStreamerAudioCaptureSource.h" >+#include "GStreamerCaptureDevice.h" >+#include "GStreamerCaptureDeviceManager.h" >+#include "GStreamerVideoCaptureSource.h" >+#include <wtf/MainThread.h> >+ >+namespace WebCore { >+ >+RealtimeMediaSource::AudioCaptureFactory& RealtimeMediaSourceCenterLibWebRTC::audioCaptureSourceFactory() >+{ >+ return RealtimeMediaSourceCenterLibWebRTC::singleton().audioFactory(); >+} >+ >+RealtimeMediaSourceCenterLibWebRTC& RealtimeMediaSourceCenterLibWebRTC::singleton() >+{ >+ ASSERT(isMainThread()); >+ static NeverDestroyed<RealtimeMediaSourceCenterLibWebRTC> center; >+ return center; >+} >+ >+RealtimeMediaSourceCenter& RealtimeMediaSourceCenter::platformCenter() >+{ >+ return RealtimeMediaSourceCenterLibWebRTC::singleton(); >+} >+ >+RealtimeMediaSourceCenterLibWebRTC::RealtimeMediaSourceCenterLibWebRTC() >+ : m_libWebRTCProvider(LibWebRTCProvider::create()) >+{ >+} >+ >+RealtimeMediaSourceCenterLibWebRTC::~RealtimeMediaSourceCenterLibWebRTC() >+{ >+} >+ >+RealtimeMediaSource::AudioCaptureFactory& RealtimeMediaSourceCenterLibWebRTC::audioFactory() >+{ >+ if (m_audioFactoryOverride) >+ return *m_audioFactoryOverride; >+ >+ return GStreamerAudioCaptureSource::factory(); >+} >+ >+RealtimeMediaSource::VideoCaptureFactory& RealtimeMediaSourceCenterLibWebRTC::videoFactory() >+{ >+ return GStreamerVideoCaptureSource::factory(); >+} >+ >+CaptureDeviceManager& RealtimeMediaSourceCenterLibWebRTC::audioCaptureDeviceManager() >+{ >+ return GStreamerAudioCaptureDeviceManager::singleton(); >+} >+ >+CaptureDeviceManager& RealtimeMediaSourceCenterLibWebRTC::videoCaptureDeviceManager() >+{ >+ return GStreamerVideoCaptureDeviceManager::singleton(); >+} >+ >+CaptureDeviceManager& RealtimeMediaSourceCenterLibWebRTC::displayCaptureDeviceManager() >+{ >+ return GStreamerDisplayCaptureDeviceManager::singleton(); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.h b/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.h >new file mode 100644 >index 00000000000..921284e1780 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.h >@@ -0,0 +1,68 @@ >+ /* >+ * Copyright (C) 2018 Metrological Group B.V. >+ * Author: Thibault Saunier <tsaunier@igalia.com> >+ * Author: Alejandro G. Castro <alex@igalia.com> >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >+ >+#include "CaptureDeviceManager.h" >+#include "LibWebRTCProvider.h" >+#include "RealtimeMediaSource.h" >+#include "RealtimeMediaSourceCenter.h" >+ >+#include <webrtc/api/peerconnectioninterface.h> >+#include <wtf/RefPtr.h> >+#include <wtf/text/WTFString.h> >+ >+namespace WebCore { >+ >+class RealtimeMediaSourceCenterLibWebRTC final : public RealtimeMediaSourceCenter { >+public: >+ WEBCORE_EXPORT static RealtimeMediaSourceCenterLibWebRTC& singleton(); >+ >+ webrtc::PeerConnectionFactoryInterface* factory() { return m_libWebRTCProvider->factory(); } >+ >+ static RealtimeMediaSource::VideoCaptureFactory& videoCaptureSourceFactory(); >+ static RealtimeMediaSource::AudioCaptureFactory& audioCaptureSourceFactory(); >+ >+private: >+ friend class NeverDestroyed<RealtimeMediaSourceCenterLibWebRTC>; >+ RealtimeMediaSourceCenterLibWebRTC(); >+ ~RealtimeMediaSourceCenterLibWebRTC(); >+ >+ void setAudioFactory(RealtimeMediaSource::AudioCaptureFactory& factory) final { m_audioFactoryOverride = &factory; } >+ void unsetAudioFactory(RealtimeMediaSource::AudioCaptureFactory&) final { m_audioFactoryOverride = nullptr; } >+ >+ RealtimeMediaSource::AudioCaptureFactory& audioFactory() final; >+ RealtimeMediaSource::VideoCaptureFactory& videoFactory() final; >+ >+ CaptureDeviceManager& audioCaptureDeviceManager() final; >+ CaptureDeviceManager& videoCaptureDeviceManager() final; >+ CaptureDeviceManager& displayCaptureDeviceManager() final; >+ >+ RealtimeMediaSource::AudioCaptureFactory* m_audioFactoryOverride { nullptr }; >+ UniqueRef<LibWebRTCProvider> m_libWebRTCProvider; >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >+ >diff --git a/Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp b/Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp >index c50abcb7a32..9c6024e53a8 100644 >--- a/Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp >+++ b/Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp >@@ -55,7 +55,7 @@ public: > } > }; > >-#if !PLATFORM(MAC) && !PLATFORM(IOS) >+#if !PLATFORM(MAC) && !PLATFORM(IOS) && !(USE(GSTREAMER) && USE(LIBWEBRTC)) > CaptureSourceOrError MockRealtimeAudioSource::create(const String& deviceID, const String& name, const MediaConstraints* constraints) > { > auto source = adoptRef(*new MockRealtimeAudioSource(deviceID, name)); >diff --git a/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp b/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp >index 9e860d6cf5a..ededa83a0e3 100644 >--- a/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp >+++ b/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp >@@ -82,7 +82,7 @@ private: > #endif > }; > >-#if !PLATFORM(MAC) && !PLATFORM(IOS) >+#if !PLATFORM(MAC) && !PLATFORM(IOS) && !(USE(GSTREAMER) && USE(LIBWEBRTC)) > CaptureSourceOrError MockRealtimeVideoSource::create(const String& deviceID, const String& name, const MediaConstraints* constraints) > { > auto source = adoptRef(*new MockRealtimeVideoSource(deviceID, name)); >diff --git a/Tools/ChangeLog b/Tools/ChangeLog >index e2d4218a522..41d3b29f965 100644 >--- a/Tools/ChangeLog >+++ b/Tools/ChangeLog >@@ -1,3 +1,15 @@ >+2018-05-18 Thibault Saunier <tsaunier@igalia.com> >+ >+ [GTK][WPE] Start implementing MediaStream API >+ https://bugs.webkit.org/show_bug.cgi?id=185787 >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ * Scripts/webkitpy/style/checker.py: Apply special formatting rules for new GObject subclasses. >+ * gstreamer/jhbuild.modules: Added a patch for the gst-plugins-base. >+ * gstreamer/patches/gst-plugins-base-0001-parsebin-Post-STREAM_COLLECTION-on-EVENT_STREAM_COLL.patch: >+ Added this fix to gst-plugings-base to fix the decodebin3. Merged as 89d0e9cc92a86aa0227ee87406737b6d31670aea >+ > 2018-06-07 Fujii Hironori <Hironori.Fujii@sony.com> > > [Win][MiniBrowser] Support multiple windows properly >diff --git a/Tools/Scripts/webkitpy/style/checker.py b/Tools/Scripts/webkitpy/style/checker.py >index 333dce26567..a5908e6016a 100644 >--- a/Tools/Scripts/webkitpy/style/checker.py >+++ b/Tools/Scripts/webkitpy/style/checker.py >@@ -207,6 +207,8 @@ _PATH_RULES_SPECIFIER = [ > os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'VideoSinkGStreamer.cpp'), > os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'WebKitWebSourceGStreamer.cpp'), > os.path.join('Source', 'WebCore', 'platform', 'audio', 'gstreamer', 'WebKitWebAudioSourceGStreamer.cpp'), >+ os.path.join('Source', 'WebCore', 'platform', 'mediastream', 'gstreamer', 'GStreamerMediaStreamSource.h'), >+ os.path.join('Source', 'WebCore', 'platform', 'mediastream', 'gstreamer', 'GStreamerMediaStreamSource.cpp'), > os.path.join('Source', 'WebCore', 'platform', 'network', 'soup', 'ProxyResolverSoup.cpp'), > os.path.join('Source', 'WebCore', 'platform', 'network', 'soup', 'ProxyResolverSoup.h')], > ["-readability/naming", >diff --git a/Tools/gstreamer/jhbuild.modules b/Tools/gstreamer/jhbuild.modules >index 5aeb756ec0a..c90a0cb468c 100644 >--- a/Tools/gstreamer/jhbuild.modules >+++ b/Tools/gstreamer/jhbuild.modules >@@ -57,7 +57,9 @@ > <dependencies> > <dep package="gstreamer"/> > </dependencies> >- <branch hash="sha:1026c7c3082d825d9b5d034c1a6dd8a4ebab60eb3738b0a0afde4ad2dc0b0db5" module="gst-plugins-base/gst-plugins-base-${version}.tar.xz" repo="gstreamer" version="1.14.1" /> >+ <branch hash="sha:1026c7c3082d825d9b5d034c1a6dd8a4ebab60eb3738b0a0afde4ad2dc0b0db5" module="gst-plugins-base/gst-plugins-base-${version}.tar.xz" repo="gstreamer" version="1.14.1" > >+ <patch file="gst-plugins-base-0001-parsebin-Post-STREAM_COLLECTION-on-EVENT_STREAM_COLL.patch" strip="1"/> <!-- Merged as 89d0e9cc92a86aa0227ee87406737b6d31670aea --> >+ </branch> > </autotools> > > <autotools id="gst-plugins-good" autogen-sh="configure" autogenargs="--disable-examples --disable-gtk-doc --disable-introspection"> >diff --git a/Tools/gstreamer/patches/gst-plugins-base-0001-parsebin-Post-STREAM_COLLECTION-on-EVENT_STREAM_COLL.patch b/Tools/gstreamer/patches/gst-plugins-base-0001-parsebin-Post-STREAM_COLLECTION-on-EVENT_STREAM_COLL.patch >new file mode 100644 >index 00000000000..680bc2db464 >--- /dev/null >+++ b/Tools/gstreamer/patches/gst-plugins-base-0001-parsebin-Post-STREAM_COLLECTION-on-EVENT_STREAM_COLL.patch >@@ -0,0 +1,33 @@ >+From 89d0e9cc92a86aa0227ee87406737b6d31670aea Mon Sep 17 00:00:00 2001 >+From: Thibault Saunier <tsaunier@igalia.com> >+Date: Wed, 18 Apr 2018 16:06:07 -0300 >+Subject: [PATCH] parsebin: Post STREAM_COLLECTION on EVENT_STREAM_COLLECTION >+ >+Otherwise decodebin won't get notified about STREAM_COLLECTION comming >+from the sources and thus will never get informored about it. Without >+being informed about the stream collection decodebin won't be able to >+select any streams. It ends up not creating any output for the streams >+defined from outside parserbin. >+ >+https://bugzilla.gnome.org/show_bug.cgi?id=795364 >+--- >+ gst/playback/gstparsebin.c | 3 +++ >+ 1 file changed, 3 insertions(+) >+ >+diff --git a/gst/playback/gstparsebin.c b/gst/playback/gstparsebin.c >+index 4c5bfc19d..f9662eb49 100644 >+--- a/gst/playback/gstparsebin.c >++++ b/gst/playback/gstparsebin.c >+@@ -4046,6 +4046,9 @@ gst_parse_pad_event (GstPad * pad, GstPadProbeInfo * info, gpointer user_data) >+ GstStreamCollection *collection = NULL; >+ gst_event_parse_stream_collection (event, &collection); >+ gst_parse_pad_update_stream_collection (parsepad, collection); >++ gst_element_post_message (GST_ELEMENT (parsepad->parsebin), >++ gst_message_new_stream_collection (GST_OBJECT (parsepad->parsebin), >++ collection)); >+ break; >+ } >+ case GST_EVENT_EOS:{ >+-- >+2.17.0 >+ >-- >2.17.1
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Formatted Diff
|
Diff
Attachments on
bug 185787
:
341758
|
342085
|
342160
|
342168
|
342187