WebKit Bugzilla
Attachment 340841 Details for
Bug 185761
: [GTK][WPE] Add mediaDevices.enumerateDevices support
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
Remember
[x]
|
Forgot Password
Login:
[x]
[patch]
Patch
bug-185761-20180521121034.patch (text/plain), 116.83 KB, created by
Alejandro G. Castro
on 2018-05-21 03:10:36 PDT
(
hide
)
Description:
Patch
Filename:
MIME Type:
Creator:
Alejandro G. Castro
Created:
2018-05-21 03:10:36 PDT
Size:
116.83 KB
patch
obsolete
>Subversion Revision: 231947 >diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog >index 385dd61ab755a5d049b333bcb2ce9f0b359a9764..4ccb4f43c75636c95f9ebde9ef6db30a7e9b462f 100644 >--- a/Source/WebCore/ChangeLog >+++ b/Source/WebCore/ChangeLog >@@ -1,3 +1,85 @@ >+2018-05-18 Alejandro G. Castro <alex@igalia.com> and Thibault Saunier <tsaunier@igalia.com> >+ >+ [GTK][WPE] Add mediaDevices.enumerateDevices support >+ https://bugs.webkit.org/show_bug.cgi?id=185761 >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ We are adding all the required classes to make the >+ enumerateDevices work, that means our own >+ RealtimeMediaSourceCenterLibWebRTC for the platform, the >+ GStreamerCaptureDeviceManager, the audio/video capturers and their >+ respective audio/video sources. We are using the GstDeviceMonitor >+ monitor to request the information from the system. >+ >+ Unskipped the relevant tests for the enumerateDevices. >+ >+ * Modules/mediastream/MediaDevicesRequest.cpp: >+ (WebCore::MediaDevicesRequest::filterDeviceList): Add support for >+ GTK and WPE platforms when filtering devices if there is no >+ persistent access to the origin. >+ * platform/GStreamer.cmake: Added the new files to the compilation. >+ * platform/audio/AudioStreamDescription.h: Added new GStreamer type. >+ * platform/audio/PlatformAudioData.h: Added new GStreamer type for >+ the GStreamerAudioData class. >+ * platform/graphics/gstreamer/GStreamerCommon.cpp: >+ (WebCore::simpleBusMessageCallback): This function and the next >+ one help us connect a bus message callback to a pipeline for >+ debugging. >+ (WebCore::connectSimpleBusMessageCallback): Ditto. >+ * platform/graphics/gstreamer/GStreamerCommon.h: Ditto. >+ * platform/mediastream/RealtimeMediaSourceCenter.cpp: >+ (WebCore::RealtimeMediaSourceCenter::singleton): Remove the code >+ used for compilation for the platform when we do not have a >+ RealtimeMediaSourceCenterLibWebRTC. Now we return the proper class >+ for the platform. >+ * platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp: >+ Added class representing the RealtimeMediaSource for the Audio >+ with GStreamer. >+ * platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h: >+ Ditto. >+ * platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp: Added >+ this class that represents the GStreamer pipeline that captures >+ audio from the system devices, it inherits from GStreamerCapturer. >+ * platform/mediastream/gstreamer/GStreamerAudioCapturer.h: Dito. >+ * platform/mediastream/gstreamer/GStreamerAudioData.h: Added this >+ class implementing PlatformAudioData for the GStreamer platform, >+ used to pass the samples information. >+ * platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h: >+ Added this class implementing AudioStreamDescription to export the >+ information about the audio stream to libwebrtc. >+ * platform/mediastream/gstreamer/GStreamerCaptureDevice.h: Added >+ this base class for the audio and video capturing devices, it >+ implements general WebKit CaptureDevice class. >+ * platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp: >+ Added this class that implements the system monitor to get the >+ list of available devices in the system. It uses GstDeviceMonitor >+ to handle the operation. It uses two singleton device managers one >+ for audio and another one for video, the design required by the >+ RealtimeMediaSourceCenter. >+ * platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h: Ditto. >+ * platform/mediastream/gstreamer/GStreamerCapturer.cpp: Added this >+ base class representing how GStreamer captures the media from the >+ input devices in the system. Two classes inherit from this one to >+ capture audio and video. It setups the GStreamer pipeline and adds >+ functions to control it. >+ * platform/mediastream/gstreamer/GStreamerCapturer.h: Ditto. >+ * platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp: >+ Added this RealtimeMediaSource representing the source of the >+ video data for the GStreamer platform. It handles the settings and >+ capabilities of the source and creates the capturer used to >+ control the operation of the stream. >+ * platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h: Ditto. >+ * platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp: Added >+ this class that inherits from the GStreamerCapturer and controls >+ the GStreamer pipelines of the video streams of the system. >+ * platform/mediastream/gstreamer/GStreamerVideoCapturer.h: Ditto. >+ * platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp: >+ Added this class that implements the key RealtimeMediaSourceCenter >+ functions to configure the base class for the platform, using the >+ other GStreamer classes. >+ * platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.h: Ditto >+ > 2018-05-17 David Kilzer <ddkilzer@apple.com> > > Lazily create WebCore::Timer for WebCore::Image >diff --git a/Source/WebKit/ChangeLog b/Source/WebKit/ChangeLog >index 15471e939d0e099108ee92aeabb105e22b74ce54..3f61fb65bf7c56cfc200fc1697f4ff66544cd42e 100644 >--- a/Source/WebKit/ChangeLog >+++ b/Source/WebKit/ChangeLog >@@ -1,3 +1,73 @@ >+2018-05-18 Alejandro G. Castro <alex@igalia.com> and Thibault Saunier <tsaunier@igalia.com> >+ >+ [GTK][WPE] Add mediaDevices.enumerateDevices support >+ https://bugs.webkit.org/show_bug.cgi?id=185761 >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ In order to implement the checkUserMediaPermissionForOrigin >+ function that is called when the webpage wants to enumerate the >+ list of devices in the system we added some new APIs for the GTK >+ and WPE ports. We have extended the >+ WebKitUserMediaPermissionRequest class to support a new kind of >+ request, the checkRequest. With this addition WebKit can >+ create a WebKitUserMediaPermissionRequest with a permission check >+ proxy, if that is the case the WebKitWebView widget launches >+ WebKitWebView::permission-check signal and the browser can >+ connnect to it to resolve it with the new >+ webkit_permission_request_resolve_check API, adding the hash salt >+ and the decision about allowing access to the devices. Main point >+ of this API is to make sure that the user can control the webpages >+ and that they don't get smart trying to fingerprint the system. There is a >+ default implementation in the widget that basically always >+ denies access and generates a random hashsalt. >+ >+ * UIProcess/API/glib/WebKitPermissionRequest.cpp: >+ (webkit_permission_request_resolve_check): Added the API to >+ resolve the permission-check requested by the engine. >+ * UIProcess/API/glib/WebKitUIClient.cpp: >+ (checkUserMediaPermissionForOrigin): Added the implementation for >+ the platform used when the webpage calls the enumerateDevices API. >+ * UIProcess/API/glib/WebKitUserMediaPermissionRequest.cpp: >+ (webkitUserMediaPermissionResolveCheck): Added the function that >+ resolves the permission check passing the hash salt and the >+ boolean specifying if the operation is allowed to the completion >+ handler in the request. >+ (webkit_permission_request_interface_init): Added the new class >+ function. >+ (webkitUserMediaPermissionRequestDispose): Make sure we dispose >+ correctly now that we have a new checkRequest, in that situation >+ request is null. >+ (webkit_user_media_permission_is_for_audio_device): Control the >+ browser does not call this API when we are doing a permission check. >+ (webkit_user_media_permission_is_for_video_device): Ditto. >+ (webkitUserMediaPermissionCheckCreate): Added new builder for the >+ WebKitPermissionRequest class using a UserMediaPermissionCheckProxy. >+ * UIProcess/API/glib/WebKitUserMediaPermissionRequestPrivate.h: >+ Added the new API for the engine to create a WebKitPermissionRequest. >+ (webkitUserMediaPermissionCheckCreate): >+ * UIProcess/API/glib/WebKitWebView.cpp: >+ (webkitWebViewPermissionCheck): Added default implementation for >+ the signal callback. >+ (webkit_web_view_class_init): Set the default implementation for >+ the permission-check signal. >+ (webkitWebViewMakePermissionCheck): Added new API to launch ask >+ for permission check, this is private and used by the engine. >+ * UIProcess/API/glib/WebKitWebViewPrivate.h: Define the new >+ private API webkitWebViewMakePermissionCheck. >+ * UIProcess/API/gtk/WebKitPermissionRequest.h: Added the new API >+ to resolve the permission check in the engine. Two minor style >+ modifications. >+ * UIProcess/API/gtk/WebKitWebView.h: Added the new class function >+ for the permission-check signal. >+ * UIProcess/API/gtk/docs/webkit2gtk-4.0-sections.txt: Added the >+ new symbol to the documentation: >+ webkit_permission_request_resolve_check. >+ * UIProcess/API/wpe/WebKitPermissionRequest.h: Added the new API >+ to resolve the permission check in the engine. >+ * UIProcess/API/wpe/WebKitWebView.h: Added the new class function >+ for the permission-check signal. >+ > 2018-05-18 Fujii Hironori <Hironori.Fujii@sony.com> > > [Curl] Remove unused SystemProxyWin.cpp >diff --git a/Source/WebCore/Modules/mediastream/MediaDevicesRequest.cpp b/Source/WebCore/Modules/mediastream/MediaDevicesRequest.cpp >index 12400dcf2deed8fbc25e1bfa42398822326f4183..4bb0626c205f44566e69312285b21da25b8d99dd 100644 >--- a/Source/WebCore/Modules/mediastream/MediaDevicesRequest.cpp >+++ b/Source/WebCore/Modules/mediastream/MediaDevicesRequest.cpp >@@ -81,9 +81,7 @@ void MediaDevicesRequest::contextDestroyed() > > void MediaDevicesRequest::filterDeviceList(Vector<Ref<MediaDeviceInfo>>& devices) > { >-#if !PLATFORM(COCOA) >- UNUSED_PARAM(devices); >-#else >+#if PLATFORM(COCOA) || PLATFORM(GTK) || PLATFORM(WPE) > > #if PLATFORM(IOS) > static const int defaultCameraCount = 2; >@@ -91,6 +89,10 @@ void MediaDevicesRequest::filterDeviceList(Vector<Ref<MediaDeviceInfo>>& devices > #if PLATFORM(MAC) > static const int defaultCameraCount = 1; > #endif >+#if PLATFORM(GTK) || PLATFORM(WPE) >+ static const int defaultCameraCount = 1; >+#endif >+ > static const int defaultMicrophoneCount = 1; > > int cameraCount = 0; >@@ -103,7 +105,8 @@ void MediaDevicesRequest::filterDeviceList(Vector<Ref<MediaDeviceInfo>>& devices > > return false; > }); >- >+#else >+ UNUSED_PARAM(devices); > #endif > } > >diff --git a/Source/WebCore/platform/GStreamer.cmake b/Source/WebCore/platform/GStreamer.cmake >index 3d5a6e0025a52bb7b7a6f71d34719f2bdc2c8e5d..d087be7542a9974d5eb84917c1db0b41ee59f0fd 100644 >--- a/Source/WebCore/platform/GStreamer.cmake >+++ b/Source/WebCore/platform/GStreamer.cmake >@@ -33,6 +33,19 @@ if (ENABLE_VIDEO OR ENABLE_WEB_AUDIO) > platform/graphics/gstreamer/mse/PlaybackPipeline.cpp > platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp > platform/graphics/gstreamer/mse/WebKitMediaSourceGStreamer.cpp >+ >+ platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp >+ platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp >+ platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp >+ platform/mediastream/gstreamer/GStreamerCapturer.cpp >+ platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp >+ platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp >+ platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp >+ >+ platform/mock/MockRealtimeAudioSource.cpp >+ platform/mock/MockRealtimeMediaSource.cpp >+ platform/mock/MockRealtimeMediaSourceCenter.cpp >+ platform/mock/MockRealtimeVideoSource.cpp > ) > > list(APPEND WebCore_SYSTEM_INCLUDE_DIRECTORIES >diff --git a/Source/WebCore/platform/audio/AudioStreamDescription.h b/Source/WebCore/platform/audio/AudioStreamDescription.h >index e440dc90a3e84dfd3b9ed5876046dba71c97fe3b..2ba585c9c0567111d75709d897cd7cb980363240 100644 >--- a/Source/WebCore/platform/audio/AudioStreamDescription.h >+++ b/Source/WebCore/platform/audio/AudioStreamDescription.h >@@ -35,6 +35,7 @@ struct PlatformDescription { > enum { > None, > CAAudioStreamBasicType, >+ GStreamerAudioStreamDescription, > } type; > Variant<std::nullptr_t, const AudioStreamBasicDescription*> description; > }; >diff --git a/Source/WebCore/platform/audio/PlatformAudioData.h b/Source/WebCore/platform/audio/PlatformAudioData.h >index e4544f63254a87876d99d8c3061203aa81679d98..cda72e8ab3c4cdc0a868faac4f05ce61c44402d0 100644 >--- a/Source/WebCore/platform/audio/PlatformAudioData.h >+++ b/Source/WebCore/platform/audio/PlatformAudioData.h >@@ -34,6 +34,7 @@ public: > enum class Kind { > None, > WebAudioBufferList, >+ GStreamerAudioData, > }; > > virtual Kind kind() const { return Kind::None; } >diff --git a/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp b/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp >index e514fe3428de52898d4a07f6bc68a0c1d96ede36..435057a9efbaf2ae26888e6bbbe69685e9fc846c 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp >+++ b/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp >@@ -30,6 +30,7 @@ > #include <mutex> > #include <wtf/glib/GLibUtilities.h> > #include <wtf/glib/GUniquePtr.h> >+#include <wtf/glib/RunLoopSourcePriority.h> > > #if ENABLE(VIDEO_TRACK) && USE(GSTREAMER_MPEGTS) > #define GST_USE_UNSTABLE_API >@@ -320,6 +321,46 @@ bool gstRegistryHasElementForMediaType(GList* elementFactories, const char* caps > return result; > } > >+static void simpleBusMessageCallback(GstBus*, GstMessage* message, GstBin* pipeline) >+{ >+ switch (GST_MESSAGE_TYPE(message)) { >+ case GST_MESSAGE_ERROR: >+ GST_ERROR_OBJECT(pipeline, "Got message: %" GST_PTR_FORMAT, message); >+ { >+ WTF::String dotFileName = String::format("%s_error", GST_OBJECT_NAME(pipeline)); >+ GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(pipeline, GST_DEBUG_GRAPH_SHOW_ALL, dotFileName.utf8().data()); >+ } >+ break; >+ case GST_MESSAGE_STATE_CHANGED: >+ if (GST_MESSAGE_SRC(message) == GST_OBJECT(pipeline)) { >+ GstState oldState, newState, pending; >+ gst_message_parse_state_changed(message, &oldState, &newState, &pending); >+ >+ GST_INFO_OBJECT(pipeline, "State changed (old: %s, new: %s, pending: %s)", >+ gst_element_state_get_name(oldState), >+ gst_element_state_get_name(newState), >+ gst_element_state_get_name(pending)); >+ >+ WTF::String dotFileName = String::format("%s_%s_%s", >+ GST_OBJECT_NAME(pipeline), >+ gst_element_state_get_name(oldState), >+ gst_element_state_get_name(newState)); >+ >+ GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(GST_BIN(pipeline), GST_DEBUG_GRAPH_SHOW_ALL, dotFileName.utf8().data()); >+ } >+ break; >+ default: >+ break; >+ } >+} >+ >+void connectSimpleBusMessageCallback(GstElement *pipeline) >+{ >+ GRefPtr<GstBus> bus = adoptGRef(gst_pipeline_get_bus(GST_PIPELINE(pipeline))); >+ gst_bus_add_signal_watch_full(bus.get(), RunLoopSourcePriority::RunLoopDispatcher); >+ g_signal_connect(bus.get(), "message", G_CALLBACK(simpleBusMessageCallback), pipeline); >+} >+ > } > > #endif // USE(GSTREAMER) >diff --git a/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h b/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h >index 259255e741e415a314a123d34d0188ad4690e64d..3b258586fe4a1add888c0bb1ce45edc1970abdfc 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h >+++ b/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h >@@ -82,6 +82,8 @@ inline GstClockTime toGstClockTime(const MediaTime &mediaTime) > } > > bool gstRegistryHasElementForMediaType(GList* elementFactories, const char* capsString); >+void connectSimpleBusMessageCallback(GstElement *pipeline); >+ > } > > #ifndef GST_BUFFER_DTS_OR_PTS >diff --git a/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp b/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp >index 2efb82d1e58c71d9da98ebca402db81d254bef45..3886ff875ce89f5c108c437bb9caa2f4d7f1602a 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp >+++ b/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp >@@ -64,14 +64,7 @@ RealtimeMediaSourceCenter& RealtimeMediaSourceCenter::singleton() > RealtimeMediaSourceCenter* override = mediaStreamCenterOverride(); > if (override) > return *override; >-#if PLATFORM(GTK) >- ASSERT(isMainThread()); >- notImplemented(); // Return MockRealtimeMediaSourceCenter to avoid crash. >- static NeverDestroyed<MockRealtimeMediaSourceCenter> center; >- return center; >-#else > return RealtimeMediaSourceCenter::platformCenter(); >-#endif > } > > void RealtimeMediaSourceCenter::setSharedStreamCenterOverride(RealtimeMediaSourceCenter* center) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..97aa87a26093c052df7a6128f021babed8ac191d >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp >@@ -0,0 +1,205 @@ >+/* >+ * Copyright (C) 2017 Apple Inc. All rights reserved. >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerAudioCaptureSource.h" >+ >+#include "GStreamerAudioData.h" >+#include "GStreamerAudioStreamDescription.h" >+#include "GStreamerCaptureDeviceManager.h" >+ >+#include <gst/app/gstappsink.h> >+#include <gst/gst.h> >+#include <wtf/NeverDestroyed.h> >+ >+namespace WebCore { >+ >+const static CapabilityValueOrRange defaultVolumeCapability = CapabilityValueOrRange(0.0, 1.0); >+const static RealtimeMediaSourceCapabilities::EchoCancellation defaultEchoCancellationCapability = RealtimeMediaSourceCapabilities::EchoCancellation::ReadWrite; >+ >+GST_DEBUG_CATEGORY(webkit_audio_capture_source_debug); >+#define GST_CAT_DEFAULT webkit_audio_capture_source_debug >+ >+static void initializeGStreamerDebug() >+{ >+ static std::once_flag debugRegisteredFlag; >+ std::call_once(debugRegisteredFlag, [] { >+ GST_DEBUG_CATEGORY_INIT(webkit_audio_capture_source_debug, "webkitaudiocapturesource", 0, "WebKit Audio Capture Source."); >+ }); >+} >+ >+class GStreamerAudioCaptureSourceFactory : public RealtimeMediaSource::AudioCaptureFactory { >+public: >+ CaptureSourceOrError createAudioCaptureSource(const CaptureDevice& device, const MediaConstraints* constraints) final >+ { >+ return GStreamerAudioCaptureSource::create(device.persistentId(), constraints); >+ } >+}; >+ >+static GStreamerAudioCaptureSourceFactory& libWebRTCAudioCaptureSourceFactory() >+{ >+ static NeverDestroyed<GStreamerAudioCaptureSourceFactory> factory; >+ return factory.get(); >+} >+ >+CaptureSourceOrError GStreamerAudioCaptureSource::create(const String& deviceID, const MediaConstraints* constraints) >+{ >+ auto device = GStreamerAudioCaptureDeviceManager::singleton().gstreamerDeviceWithUID(deviceID); >+ if (!device) { >+ auto errorMessage = String::format("GStreamerAudioCaptureSource::create(): GStreamer did not find the device: %s.", deviceID.utf8().data()); >+ return CaptureSourceOrError(WTFMove(errorMessage)); >+ } >+ >+ auto source = adoptRef(*new GStreamerAudioCaptureSource(device.value())); >+ >+ if (constraints) { >+ auto result = source->applyConstraints(*constraints); >+ if (result) >+ return WTFMove(result.value().first); >+ } >+ return CaptureSourceOrError(WTFMove(source)); >+} >+ >+RealtimeMediaSource::AudioCaptureFactory& GStreamerAudioCaptureSource::factory() >+{ >+ return libWebRTCAudioCaptureSourceFactory(); >+} >+ >+GStreamerAudioCaptureSource::GStreamerAudioCaptureSource(GStreamerCaptureDevice device) >+ : RealtimeMediaSource(device.persistentId(), RealtimeMediaSource::Type::Audio, device.label()) >+ , m_capturer(std::make_unique<GStreamerAudioCapturer>(device)) >+{ >+ initializeGStreamerDebug(); >+} >+ >+GStreamerAudioCaptureSource::GStreamerAudioCaptureSource(const String& deviceID, const String& name) >+ : RealtimeMediaSource(deviceID, RealtimeMediaSource::Type::Audio, name) >+ , m_capturer(std::make_unique<GStreamerAudioCapturer>()) >+{ >+ initializeGStreamerDebug(); >+} >+ >+GStreamerAudioCaptureSource::~GStreamerAudioCaptureSource() >+{ >+} >+ >+void GStreamerAudioCaptureSource::startProducingData() >+{ >+ m_capturer->setupPipeline(); >+ m_capturer->setSampleRate(sampleRate()); >+ g_signal_connect(m_capturer->sink(), "new-sample", G_CALLBACK(newSampleCallback), this); >+ m_capturer->play(); >+} >+ >+GstFlowReturn GStreamerAudioCaptureSource::newSampleCallback(GstElement* sink, GStreamerAudioCaptureSource* source) >+{ >+ auto sample = adoptGRef(gst_app_sink_pull_sample(GST_APP_SINK(sink))); >+ >+ // FIXME - figure out a way to avoid copying (on write) the data. >+ GstBuffer* buf = gst_sample_get_buffer(sample.get()); >+ auto frames(std::unique_ptr<GStreamerAudioData>(new GStreamerAudioData(WTFMove(sample)))); >+ auto streamDesc(std::unique_ptr<GStreamerAudioStreamDescription>(new GStreamerAudioStreamDescription(frames->getAudioInfo()))); >+ >+ source->audioSamplesAvailable( >+ MediaTime(GST_TIME_AS_USECONDS(GST_BUFFER_PTS(buf)), G_USEC_PER_SEC), >+ *frames, *streamDesc, gst_buffer_get_size(buf) / frames->getAudioInfo().bpf); >+ >+ return GST_FLOW_OK; >+} >+ >+void GStreamerAudioCaptureSource::stopProducingData() >+{ >+ m_capturer->stop(); >+} >+ >+const RealtimeMediaSourceCapabilities& GStreamerAudioCaptureSource::capabilities() const >+{ >+ if (!m_capabilities) { >+ GRefPtr<GstCaps> caps = m_capturer->caps(); >+ int minSampleRate = 0, maxSampleRate = 0; >+ uint i; >+ >+ for (i = 0; i < gst_caps_get_size(caps.get()); i++) { >+ int capabilityMinSampleRate = 0, capabilityMaxSampleRate = 0; >+ GstStructure* str = gst_caps_get_structure(caps.get(), i); >+ >+ // Only accept raw audio for now. >+ if (!gst_structure_has_name(str, "audio/x-raw")) >+ continue; >+ >+ gst_structure_get(str, "rate", GST_TYPE_INT_RANGE, &capabilityMinSampleRate, &capabilityMaxSampleRate, nullptr); >+ if (i > 0) { >+ minSampleRate = std::min(minSampleRate, capabilityMinSampleRate); >+ maxSampleRate = std::max(maxSampleRate, capabilityMaxSampleRate); >+ } else { >+ minSampleRate = capabilityMinSampleRate; >+ maxSampleRate = capabilityMaxSampleRate; >+ } >+ } >+ >+ RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints()); >+ capabilities.setDeviceId(id()); >+ capabilities.setEchoCancellation(defaultEchoCancellationCapability); >+ capabilities.setVolume(defaultVolumeCapability); >+ capabilities.setSampleRate(CapabilityValueOrRange(minSampleRate, maxSampleRate)); >+ m_capabilities = WTFMove(capabilities); >+ } >+ return m_capabilities.value(); >+} >+ >+bool GStreamerAudioCaptureSource::applySampleRate(int sampleRate) >+{ >+ return m_capturer->setSampleRate(sampleRate); >+} >+ >+const RealtimeMediaSourceSettings& GStreamerAudioCaptureSource::settings() const >+{ >+ if (!m_currentSettings) { >+ RealtimeMediaSourceSettings settings; >+ settings.setDeviceId(id()); >+ >+ RealtimeMediaSourceSupportedConstraints supportedConstraints; >+ supportedConstraints.setSupportsDeviceId(true); >+ supportedConstraints.setSupportsEchoCancellation(true); >+ supportedConstraints.setSupportsVolume(true); >+ supportedConstraints.setSupportsSampleRate(true); >+ settings.setSupportedConstraints(supportedConstraints); >+ >+ m_currentSettings = WTFMove(settings); >+ } >+ >+ m_currentSettings->setVolume(volume()); >+ m_currentSettings->setSampleRate(sampleRate()); >+ m_currentSettings->setEchoCancellation(echoCancellation()); >+ >+ return m_currentSettings.value(); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h >new file mode 100644 >index 0000000000000000000000000000000000000000..af929663abc791cd82d60bf16bc94f62e614d401 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h >@@ -0,0 +1,66 @@ >+/* >+ * Copyright (C) 2017 Apple Inc. All rights reserved. >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerAudioCapturer.h" >+#include "GStreamerCaptureDevice.h" >+#include "RealtimeMediaSource.h" >+ >+namespace WebCore { >+ >+class GStreamerAudioCaptureSource : public RealtimeMediaSource { >+public: >+ static CaptureSourceOrError create(const String& deviceID, const MediaConstraints*); >+ WEBCORE_EXPORT static AudioCaptureFactory& factory(); >+ >+ const RealtimeMediaSourceCapabilities& capabilities() const final; >+ const RealtimeMediaSourceSettings& settings() const final; >+ >+private: >+ GStreamerAudioCaptureSource(GStreamerCaptureDevice); >+ GStreamerAudioCaptureSource(const String& deviceID, const String& name); >+ virtual ~GStreamerAudioCaptureSource(); >+ >+ bool applySampleRate(int) final; >+ bool isCaptureSource() const final { return true; } >+ void startProducingData() final; >+ void stopProducingData() final; >+ bool applyVolume(double) final { return true; } >+ >+ std::unique_ptr<GStreamerAudioCapturer> m_capturer; >+ >+ static GstFlowReturn newSampleCallback(GstElement*, GStreamerAudioCaptureSource*); >+ void triggerSampleAvailable(GstSample*); >+ >+ mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >+ mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings; >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..0f860736ec61dc7634306a1d6f7f26ef0a249f33 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp >@@ -0,0 +1,74 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerAudioCapturer.h" >+ >+#include "LibWebRTCAudioFormat.h" >+ >+#include <gst/app/gstappsink.h> >+ >+namespace WebCore { >+ >+GStreamerAudioCapturer::GStreamerAudioCapturer(GStreamerCaptureDevice device) >+ : GStreamerCapturer(device, adoptGRef(gst_caps_new_simple("audio/x-raw", "rate", G_TYPE_INT, LibWebRTCAudioFormat::sampleRate, nullptr))) >+{ >+} >+ >+GStreamerAudioCapturer::GStreamerAudioCapturer() >+ : GStreamerCapturer("audiotestsrc", adoptGRef(gst_caps_new_simple("audio/x-raw", "rate", G_TYPE_INT, LibWebRTCAudioFormat::sampleRate, nullptr))) >+{ >+} >+ >+GstElement* GStreamerAudioCapturer::createConverter() >+{ >+ // FIXME Handle errors. >+ return gst_parse_bin_from_description("audioconvert ! audioresample", TRUE, nullptr); >+} >+ >+bool GStreamerAudioCapturer::setSampleRate(int sampleRate) >+{ >+ >+ if (sampleRate > 0) { >+ GST_INFO_OBJECT(m_pipeline.get(), "Setting SampleRate %d", sampleRate); >+ m_caps = adoptGRef(gst_caps_new_simple("audio/x-raw", "rate", G_TYPE_INT, sampleRate, nullptr)); >+ } else { >+ GST_INFO_OBJECT(m_pipeline.get(), "Not forcing sample rate"); >+ m_caps = adoptGRef(gst_caps_new_empty_simple("audio/x-raw")); >+ } >+ >+ if (!m_capsfilter.get()) >+ return false; >+ >+ g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr); >+ >+ return true; >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.h >new file mode 100644 >index 0000000000000000000000000000000000000000..c2de444a883f4b6063cc020175e1bf9e6ddd94ae >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.h >@@ -0,0 +1,47 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GStreamerCapturer.h" >+ >+namespace WebCore { >+ >+class GStreamerAudioCapturer : public GStreamerCapturer { >+public: >+ GStreamerAudioCapturer(GStreamerCaptureDevice); >+ GStreamerAudioCapturer(); >+ >+ GstElement* createConverter() final; >+ const char* name() final { return "Audio"; } >+ >+ bool setSampleRate(int); >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioData.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioData.h >new file mode 100644 >index 0000000000000000000000000000000000000000..706d448aa6347dab04122fd4e21f3fdffc30707f >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioData.h >@@ -0,0 +1,57 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Library General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Library General Public License for more details. >+ * >+ * You should have received a copy of the GNU Library General Public License >+ * aint with this library; see the file COPYING.LIB. If not, write to >+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, >+ * Boston, MA 02110-1301, USA. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GRefPtrGStreamer.h" >+#include "PlatformAudioData.h" >+ >+#include <gst/audio/audio.h> >+ >+namespace WebCore { >+class GStreamerAudioData : public PlatformAudioData { >+public: >+ GStreamerAudioData(GRefPtr<GstSample>&& sample, GstAudioInfo info) >+ : m_sample(WTFMove(sample)) >+ , m_audioInfo(info) >+ { >+ } >+ >+ GStreamerAudioData(GRefPtr<GstSample>&& sample) >+ : m_sample(WTFMove(sample)) >+ { >+ gst_audio_info_from_caps(&m_audioInfo, gst_sample_get_caps(m_sample.get())); >+ } >+ >+ GstSample* getSample() { return m_sample.get(); } >+ GstAudioInfo getAudioInfo() {return m_audioInfo; } >+ >+private: >+ Kind kind() const { return Kind::GStreamerAudioData; } >+ GRefPtr<GstSample> m_sample; >+ GRefPtr<GstCaps> m_caps; >+ >+ GstAudioInfo m_audioInfo; >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h >new file mode 100644 >index 0000000000000000000000000000000000000000..1aff44923454c18c9d76ccfb5660ff7ad53e076d >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h >@@ -0,0 +1,109 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "AudioStreamDescription.h" >+#include <gst/audio/audio.h> >+ >+namespace WebCore { >+ >+class GStreamerAudioStreamDescription final: public AudioStreamDescription { >+public: >+ GStreamerAudioStreamDescription(GstAudioInfo info) >+ : m_info(info) >+ , m_caps(adoptGRef(gst_audio_info_to_caps(&m_info))) >+ { >+ } >+ >+ GStreamerAudioStreamDescription(GstAudioInfo *info) >+ : m_info(*info) >+ , m_caps(adoptGRef(gst_audio_info_to_caps(&m_info))) >+ { >+ } >+ >+ GStreamerAudioStreamDescription() >+ { >+ gst_audio_info_init(&m_info); >+ } >+ >+ WEBCORE_EXPORT ~GStreamerAudioStreamDescription() { }; >+ >+ const PlatformDescription& platformDescription() const >+ { >+ m_platformDescription = { PlatformDescription::GStreamerAudioStreamDescription, (AudioStreamBasicDescription*) &m_info }; >+ >+ return m_platformDescription; >+ } >+ >+ WEBCORE_EXPORT PCMFormat format() const final { >+ switch (GST_AUDIO_INFO_FORMAT(&m_info)) { >+ case GST_AUDIO_FORMAT_S16LE: >+ case GST_AUDIO_FORMAT_S16BE: >+ return Int16; >+ case GST_AUDIO_FORMAT_S32LE: >+ case GST_AUDIO_FORMAT_S32BE: >+ return Int32; >+ case GST_AUDIO_FORMAT_F32LE: >+ case GST_AUDIO_FORMAT_F32BE: >+ return Float32; >+ case GST_AUDIO_FORMAT_F64LE: >+ case GST_AUDIO_FORMAT_F64BE: >+ return Float64; >+ default: >+ break; >+ } >+ return None; >+ } >+ >+ double sampleRate() const final { return GST_AUDIO_INFO_RATE (&m_info); } >+ bool isPCM() const final { return format() != None; } >+ bool isInterleaved() const final { return GST_AUDIO_INFO_LAYOUT (&m_info) == GST_AUDIO_LAYOUT_INTERLEAVED; } >+ bool isSignedInteger() const final { return GST_AUDIO_INFO_IS_INTEGER (&m_info); } >+ bool isNativeEndian() const final { return GST_AUDIO_INFO_ENDIANNESS (&m_info) == G_BYTE_ORDER; } >+ bool isFloat() const final { return GST_AUDIO_INFO_IS_FLOAT (&m_info); } >+ >+ uint32_t numberOfInterleavedChannels() const final { return isInterleaved() ? GST_AUDIO_INFO_CHANNELS (&m_info) : TRUE; } >+ uint32_t numberOfChannelStreams() const final { return GST_AUDIO_INFO_CHANNELS (&m_info); } >+ uint32_t numberOfChannels() const final { return GST_AUDIO_INFO_CHANNELS (&m_info); } >+ uint32_t sampleWordSize() const final { return GST_AUDIO_INFO_BPS (&m_info); } >+ >+ bool operator==(const GStreamerAudioStreamDescription& other) { return gst_audio_info_is_equal (&m_info, &other.m_info); } >+ bool operator!=(const GStreamerAudioStreamDescription& other) { return !operator == (other); } >+ >+ GstCaps* caps() { return m_caps.get(); } >+ GstAudioInfo* getInfo() { return &m_info; } >+ >+private: >+ GstAudioInfo m_info; >+ GRefPtr<GstCaps> m_caps; >+ mutable PlatformDescription m_platformDescription; >+}; >+ >+} // WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDevice.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDevice.h >new file mode 100644 >index 0000000000000000000000000000000000000000..e418aead73f970b6b8df468fee80d363123d874b >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDevice.h >@@ -0,0 +1,47 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Lesser General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Lesser General Public License for more details. >+ * >+ * You should have received a copy of the GNU Lesser General Public >+ * License along with this library; if not, write to the Free Software >+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >+ >+#include "CaptureDevice.h" >+#include "GRefPtrGStreamer.h" >+ >+#include <gst/gst.h> >+ >+namespace WebCore { >+ >+class GStreamerCaptureDevice : public CaptureDevice { >+public: >+ GStreamerCaptureDevice(GRefPtr<GstDevice>&& device, const String& persistentId, DeviceType type, const String& label, const String& groupId = emptyString()) >+ : CaptureDevice(persistentId, type, label, groupId) >+ , m_device(device) >+ { >+ } >+ >+ GstCaps* caps() const { return gst_device_get_caps(m_device.get()); } >+ GstDevice * device() { return m_device.get(); } >+ >+private: >+ GRefPtr<GstDevice> m_device; >+}; >+ >+} >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..df26b647db7569b9ff9b5080f93a2788aeb79117 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp >@@ -0,0 +1,138 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >+#include "GStreamerCaptureDeviceManager.h" >+ >+#include "GStreamerCommon.h" >+ >+namespace WebCore { >+ >+GStreamerAudioCaptureDeviceManager& GStreamerAudioCaptureDeviceManager::singleton() >+{ >+ static NeverDestroyed<GStreamerAudioCaptureDeviceManager> manager; >+ return manager; >+} >+ >+GStreamerVideoCaptureDeviceManager& GStreamerVideoCaptureDeviceManager::singleton() >+{ >+ static NeverDestroyed<GStreamerVideoCaptureDeviceManager> manager; >+ return manager; >+} >+ >+GStreamerDisplayCaptureDeviceManager& GStreamerDisplayCaptureDeviceManager::singleton() >+{ >+ static NeverDestroyed<GStreamerDisplayCaptureDeviceManager> manager; >+ return manager; >+} >+ >+std::optional<GStreamerCaptureDevice> GStreamerCaptureDeviceManager::gstreamerDeviceWithUID(const String& deviceID) >+{ >+ captureDevices(); >+ >+ for (auto& device : m_gstreamerDevices) { >+ if (device.persistentId() == deviceID) >+ return device; >+ } >+ return std::nullopt; >+} >+ >+const Vector<CaptureDevice>& GStreamerCaptureDeviceManager::captureDevices() >+{ >+ initializeGStreamer(); >+ if (m_devices.isEmpty()) >+ refreshCaptureDevices(); >+ >+ return m_devices; >+} >+ >+void GStreamerCaptureDeviceManager::deviceAdded(GRefPtr<GstDevice>&& device) >+{ >+ GUniquePtr<GstStructure> properties(gst_device_get_properties(device.get())); >+ const char* klass = gst_structure_get_string(properties.get(), "device.class"); >+ >+ if (klass && !g_strcmp0(klass, "monitor")) >+ return; >+ >+ CaptureDevice::DeviceType type = deviceType(); >+ GUniquePtr<char> deviceClassChar(gst_device_get_device_class(device.get())); >+ String deviceClass(String(deviceClassChar.get())); >+ if (type == CaptureDevice::DeviceType::Microphone && !deviceClass.startsWith("Audio")) >+ return; >+ if (type == CaptureDevice::DeviceType::Camera && !deviceClass.startsWith("Video")) >+ return; >+ >+ // This isn't really a UID but should be good enough (libwebrtc >+ // itself does that at least for pulseaudio devices). >+ GUniquePtr<char> deviceName(gst_device_get_display_name(device.get())); >+ String identifier = String::fromUTF8(deviceName.get()); >+ >+ auto gstCaptureDevice = GStreamerCaptureDevice(WTFMove(device), identifier, type, identifier); >+ gstCaptureDevice.setEnabled(true); >+ m_gstreamerDevices.append(WTFMove(gstCaptureDevice)); >+ // FIXME: We need a CaptureDevice copy in other vector just for captureDevices API. >+ auto captureDevice = CaptureDevice(identifier, type, identifier); >+ captureDevice.setEnabled(true); >+ m_devices.append(WTFMove(captureDevice)); >+} >+ >+void GStreamerCaptureDeviceManager::refreshCaptureDevices() >+{ >+ if (!m_deviceMonitor) { >+ m_deviceMonitor = adoptGRef(gst_device_monitor_new()); >+ >+ CaptureDevice::DeviceType type = deviceType(); >+ if (type == CaptureDevice::DeviceType::Camera) { >+ GRefPtr<GstCaps> caps = adoptGRef(gst_caps_new_empty_simple("video/x-raw")); >+ gst_device_monitor_add_filter(m_deviceMonitor.get(), "Video/Source", caps.get()); >+ } else if (type == CaptureDevice::DeviceType::Microphone) { >+ GRefPtr<GstCaps> caps = adoptGRef(gst_caps_new_empty_simple("audio/x-raw")); >+ gst_device_monitor_add_filter(m_deviceMonitor.get(), "Audio/Source", caps.get()); >+ } >+ // FIXME: Add monitor for added/removed messages on the bus. >+ } >+ >+ if (!gst_device_monitor_start(m_deviceMonitor.get())) { >+ GST_WARNING_OBJECT(m_deviceMonitor.get(), "Could not start device monitor"); >+ m_deviceMonitor = nullptr; >+ >+ return; >+ } >+ >+ GList* devices = gst_device_monitor_get_devices(m_deviceMonitor.get()); >+ while (devices) { >+ GRefPtr<GstDevice> device = adoptGRef(GST_DEVICE_CAST(devices->data)); >+ deviceAdded(WTFMove(device)); >+ devices = g_list_delete_link(devices, devices); >+ } >+ >+ gst_device_monitor_stop(m_deviceMonitor.get()); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h >new file mode 100644 >index 0000000000000000000000000000000000000000..a185ed7e50c15a1d25128d318db4b1a5d004ce6c >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h >@@ -0,0 +1,78 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * This library is free software; you can redistribute it and/or >+ * modify it under the terms of the GNU Lesser General Public >+ * License as published by the Free Software Foundation; either >+ * version 2 of the License, or (at your option) any later version. >+ * >+ * This library is distributed in the hope that it will be useful, >+ * but WITHOUT ANY WARRANTY; without even the implied warranty of >+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >+ * Lesser General Public License for more details. >+ * >+ * You should have received a copy of the GNU Lesser General Public >+ * License along with this library; if not, write to the Free Software >+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >+ >+#include "CaptureDeviceManager.h" >+#include "GRefPtrGStreamer.h" >+#include "GStreamerCaptureDevice.h" >+ >+namespace WebCore { >+ >+class GStreamerCaptureDeviceManager : public CaptureDeviceManager { >+public: >+ std::optional<GStreamerCaptureDevice> gstreamerDeviceWithUID(const String&); >+ >+ void refreshCaptureDevices() final; >+ const Vector<CaptureDevice>& captureDevices() final; >+ virtual CaptureDevice::DeviceType deviceType() = 0; >+ >+private: >+ void deviceAdded(GRefPtr<GstDevice>&&); >+ >+ GRefPtr<GstDeviceMonitor> m_deviceMonitor; >+ Vector<GStreamerCaptureDevice> m_gstreamerDevices; >+ Vector<CaptureDevice> m_devices; >+}; >+ >+class GStreamerAudioCaptureDeviceManager final : public GStreamerCaptureDeviceManager { >+ friend class NeverDestroyed<GStreamerAudioCaptureDeviceManager>; >+public: >+ static GStreamerAudioCaptureDeviceManager& singleton(); >+ CaptureDevice::DeviceType deviceType() final { return CaptureDevice::DeviceType::Microphone; } >+private: >+ GStreamerAudioCaptureDeviceManager() = default; >+ ~GStreamerAudioCaptureDeviceManager() = default; >+}; >+ >+class GStreamerVideoCaptureDeviceManager final : public GStreamerCaptureDeviceManager { >+ friend class NeverDestroyed<GStreamerVideoCaptureDeviceManager>; >+public: >+ static GStreamerVideoCaptureDeviceManager& singleton(); >+ static RealtimeMediaSource::VideoCaptureFactory& videoFactory(); >+ CaptureDevice::DeviceType deviceType() final { return CaptureDevice::DeviceType::Camera; } >+private: >+ GStreamerVideoCaptureDeviceManager() = default; >+ ~GStreamerVideoCaptureDeviceManager() = default; >+}; >+ >+class GStreamerDisplayCaptureDeviceManager final : public GStreamerCaptureDeviceManager { >+ friend class NeverDestroyed<GStreamerDisplayCaptureDeviceManager>; >+public: >+ static GStreamerDisplayCaptureDeviceManager& singleton(); >+ CaptureDevice::DeviceType deviceType() final { return CaptureDevice::DeviceType::Screen; } >+private: >+ GStreamerDisplayCaptureDeviceManager() = default; >+ ~GStreamerDisplayCaptureDeviceManager() = default; >+}; >+ >+} >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..9ca52b51180f7b1aca00eabf86585d5387a02839 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp >@@ -0,0 +1,181 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerCapturer.h" >+ >+#include <gst/app/gstappsink.h> >+#include <mutex> >+#include <webrtc/api/mediastreaminterface.h> >+ >+GST_DEBUG_CATEGORY(webkit_capturer_debug); >+#define GST_CAT_DEFAULT webkit_capturer_debug >+ >+namespace WebCore { >+ >+static void initializeGStreamerAndDebug() >+{ >+ initializeGStreamer(); >+ >+ static std::once_flag debugRegisteredFlag; >+ std::call_once(debugRegisteredFlag, [] { >+ GST_DEBUG_CATEGORY_INIT(webkit_capturer_debug, "webkitcapturer", 0, "WebKit Capturer"); >+ }); >+} >+ >+GStreamerCapturer::GStreamerCapturer(GStreamerCaptureDevice device, GRefPtr<GstCaps> caps) >+ : m_device(device.device()) >+ , m_caps(caps) >+ , m_sourceFactory(nullptr) >+{ >+ initializeGStreamerAndDebug(); >+} >+ >+GStreamerCapturer::GStreamerCapturer(const char* sourceFactory, GRefPtr<GstCaps> caps) >+ : m_device(nullptr) >+ , m_caps(caps) >+ , m_sourceFactory(sourceFactory) >+{ >+ initializeGStreamerAndDebug(); >+} >+ >+GstElement* GStreamerCapturer::createSource() >+{ >+ if (m_sourceFactory) { >+ m_src = adoptGRef(makeElement(m_sourceFactory)); >+ ASSERT(m_src); >+ >+ return m_src.get(); >+ } >+ >+ ASSERT(m_device); >+ GUniquePtr<char> sourceName(g_strdup_printf("%s_%p", name(), this)); >+ m_src = gst_device_create_element(m_device.get(), sourceName.get()); >+ ASSERT(m_src); >+ >+ return m_src.get(); >+} >+ >+GstCaps* GStreamerCapturer::caps() >+{ >+ if (m_sourceFactory) { >+ GRefPtr<GstElement> element = makeElement(m_sourceFactory); >+ auto pad = adoptGRef(gst_element_get_static_pad(element.get(), "src")); >+ >+ return gst_pad_query_caps(pad.get(), nullptr); >+ } >+ >+ ASSERT(m_device); >+ return gst_device_get_caps(m_device.get()); >+} >+ >+void GStreamerCapturer::setupPipeline() >+{ >+ m_pipeline = makeElement("pipeline"); >+ >+ GRefPtr<GstElement> source = createSource(); >+ GRefPtr<GstElement> converter = createConverter(); >+ >+ m_capsfilter = makeElement("capsfilter"); >+ m_tee = makeElement("tee"); >+ m_sink = makeElement("appsink"); >+ >+ gst_app_sink_set_emit_signals(GST_APP_SINK(m_sink.get()), TRUE); >+ g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr); >+ >+ gst_bin_add_many(GST_BIN(m_pipeline.get()), source.get(), converter.get(), m_capsfilter.get(), m_tee.get(), nullptr); >+ gst_element_link_many(source.get(), converter.get(), m_capsfilter.get(), m_tee.get(), NULL); >+ >+ addSink(m_sink.get()); >+ >+ connectSimpleBusMessageCallback(pipeline()); >+} >+ >+GstElement* GStreamerCapturer::makeElement(const char* factoryName) >+{ >+ auto element = gst_element_factory_make(factoryName, nullptr); >+ GUniquePtr<char> capturerName(g_strdup_printf("%s_capturer_%s_%p", name(), GST_OBJECT_NAME(element), this)); >+ gst_object_set_name(GST_OBJECT(element), capturerName.get()); >+ >+ return element; >+} >+ >+void GStreamerCapturer::addSink(GstElement *newSink) >+{ >+ if (!m_pipeline || !m_tee) >+ return; >+ >+ auto queue = makeElement("queue"); >+ gst_bin_add_many(GST_BIN(pipeline()), queue, newSink, nullptr); >+ gst_element_sync_state_with_parent(queue); >+ gst_element_sync_state_with_parent(newSink); >+ >+ if (!gst_element_link_pads(m_tee.get(), "src_%u", queue, "sink")) >+ ASSERT_NOT_REACHED(); >+ >+ if (!gst_element_link(queue, newSink)) >+ ASSERT_NOT_REACHED(); >+ >+ if (newSink == sink()) { >+ GST_INFO_OBJECT(pipeline(), "Setting queue as leaky upstream", >+ " so that the player can set the sink as to PAUSED without " >+ " setting the whole capturer to PAUSED"); >+ g_object_set(queue, "leaky", 2 /* upstream */, nullptr); >+ } >+ >+ GST_INFO_OBJECT(pipeline(), "Adding sink: %" GST_PTR_FORMAT, newSink); >+ >+ GUniquePtr<char> dumpName(g_strdup_printf("%s_sink_%s_added", GST_OBJECT_NAME(pipeline()), GST_OBJECT_NAME(newSink))); >+ GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(GST_BIN(pipeline()), GST_DEBUG_GRAPH_SHOW_ALL, dumpName.get()); >+} >+ >+void GStreamerCapturer::play() >+{ >+ ASSERT(m_pipeline); >+ >+ GST_INFO_OBJECT((gpointer) pipeline(), "Going to PLAYING!"); >+ >+ gst_element_set_state(pipeline(), GST_STATE_PLAYING); >+} >+ >+void GStreamerCapturer::stop() >+{ >+ ASSERT(m_pipeline); >+ >+ GST_INFO_OBJECT((gpointer) pipeline(), "Tearing down!"); >+ >+ // Make sure to remove sync handler before tearing down, avoiding >+ // possible deadlocks. >+ GRefPtr<GstBus> bus = adoptGRef(gst_pipeline_get_bus(GST_PIPELINE(pipeline()))); >+ gst_bus_set_sync_handler(bus.get(), nullptr, nullptr, nullptr); >+ >+ gst_element_set_state(pipeline(), GST_STATE_NULL); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h >new file mode 100644 >index 0000000000000000000000000000000000000000..d734b81d1b83ae0898dffcf8112fd3e185b637c8 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h >@@ -0,0 +1,77 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GStreamerCommon.h" >+ >+#include "LibWebRTCMacros.h" >+#include "gstreamer/GStreamerCaptureDevice.h" >+ >+#include <gst/gst.h> >+ >+#pragma once >+ >+namespace WebCore { >+ >+class GStreamerCapturer { >+public: >+ GStreamerCapturer(GStreamerCaptureDevice, GRefPtr<GstCaps>); >+ GStreamerCapturer(const char* sourceFactory, GRefPtr<GstCaps>); >+ >+ void setupPipeline(); >+ virtual void play(); >+ virtual void stop(); >+ GstCaps* caps(); >+ void addSink(GstElement *newSink); >+ GstElement* makeElement(const char* factoryName); >+ GstElement* createSource(); >+ virtual const char* name() = 0; >+ >+ GstElement* sink() const { return m_sink.get(); } >+ void setSink(GstElement* sink) { m_sink = adoptGRef(sink); }; >+ >+ GstElement* pipeline() const { return m_pipeline.get(); } >+ virtual GstElement* createConverter() = 0; >+ >+protected: >+ GRefPtr<GstElement> m_sink; >+ GRefPtr<GstElement> m_src; >+ GRefPtr<GstElement> m_tee; >+ GRefPtr<GstElement> m_capsfilter; >+ GRefPtr<GstDevice> m_device; >+ GRefPtr<GstCaps> m_caps; >+ GRefPtr<GstElement> m_pipeline; >+ >+private: >+ const char* m_sourceFactory; >+ >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..3bd7581a2e4753fd7f00cc90bdff60f8ccd63616 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp >@@ -0,0 +1,269 @@ >+/* >+ * Copyright (C) 2017 Apple Inc. All rights reserved. >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerVideoCaptureSource.h" >+ >+#include "GStreamerCaptureDeviceManager.h" >+#include "MediaSampleGStreamer.h" >+ >+#include <gst/app/gstappsink.h> >+#include <webrtc/api/mediastreaminterface.h> >+#include <webrtc/api/peerconnectioninterface.h> >+#include <webrtc/media/base/videocommon.h> >+#include <webrtc/media/engine/webrtcvideocapturer.h> >+#include <webrtc/media/engine/webrtcvideocapturerfactory.h> >+#include <webrtc/modules/video_capture/video_capture_defines.h> >+ >+namespace WebCore { >+ >+const static int defaultWidth = 640; >+const static int defaultHeight = 480; >+ >+GST_DEBUG_CATEGORY(webkit_video_capture_source_debug); >+#define GST_CAT_DEFAULT webkit_video_capture_source_debug >+ >+static void initializeGStreamerDebug() >+{ >+ static std::once_flag debugRegisteredFlag; >+ std::call_once(debugRegisteredFlag, [] { >+ GST_DEBUG_CATEGORY_INIT(webkit_video_capture_source_debug, "webkitvideocapturesource", 0, >+ "WebKit Video Capture Source."); >+ }); >+} >+ >+class GStreamerVideoCaptureSourceFactory final : public RealtimeMediaSource::VideoCaptureFactory { >+public: >+ CaptureSourceOrError createVideoCaptureSource(const CaptureDevice& device, const MediaConstraints* constraints) final >+ { >+ return GStreamerVideoCaptureSource::create(device.persistentId(), constraints); >+ } >+}; >+ >+RealtimeMediaSource::VideoCaptureFactory& libWebRTCVideoCaptureSourceFactory() >+{ >+ static NeverDestroyed<GStreamerVideoCaptureSourceFactory> factory; >+ return factory.get(); >+} >+ >+CaptureSourceOrError GStreamerVideoCaptureSource::create(const String& deviceID, const MediaConstraints* constraints) >+{ >+ auto device = GStreamerVideoCaptureDeviceManager::singleton().gstreamerDeviceWithUID(deviceID); >+ if (!device) { >+ auto errorMessage = String::format("GStreamerVideoCaptureSource::create(): GStreamer did not find the device: %s.", deviceID.utf8().data()); >+ return CaptureSourceOrError(WTFMove(errorMessage)); >+ } >+ >+ auto source = adoptRef(*new GStreamerVideoCaptureSource(device.value())); >+ >+ if (constraints) { >+ auto result = source->applyConstraints(*constraints); >+ if (result) >+ return WTFMove(result.value().first); >+ } >+ return CaptureSourceOrError(WTFMove(source)); >+} >+ >+RealtimeMediaSource::VideoCaptureFactory& GStreamerVideoCaptureSource::factory() >+{ >+ return libWebRTCVideoCaptureSourceFactory(); >+} >+ >+GStreamerVideoCaptureSource::GStreamerVideoCaptureSource(const String& deviceID, const String& name, const gchar *source_factory) >+ : RealtimeMediaSource(deviceID, RealtimeMediaSource::Type::Video, name) >+ , m_capturer(std::make_unique<GStreamerVideoCapturer>(source_factory)) >+{ >+ initializeGStreamerDebug(); >+} >+ >+GStreamerVideoCaptureSource::GStreamerVideoCaptureSource(GStreamerCaptureDevice device) >+ : RealtimeMediaSource(device.persistentId(), RealtimeMediaSource::Type::Video, device.label()) >+ , m_capturer(std::make_unique<GStreamerVideoCapturer>(device)) >+{ >+ initializeGStreamerDebug(); >+} >+ >+GStreamerVideoCaptureSource::~GStreamerVideoCaptureSource() >+{ >+} >+ >+bool GStreamerVideoCaptureSource::applySize(const IntSize &size) >+{ >+ m_capturer->setSize(size.width(), size.height()); >+ >+ return true; >+} >+ >+bool GStreamerVideoCaptureSource::applyFrameRate(double framerate) >+{ >+ m_capturer->setFrameRate(framerate); >+ >+ return true; >+} >+ >+void GStreamerVideoCaptureSource::startProducingData() >+{ >+ m_capturer->setupPipeline(); >+ m_capturer->setSize(size().width(), size().height()); >+ m_capturer->setFrameRate(frameRate()); >+ g_signal_connect(m_capturer->sink(), "new-sample", G_CALLBACK(newSampleCallback), this); >+ m_capturer->play(); >+} >+ >+GstFlowReturn GStreamerVideoCaptureSource::newSampleCallback(GstElement* sink, GStreamerVideoCaptureSource* source) >+{ >+ auto gstSample = adoptGRef(gst_app_sink_pull_sample(GST_APP_SINK(sink))); >+ auto mediaSample = MediaSampleGStreamer::create(WTFMove(gstSample), WebCore::FloatSize(), String()); >+ >+ // FIXME - Check how presentationSize is supposed to be used here. >+ callOnMainThread([protectedThis = makeRef(*source), mediaSample = WTFMove(mediaSample)] { >+ protectedThis->videoSampleAvailable(mediaSample.get()); >+ }); >+ >+ return GST_FLOW_OK; >+} >+ >+void GStreamerVideoCaptureSource::stopProducingData() >+{ >+ m_capturer->stop(); >+ >+ GST_INFO("Reset height and width after stopping source"); >+ setHeight(defaultHeight); >+ setWidth(defaultWidth); >+} >+ >+const RealtimeMediaSourceCapabilities& GStreamerVideoCaptureSource::capabilities() const >+{ >+ if (!m_capabilities) { >+ RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints()); >+ GRefPtr<GstCaps> caps = adoptGRef(m_capturer->caps()); >+ int32_t minWidth = G_MAXINT32, minHeight = G_MAXINT32, minFramerate = G_MAXINT32; >+ int32_t maxWidth = G_MININT32, maxHeight = G_MININT32, maxFramerate = G_MININT32; >+ >+ for (guint i = 0; i < gst_caps_get_size(caps.get()); i++) { >+ GstStructure* str = gst_caps_get_structure(caps.get(), i); >+ >+ // Only accept raw video for now. >+ if (!gst_structure_has_name(str, "video/x-raw")) >+ continue; >+ >+ int32_t tmpMinWidth, tmpMinHeight, tmpMinFPSNumerator, tmpMinFPSDenominator, tmpMinFramerate; >+ int32_t tmpMaxWidth, tmpMaxHeight, tmpMaxFPSNumerator, tmpMaxFPSDenominator, tmpMaxFramerate; >+ >+ if (!gst_structure_get(str, "width", GST_TYPE_INT_RANGE, &tmpMinWidth, &tmpMaxWidth, "height", GST_TYPE_INT_RANGE, &tmpMinHeight, &tmpMaxHeight, nullptr)) { >+ >+ g_assert(gst_structure_get(str, "width", G_TYPE_INT, &tmpMinWidth, "height", G_TYPE_INT, &tmpMinHeight, nullptr)); >+ tmpMaxWidth = tmpMinWidth; >+ tmpMaxHeight = tmpMinHeight; >+ } >+ >+ if (gst_structure_get(str, "framerate", GST_TYPE_FRACTION_RANGE, &tmpMinFPSNumerator, &tmpMinFPSDenominator, &tmpMaxFPSNumerator, &tmpMaxFPSDenominator, nullptr)) { >+ tmpMinFramerate = (int)(tmpMinFPSNumerator / tmpMinFPSDenominator); >+ tmpMaxFramerate = (int)(tmpMaxFPSNumerator / tmpMaxFPSDenominator); >+ } else if (gst_structure_get(str, >+ "framerate", GST_TYPE_FRACTION, &tmpMinFPSNumerator, &tmpMinFPSDenominator, nullptr)) { >+ tmpMaxFPSNumerator = tmpMinFPSNumerator; >+ tmpMaxFPSDenominator = tmpMinFPSDenominator; >+ tmpMinFramerate = (int)(tmpMinFPSNumerator / tmpMinFPSDenominator); >+ tmpMaxFramerate = (int)(tmpMaxFPSNumerator / tmpMaxFPSDenominator); >+ } else { >+ const GValue* frameRates(gst_structure_get_value(str, "framerate")); >+ tmpMinFramerate = G_MAXINT; >+ tmpMaxFramerate = 0; >+ >+ guint frameRatesLength = static_cast<guint>(gst_value_list_get_size(frameRates)) - 1; >+ >+ for (guint i = 0; i < frameRatesLength; i++) { >+ const GValue* val = gst_value_list_get_value(frameRates, i); >+ >+ g_assert(G_VALUE_TYPE(val) == GST_TYPE_FRACTION); >+ gint framerate = (int)(gst_value_get_fraction_numerator(val) / gst_value_get_fraction_denominator(val)); >+ >+ tmpMinFramerate = std::min(tmpMinFramerate, framerate); >+ tmpMaxFramerate = std::max(tmpMaxFramerate, framerate); >+ } >+ >+ if (i > 0) { >+ minWidth = std::min(tmpMinWidth, minWidth); >+ minHeight = std::min(tmpMinHeight, minHeight); >+ minFramerate = std::min(tmpMinFramerate, minFramerate); >+ >+ maxWidth = std::max(tmpMaxWidth, maxWidth); >+ maxHeight = std::max(tmpMaxHeight, maxHeight); >+ maxFramerate = std::max(tmpMaxFramerate, maxFramerate); >+ } else { >+ minWidth = tmpMinWidth; >+ minHeight = tmpMinHeight; >+ minFramerate = tmpMinFramerate; >+ >+ maxWidth = tmpMaxWidth; >+ maxHeight = tmpMaxHeight; >+ maxFramerate = tmpMaxFramerate; >+ } >+ } >+ >+ capabilities.setDeviceId(id()); >+ capabilities.setWidth(CapabilityValueOrRange(minWidth, maxWidth)); >+ capabilities.setHeight(CapabilityValueOrRange(minHeight, maxHeight)); >+ capabilities.setFrameRate(CapabilityValueOrRange(minFramerate, maxFramerate)); >+ m_capabilities = WTFMove(capabilities); >+ } >+ } >+ >+ return m_capabilities.value(); >+} >+ >+const RealtimeMediaSourceSettings& GStreamerVideoCaptureSource::settings() const >+{ >+ if (!m_currentSettings) { >+ RealtimeMediaSourceSettings settings; >+ settings.setDeviceId(id()); >+ >+ RealtimeMediaSourceSupportedConstraints supportedConstraints; >+ supportedConstraints.setSupportsDeviceId(true); >+ supportedConstraints.setSupportsFacingMode(true); >+ supportedConstraints.setSupportsWidth(true); >+ supportedConstraints.setSupportsHeight(true); >+ supportedConstraints.setSupportsAspectRatio(true); >+ supportedConstraints.setSupportsFrameRate(true); >+ settings.setSupportedConstraints(supportedConstraints); >+ >+ m_currentSettings = WTFMove(settings); >+ } >+ >+ m_currentSettings->setWidth(size().width()); >+ m_currentSettings->setHeight(size().height()); >+ m_currentSettings->setFrameRate(frameRate()); >+ m_currentSettings->setAspectRatio(aspectRatio()); >+ m_currentSettings->setFacingMode(facingMode()); >+ return m_currentSettings.value(); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h >new file mode 100644 >index 0000000000000000000000000000000000000000..4fae63a696d03256117e9eb9b1d45e296141be16 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h >@@ -0,0 +1,66 @@ >+/* >+ * Copyright (C) 2017 Apple Inc. All rights reserved. >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerVideoCapturer.h" >+#include "RealtimeMediaSource.h" >+ >+namespace WebCore { >+ >+class GStreamerVideoCaptureSource : public RealtimeMediaSource { >+public: >+ static CaptureSourceOrError create(const String& deviceID, const MediaConstraints*); >+ WEBCORE_EXPORT static VideoCaptureFactory& factory(); >+ >+ const RealtimeMediaSourceCapabilities& capabilities() const final; >+ const RealtimeMediaSourceSettings& settings() const final; >+ >+private: >+ GStreamerVideoCaptureSource(const String& deviceID, const String& name, const gchar * source_factory); >+ virtual ~GStreamerVideoCaptureSource(); >+ >+ GStreamerVideoCaptureSource(GStreamerCaptureDevice); >+ >+ static GstFlowReturn newSampleCallback(GstElement*, GStreamerVideoCaptureSource*); >+ >+ void startProducingData() final; >+ void stopProducingData() final; >+ >+ bool isCaptureSource() const final { return true; } >+ bool applySize(const IntSize&) final; >+ bool applyFrameRate(double) final; >+ bool applyAspectRatio(double) final { return true; } >+ >+ mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >+ mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings; >+ std::unique_ptr<GStreamerVideoCapturer> m_capturer; >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..13ede5baaf83f236fd332edb1f1ba1f1ff5cf70b >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp >@@ -0,0 +1,105 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+#include "GStreamerVideoCapturer.h" >+ >+#include <gst/app/gstappsink.h> >+ >+namespace WebCore { >+ >+GStreamerVideoCapturer::GStreamerVideoCapturer(GStreamerCaptureDevice device) >+ : GStreamerCapturer(device, adoptGRef(gst_caps_new_empty_simple("video/x-raw"))) >+{ >+} >+ >+GStreamerVideoCapturer::GStreamerVideoCapturer(const char* sourceFactory) >+ : GStreamerCapturer(sourceFactory, adoptGRef(gst_caps_new_empty_simple("video/x-raw"))) >+{ >+} >+ >+GstElement* GStreamerVideoCapturer::createConverter() >+{ >+ // FIXME Handle errors. >+ return gst_parse_bin_from_description("videoscale ! videoconvert ! videorate", TRUE, nullptr); >+} >+ >+GstVideoInfo GStreamerVideoCapturer::getBestFormat() >+{ >+ GRefPtr<GstCaps> caps = adoptGRef(gst_caps_fixate(gst_device_get_caps(m_device.get()))); >+ GstVideoInfo info; >+ gst_video_info_from_caps(&info, caps.get()); >+ >+ return info; >+} >+ >+bool GStreamerVideoCapturer::setSize(int width, int height) >+{ >+ if (!width || !height) >+ return false; >+ >+ m_caps = adoptGRef(gst_caps_copy(m_caps.get())); >+ gst_caps_set_simple(m_caps.get(), "width", G_TYPE_INT, width, "height", G_TYPE_INT, height, nullptr); >+ >+ if (!m_capsfilter) >+ return false; >+ >+ g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr); >+ >+ return true; >+} >+ >+bool GStreamerVideoCapturer::setFrameRate(double frameRate) >+{ >+ int numerator, denominator; >+ >+ gst_util_double_to_fraction(frameRate, &numerator, &denominator); >+ >+ if (numerator < -G_MAXINT) { >+ GST_INFO_OBJECT(m_pipeline.get(), "Framerate %f not allowed", frameRate); >+ return false; >+ } >+ >+ if (!numerator) { >+ GST_ERROR_OBJECT(m_pipeline.get(), "Do not force variable framerate"); >+ return false; >+ } >+ >+ m_caps = adoptGRef(gst_caps_copy(m_caps.get())); >+ gst_caps_set_simple(m_caps.get(), "framerate", GST_TYPE_FRACTION, numerator, denominator, nullptr); >+ >+ if (!m_capsfilter) >+ return false; >+ >+ g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr); >+ >+ return true; >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h >new file mode 100644 >index 0000000000000000000000000000000000000000..2475ac017e9c8c51ed7f92c77ed8b771b889f29b >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h >@@ -0,0 +1,50 @@ >+/* >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >+ >+#include "GStreamerCapturer.h" >+ >+#include <gst/video/video.h> >+ >+namespace WebCore { >+ >+class GStreamerVideoCapturer : public GStreamerCapturer { >+public: >+ GStreamerVideoCapturer(GStreamerCaptureDevice); >+ GStreamerVideoCapturer(const char* source_factory); >+ >+ GstElement* createConverter() final; >+ const char* name() final { return "Video"; } >+ >+ bool setSize(int width, int height); >+ bool setFrameRate(double); >+ GstVideoInfo getBestFormat(); >+}; >+ >+} // namespace WebCore >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) && USE(GSTREAMER) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp b/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..59de912c51505a29e523a03a66ad1b4b9e6bfb80 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp >@@ -0,0 +1,98 @@ >+/* >+ * Copyright (C) 2013-2016 Apple, Inc. All rights reserved. >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer >+ * in the documentation and/or other materials provided with the >+ * distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT >+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR >+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT >+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, >+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT >+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, >+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY >+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >+#include "RealtimeMediaSourceCenterLibWebRTC.h" >+ >+#include "GStreamerAudioCaptureSource.h" >+#include "GStreamerCaptureDevice.h" >+#include "GStreamerCaptureDeviceManager.h" >+#include "GStreamerVideoCaptureSource.h" >+#include <wtf/MainThread.h> >+ >+namespace WebCore { >+ >+RealtimeMediaSource::AudioCaptureFactory& RealtimeMediaSourceCenterLibWebRTC::audioCaptureSourceFactory() >+{ >+ return RealtimeMediaSourceCenterLibWebRTC::singleton().audioFactory(); >+} >+ >+RealtimeMediaSourceCenterLibWebRTC& RealtimeMediaSourceCenterLibWebRTC::singleton() >+{ >+ ASSERT(isMainThread()); >+ static NeverDestroyed<RealtimeMediaSourceCenterLibWebRTC> center; >+ return center; >+} >+ >+RealtimeMediaSourceCenter& RealtimeMediaSourceCenter::platformCenter() >+{ >+ return RealtimeMediaSourceCenterLibWebRTC::singleton(); >+} >+ >+RealtimeMediaSourceCenterLibWebRTC::RealtimeMediaSourceCenterLibWebRTC() >+ : m_libWebRTCProvider(LibWebRTCProvider::create()) >+{ >+} >+ >+RealtimeMediaSourceCenterLibWebRTC::~RealtimeMediaSourceCenterLibWebRTC() >+{ >+} >+ >+RealtimeMediaSource::AudioCaptureFactory& RealtimeMediaSourceCenterLibWebRTC::audioFactory() >+{ >+ if (m_audioFactoryOverride) >+ return *m_audioFactoryOverride; >+ >+ return GStreamerAudioCaptureSource::factory(); >+} >+ >+RealtimeMediaSource::VideoCaptureFactory& RealtimeMediaSourceCenterLibWebRTC::videoFactory() >+{ >+ return GStreamerVideoCaptureSource::factory(); >+} >+ >+CaptureDeviceManager& RealtimeMediaSourceCenterLibWebRTC::audioCaptureDeviceManager() >+{ >+ return GStreamerAudioCaptureDeviceManager::singleton(); >+} >+ >+CaptureDeviceManager& RealtimeMediaSourceCenterLibWebRTC::videoCaptureDeviceManager() >+{ >+ return GStreamerVideoCaptureDeviceManager::singleton(); >+} >+ >+CaptureDeviceManager& RealtimeMediaSourceCenterLibWebRTC::displayCaptureDeviceManager() >+{ >+ return GStreamerDisplayCaptureDeviceManager::singleton(); >+} >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >diff --git a/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.h b/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.h >new file mode 100644 >index 0000000000000000000000000000000000000000..4b22d83c162b4538e345cdcc38fbb780c59ea882 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.h >@@ -0,0 +1,75 @@ >+/* >+ * Copyright (C) 2013-2016 Apple, Inc. All rights reserved. >+ * Copyright (C) 2017 Igalia S.L. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer >+ * in the documentation and/or other materials provided with the >+ * distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT >+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR >+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT >+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, >+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT >+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, >+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY >+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >+ >+#include "CaptureDeviceManager.h" >+#include "LibWebRTCProvider.h" >+#include "RealtimeMediaSource.h" >+#include "RealtimeMediaSourceCenter.h" >+ >+#include <webrtc/api/peerconnectioninterface.h> >+#include <wtf/RefPtr.h> >+#include <wtf/text/WTFString.h> >+ >+namespace WebCore { >+ >+class RealtimeMediaSourceCenterLibWebRTC final : public RealtimeMediaSourceCenter { >+public: >+ WEBCORE_EXPORT static RealtimeMediaSourceCenterLibWebRTC& singleton(); >+ >+ webrtc::PeerConnectionFactoryInterface* factory() { return m_libWebRTCProvider->factory(); } >+ >+ static RealtimeMediaSource::VideoCaptureFactory& videoCaptureSourceFactory(); >+ static RealtimeMediaSource::AudioCaptureFactory& audioCaptureSourceFactory(); >+ >+private: >+ friend class NeverDestroyed<RealtimeMediaSourceCenterLibWebRTC>; >+ RealtimeMediaSourceCenterLibWebRTC(); >+ ~RealtimeMediaSourceCenterLibWebRTC(); >+ >+ void setAudioFactory(RealtimeMediaSource::AudioCaptureFactory& factory) final { m_audioFactoryOverride = &factory; } >+ void unsetAudioFactory(RealtimeMediaSource::AudioCaptureFactory&) final { m_audioFactoryOverride = nullptr; } >+ >+ RealtimeMediaSource::AudioCaptureFactory& audioFactory() final; >+ RealtimeMediaSource::VideoCaptureFactory& videoFactory() final; >+ >+ CaptureDeviceManager& audioCaptureDeviceManager() final; >+ CaptureDeviceManager& videoCaptureDeviceManager() final; >+ CaptureDeviceManager& displayCaptureDeviceManager() final; >+ >+ RealtimeMediaSource::AudioCaptureFactory* m_audioFactoryOverride { nullptr }; >+ UniqueRef<LibWebRTCProvider> m_libWebRTCProvider; >+}; >+ >+} // namespace WebCore >+ >+#endif // ENABLE(MEDIA_STREAM) && USE(LIBWEBRTC) >+ >diff --git a/Source/WebKit/UIProcess/API/glib/WebKitPermissionRequest.cpp b/Source/WebKit/UIProcess/API/glib/WebKitPermissionRequest.cpp >index 92235cef17e563a30b4d3c146d6c4579e9860250..ed20560b7207bb42898be586e3ad6659d1a8064c 100644 >--- a/Source/WebKit/UIProcess/API/glib/WebKitPermissionRequest.cpp >+++ b/Source/WebKit/UIProcess/API/glib/WebKitPermissionRequest.cpp >@@ -70,3 +70,31 @@ void webkit_permission_request_deny(WebKitPermissionRequest* request) > if (iface->deny) > iface->deny(request); > } >+ >+/** >+ * webkit_permission_request_resolve_check: >+ * @request: a #WebKitPermissionRequest representing a situation where >+ * the engine needs to check if the user gave permission to do certain >+ * type of operations for an origin. >+ * @salt: random data generated by the embedder >+ * that the engine will use to hash data in case the user does not >+ * allow the origin to access certain information. >+ * @allowed: a boolean specifying if the user allows the requested >+ * operation, if it does the engine will send more information to the >+ * origin, if it doesn't it will use the salt to hash the information >+ * to generate >+ * >+ * We have to call this function to specify if a request is allowed >+ * using the @allowed parameter. Also we add the random data that >+ * would be used as a hash salt in case the user did not allow the >+ * origin for this kind of requests, using the >+ * @salt parameter. >+ */ >+void webkit_permission_request_resolve_check(WebKitPermissionRequest *request, const gchar* mediaIdentifierHashSalt, gboolean allowed) >+{ >+ g_return_if_fail(WEBKIT_IS_PERMISSION_REQUEST(request)); >+ >+ WebKitPermissionRequestIface* iface = WEBKIT_PERMISSION_REQUEST_GET_IFACE(request); >+ if (iface->resolve_check) >+ iface->resolve_check(request, mediaIdentifierHashSalt, allowed); >+} >diff --git a/Source/WebKit/UIProcess/API/glib/WebKitUIClient.cpp b/Source/WebKit/UIProcess/API/glib/WebKitUIClient.cpp >index 42bfa9ff05fe9027e854f7db45f143352b467d5a..ee998e8739a1a22ca8cc9678e822cac9eab8c0a9 100644 >--- a/Source/WebKit/UIProcess/API/glib/WebKitUIClient.cpp >+++ b/Source/WebKit/UIProcess/API/glib/WebKitUIClient.cpp >@@ -191,6 +191,13 @@ private: > return true; > } > >+ bool checkUserMediaPermissionForOrigin(WebPageProxy&, WebFrameProxy&, API::SecurityOrigin& userMediaDocumentOrigin, API::SecurityOrigin& topLevelDocumentOrigin, UserMediaPermissionCheckProxy& permissionRequest) override >+ { >+ GRefPtr<WebKitUserMediaPermissionRequest> userMediaPermissionRequest = adoptGRef(webkitUserMediaPermissionCheckCreate(permissionRequest, userMediaDocumentOrigin, topLevelDocumentOrigin)); >+ webkitWebViewMakePermissionCheck(m_webView, WEBKIT_PERMISSION_REQUEST(userMediaPermissionRequest.get())); >+ return true; >+ } >+ > void decidePolicyForNotificationPermissionRequest(WebPageProxy&, API::SecurityOrigin&, Function<void(bool)>&& completionHandler) final > { > GRefPtr<WebKitNotificationPermissionRequest> notificationPermissionRequest = adoptGRef(webkitNotificationPermissionRequestCreate(NotificationPermissionRequest::create(WTFMove(completionHandler)).ptr())); >diff --git a/Source/WebKit/UIProcess/API/glib/WebKitUserMediaPermissionRequest.cpp b/Source/WebKit/UIProcess/API/glib/WebKitUserMediaPermissionRequest.cpp >index f08bfc110f8a087161fe2af921810d1981b84e55..728a81b0cdcadbda9154454070789b2051a8bb5e 100644 >--- a/Source/WebKit/UIProcess/API/glib/WebKitUserMediaPermissionRequest.cpp >+++ b/Source/WebKit/UIProcess/API/glib/WebKitUserMediaPermissionRequest.cpp >@@ -19,6 +19,7 @@ > #include "config.h" > #include "WebKitUserMediaPermissionRequest.h" > >+#include "UserMediaPermissionCheckProxy.h" > #include "UserMediaPermissionRequestProxy.h" > #include "WebKitPermissionRequest.h" > #include "WebKitUserMediaPermissionRequestPrivate.h" >@@ -53,6 +54,7 @@ static void webkit_permission_request_interface_init(WebKitPermissionRequestIfac > > struct _WebKitUserMediaPermissionRequestPrivate { > RefPtr<UserMediaPermissionRequestProxy> request; >+ RefPtr<UserMediaPermissionCheckProxy> checkRequest; > bool madeDecision; > }; > >@@ -95,16 +97,28 @@ static void webkitUserMediaPermissionRequestDeny(WebKitPermissionRequest* reques > priv->request->deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied); > } > >+static void webkitUserMediaPermissionResolveCheck(WebKitPermissionRequest* request, const gchar* salt, gboolean allowed) >+{ >+ ASSERT(WEBKIT_IS_USER_MEDIA_PERMISSION_REQUEST(request)); >+ >+ WebKitUserMediaPermissionRequestPrivate* priv = WEBKIT_USER_MEDIA_PERMISSION_REQUEST(request)->priv; >+ >+ priv->checkRequest->setUserMediaAccessInfo(String::fromUTF8(salt), allowed); >+} >+ > static void webkit_permission_request_interface_init(WebKitPermissionRequestIface* iface) > { > iface->allow = webkitUserMediaPermissionRequestAllow; > iface->deny = webkitUserMediaPermissionRequestDeny; >+ iface->resolve_check = webkitUserMediaPermissionResolveCheck; > } > > static void webkitUserMediaPermissionRequestDispose(GObject* object) > { > // Default behaviour when no decision has been made is denying the request. >- webkitUserMediaPermissionRequestDeny(WEBKIT_PERMISSION_REQUEST(object)); >+ WebKitUserMediaPermissionRequestPrivate* priv = WEBKIT_USER_MEDIA_PERMISSION_REQUEST(object)->priv; >+ if (priv->request) >+ webkitUserMediaPermissionRequestDeny(WEBKIT_PERMISSION_REQUEST(object)); > G_OBJECT_CLASS(webkit_user_media_permission_request_parent_class)->dispose(object); > } > >@@ -118,7 +132,9 @@ static void webkitUserMediaPermissionRequestDispose(GObject* object) > */ > gboolean webkit_user_media_permission_is_for_audio_device(WebKitUserMediaPermissionRequest* request) > { >- g_return_val_if_fail(request->priv->request, FALSE); >+ g_return_val_if_fail(request->priv->request || request->priv->checkRequest, FALSE); >+ if (request->priv->checkRequest) >+ return FALSE; > return request->priv->request->requiresAudioCapture(); > } > >@@ -132,7 +148,9 @@ gboolean webkit_user_media_permission_is_for_audio_device(WebKitUserMediaPermiss > */ > gboolean webkit_user_media_permission_is_for_video_device(WebKitUserMediaPermissionRequest* request) > { >- g_return_val_if_fail(request->priv->request, FALSE); >+ g_return_val_if_fail(request->priv->request || request->priv->checkRequest, FALSE); >+ if (request->priv->checkRequest) >+ return FALSE; > return request->priv->request->requiresVideoCapture(); > } > >@@ -196,3 +214,14 @@ WebKitUserMediaPermissionRequest* webkitUserMediaPermissionRequestCreate(UserMed > usermediaPermissionRequest->priv->request = &request; > return usermediaPermissionRequest; > } >+ >+WebKitUserMediaPermissionRequest* webkitUserMediaPermissionCheckCreate(UserMediaPermissionCheckProxy& request, API::SecurityOrigin& userMediaDocumentOrigin, API::SecurityOrigin& topLevelDocumentOrigin) >+{ >+ WebKitUserMediaPermissionRequest* usermediaPermissionRequest = WEBKIT_USER_MEDIA_PERMISSION_REQUEST(g_object_new(WEBKIT_TYPE_USER_MEDIA_PERMISSION_REQUEST, nullptr)); >+ >+ UNUSED_PARAM(userMediaDocumentOrigin); >+ UNUSED_PARAM(topLevelDocumentOrigin); >+ >+ usermediaPermissionRequest->priv->checkRequest = &request; >+ return usermediaPermissionRequest; >+} >diff --git a/Source/WebKit/UIProcess/API/glib/WebKitUserMediaPermissionRequestPrivate.h b/Source/WebKit/UIProcess/API/glib/WebKitUserMediaPermissionRequestPrivate.h >index 904a15be76ee75f6c9685e97cb65c3b969dc186c..29977268d424fbb147b833fa1ea3185399059e4e 100644 >--- a/Source/WebKit/UIProcess/API/glib/WebKitUserMediaPermissionRequestPrivate.h >+++ b/Source/WebKit/UIProcess/API/glib/WebKitUserMediaPermissionRequestPrivate.h >@@ -22,3 +22,4 @@ > #include "WebKitUserMediaPermissionRequest.h" > > WebKitUserMediaPermissionRequest* webkitUserMediaPermissionRequestCreate(WebKit::UserMediaPermissionRequestProxy&, API::SecurityOrigin&, API::SecurityOrigin&); >+WebKitUserMediaPermissionRequest* webkitUserMediaPermissionCheckCreate(WebKit::UserMediaPermissionCheckProxy&, API::SecurityOrigin&, API::SecurityOrigin&); >diff --git a/Source/WebKit/UIProcess/API/glib/WebKitWebView.cpp b/Source/WebKit/UIProcess/API/glib/WebKitWebView.cpp >index 3b7f2871520b634b8f97b7068b1b773705a536f0..ed670479cbef07cf80a174d3e50e8d6833441c9e 100644 >--- a/Source/WebKit/UIProcess/API/glib/WebKitWebView.cpp >+++ b/Source/WebKit/UIProcess/API/glib/WebKitWebView.cpp >@@ -69,6 +69,8 @@ > #include <WebCore/RefPtrCairo.h> > #include <WebCore/URL.h> > #include <glib/gi18n-lib.h> >+#include <wtf/CryptographicallyRandomNumber.h> >+#include <wtf/HexNumber.h> > #include <wtf/SetForScope.h> > #include <wtf/glib/GRefPtr.h> > #include <wtf/glib/WTFGType.h> >@@ -124,6 +126,7 @@ enum { > > DECIDE_POLICY, > PERMISSION_REQUEST, >+ PERMISSION_CHECK, > > MOUSE_TARGET_CHANGED, > >@@ -212,6 +215,7 @@ struct _WebKitWebViewPrivate { > CString title; > CString customTextEncoding; > CString activeURI; >+ String mediaIdentifierHashSalt; > bool isLoading; > bool isEphemeral; > bool isControlledByAutomation; >@@ -428,6 +432,26 @@ static gboolean webkitWebViewPermissionRequest(WebKitWebView*, WebKitPermissionR > return TRUE; > } > >+static gboolean webkitWebViewPermissionCheck(WebKitWebView* webView, WebKitPermissionRequest* request) >+{ >+ WebKitWebViewPrivate* priv = webView->priv; >+ if (priv->mediaIdentifierHashSalt.isNull()) { >+ static int hashSaltSize = 128; >+ static int randomDataSize = hashSaltSize / 16; >+ uint64_t randomData[randomDataSize]; >+ cryptographicallyRandomValues(reinterpret_cast<unsigned char*>(randomData), sizeof(randomData)); >+ >+ StringBuilder builder; >+ builder.reserveCapacity(128); >+ for (int i = 0; i < randomDataSize; i++) >+ appendUnsigned64AsHex(randomData[0], builder); >+ priv->mediaIdentifierHashSalt = builder.toString(); >+ } >+ >+ webkit_permission_request_resolve_check(request, priv->mediaIdentifierHashSalt.utf8().data(), false); >+ return TRUE; >+} >+ > static void allowModalDialogsChanged(WebKitSettings* settings, GParamSpec*, WebKitWebView* webView) > { > getPage(webView).setCanRunModal(webkit_settings_get_allow_modal_dialogs(settings)); >@@ -835,6 +859,7 @@ static void webkit_web_view_class_init(WebKitWebViewClass* webViewClass) > webViewClass->script_dialog = webkitWebViewScriptDialog; > webViewClass->decide_policy = webkitWebViewDecidePolicy; > webViewClass->permission_request = webkitWebViewPermissionRequest; >+ webViewClass->permission_check = webkitWebViewPermissionCheck; > webViewClass->run_file_chooser = webkitWebViewRunFileChooser; > webViewClass->authenticate = webkitWebViewAuthenticate; > webViewClass->show_notification = webkitWebViewShowNotification; >@@ -1492,6 +1517,48 @@ static void webkit_web_view_class_init(WebKitWebViewClass* webViewClass) > g_cclosure_marshal_generic, > G_TYPE_BOOLEAN, 1, /* number of parameters */ > WEBKIT_TYPE_PERMISSION_REQUEST); >+ >+ /** >+ * WebKitWebView::permission-check: >+ * @web_view: the #WebKitWebView on which the signal is emitted >+ * @request: the #WebKitPermissionRequest >+ * >+ * This signal is emitted when WebKit wants to check the >+ * permissions already given to an origin by a user, such as >+ * allowing the browser to use the microphone or the >+ * camera. Usually the embedder does not ask the user to resolve >+ * the request but uses what the user answered previously, denying >+ * by default if the user did not give permissions specifically to >+ * this origin. >+ * >+ * For more information about how to use the @request check >+ * permission-request signal documentation. >+ * >+ * If the signal is not handled, the @request will be denied and a >+ * random hash salt will be generated. Usually a embedder wants to >+ * connect to the signal and call >+ * webkit_permission_request_resolve_check() answering based on >+ * what the user already decided about some previous >+ * #WebKitPermissionRequest, that means storing the previous >+ * decisions of the user about each origin. It also should use a >+ * random salt that must be stored for each origin to make sure >+ * the webpage always gets the same ID for the devices, or some >+ * applications could not work properly. >+ * >+ * Returns: %TRUE to stop other handlers from being invoked for the event. >+ * %FALSE to propagate the event further. >+ * >+ */ >+ signals[PERMISSION_CHECK] = g_signal_new( >+ "permission-check", >+ G_TYPE_FROM_CLASS(webViewClass), >+ G_SIGNAL_RUN_LAST, >+ G_STRUCT_OFFSET(WebKitWebViewClass, permission_check), >+ g_signal_accumulator_true_handled, nullptr /* accumulator data */, >+ g_cclosure_marshal_generic, >+ G_TYPE_BOOLEAN, 1, /* number of parameters */ >+ WEBKIT_TYPE_PERMISSION_REQUEST); >+ > /** > * WebKitWebView::mouse-target-changed: > * @web_view: the #WebKitWebView on which the signal is emitted >@@ -2200,6 +2267,12 @@ void webkitWebViewMakePermissionRequest(WebKitWebView* webView, WebKitPermission > g_signal_emit(webView, signals[PERMISSION_REQUEST], 0, request, &returnValue); > } > >+void webkitWebViewMakePermissionCheck(WebKitWebView* webView, WebKitPermissionRequest* request) >+{ >+ gboolean returnValue; >+ g_signal_emit(webView, signals[PERMISSION_CHECK], 0, request, &returnValue); >+} >+ > void webkitWebViewMouseTargetChanged(WebKitWebView* webView, const WebHitTestResultData& hitTestResult, WebEvent::Modifiers modifiers) > { > #if PLATFORM(GTK) >diff --git a/Source/WebKit/UIProcess/API/glib/WebKitWebViewPrivate.h b/Source/WebKit/UIProcess/API/glib/WebKitWebViewPrivate.h >index 3605a915ed5010866f592a554325e2f65de0bf55..cc456fae8bc17317d05d307b9f95d29fee14e10b 100644 >--- a/Source/WebKit/UIProcess/API/glib/WebKitWebViewPrivate.h >+++ b/Source/WebKit/UIProcess/API/glib/WebKitWebViewPrivate.h >@@ -63,6 +63,7 @@ void webkitWebViewAcceptCurrentScriptDialog(WebKitWebView*); > void webkitWebViewDismissCurrentScriptDialog(WebKitWebView*); > std::optional<WebKitScriptDialogType> webkitWebViewGetCurrentScriptDialogType(WebKitWebView*); > void webkitWebViewMakePermissionRequest(WebKitWebView*, WebKitPermissionRequest*); >+void webkitWebViewMakePermissionCheck(WebKitWebView*, WebKitPermissionRequest*); > void webkitWebViewMakePolicyDecision(WebKitWebView*, WebKitPolicyDecisionType, WebKitPolicyDecision*); > void webkitWebViewMouseTargetChanged(WebKitWebView*, const WebKit::WebHitTestResultData&, WebKit::WebEvent::Modifiers); > void webkitWebViewHandleDownloadRequest(WebKitWebView*, WebKit::DownloadProxy*); >diff --git a/Source/WebKit/UIProcess/API/gtk/WebKitPermissionRequest.h b/Source/WebKit/UIProcess/API/gtk/WebKitPermissionRequest.h >index a62f1f1a5b6d706253b307e26f43afa294891d86..1bdaeab26d345ed53cf3b820cdba677d8c7c4076 100644 >--- a/Source/WebKit/UIProcess/API/gtk/WebKitPermissionRequest.h >+++ b/Source/WebKit/UIProcess/API/gtk/WebKitPermissionRequest.h >@@ -42,16 +42,24 @@ struct _WebKitPermissionRequestIface { > > void (* allow) (WebKitPermissionRequest *request); > void (* deny) (WebKitPermissionRequest *request); >+ void (* resolve_check) (WebKitPermissionRequest *request, >+ const gchar* salt, >+ gboolean allowed); > }; > > WEBKIT_API GType > webkit_permission_request_get_type (void); > > WEBKIT_API void >-webkit_permission_request_allow (WebKitPermissionRequest *request); >+webkit_permission_request_allow (WebKitPermissionRequest *request); > > WEBKIT_API void >-webkit_permission_request_deny (WebKitPermissionRequest *request); >+webkit_permission_request_deny (WebKitPermissionRequest *request); >+ >+WEBKIT_API void >+webkit_permission_request_resolve_check (WebKitPermissionRequest *request, >+ const gchar* salt, >+ gboolean allowed); > > G_END_DECLS > >diff --git a/Source/WebKit/UIProcess/API/gtk/WebKitWebView.h b/Source/WebKit/UIProcess/API/gtk/WebKitWebView.h >index 03dfea538bf179eb45db19251cf4f54dfe183721..107ee3fa7652c14ede5cda9ccedb0fe23663b410 100644 >--- a/Source/WebKit/UIProcess/API/gtk/WebKitWebView.h >+++ b/Source/WebKit/UIProcess/API/gtk/WebKitWebView.h >@@ -231,6 +231,8 @@ struct _WebKitWebViewClass { > WebKitPolicyDecisionType type); > gboolean (* permission_request) (WebKitWebView *web_view, > WebKitPermissionRequest *permission_request); >+ gboolean (* permission_check) (WebKitWebView *web_view, >+ WebKitPermissionRequest *permission_request); > void (* mouse_target_changed) (WebKitWebView *web_view, > WebKitHitTestResult *hit_test_result, > guint modifiers); >diff --git a/Source/WebKit/UIProcess/API/gtk/docs/webkit2gtk-4.0-sections.txt b/Source/WebKit/UIProcess/API/gtk/docs/webkit2gtk-4.0-sections.txt >index 8ac042007860d6045e35a7d50487e1652480f9b0..bf2b8c2c15a6cd053f752d2e0c5644cca87ed936 100644 >--- a/Source/WebKit/UIProcess/API/gtk/docs/webkit2gtk-4.0-sections.txt >+++ b/Source/WebKit/UIProcess/API/gtk/docs/webkit2gtk-4.0-sections.txt >@@ -596,6 +596,7 @@ webkit_download_get_type > WebKitPermissionRequest > webkit_permission_request_allow > webkit_permission_request_deny >+webkit_permission_request_resolve_check > > <SUBSECTION Standard> > WebKitPermissionRequestIface >diff --git a/Source/WebKit/UIProcess/API/wpe/WebKitPermissionRequest.h b/Source/WebKit/UIProcess/API/wpe/WebKitPermissionRequest.h >index 8ae7598e2ec86c548f10391993cb6c03d8414c0c..6609beeefa4bfe5cd8f8806d5df5bf167f8a8992 100644 >--- a/Source/WebKit/UIProcess/API/wpe/WebKitPermissionRequest.h >+++ b/Source/WebKit/UIProcess/API/wpe/WebKitPermissionRequest.h >@@ -42,6 +42,9 @@ struct _WebKitPermissionRequestIface { > > void (* allow) (WebKitPermissionRequest *request); > void (* deny) (WebKitPermissionRequest *request); >+ void (* resolve_check) (WebKitPermissionRequest *request, >+ const gchar* salt, >+ gboolean allowed); > }; > > WEBKIT_API GType >@@ -53,6 +56,11 @@ webkit_permission_request_allow (WebKitPermissionRequest *request); > WEBKIT_API void > webkit_permission_request_deny (WebKitPermissionRequest *request); > >+WEBKIT_API void >+webkit_permission_request_resolve_check (WebKitPermissionRequest *request, >+ const gchar* salt, >+ gboolean allowed); >+ > G_END_DECLS > > #endif >diff --git a/Source/WebKit/UIProcess/API/wpe/WebKitWebView.h b/Source/WebKit/UIProcess/API/wpe/WebKitWebView.h >index 3c02f0d907327295314fe2f009c5b20e05bf9710..a3fc3a47bfb9ad5fc112ce3b605b86821a697c9a 100644 >--- a/Source/WebKit/UIProcess/API/wpe/WebKitWebView.h >+++ b/Source/WebKit/UIProcess/API/wpe/WebKitWebView.h >@@ -197,6 +197,8 @@ struct _WebKitWebViewClass { > WebKitPolicyDecisionType type); > gboolean (* permission_request) (WebKitWebView *web_view, > WebKitPermissionRequest *permission_request); >+ gboolean (* permission_check) (WebKitWebView *web_view, >+ WebKitPermissionRequest *permission_request); > void (* mouse_target_changed) (WebKitWebView *web_view, > WebKitHitTestResult *hit_test_result, > guint modifiers); >diff --git a/Tools/ChangeLog b/Tools/ChangeLog >index 7f293dcefe335367ace20b2ebcf26a67b50a625a..85b81813e4d116a0a5a6e6edf24bb798ca0c4e27 100644 >--- a/Tools/ChangeLog >+++ b/Tools/ChangeLog >@@ -1,3 +1,20 @@ >+2018-05-18 Alejandro G. Castro <alex@igalia.com> and Thibault Saunier <tsaunier@igalia.com> >+ >+ [GTK][WPE] Add mediaDevices.enumerateDevices support >+ https://bugs.webkit.org/show_bug.cgi?id=185761 >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ Added new API test for the mediaDevices.enumerateDevices: >+ usermedia-enumeratedevices-permission-check. >+ >+ * TestWebKitAPI/Tests/WebKitGLib/TestUIClient.cpp: >+ (permissionCheck): Implemented the permission check resolution. >+ (testWebViewUserMediaEnumerateDevicesPermissionCheck): Added the >+ new test checking the enumerateDevices API when permission is >+ denied and when permission is allowed for the origin. >+ (beforeAll): Defined the new test. >+ > 2018-05-17 Nan Wang <n_wang@apple.com> > > AX: [macOS] Expose the primary screen height through AX API >diff --git a/Tools/TestWebKitAPI/Tests/WebKitGLib/TestUIClient.cpp b/Tools/TestWebKitAPI/Tests/WebKitGLib/TestUIClient.cpp >index 6939102fd318dfbe40e493be3c7a375e7cd5cc49..cfa254f24d32e3cc3a6b6cb8600a86fdd36099d6 100644 >--- a/Tools/TestWebKitAPI/Tests/WebKitGLib/TestUIClient.cpp >+++ b/Tools/TestWebKitAPI/Tests/WebKitGLib/TestUIClient.cpp >@@ -231,6 +231,25 @@ public: > return TRUE; > } > >+ static gboolean permissionCheck(WebKitWebView*, WebKitPermissionRequest* request, UIClientTest* test) >+ { >+ g_assert(WEBKIT_IS_PERMISSION_REQUEST(request)); >+ test->assertObjectIsDeletedWhenTestFinishes(G_OBJECT(request)); >+ >+ if (test->m_verifyMediaTypes && WEBKIT_IS_USER_MEDIA_PERMISSION_REQUEST(request)) { >+ WebKitUserMediaPermissionRequest* userMediaRequest = WEBKIT_USER_MEDIA_PERMISSION_REQUEST(request); >+ g_assert(!webkit_user_media_permission_is_for_audio_device(userMediaRequest)); >+ g_assert(!webkit_user_media_permission_is_for_video_device(userMediaRequest)); >+ } >+ >+ if (test->m_allowPermissionRequests) >+ webkit_permission_request_resolve_check(request, "randomHashSalt", true); >+ else >+ webkit_permission_request_resolve_check(request, "randomHashSalt", false); >+ >+ return TRUE; >+ } >+ > static void permissionResultMessageReceivedCallback(WebKitUserContentManager* userContentManager, WebKitJavascriptResult* javascriptResult, UIClientTest* test) > { > test->m_permissionResult.reset(WebViewTest::javascriptResultToCString(javascriptResult)); >@@ -251,6 +270,7 @@ public: > g_signal_connect(m_webView, "script-dialog", G_CALLBACK(scriptDialog), this); > g_signal_connect(m_webView, "mouse-target-changed", G_CALLBACK(mouseTargetChanged), this); > g_signal_connect(m_webView, "permission-request", G_CALLBACK(permissionRequested), this); >+ g_signal_connect(m_webView, "permission-check", G_CALLBACK(permissionCheck), this); > webkit_user_content_manager_register_script_message_handler(m_userContentManager.get(), "permission"); > g_signal_connect(m_userContentManager.get(), "script-message-received::permission", G_CALLBACK(permissionResultMessageReceivedCallback), this); > } >@@ -818,6 +838,47 @@ static void testWebViewGeolocationPermissionRequests(UIClientTest* test, gconstp > #endif // ENABLE(GEOLOCATION) > > #if ENABLE(MEDIA_STREAM) >+static void testWebViewUserMediaEnumerateDevicesPermissionCheck(UIClientTest* test, gconstpointer) >+{ >+ WebKitSettings* settings = webkit_web_view_get_settings(test->m_webView); >+ gboolean enabled = webkit_settings_get_enable_media_stream(settings); >+ webkit_settings_set_enable_media_stream(settings, TRUE); >+ >+#if PLATFORM(GTK) >+ test->showInWindowAndWaitUntilMapped(); >+#endif >+ static const char* userMediaRequestHTML = >+ "<html>" >+ " <script>" >+ " function runTest()" >+ " {" >+ " navigator.mediaDevices.enumerateDevices().then(" >+ " function(devices) { " >+ " devices.forEach(function(device) {" >+ " if (device.label) document.title = \"OK\";" >+ " else document.title = \"Permission denied\";" >+ " })" >+ " })" >+ " }" >+ " </script>" >+ " <body onload='runTest();'></body>" >+ "</html>"; >+ >+ test->m_verifyMediaTypes = TRUE; >+ >+ // Test denying a permission request. >+ test->m_allowPermissionRequests = false; >+ test->loadHtml(userMediaRequestHTML, nullptr); >+ test->waitUntilTitleChangedTo("Permission denied"); >+ >+ // Test allowing a permission request. >+ test->m_allowPermissionRequests = true; >+ test->loadHtml(userMediaRequestHTML, nullptr); >+ test->waitUntilTitleChangedTo("OK"); >+ >+ webkit_settings_set_enable_media_stream(settings, enabled); >+} >+ > static void testWebViewUserMediaPermissionRequests(UIClientTest* test, gconstpointer) > { > WebKitSettings* settings = webkit_web_view_get_settings(test->m_webView); >@@ -1140,6 +1201,7 @@ void beforeAll() > UIClientTest::add("WebKitWebView", "geolocation-permission-requests", testWebViewGeolocationPermissionRequests); > #endif > #if ENABLE(MEDIA_STREAM) >+ UIClientTest::add("WebKitWebView", "usermedia-enumeratedevices-permission-check", testWebViewUserMediaEnumerateDevicesPermissionCheck); > UIClientTest::add("WebKitWebView", "usermedia-permission-requests", testWebViewUserMediaPermissionRequests); > UIClientTest::add("WebKitWebView", "audio-usermedia-permission-request", testWebViewAudioOnlyUserMediaPermissionRequests); > #endif >diff --git a/LayoutTests/ChangeLog b/LayoutTests/ChangeLog >index 29cac6a6ca048553fa419ab7aad7e0939a0609b2..34310d620d8cf09236b4fcdbf594739cd874df66 100644 >--- a/LayoutTests/ChangeLog >+++ b/LayoutTests/ChangeLog >@@ -1,3 +1,23 @@ >+2018-05-18 Alejandro G. Castro <alex@igalia.com> and Thibault Saunier <tsaunier@igalia.com> >+ >+ [GTK][WPE] Add mediaDevices.enumerateDevices support >+ https://bugs.webkit.org/show_bug.cgi?id=185761 >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ Remove the Failure expectation for the test >+ media-devices-enumerate-devices.html. Just for the record we have >+ also solved the following tests, but we are not removing the >+ general skip of the mediastream directory until the next patch: >+ >+ fast/mediastream/media-devices-enumerate-devices.html >+ fast/mediastream/resources/enumerate-devices-frame.html >+ http/tests/media/media-stream/enumerate-devices-source-id.html >+ http/tests/media/media-stream/enumerate-devices-source-id-persistent.html >+ http/tests/media/media-stream/resources/enumerate-devices-source-id-frame.html >+ >+ * platform/gtk/TestExpectations: >+ > 2018-05-17 Michael Saboff <msaboff@apple.com> > > We don't throw SyntaxErrors for runtime generated regular expressions with errors >diff --git a/LayoutTests/platform/gtk/TestExpectations b/LayoutTests/platform/gtk/TestExpectations >index 8c62f2c8fe98752d41e8a9e01e2805ca4b812343..b7818a8e49acb79271718300b4f54d8c3b892e59 100644 >--- a/LayoutTests/platform/gtk/TestExpectations >+++ b/LayoutTests/platform/gtk/TestExpectations >@@ -636,7 +636,6 @@ webkit.org/b/79203 imported/w3c/web-platform-tests/mediacapture-streams/MediaStr > webkit.org/b/151344 fast/mediastream/MediaStream-add-ended-tracks.html [ Timeout ] > # Crash is bug #176801 > webkit.org/b/160996 fast/mediastream/MediaStream-video-element-video-tracks-disabled.html [ ImageOnlyFailure Crash ] >-webkit.org/b/172269 fast/mediastream/media-devices-enumerate-devices.html [ Failure ] > webkit.org/b/173257 fast/mediastream/getUserMedia-grant-persistency3.html [ Pass Failure ] > > # Canvas captureStream support is not implemented
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Flags:
mcatanzaro
:
review-
Actions:
View
|
Formatted Diff
|
Diff
Attachments on
bug 185761
:
340841
|
346900
|
351077
|
351080
|
351081
|
351526
|
351531
|
351533
|
351591
|
351952
|
352030