WebKit Bugzilla
Attachment 339721 Details for
Bug 181855
: [EME][GStreamer] Move the decryptor from AppendPipeline to PlaybackPipeline.
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
Remember
[x]
|
Forgot Password
Login:
[x]
[patch]
Patch
bug-181855-20180507171655.patch (text/plain), 19.34 KB, created by
Yacine Bandou
on 2018-05-07 08:16:54 PDT
(
hide
)
Description:
Patch
Filename:
MIME Type:
Creator:
Yacine Bandou
Created:
2018-05-07 08:16:54 PDT
Size:
19.34 KB
patch
obsolete
>Subversion Revision: 231044 >diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog >index 1873cd4e8cc2eb7d0c5a81554c341ec4728bcb67..e182b0a8cb8d7a86ed5f59a819f45fc55e10fd7e 100644 >--- a/Source/WebCore/ChangeLog >+++ b/Source/WebCore/ChangeLog >@@ -1,3 +1,53 @@ >+2018-01-24 Yacine Bandou <yacine.bandou_ext@softathome.com> >+ >+ [EME][GStreamer] Move the decryptor from AppendPipeline to PlaybackPipeline. >+ https://bugs.webkit.org/show_bug.cgi?id=181855 >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ The goal of this move is to handle the limitation of SVP (Secure Video Path) memory size. >+ >+ When the decryptor is in the AppendPipeline and we use SVP, we buffer in MediaSource queue >+ the decrypted GstBuffers that are in SVP memory. >+ This behavior cause an out-of-memory error, because we are limited in SVP memory size. >+ >+ By moving the decryptor in PlaybackPipeline, we avoid to buffer the decrypted GstBuffers >+ which use the SVP memory and we buffer the encrypted GstBuffers that are in system memory. >+ >+ This new architecture also allows to start the buffering before obtaining the DRM license >+ and it makes easier to manage dynamic change of the license or Key. >+ >+ The decryptor is auto plugged by GStreamer playbin in PlaybackPipeline. >+ >+ SVP: Secure Video Path also named trusted or protected video path, it is a memory which is >+ protected by a hardware access control engine, it is not accessible to other unauthorised >+ software or hardware components. >+ >+ Tests: >+ media/encrypted-media/clearKey/clearKey-cenc-audio-playback-mse.html >+ media/encrypted-media/clearKey/clearKey-cenc-video-playback-mse.html >+ >+ * platform/graphics/gstreamer/eme/WebKitCommonEncryptionDecryptorGStreamer.cpp: >+ (webkitMediaCommonEncryptionDecryptSinkEventHandler): >+ * platform/graphics/gstreamer/mse/AppendPipeline.cpp: >+ (WebCore::dumpAppendState): >+ (WebCore::AppendPipeline::AppendPipeline): >+ (WebCore::AppendPipeline::handleNeedContextSyncMessage): >+ (WebCore::AppendPipeline::handleAppsrcNeedDataReceived): >+ (WebCore::AppendPipeline::setAppendState): >+ (WebCore::AppendPipeline::parseDemuxerSrcPadCaps): >+ (WebCore::AppendPipeline::appsinkNewSample): >+ (WebCore::AppendPipeline::connectDemuxerSrcPadToAppsinkFromAnyThread): >+ (WebCore::AppendPipeline::disconnectDemuxerSrcPadFromAppsinkFromAnyThread): >+ (WebCore::appendPipelineElementMessageCallback): Deleted. >+ (WebCore::AppendPipeline::handleElementMessage): Deleted. >+ (WebCore::AppendPipeline::dispatchPendingDecryptionStructure): Deleted. >+ (WebCore::AppendPipeline::dispatchDecryptionStructure): Deleted. >+ * platform/graphics/gstreamer/mse/AppendPipeline.h: >+ * platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp: >+ (WebCore::MediaPlayerPrivateGStreamerMSE::attemptToDecryptWithInstance): >+ * platform/graphics/gstreamer/mse/PlaybackPipeline.cpp: >+ > 2018-01-19 Yacine Bandou <yacine.bandou_ext@softathome.com> > > [EME][GStreamer] Add a new message "decrypt-key-needed" send from the decryptor to the application. >diff --git a/Source/WebCore/platform/graphics/gstreamer/eme/WebKitCommonEncryptionDecryptorGStreamer.cpp b/Source/WebCore/platform/graphics/gstreamer/eme/WebKitCommonEncryptionDecryptorGStreamer.cpp >index 33a330aaf6462c607bf5938b0f37689b9f7860be..8d1b84e6e1b9861e4e9d385ced1d90beaa3f15a5 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/eme/WebKitCommonEncryptionDecryptorGStreamer.cpp >+++ b/Source/WebCore/platform/graphics/gstreamer/eme/WebKitCommonEncryptionDecryptorGStreamer.cpp >@@ -299,23 +299,6 @@ static gboolean webkitMediaCommonEncryptionDecryptSinkEventHandler(GstBaseTransf > gboolean result = FALSE; > > switch (GST_EVENT_TYPE(event)) { >- case GST_EVENT_PROTECTION: { >- const char* systemId = nullptr; >- >- gst_event_parse_protection(event, &systemId, nullptr, nullptr); >- GST_TRACE_OBJECT(self, "received protection event for %s", systemId); >- >- if (!g_strcmp0(systemId, klass->protectionSystemId)) { >- GST_DEBUG_OBJECT(self, "sending protection event to the pipeline"); >- gst_element_post_message(GST_ELEMENT(self), >- gst_message_new_element(GST_OBJECT(self), >- gst_structure_new("drm-key-needed", "event", GST_TYPE_EVENT, event, nullptr))); >- } >- >- gst_event_unref(event); >- result = TRUE; >- break; >- } > case GST_EVENT_CUSTOM_DOWNSTREAM_OOB: { > if (klass->handleKeyResponse(self, event)) { > GST_DEBUG_OBJECT(self, "key received"); >diff --git a/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp b/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp >index c2508c75c9f95c6fca489be9a9b6408ed8f93a2a..691f0c8ac593d4ee49ed8448a83129074d1e996c 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp >+++ b/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp >@@ -55,8 +55,6 @@ static const char* dumpAppendState(AppendPipeline::AppendState appendState) > return "NotStarted"; > case AppendPipeline::AppendState::Ongoing: > return "Ongoing"; >- case AppendPipeline::AppendState::KeyNegotiation: >- return "KeyNegotiation"; > case AppendPipeline::AppendState::DataStarve: > return "DataStarve"; > case AppendPipeline::AppendState::Sampling: >@@ -93,13 +91,6 @@ static void appendPipelineApplicationMessageCallback(GstBus*, GstMessage* messag > appendPipeline->handleApplicationMessage(message); > } > >-#if ENABLE(ENCRYPTED_MEDIA) >-static void appendPipelineElementMessageCallback(GstBus*, GstMessage* message, AppendPipeline* appendPipeline) >-{ >- appendPipeline->handleElementMessage(message); >-} >-#endif >- > static void appendPipelineStateChangeMessageCallback(GstBus*, GstMessage* message, AppendPipeline* appendPipeline) > { > appendPipeline->handleStateChangeMessage(message); >@@ -131,9 +122,6 @@ AppendPipeline::AppendPipeline(Ref<MediaSourceClientGStreamerMSE> mediaSourceCli > > g_signal_connect(m_bus.get(), "sync-message::need-context", G_CALLBACK(appendPipelineNeedContextMessageCallback), this); > g_signal_connect(m_bus.get(), "message::application", G_CALLBACK(appendPipelineApplicationMessageCallback), this); >-#if ENABLE(ENCRYPTED_MEDIA) >- g_signal_connect(m_bus.get(), "message::element", G_CALLBACK(appendPipelineElementMessageCallback), this); >-#endif > g_signal_connect(m_bus.get(), "message::state-changed", G_CALLBACK(appendPipelineStateChangeMessageCallback), this); > > // We assign the created instances here instead of adoptRef() because gst_bin_add_many() >@@ -275,9 +263,6 @@ void AppendPipeline::handleNeedContextSyncMessage(GstMessage* message) > const gchar* contextType = nullptr; > gst_message_parse_context_type(message, &contextType); > GST_TRACE("context type: %s", contextType); >- if (!g_strcmp0(contextType, "drm-preferred-decryption-system-id") >- && m_appendState != AppendPipeline::AppendState::KeyNegotiation) >- setAppendState(AppendPipeline::AppendState::KeyNegotiation); > > // MediaPlayerPrivateGStreamerBase will take care of setting up encryption. > if (m_playerPrivate) >@@ -329,25 +314,6 @@ void AppendPipeline::handleApplicationMessage(GstMessage* message) > ASSERT_NOT_REACHED(); > } > >-#if ENABLE(ENCRYPTED_MEDIA) >-void AppendPipeline::handleElementMessage(GstMessage* message) >-{ >- ASSERT(WTF::isMainThread()); >- >- const GstStructure* structure = gst_message_get_structure(message); >- GST_TRACE("%s message from %s", gst_structure_get_name(structure), GST_MESSAGE_SRC_NAME(message)); >- if (m_playerPrivate && gst_structure_has_name(structure, "drm-key-needed")) { >- if (m_appendState != AppendPipeline::AppendState::KeyNegotiation) >- setAppendState(AppendPipeline::AppendState::KeyNegotiation); >- >- GST_DEBUG("sending drm-key-needed message from %s to the player", GST_MESSAGE_SRC_NAME(message)); >- GRefPtr<GstEvent> event; >- gst_structure_get(structure, "event", GST_TYPE_EVENT, &event.outPtr(), nullptr); >- m_playerPrivate->handleProtectionEvent(event.get()); >- } >-} >-#endif >- > void AppendPipeline::handleStateChangeMessage(GstMessage* message) > { > ASSERT(WTF::isMainThread()); >@@ -373,7 +339,7 @@ void AppendPipeline::handleAppsrcNeedDataReceived() > return; > } > >- ASSERT(m_appendState == AppendState::KeyNegotiation || m_appendState == AppendState::Ongoing || m_appendState == AppendState::Sampling); >+ ASSERT(m_appendState == AppendState::Ongoing || m_appendState == AppendState::Sampling); > ASSERT(!m_appsrcNeedDataReceived); > > GST_TRACE("received need-data from appsrc"); >@@ -431,8 +397,7 @@ void AppendPipeline::setAppendState(AppendState newAppendState) > // NotStarted-->Ongoing-->DataStarve-->NotStarted > // | | `->Aborting-->NotStarted > // | `->Sampling-···->Sampling-->LastSample-->NotStarted >- // | | `->Aborting-->NotStarted >- // | `->KeyNegotiation-->Ongoing-->[...] >+ // | `->Aborting-->NotStarted > // `->Aborting-->NotStarted > AppendState oldAppendState = m_appendState; > AppendState nextAppendState = AppendState::Invalid; >@@ -468,19 +433,8 @@ void AppendPipeline::setAppendState(AppendState newAppendState) > break; > } > break; >- case AppendState::KeyNegotiation: >- switch (newAppendState) { >- case AppendState::Ongoing: >- case AppendState::Invalid: >- ok = true; >- break; >- default: >- break; >- } >- break; > case AppendState::Ongoing: > switch (newAppendState) { >- case AppendState::KeyNegotiation: > case AppendState::Sampling: > case AppendState::Invalid: > ok = true; >@@ -582,18 +536,7 @@ void AppendPipeline::parseDemuxerSrcPadCaps(GstCaps* demuxerSrcPadCaps) > > m_demuxerSrcPadCaps = adoptGRef(demuxerSrcPadCaps); > m_streamType = WebCore::MediaSourceStreamTypeGStreamer::Unknown; >-#if ENABLE(ENCRYPTED_MEDIA) >- if (areEncryptedCaps(m_demuxerSrcPadCaps.get())) { >- // Any previous decryptor should have been removed from the pipeline by disconnectFromAppSinkFromStreamingThread() >- ASSERT(!m_decryptor); >- GstStructure* structure = gst_caps_get_structure(m_demuxerSrcPadCaps.get(), 0); >- m_decryptor = GStreamerEMEUtilities::createDecryptor(gst_structure_get_string(structure, "protection-system")); >- if (!m_decryptor) { >- GST_ERROR("decryptor not found for caps: %" GST_PTR_FORMAT, m_demuxerSrcPadCaps.get()); >- return; >- } >- } >-#endif >+ > const char* originalMediaType = capsMediaType(m_demuxerSrcPadCaps.get()); > if (!MediaPlayerPrivateGStreamerMSE::supportsCodec(originalMediaType)) { > m_presentationSize = WebCore::FloatSize(); >@@ -673,10 +616,6 @@ void AppendPipeline::appsinkNewSample(GstSample* sample) > { > LockHolder locker(m_newSampleLock); > >- // If we were in KeyNegotiation but samples are coming, assume we're already OnGoing >- if (m_appendState == AppendState::KeyNegotiation) >- setAppendState(AppendState::Ongoing); >- > // Ignore samples if we're not expecting them. Refuse processing if we're in Invalid state. > if (m_appendState != AppendState::Ongoing && m_appendState != AppendState::Sampling) { > GST_WARNING("Unexpected sample, appendState=%s", dumpAppendState(m_appendState)); >@@ -1009,28 +948,10 @@ void AppendPipeline::connectDemuxerSrcPadToAppsinkFromAnyThread(GstPad* demuxerS > currentSrcPad = parserSrcPad; > } > >-#if ENABLE(ENCRYPTED_MEDIA) >- if (m_decryptor) { >- gst_object_ref(m_decryptor.get()); >- gst_bin_add(GST_BIN(m_pipeline.get()), m_decryptor.get()); >- gst_element_sync_state_with_parent(m_decryptor.get()); >- >- GRefPtr<GstPad> decryptorSinkPad = adoptGRef(gst_element_get_static_pad(m_decryptor.get(), "sink")); >- GRefPtr<GstPad> decryptorSrcPad = adoptGRef(gst_element_get_static_pad(m_decryptor.get(), "src")); >- >- gst_pad_link(currentSrcPad.get(), decryptorSinkPad.get()); >- currentSrcPad = decryptorSrcPad; >- } >-#endif >- > gst_pad_link(currentSrcPad.get(), appsinkSinkPad.get()); > > gst_element_sync_state_with_parent(m_appsink.get()); > >-#if ENABLE(ENCRYPTED_MEDIA) >- if (m_pendingDecryptionStructure) >- dispatchPendingDecryptionStructure(); >-#endif > gst_element_set_state(m_pipeline.get(), GST_STATE_PAUSED); > gst_element_sync_state_with_parent(m_appsink.get()); > >@@ -1109,14 +1030,6 @@ void AppendPipeline::disconnectDemuxerSrcPadFromAppsinkFromAnyThread(GstPad*) > > GST_DEBUG("Disconnecting appsink"); > >-#if ENABLE(ENCRYPTED_MEDIA) >- if (m_decryptor) { >- gst_element_set_state(m_decryptor.get(), GST_STATE_NULL); >- gst_bin_remove(GST_BIN(m_pipeline.get()), m_decryptor.get()); >- m_decryptor = nullptr; >- } >-#endif >- > if (m_parser) { > gst_element_set_state(m_parser.get(), GST_STATE_NULL); > gst_bin_remove(GST_BIN(m_pipeline.get()), m_parser.get()); >@@ -1126,35 +1039,6 @@ void AppendPipeline::disconnectDemuxerSrcPadFromAppsinkFromAnyThread(GstPad*) > GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(GST_BIN(m_pipeline.get()), GST_DEBUG_GRAPH_SHOW_ALL, "pad-removed-after"); > } > >-#if ENABLE(ENCRYPTED_MEDIA) >-void AppendPipeline::dispatchPendingDecryptionStructure() >-{ >- ASSERT(m_decryptor); >- ASSERT(m_pendingDecryptionStructure); >- ASSERT(m_appendState == AppendState::KeyNegotiation); >- GST_TRACE("dispatching key to append pipeline %p", this); >- >- // Release the m_pendingDecryptionStructure object since >- // gst_event_new_custom() takes over ownership of it. >- gst_element_send_event(m_pipeline.get(), gst_event_new_custom(GST_EVENT_CUSTOM_DOWNSTREAM_OOB, m_pendingDecryptionStructure.release())); >- >- setAppendState(AppendState::Ongoing); >-} >- >-void AppendPipeline::dispatchDecryptionStructure(GUniquePtr<GstStructure>&& structure) >-{ >- if (m_appendState == AppendState::KeyNegotiation) { >- GST_TRACE("append pipeline %p in key negotiation", this); >- m_pendingDecryptionStructure = WTFMove(structure); >- if (m_decryptor) >- dispatchPendingDecryptionStructure(); >- else >- GST_TRACE("no decryptor yet, waiting for it"); >- } else >- GST_TRACE("append pipeline %p not in key negotiation", this); >-} >-#endif >- > static void appendPipelineAppsinkCapsChanged(GObject* appsinkPad, GParamSpec*, AppendPipeline* appendPipeline) > { > GstStructure* structure = gst_structure_new_empty("appsink-caps-changed"); >diff --git a/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.h b/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.h >index 83304661b482a9cb927d3c70ee0a749d8953240e..995c12975d94ec4c007ffeb2d79cfa7a45f53894 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.h >+++ b/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.h >@@ -42,7 +42,7 @@ struct PadProbeInformation { > > class AppendPipeline : public ThreadSafeRefCounted<AppendPipeline> { > public: >- enum class AppendState { Invalid, NotStarted, Ongoing, KeyNegotiation, DataStarve, Sampling, LastSample, Aborting }; >+ enum class AppendState { Invalid, NotStarted, Ongoing, DataStarve, Sampling, LastSample, Aborting }; > > AppendPipeline(Ref<MediaSourceClientGStreamerMSE>, Ref<SourceBufferPrivateGStreamer>, MediaPlayerPrivateGStreamerMSE&); > virtual ~AppendPipeline(); >@@ -50,9 +50,6 @@ public: > void handleNeedContextSyncMessage(GstMessage*); > void handleApplicationMessage(GstMessage*); > void handleStateChangeMessage(GstMessage*); >-#if ENABLE(ENCRYPTED_MEDIA) >- void handleElementMessage(GstMessage*); >-#endif > > gint id(); > AppendState appendState() { return m_appendState; } >@@ -60,9 +57,6 @@ public: > > GstFlowReturn handleNewAppsinkSample(GstElement*); > GstFlowReturn pushNewBuffer(GstBuffer*); >-#if ENABLE(ENCRYPTED_MEDIA) >- void dispatchDecryptionStructure(GUniquePtr<GstStructure>&&); >-#endif > > // Takes ownership of caps. > void parseDemuxerSrcPadCaps(GstCaps*); >@@ -98,9 +92,6 @@ private: > void handleAppsrcNeedDataReceived(); > void removeAppsrcDataLeavingProbe(); > void setAppsrcDataLeavingProbe(); >-#if ENABLE(ENCRYPTED_MEDIA) >- void dispatchPendingDecryptionStructure(); >-#endif > > Ref<MediaSourceClientGStreamerMSE> m_mediaSourceClient; > Ref<SourceBufferPrivateGStreamer> m_sourceBufferPrivate; >@@ -118,9 +109,6 @@ private: > GRefPtr<GstElement> m_appsrc; > GRefPtr<GstElement> m_demux; > GRefPtr<GstElement> m_parser; // Optional. >-#if ENABLE(ENCRYPTED_MEDIA) >- GRefPtr<GstElement> m_decryptor; >-#endif > // The demuxer has one src stream only, so only one appsink is needed and linked to it. > GRefPtr<GstElement> m_appsink; > >@@ -155,9 +143,6 @@ private: > RefPtr<WebCore::TrackPrivateBase> m_track; > > GRefPtr<GstBuffer> m_pendingBuffer; >-#if ENABLE(ENCRYPTED_MEDIA) >- GUniquePtr<GstStructure> m_pendingDecryptionStructure; >-#endif > }; > > } // namespace WebCore. >diff --git a/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp b/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp >index a30811f658223467babb2af6501eb08c169364eb..65a1ba7e2fcd0990bc3fa8360e59511463b48cd3 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp >+++ b/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp >@@ -955,8 +955,7 @@ void MediaPlayerPrivateGStreamerMSE::attemptToDecryptWithInstance(CDMInstance& i > gst_structure_set_value(structure.get(), "key-ids", &keyIDList); > gst_structure_set_value(structure.get(), "key-values", &keyValueList); > >- for (auto it : m_appendPipelinesMap) >- it.value->dispatchDecryptionStructure(GUniquePtr<GstStructure>(gst_structure_copy(structure.get()))); >+ gst_element_send_event(m_playbackPipeline->pipeline(), gst_event_new_custom(GST_EVENT_CUSTOM_DOWNSTREAM_OOB, structure.release())); > } > } > #endif >diff --git a/Source/WebCore/platform/graphics/gstreamer/mse/PlaybackPipeline.cpp b/Source/WebCore/platform/graphics/gstreamer/mse/PlaybackPipeline.cpp >index 06d5862c5063dbde8d0295c0a0e69c395085e576..d15e8addc45c08d35776ae6a4fb790a8f94d060b 100644 >--- a/Source/WebCore/platform/graphics/gstreamer/mse/PlaybackPipeline.cpp >+++ b/Source/WebCore/platform/graphics/gstreamer/mse/PlaybackPipeline.cpp >@@ -180,7 +180,10 @@ void PlaybackPipeline::attachTrack(RefPtr<SourceBufferPrivateGStreamer> sourceBu > > GUniquePtr<gchar> parserBinName(g_strdup_printf("streamparser%u", padId)); > >- if (!g_strcmp0(mediaType, "video/x-h264")) { >+ if (areEncryptedCaps(caps)) { >+ GST_DEBUG("It's encrypted content, parsers are not needed before decrypting the content"); >+ stream->parser = nullptr; >+ } else if (!g_strcmp0(mediaType, "video/x-h264")) { > GRefPtr<GstCaps> filterCaps = adoptGRef(gst_caps_new_simple("video/x-h264", "alignment", G_TYPE_STRING, "au", nullptr)); > GstElement* capsfilter = gst_element_factory_make("capsfilter", nullptr); > g_object_set(capsfilter, "caps", filterCaps.get(), nullptr);
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Formatted Diff
|
Diff
Attachments on
bug 181855
:
332158
|
333486
|
333487
|
333490
|
338892
|
339486
|
339540
|
339542
| 339721 |
339727