Bug 31673 – [Qt] need an API to suspend and resume active Javascript DOM objects.

• Component: New Bugs
• Status: RESOLVED INVALID

[Yongjun Zhang]

QtWebKit client would be interested to suspend/resume JavaScript DOM objects in some cases.  For example, in mobile, the client can select to suspend an invisible long running QWebPage to save battery energy and network bandwidth, and resume it when it comes to the foreground.  In WebCore, ScriptExecutionContext::suspendDOMActiveObjects()/resumeDOMActiveObjects() are designed for this purpose.  It would be nice to have an API in QWebFrame to expose the feature.

[Simon Hausmann]

We had a long discussion on IRC about all sorts of suspension mechanisms (from suspending the entire process to being able to suspend individual animations or scripts).

Unfortunately we did not reach a conclusion.

My guts feeling is that we may be able to define "active objects" in the documentation as a combination of active DOM objects (xhr, etc. - DOMActiveObject subclasses in WebCore) and animated graphical objects visible in the viewport. Then we could wrap that behind suspendActiveObjects() and resumeActiveObjects().

[Yongjun Zhang]

(In reply to comment #1)
> We had a long discussion on IRC about all sorts of suspension mechanisms (from
> suspending the entire process to being able to suspend individual animations or
> scripts).
> 
> Unfortunately we did not reach a conclusion.
> 
> My guts feeling is that we may be able to define "active objects" in the
> documentation as a combination of active DOM objects (xhr, etc. -
> DOMActiveObject subclasses in WebCore) and animated graphical objects visible
> in the viewport. Then we could wrap that behind suspendActiveObjects() and
> resumeActiveObjects().

Thanks, Simon.

My apologies that I missed the IRC weekly meeting - I was thing it was 8:00 AM Boston time ;)

I think originally the idea is to suspend DOMTimer which initialized by JS setTimeOut(). Certainly, it wouldn't hurt if we add options to suspend animated GIFs and plugins in this API.

[Simon Hausmann]

This seems to be related to the following requirement: http://bugreports.qt.nokia.com/browse/QTWEBKIT-6

[Yongjun Zhang]

Created attachment 45073 [details]
first attempt to introduce the suspend/resume DOM objects API.

the patch only duplicates the names from ScriptExcutionContext.

[Laszlo Gombos]

I'm not sure if 4.6.1 is the correct version info. 

I would also think that we want to suspend more than just DOMTimers.

[Laszlo Gombos]

Created attachment 45918 [details]
private API prepared to merge to 4.6

[Kenneth Rohde Christiansen]

(In reply to comment #6)
> Created an attachment (id=45918) [details]
> private API prepared to merge to 4.6

LGTM, r=me if you want it committed.

Jesus is investigating the use-cases and should come up with a nice API for suspending/resuming DOM objects, GIF animations etc.

[Laszlo Gombos]

Created attachment 45919 [details]
private API prepared to merge to 4.6 - 2nd try

[Laszlo Gombos]

We might want to consider committing https://bugs.webkit.org/attachment.cgi?id=45919 to the trunk as well, just to keep the trunk in sync with 4.6.

[Laszlo Gombos]

(In reply to comment #4)
> Created an attachment (id=45073) [details]
> first attempt to introduce the suspend/resume DOM objects API.
> 
> the patch only duplicates the names from ScriptExcutionContext.

Yongjun, for the proper public API solution for the trunk, we will also need API tests.

[Kenneth Rohde Christiansen]

(In reply to comment #1)
> We had a long discussion on IRC about all sorts of suspension mechanisms (from
> suspending the entire process to being able to suspend individual animations or
> scripts).
> 
> Unfortunately we did not reach a conclusion.
> 
> My guts feeling is that we may be able to define "active objects" in the
> documentation as a combination of active DOM objects (xhr, etc. -
> DOMActiveObject subclasses in WebCore) and animated graphical objects visible
> in the viewport. Then we could wrap that behind suspendActiveObjects() and
> resumeActiveObjects().

Hi, let me chime in. Basically, I believe that we only have one use case for the moment; not wasting CPU cycles for pages/"dashboard like widgets" that are not visible to the user.

Disabling the entire process might not be good as it might break loading.

It seems that we do not need fine-grained API that makes it possible to suspend/resume individual active objects such as CSS animations, GIF animations, Script execution on objects etc, and as such I support the suggested suspend/resumeActiveObjects API with proper documentation.

If we in the future need more fine-grained API, that can easily be added without making the suspend/resumeActiveObjects obsolete.

[Jesus Sanchez-Palencia]

I've been investigating the use cases and discussing with Kenneth about a good API, but I'd like to have some more input from you guys.

So, I think that we could define Active DOM Objects as Plugins, JavaScript, Animated GIFs, DOM events, CSS animations, DOMActiveObject subclasses and, as Simons stated, wrap them behind the API. Is there anything else (active objects, non-active objects) that we should be able to suspend/resume by now?

Kenneth and I were discussing, but we didn't reach a conclusion about all possible use-cases. Do we want to always stop the execution of all active DOM objects or do we want to have "options"? IMHO, it makes sense to stop everything for the use case that a browser is running on background, for instance, but it also makes sense to be able to suspend scripts only. I'm not sure about the other DOM Objects (animated GIFs, CSS animations, etc), though.

About the API, I think that we could have methods (suspend/resume) on QWebFrame, as Yongjun Zhang proposed, but I'm also willing to have a convenience API on QWebPage so we can suspend/resume the execution of all its sub-frames, in a more correct way.

[Kenneth Rohde Christiansen]

> So, I think that we could define Active DOM Objects as Plugins, JavaScript,
> Animated GIFs, DOM events, CSS animations, DOMActiveObject subclasses and, as
> Simons stated, wrap them behind the API. Is there anything else (active
> objects, non-active objects) that we should be able to suspend/resume by now?

I guess that the ScriptExecutionContext::suspend... takes care of the DOM events, JS Objects etc. so I guess we only have plugins, GIF/(APNG?..) animations, and CSS animations.
  
> About the API, I think that we could have methods (suspend/resume) on
> QWebFrame, as Yongjun Zhang proposed, but I'm also willing to have a
> convenience API on QWebPage so we can suspend/resume the execution of all its
> sub-frames, in a more correct way.

I guess we need it on the QWebFrame in order to be able to use it for "dashboard like widgets" etc and as it is a specialized method that will not be used that much by the general public, I guess we could leave it on the QWebFrame.

[Antonio Gomes]

> I guess that the ScriptExecutionContext::suspend... takes care of the DOM
> events, JS Objects etc. so I guess we only have plugins, GIF/(APNG?..)
> animations, and CSS animations.

I'd add SMIL to the list.

> > About the API, I think that we could have methods (suspend/resume) on
> > QWebFrame, as Yongjun Zhang proposed, but I'm also willing to have a
> > convenience API on QWebPage so we can suspend/resume the execution of all its
> > sub-frames, in a more correct way.

I like have it per-qwebpage too, hence pausing main and sub frame at once, if one wants so.

[Kenneth Rohde Christiansen]

(In reply to comment #14)
> > I guess that the ScriptExecutionContext::suspend... takes care of the DOM
> > events, JS Objects etc. so I guess we only have plugins, GIF/(APNG?..)
> > animations, and CSS animations.
> 
> I'd add SMIL to the list.
> 
> > > About the API, I think that we could have methods (suspend/resume) on
> > > QWebFrame, as Yongjun Zhang proposed, but I'm also willing to have a
> > > convenience API on QWebPage so we can suspend/resume the execution of all its
> > > sub-frames, in a more correct way.
> 
> I like have it per-qwebpage too, hence pausing main and sub frame at once, if
> one wants so.

Of course, the API should include subframes. like if you suspend active objects for a frame, it will suspend for all subframes, and likewise for resume, thus the only difference becomes calling page->mainFrame()->suspend...() instead of page->suspend...().

[Jesus Sanchez-Palencia]

(In reply to comment #2)
> (In reply to comment #1)
> > We had a long discussion on IRC about all sorts of suspension mechanisms 
> I think originally the idea is to suspend DOMTimer which initialized by JS
> setTimeOut(). Certainly, it wouldn't hurt if we add options to suspend animated
> GIFs and plugins in this API.

You should note that according to the comments in bindings/js/JSDOMBinding.cpp, JS objects created by setTimeout do not have JS wrappers and are, AFAICT, not included in the ActiveDOMObjects and thus, not suspended at all.

Have you verified this? The comments even have a "// FIXME: perhaps need to make sure even timers have a markable 'wrapper'.".

Just check markActiveObjectsForContext for more info.

[Jesus Sanchez-Palencia]

(In reply to comment #16)
> You should note that according to the comments in bindings/js/JSDOMBinding.cpp,
> JS objects created by setTimeout do not have JS wrappers and are, AFAICT, not
> included in the ActiveDOMObjects and thus, not suspended at all.
> 
> Have you verified this? The comments even have a "// FIXME: perhaps need to
> make sure even timers have a markable 'wrapper'.".
> 
> Just check markActiveObjectsForContext for more info.

Ok, just found in WebCore/page/DOMTimer.cpp:87  :
// DOMTimer constructor links the new timer into a list of ActiveDOMObjects held by the 'context'.

And looking at the constructor, line 64, found: scriptExecutionContext()->addTimeout(m_timeoutId, this);
Taking a look at the commit 177db693, which introduced this to DOMTimer, I can just conclude that the previous comment that I stated is no longer valid. Patching it right now. (and we are back to the original topic)

[Laszlo Gombos]

(In reply to comment #9)
> We might want to consider committing
> https://bugs.webkit.org/attachment.cgi?id=45919 to the trunk as well, just to
> keep the trunk in sync with 4.6.

The patch will need changes in the Symbian def files as well.

[Simon Hausmann]

(In reply to comment #18)
> (In reply to comment #9)
> > We might want to consider committing
> > https://bugs.webkit.org/attachment.cgi?id=45919 to the trunk as well, just to
> > keep the trunk in sync with 4.6.
> 
> The patch will need changes in the Symbian def files as well.

Yep. I'll submit the change into the trunk, cherry-pick it into qtwebkit-4.6, update the def files there (weekly?), submit them also back into the trunk.

Once we remove the private API we mark those symbols as absent in the def files.

[Simon Hausmann]

Landed in r52914 in the trunk and cherry-picked as 7f2f0abd29e5363012b24c2465226e8ec2f33183 into qtwebkit-4.6

[Kenneth Rohde Christiansen]

(In reply to comment #20)
> Landed in r52914 in the trunk and cherry-picked as
> 7f2f0abd29e5363012b24c2465226e8ec2f33183 into qtwebkit-4.6

Is this enough for the JIRA task? What is holding us from settling on a public API and add autotests?

[Simon Hausmann]

Benjamin, Tor Arne: What do you think about the currently proposed names?

QWebFrame::suspendActiveDOMObjects
QWebFrame::resumeActiveDOMObjects

(see also Jesus' analysis)

[Tor Arne Vestbø]

(In reply to comment #12)
> So, I think that we could define Active DOM Objects as Plugins, JavaScript,
> Animated GIFs, DOM events, CSS animations, DOMActiveObject subclasses and, as
> Simons stated, wrap them behind the API. Is there anything else (active
> objects, non-active objects) that we should be able to suspend/resume by now?

Does that mean frame->document()->suspendActiveDOMObjects() will include all those types of activities you mentioned, including plugins, animations, etc, or do we have to do some additional suspending of our own?

API-naming-vise I don't like "Active" as a prefix, as it somehow implies that those objects that are currently inactive are not suspended. What if they then wake up/become active, are they then not part of the group of suspended objects?

I also dislike "DOMObjects" as a catch-all representation of the classes you mentioned above, can we find something more descriptive? Or perhaps do we even need a postfix?

QWebFrame::suspend()
QWebFrame::resume()


?

[Jesus Sanchez-Palencia]

(In reply to comment #23)
> Does that mean frame->document()->suspendActiveDOMObjects() will include all
> those types of activities you mentioned, including plugins, animations, etc, or
> do we have to do some additional suspending of our own?

My feelings are that our API should include everything. All Active DOM Objects, at least. I'm not sure right now if animated GIFs and CSS animations are ActiveDOMObjects, but I've already checked the others. XHR inherits ActiveDOMObject but it doesn't re-implement ActiveDOMObject::suspend() and ActiveDOMObject::resume() and, therefore, wouldn't be suspended/resumed. Maybe I should open a new bug for this?

> 
> API-naming-vise I don't like "Active" as a prefix, as it somehow implies that
> those objects that are currently inactive are not suspended. What if they then
> wake up/become active, are they then not part of the group of suspended
> objects?
> 
> I also dislike "DOMObjects" as a catch-all representation of the classes you
> mentioned above, can we find something more descriptive? Or perhaps do we even
> need a postfix?
> 
> QWebFrame::suspend()
> QWebFrame::resume()

Ok, I got it. Let me propose 4 API's with pros and cons:

- QWebFrame::suspendActiveDOMObjects() and QWebFrame::resumeActiveDOMObjects()
It's clear enough, but too long and can confuse because of the inactive objects that we also need to suspend.

- QWebFrame::suspend() and QWebFrame::resume()
Seems nice, but when I suspend all DOMObjects of a frame can I still click on hyperlinks? If so, the frame isn't completely suspended and the API would be inconsistent, IMHO. But yes, I have to check this.

- QWebFrame::suspendObjectsExecution() and QWebFrame::resumeObjectsExecution()
It's not too long and does not use Active or DOM as prefix or postfix.

And a brand new idea (thanks, Kenneth):
- QWebFrame::setExecutionMode(ExecutionMode)
Where 'ExecutionMode' could be 'Default', 'PowerSaving', etc.
This is consistent with this bug's description and can be extended when more options and use-cases arrive (i.e.: if someday we need to just suspend scripts, animations, etc)

[Sriram Neelakandan]

I think the bug mentions about everything that consumes CPU-cycles/battery-life when in background .. 

--> JS Timers, DOM Events, Plugins, CSS animations, GIF animations

What about HTML5 audio/video playback ?

[Allan Sandfeld Jensen]

WebKit2 now has an API like this between WebProcess and UIProcess. It is currently called suspendActiveDOMObjectsAndAnimations(), since that is what it does.

[Jocelyn Turcotte]

=== Bulk closing of Qt bugs ===

If you believe that this bug report is still relevant for a non-Qt port of webkit.org, please re-open it and remove [Qt] from the summary.

If you believe that this is still an important QtWebKit bug, please fill a new report at https://bugreports.qt-project.org and add a link to this issue. See http://qt-project.org/wiki/ReportingBugsInQt for additional guidelines.