Bug 31673 - [Qt] need an API to suspend and resume active Javascript DOM objects.
Summary: [Qt] need an API to suspend and resume active Javascript DOM objects.
Status: RESOLVED INVALID
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P3 Normal
Assignee: Yongjun Zhang
URL:
Keywords: Qt
Depends on:
Blocks:
 
Reported: 2009-11-19 10:03 PST by Yongjun Zhang
Modified: 2014-02-03 03:15 PST (History)
11 users (show)

See Also:


Attachments
first attempt to introduce the suspend/resume DOM objects API. (2.22 KB, patch)
2009-12-17 08:27 PST, Yongjun Zhang
no flags Details | Formatted Diff | Diff
private API prepared to merge to 4.6 (1.68 KB, patch)
2010-01-05 12:14 PST, Laszlo Gombos
no flags Details | Formatted Diff | Diff
private API prepared to merge to 4.6 - 2nd try (1.77 KB, patch)
2010-01-05 12:30 PST, Laszlo Gombos
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Yongjun Zhang 2009-11-19 10:03:03 PST
QtWebKit client would be interested to suspend/resume JavaScript DOM objects in some cases.  For example, in mobile, the client can select to suspend an invisible long running QWebPage to save battery energy and network bandwidth, and resume it when it comes to the foreground.  In WebCore, ScriptExecutionContext::suspendDOMActiveObjects()/resumeDOMActiveObjects() are designed for this purpose.  It would be nice to have an API in QWebFrame to expose the feature.
Comment 1 Simon Hausmann 2009-11-23 08:23:03 PST
We had a long discussion on IRC about all sorts of suspension mechanisms (from suspending the entire process to being able to suspend individual animations or scripts).

Unfortunately we did not reach a conclusion.

My guts feeling is that we may be able to define "active objects" in the documentation as a combination of active DOM objects (xhr, etc. - DOMActiveObject subclasses in WebCore) and animated graphical objects visible in the viewport. Then we could wrap that behind suspendActiveObjects() and resumeActiveObjects().
Comment 2 Yongjun Zhang 2009-11-23 10:39:55 PST
(In reply to comment #1)
> We had a long discussion on IRC about all sorts of suspension mechanisms (from
> suspending the entire process to being able to suspend individual animations or
> scripts).
> 
> Unfortunately we did not reach a conclusion.
> 
> My guts feeling is that we may be able to define "active objects" in the
> documentation as a combination of active DOM objects (xhr, etc. -
> DOMActiveObject subclasses in WebCore) and animated graphical objects visible
> in the viewport. Then we could wrap that behind suspendActiveObjects() and
> resumeActiveObjects().

Thanks, Simon.

My apologies that I missed the IRC weekly meeting - I was thing it was 8:00 AM Boston time ;)

I think originally the idea is to suspend DOMTimer which initialized by JS setTimeOut(). Certainly, it wouldn't hurt if we add options to suspend animated GIFs and plugins in this API.
Comment 3 Simon Hausmann 2009-11-25 05:17:52 PST
This seems to be related to the following requirement: http://bugreports.qt.nokia.com/browse/QTWEBKIT-6
Comment 4 Yongjun Zhang 2009-12-17 08:27:01 PST
Created attachment 45073 [details]
first attempt to introduce the suspend/resume DOM objects API.

the patch only duplicates the names from ScriptExcutionContext.
Comment 5 Laszlo Gombos 2009-12-17 12:22:17 PST
I'm not sure if 4.6.1 is the correct version info. 

I would also think that we want to suspend more than just DOMTimers.
Comment 6 Laszlo Gombos 2010-01-05 12:14:22 PST
Created attachment 45918 [details]
private API prepared to merge to 4.6
Comment 7 Kenneth Rohde Christiansen 2010-01-05 12:17:27 PST
(In reply to comment #6)
> Created an attachment (id=45918) [details]
> private API prepared to merge to 4.6

LGTM, r=me if you want it committed.

Jesus is investigating the use-cases and should come up with a nice API for suspending/resuming DOM objects, GIF animations etc.
Comment 8 Laszlo Gombos 2010-01-05 12:30:55 PST
Created attachment 45919 [details]
private API prepared to merge to 4.6 - 2nd try
Comment 9 Laszlo Gombos 2010-01-05 12:36:46 PST
We might want to consider committing https://bugs.webkit.org/attachment.cgi?id=45919 to the trunk as well, just to keep the trunk in sync with 4.6.
Comment 10 Laszlo Gombos 2010-01-05 12:37:08 PST
(In reply to comment #4)
> Created an attachment (id=45073) [details]
> first attempt to introduce the suspend/resume DOM objects API.
> 
> the patch only duplicates the names from ScriptExcutionContext.

Yongjun, for the proper public API solution for the trunk, we will also need API tests.
Comment 11 Kenneth Rohde Christiansen 2010-01-06 05:39:43 PST
(In reply to comment #1)
> We had a long discussion on IRC about all sorts of suspension mechanisms (from
> suspending the entire process to being able to suspend individual animations or
> scripts).
> 
> Unfortunately we did not reach a conclusion.
> 
> My guts feeling is that we may be able to define "active objects" in the
> documentation as a combination of active DOM objects (xhr, etc. -
> DOMActiveObject subclasses in WebCore) and animated graphical objects visible
> in the viewport. Then we could wrap that behind suspendActiveObjects() and
> resumeActiveObjects().

Hi, let me chime in. Basically, I believe that we only have one use case for the moment; not wasting CPU cycles for pages/"dashboard like widgets" that are not visible to the user.

Disabling the entire process might not be good as it might break loading.

It seems that we do not need fine-grained API that makes it possible to suspend/resume individual active objects such as CSS animations, GIF animations, Script execution on objects etc, and as such I support the suggested suspend/resumeActiveObjects API with proper documentation.

If we in the future need more fine-grained API, that can easily be added without making the suspend/resumeActiveObjects obsolete.
Comment 12 Jesus Sanchez-Palencia 2010-01-06 05:48:42 PST
I've been investigating the use cases and discussing with Kenneth about a good API, but I'd like to have some more input from you guys.

So, I think that we could define Active DOM Objects as Plugins, JavaScript, Animated GIFs, DOM events, CSS animations, DOMActiveObject subclasses and, as Simons stated, wrap them behind the API. Is there anything else (active objects, non-active objects) that we should be able to suspend/resume by now?

Kenneth and I were discussing, but we didn't reach a conclusion about all possible use-cases. Do we want to always stop the execution of all active DOM objects or do we want to have "options"? IMHO, it makes sense to stop everything for the use case that a browser is running on background, for instance, but it also makes sense to be able to suspend scripts only. I'm not sure about the other DOM Objects (animated GIFs, CSS animations, etc), though.

About the API, I think that we could have methods (suspend/resume) on QWebFrame, as Yongjun Zhang proposed, but I'm also willing to have a convenience API on QWebPage so we can suspend/resume the execution of all its sub-frames, in a more correct way.
Comment 13 Kenneth Rohde Christiansen 2010-01-06 05:56:08 PST
> So, I think that we could define Active DOM Objects as Plugins, JavaScript,
> Animated GIFs, DOM events, CSS animations, DOMActiveObject subclasses and, as
> Simons stated, wrap them behind the API. Is there anything else (active
> objects, non-active objects) that we should be able to suspend/resume by now?

I guess that the ScriptExecutionContext::suspend... takes care of the DOM events, JS Objects etc. so I guess we only have plugins, GIF/(APNG?..) animations, and CSS animations.
  
> About the API, I think that we could have methods (suspend/resume) on
> QWebFrame, as Yongjun Zhang proposed, but I'm also willing to have a
> convenience API on QWebPage so we can suspend/resume the execution of all its
> sub-frames, in a more correct way.

I guess we need it on the QWebFrame in order to be able to use it for "dashboard like widgets" etc and as it is a specialized method that will not be used that much by the general public, I guess we could leave it on the QWebFrame.
Comment 14 Antonio Gomes 2010-01-06 06:13:47 PST
> I guess that the ScriptExecutionContext::suspend... takes care of the DOM
> events, JS Objects etc. so I guess we only have plugins, GIF/(APNG?..)
> animations, and CSS animations.

I'd add SMIL to the list.

> > About the API, I think that we could have methods (suspend/resume) on
> > QWebFrame, as Yongjun Zhang proposed, but I'm also willing to have a
> > convenience API on QWebPage so we can suspend/resume the execution of all its
> > sub-frames, in a more correct way.

I like have it per-qwebpage too, hence pausing main and sub frame at once, if one wants so.
Comment 15 Kenneth Rohde Christiansen 2010-01-06 06:25:21 PST
(In reply to comment #14)
> > I guess that the ScriptExecutionContext::suspend... takes care of the DOM
> > events, JS Objects etc. so I guess we only have plugins, GIF/(APNG?..)
> > animations, and CSS animations.
> 
> I'd add SMIL to the list.
> 
> > > About the API, I think that we could have methods (suspend/resume) on
> > > QWebFrame, as Yongjun Zhang proposed, but I'm also willing to have a
> > > convenience API on QWebPage so we can suspend/resume the execution of all its
> > > sub-frames, in a more correct way.
> 
> I like have it per-qwebpage too, hence pausing main and sub frame at once, if
> one wants so.

Of course, the API should include subframes. like if you suspend active objects for a frame, it will suspend for all subframes, and likewise for resume, thus the only difference becomes calling page->mainFrame()->suspend...() instead of page->suspend...().
Comment 16 Jesus Sanchez-Palencia 2010-01-06 11:00:38 PST
(In reply to comment #2)
> (In reply to comment #1)
> > We had a long discussion on IRC about all sorts of suspension mechanisms 
> I think originally the idea is to suspend DOMTimer which initialized by JS
> setTimeOut(). Certainly, it wouldn't hurt if we add options to suspend animated
> GIFs and plugins in this API.

You should note that according to the comments in bindings/js/JSDOMBinding.cpp, JS objects created by setTimeout do not have JS wrappers and are, AFAICT, not included in the ActiveDOMObjects and thus, not suspended at all.

Have you verified this? The comments even have a "// FIXME: perhaps need to make sure even timers have a markable 'wrapper'.".

Just check markActiveObjectsForContext for more info.
Comment 17 Jesus Sanchez-Palencia 2010-01-06 12:00:55 PST
(In reply to comment #16)
> You should note that according to the comments in bindings/js/JSDOMBinding.cpp,
> JS objects created by setTimeout do not have JS wrappers and are, AFAICT, not
> included in the ActiveDOMObjects and thus, not suspended at all.
> 
> Have you verified this? The comments even have a "// FIXME: perhaps need to
> make sure even timers have a markable 'wrapper'.".
> 
> Just check markActiveObjectsForContext for more info.

Ok, just found in WebCore/page/DOMTimer.cpp:87  :
// DOMTimer constructor links the new timer into a list of ActiveDOMObjects held by the 'context'.

And looking at the constructor, line 64, found: scriptExecutionContext()->addTimeout(m_timeoutId, this);
Taking a look at the commit 177db693, which introduced this to DOMTimer, I can just conclude that the previous comment that I stated is no longer valid. Patching it right now. (and we are back to the original topic)
Comment 18 Laszlo Gombos 2010-01-06 14:50:45 PST
(In reply to comment #9)
> We might want to consider committing
> https://bugs.webkit.org/attachment.cgi?id=45919 to the trunk as well, just to
> keep the trunk in sync with 4.6.

The patch will need changes in the Symbian def files as well.
Comment 19 Simon Hausmann 2010-01-07 03:24:56 PST
(In reply to comment #18)
> (In reply to comment #9)
> > We might want to consider committing
> > https://bugs.webkit.org/attachment.cgi?id=45919 to the trunk as well, just to
> > keep the trunk in sync with 4.6.
> 
> The patch will need changes in the Symbian def files as well.

Yep. I'll submit the change into the trunk, cherry-pick it into qtwebkit-4.6, update the def files there (weekly?), submit them also back into the trunk.

Once we remove the private API we mark those symbols as absent in the def files.
Comment 20 Simon Hausmann 2010-01-07 03:27:40 PST
Landed in r52914 in the trunk and cherry-picked as 7f2f0abd29e5363012b24c2465226e8ec2f33183 into qtwebkit-4.6
Comment 21 Kenneth Rohde Christiansen 2010-01-07 04:10:07 PST
(In reply to comment #20)
> Landed in r52914 in the trunk and cherry-picked as
> 7f2f0abd29e5363012b24c2465226e8ec2f33183 into qtwebkit-4.6

Is this enough for the JIRA task? What is holding us from settling on a public API and add autotests?
Comment 22 Simon Hausmann 2010-01-07 05:14:13 PST
Benjamin, Tor Arne: What do you think about the currently proposed names?

QWebFrame::suspendActiveDOMObjects
QWebFrame::resumeActiveDOMObjects

(see also Jesus' analysis)
Comment 23 Tor Arne Vestbø 2010-01-07 05:51:08 PST
(In reply to comment #12)
> So, I think that we could define Active DOM Objects as Plugins, JavaScript,
> Animated GIFs, DOM events, CSS animations, DOMActiveObject subclasses and, as
> Simons stated, wrap them behind the API. Is there anything else (active
> objects, non-active objects) that we should be able to suspend/resume by now?

Does that mean frame->document()->suspendActiveDOMObjects() will include all those types of activities you mentioned, including plugins, animations, etc, or do we have to do some additional suspending of our own?

API-naming-vise I don't like "Active" as a prefix, as it somehow implies that those objects that are currently inactive are not suspended. What if they then wake up/become active, are they then not part of the group of suspended objects?

I also dislike "DOMObjects" as a catch-all representation of the classes you mentioned above, can we find something more descriptive? Or perhaps do we even need a postfix?

QWebFrame::suspend()
QWebFrame::resume()


?
Comment 24 Jesus Sanchez-Palencia 2010-01-08 03:59:37 PST
(In reply to comment #23)
> Does that mean frame->document()->suspendActiveDOMObjects() will include all
> those types of activities you mentioned, including plugins, animations, etc, or
> do we have to do some additional suspending of our own?

My feelings are that our API should include everything. All Active DOM Objects, at least. I'm not sure right now if animated GIFs and CSS animations are ActiveDOMObjects, but I've already checked the others. XHR inherits ActiveDOMObject but it doesn't re-implement ActiveDOMObject::suspend() and ActiveDOMObject::resume() and, therefore, wouldn't be suspended/resumed. Maybe I should open a new bug for this?

> 
> API-naming-vise I don't like "Active" as a prefix, as it somehow implies that
> those objects that are currently inactive are not suspended. What if they then
> wake up/become active, are they then not part of the group of suspended
> objects?
> 
> I also dislike "DOMObjects" as a catch-all representation of the classes you
> mentioned above, can we find something more descriptive? Or perhaps do we even
> need a postfix?
> 
> QWebFrame::suspend()
> QWebFrame::resume()

Ok, I got it. Let me propose 4 API's with pros and cons:

- QWebFrame::suspendActiveDOMObjects() and QWebFrame::resumeActiveDOMObjects()
It's clear enough, but too long and can confuse because of the inactive objects that we also need to suspend.

- QWebFrame::suspend() and QWebFrame::resume()
Seems nice, but when I suspend all DOMObjects of a frame can I still click on hyperlinks? If so, the frame isn't completely suspended and the API would be inconsistent, IMHO. But yes, I have to check this.

- QWebFrame::suspendObjectsExecution() and QWebFrame::resumeObjectsExecution()
It's not too long and does not use Active or DOM as prefix or postfix.

And a brand new idea (thanks, Kenneth):
- QWebFrame::setExecutionMode(ExecutionMode)
Where 'ExecutionMode' could be 'Default', 'PowerSaving', etc.
This is consistent with this bug's description and can be extended when more options and use-cases arrive (i.e.: if someday we need to just suspend scripts, animations, etc)
Comment 25 Sriram Neelakandan 2010-06-02 03:53:56 PDT
I think the bug mentions about everything that consumes CPU-cycles/battery-life when in background .. 

--> JS Timers, DOM Events, Plugins, CSS animations, GIF animations

What about HTML5 audio/video playback ?
Comment 26 Allan Sandfeld Jensen 2012-05-24 05:42:53 PDT
WebKit2 now has an API like this between WebProcess and UIProcess. It is currently called suspendActiveDOMObjectsAndAnimations(), since that is what it does.
Comment 27 Jocelyn Turcotte 2014-02-03 03:15:59 PST
=== Bulk closing of Qt bugs ===

If you believe that this bug report is still relevant for a non-Qt port of webkit.org, please re-open it and remove [Qt] from the summary.

If you believe that this is still an important QtWebKit bug, please fill a new report at https://bugreports.qt-project.org and add a link to this issue. See http://qt-project.org/wiki/ReportingBugsInQt for additional guidelines.