Bug 78309 - Perf-o-matic should process reports in background
Summary: Perf-o-matic should process reports in background
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit Website (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Ryosuke Niwa
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-02-09 18:13 PST by Ryosuke Niwa
Modified: 2012-02-10 03:26 PST (History)
2 users (show)

See Also:


Attachments
Patch (40.23 KB, patch)
2012-02-10 01:02 PST, Ryosuke Niwa
no flags Details | Formatted Diff | Diff
Removed report_logs_handler.pyc (37.02 KB, patch)
2012-02-10 01:05 PST, Ryosuke Niwa
no flags Details | Formatted Diff | Diff
Fixed yaml > html (38.81 KB, patch)
2012-02-10 01:45 PST, Ryosuke Niwa
morrita: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryosuke Niwa 2012-02-09 18:13:33 PST
I just realized that there are a whole bunch of ReportLog objects sitting in the production server because appengine ran out of time to run report_handler.py :( When appengine cold-starts, it takes extra time to compile the scripts and end up not being able to finish the request.

As someone worked on the flakiness dashboard suggested before, we should do the whole parsing, processing, etc... in background instead. We also need some UI to salvage the lost reports.
Comment 1 Ryosuke Niwa 2012-02-10 01:02:17 PST
Created attachment 126471 [details]
Patch
Comment 2 Ryosuke Niwa 2012-02-10 01:05:16 PST
Created attachment 126472 [details]
Removed report_logs_handler.pyc
Comment 3 Hajime Morrita 2012-02-10 01:14:57 PST
Comment on attachment 126472 [details]
Removed report_logs_handler.pyc

View in context: https://bugs.webkit.org/attachment.cgi?id=126472&action=review

I really want to see the unittest. for this.
You can test datastore even locally.
http://code.google.com/appengine/docs/python/tools/localunittesting.html

> Websites/webkit-perf.appspot.com/report_logs.yaml:1
> +<!DOCTYPE html>

This doesn't look like a yaml at all.
Comment 4 Ryosuke Niwa 2012-02-10 01:22:13 PST
(In reply to comment #3)
> (From update of attachment 126472 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=126472&action=review
> 
> I really want to see the unittest. for this.
> You can test datastore even locally.
> http://code.google.com/appengine/docs/python/tools/localunittesting.html

I don't think testing model code is interesting since most of the logic lives in handler classes at the moment.
 
> > Websites/webkit-perf.appspot.com/report_logs.yaml:1
> > +<!DOCTYPE html>
> 
> This doesn't look like a yaml at all.

What do you mean?
Comment 5 Ryosuke Niwa 2012-02-10 01:24:20 PST
FWIW, this code has been pushed into the production.
Comment 6 Ryosuke Niwa 2012-02-10 01:45:12 PST
Created attachment 126476 [details]
Fixed yaml > html
Comment 7 Ryosuke Niwa 2012-02-10 01:56:14 PST
Committed r107386: <http://trac.webkit.org/changeset/107386>
Comment 8 Ryosuke Niwa 2012-02-10 03:26:39 PST
Build fix landed in http://trac.webkit.org/changeset/107393.