Bug 77518 - Change Build bot with BuildAndTest bot for EFL port
Summary: Change Build bot with BuildAndTest bot for EFL port
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Gyuyoung Kim
URL:
Keywords:
Depends on: 78348 79440
Blocks:
  Show dependency treegraph
 
Reported: 2012-02-01 01:03 PST by Gyuyoung Kim
Modified: 2012-02-25 20:12 PST (History)
6 users (show)

See Also:


Attachments
Patch (1.44 KB, patch)
2012-02-01 01:05 PST, Gyuyoung Kim
no flags Details | Formatted Diff | Diff
Patch (5.76 KB, patch)
2012-02-07 20:14 PST, Gyuyoung Kim
no flags Details | Formatted Diff | Diff
screenshot of local efl buildandtest bot (126.02 KB, image/png)
2012-02-07 20:49 PST, Gyuyoung Kim
no flags Details
Patch (14.67 KB, patch)
2012-02-08 04:31 PST, Gyuyoung Kim
no flags Details | Formatted Diff | Diff
scrrenshot of local efl BuildAndTest bot (156.39 KB, image/png)
2012-02-08 04:33 PST, Gyuyoung Kim
no flags Details
Patch (14.70 KB, patch)
2012-02-08 06:48 PST, Gyuyoung Kim
no flags Details | Formatted Diff | Diff
Patch (14.70 KB, patch)
2012-02-08 10:14 PST, Gyuyoung Kim
no flags Details | Formatted Diff | Diff
screenshot of local efl buildandtest bot (152.43 KB, image/png)
2012-02-10 02:58 PST, Gyuyoung Kim
no flags Details
Patch (11.99 KB, patch)
2012-02-10 03:37 PST, Gyuyoung Kim
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Gyuyoung Kim 2012-02-01 01:03:28 PST
In order to support test bot for EFL port, EFL port needs to support test result archive.
Comment 1 Gyuyoung Kim 2012-02-01 01:05:27 PST
Created attachment 124900 [details]
Patch
Comment 2 Ryosuke Niwa 2012-02-01 01:13:14 PST
Comment on attachment 124900 [details]
Patch

You also need to update the if-else clauses around lines 66-76.
Comment 3 Gyuyoung Kim 2012-02-07 20:14:24 PST
Created attachment 125988 [details]
Patch
Comment 4 Gyuyoung Kim 2012-02-07 20:36:04 PST
Hello Ryosuke,

I decide to change EFL build bot to BuildAndTest bot. But, there are two problems.

One is jscore-test and layout-test are failed.
 - 4 test cases are failed in jscore-test.
 - layout test is exited early due to over 20 crashes.

Can these test fails be fixed after setting test bot up? 



The other is that "MasterShellCommand" step is failed. It looks this step is to extract uploaded test result. Can this step be tested locally ?
Comment 5 Ryosuke Niwa 2012-02-07 20:40:20 PST
(In reply to comment #4)
> Hello Ryosuke,
> 
> I decide to change EFL build bot to BuildAndTest bot. But, there are two problems.
> 
> One is jscore-test and layout-test are failed.
>  - 4 test cases are failed in jscore-test.
>  - layout test is exited early due to over 20 crashes.

That doesn't sound good. Can we at least fix the crash so that we can run all the tests? I think it's okay to have failing tests but run-webkit-test exiting early doesn't sound good to me.

> The other is that "MasterShellCommand" step is failed. It looks this step is to extract uploaded test result. Can this step be tested locally ?

Are you archiving test results property? i.e. does layout-test.zip (or whatever the right filename is) exit? Does the upload step succeed?
Comment 6 Gyuyoung Kim 2012-02-07 20:49:58 PST
Created attachment 125993 [details]
screenshot of local efl buildandtest bot
Comment 7 Gyuyoung Kim 2012-02-07 20:54:13 PST
(In reply to comment #5)

> > The other is that "MasterShellCommand" step is failed. It looks this step is to extract uploaded test result. Can this step be tested locally ?
> 
> Are you archiving test results property? i.e. does layout-test.zip (or whatever the right filename is) exit? Does the upload step succeed?

yes, layout-test-results.zip file is created successfully. But, I'm suspecting if this file is uploaded well. If local build bot slave upload this file to bugzilla, this is can be failed because my office is using firewall for security. If I should pass this step, I will test it in my home again.
Comment 8 Ryosuke Niwa 2012-02-07 23:08:09 PST
(In reply to comment #7)
> (In reply to comment #5)
> yes, layout-test-results.zip file is created successfully. But, I'm suspecting if this file is uploaded well. If local build bot slave upload this file to bugzilla, this is can be failed because my office is using firewall for security. If I should pass this step, I will test it in my home again.

Oh, I see. That sounds like a local security issue. We can figure it out after the fact. But we should make sure run-webkit-test isn't exiting early as you described since that makes the bot pretty useless.
Comment 9 Gyuyoung Kim 2012-02-08 04:31:31 PST
Created attachment 126055 [details]
Patch
Comment 10 Gyuyoung Kim 2012-02-08 04:33:59 PST
Created attachment 126056 [details]
scrrenshot of local efl BuildAndTest bot
Comment 11 Gyuyoung Kim 2012-02-08 04:45:45 PST
Hi Ryosuke,

There are three problems why layout test is not finished.

 1. Missing option in webkitdirs.pm
    : I add it to fix crash.

 2. Insufficient skip list for EFL port.
    : I update skip list with test cases which don't have expected result. 

 3. Disabled features in build-webkit script.
    : Though OptionEfl.cmake file enables the features, build-webkit doesn't enable them(page visibility, shared worker, video, workers).

Layout test reports 466 failures 4 flakes after solving above.
Comment 12 Gyuyoung Kim 2012-02-08 05:13:29 PST
Comment on attachment 126055 [details]
Patch

Attachment 126055 [details] did not pass efl-ews (efl):
Output: http://queues.webkit.org/results/11464382
Comment 13 Gyuyoung Kim 2012-02-08 06:48:43 PST
Created attachment 126074 [details]
Patch
Comment 14 Gyuyoung Kim 2012-02-08 07:52:51 PST
Comment on attachment 126074 [details]
Patch

Attachment 126074 [details] did not pass efl-ews (efl):
Output: http://queues.webkit.org/results/11453443
Comment 15 Gyuyoung Kim 2012-02-08 10:14:33 PST
Created attachment 126107 [details]
Patch
Comment 16 Gyuyoung Kim 2012-02-08 19:33:10 PST
GStreamer wasn't installed on EWS server. I installed it. :-)
Comment 17 Gyuyoung Kim 2012-02-10 02:58:09 PST
Created attachment 126486 [details]
screenshot of local efl buildandtest bot 

Hello Ryosuke,

I have test buildbot on linux machine. So, MasterShellCommand couldn't run on my local buildbot. As you know, *ditto* is os x command. So, I change it for linux as below,

self.command = ["unzip", "-d", self.build.getProperties().render(self.resultDirectory) , self.build.getProperties().render(self.zipFile)]

All steps are passed after changing it. Test result is shown well.

Should I test build bot on Mac? If not so, I think it is better to mention this issue to wiki page for build bot.
http://trac.webkit.org/wiki/BuildBot
Comment 18 Ryosuke Niwa 2012-02-10 03:01:17 PST
Comment on attachment 126107 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=126107&action=review

> Tools/Scripts/build-webkit:286
> -      define => "ENABLE_PAGE_VISIBILITY_API", default => 0, value => \$pageVisibilityApiSupport },
> +      define => "ENABLE_PAGE_VISIBILITY_API", default => isEfl(), value => \$pageVisibilityApiSupport },

Let's not make these changes in this patch.
Comment 19 Gyuyoung Kim 2012-02-10 03:37:38 PST
Created attachment 126491 [details]
Patch
Comment 20 Gyuyoung Kim 2012-02-10 03:40:51 PST
(In reply to comment #18)

> Let's not make these changes in this patch.

I split the change of build-webkit into new bug (Bug 78348)
Comment 21 WebKit Review Bot 2012-02-25 20:10:51 PST
The commit-queue encountered the following flaky tests while processing attachment 126491 [details]:

inspector/protocol/console-agent.html bug 79563 (authors: caseq@chromium.org, loislo@chromium.org, and pfeldman@chromium.org)
The commit-queue is continuing to process your patch.
Comment 22 WebKit Review Bot 2012-02-25 20:12:44 PST
Comment on attachment 126491 [details]
Patch

Clearing flags on attachment: 126491

Committed r108923: <http://trac.webkit.org/changeset/108923>
Comment 23 WebKit Review Bot 2012-02-25 20:12:50 PST
All reviewed patches have been landed.  Closing bug.