Change Build bot with BuildAndTest bot for EFL port

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Gyuyoung Kim]

In order to support test bot for EFL port, EFL port needs to support test result archive.

[Gyuyoung Kim]

Created attachment 124900 [details]
Patch

[Ryosuke Niwa]

Comment on attachment 124900 [details]
Patch

You also need to update the if-else clauses around lines 66-76.

[Gyuyoung Kim]

Created attachment 125988 [details]
Patch

[Gyuyoung Kim]

Hello Ryosuke,

I decide to change EFL build bot to BuildAndTest bot. But, there are two problems.

One is jscore-test and layout-test are failed.
 - 4 test cases are failed in jscore-test.
 - layout test is exited early due to over 20 crashes.

Can these test fails be fixed after setting test bot up? 



The other is that "MasterShellCommand" step is failed. It looks this step is to extract uploaded test result. Can this step be tested locally ?

[Ryosuke Niwa]

(In reply to comment #4)
> Hello Ryosuke,
> 
> I decide to change EFL build bot to BuildAndTest bot. But, there are two problems.
> 
> One is jscore-test and layout-test are failed.
>  - 4 test cases are failed in jscore-test.
>  - layout test is exited early due to over 20 crashes.

That doesn't sound good. Can we at least fix the crash so that we can run all the tests? I think it's okay to have failing tests but run-webkit-test exiting early doesn't sound good to me.

> The other is that "MasterShellCommand" step is failed. It looks this step is to extract uploaded test result. Can this step be tested locally ?

Are you archiving test results property? i.e. does layout-test.zip (or whatever the right filename is) exit? Does the upload step succeed?

[Gyuyoung Kim]

Created attachment 125993 [details]
screenshot of local efl buildandtest bot

[Gyuyoung Kim]

(In reply to comment #5)

> > The other is that "MasterShellCommand" step is failed. It looks this step is to extract uploaded test result. Can this step be tested locally ?
> 
> Are you archiving test results property? i.e. does layout-test.zip (or whatever the right filename is) exit? Does the upload step succeed?

yes, layout-test-results.zip file is created successfully. But, I'm suspecting if this file is uploaded well. If local build bot slave upload this file to bugzilla, this is can be failed because my office is using firewall for security. If I should pass this step, I will test it in my home again.

[Ryosuke Niwa]

(In reply to comment #7)
> (In reply to comment #5)
> yes, layout-test-results.zip file is created successfully. But, I'm suspecting if this file is uploaded well. If local build bot slave upload this file to bugzilla, this is can be failed because my office is using firewall for security. If I should pass this step, I will test it in my home again.

Oh, I see. That sounds like a local security issue. We can figure it out after the fact. But we should make sure run-webkit-test isn't exiting early as you described since that makes the bot pretty useless.

[Gyuyoung Kim]

Created attachment 126055 [details]
Patch

[Gyuyoung Kim]

Created attachment 126056 [details]
scrrenshot of local efl BuildAndTest bot

[Gyuyoung Kim]

Hi Ryosuke,

There are three problems why layout test is not finished.

 1. Missing option in webkitdirs.pm
    : I add it to fix crash.

 2. Insufficient skip list for EFL port.
    : I update skip list with test cases which don't have expected result. 

 3. Disabled features in build-webkit script.
    : Though OptionEfl.cmake file enables the features, build-webkit doesn't enable them(page visibility, shared worker, video, workers).

Layout test reports 466 failures 4 flakes after solving above.

[Gyuyoung Kim]

Comment on attachment 126055 [details]
Patch

Attachment 126055 [details] did not pass efl-ews (efl):
Output: http://queues.webkit.org/results/11464382

[Gyuyoung Kim]

Created attachment 126074 [details]
Patch

[Gyuyoung Kim]

Comment on attachment 126074 [details]
Patch

Attachment 126074 [details] did not pass efl-ews (efl):
Output: http://queues.webkit.org/results/11453443

[Gyuyoung Kim]

Created attachment 126107 [details]
Patch

[Gyuyoung Kim]

GStreamer wasn't installed on EWS server. I installed it. :-)

[Gyuyoung Kim]

Created attachment 126486 [details]
screenshot of local efl buildandtest bot 

Hello Ryosuke,

I have test buildbot on linux machine. So, MasterShellCommand couldn't run on my local buildbot. As you know, *ditto* is os x command. So, I change it for linux as below,

self.command = ["unzip", "-d", self.build.getProperties().render(self.resultDirectory) , self.build.getProperties().render(self.zipFile)]

All steps are passed after changing it. Test result is shown well.

Should I test build bot on Mac? If not so, I think it is better to mention this issue to wiki page for build bot.
http://trac.webkit.org/wiki/BuildBot

[Ryosuke Niwa]

Comment on attachment 126107 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=126107&action=review

> Tools/Scripts/build-webkit:286
> -      define => "ENABLE_PAGE_VISIBILITY_API", default => 0, value => \$pageVisibilityApiSupport },
> +      define => "ENABLE_PAGE_VISIBILITY_API", default => isEfl(), value => \$pageVisibilityApiSupport },

Let's not make these changes in this patch.

[Gyuyoung Kim]

Created attachment 126491 [details]
Patch

[Gyuyoung Kim]

(In reply to comment #18)

> Let's not make these changes in this patch.

I split the change of build-webkit into new bug (Bug 78348)

[WebKit Review Bot]

The commit-queue encountered the following flaky tests while processing attachment 126491 [details]:

inspector/protocol/console-agent.html bug 79563 (authors: [e-mail], [e-mail], and [e-mail])
The commit-queue is continuing to process your patch.

[WebKit Review Bot]

Comment on attachment 126491 [details]
Patch

Clearing flags on attachment: 126491

Committed r108923: <http://trac.webkit.org/changeset/108923>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.