Bug 93847

Summary:

[Qt] Very strange incremental build bug

Product:

WebKit

Reporter:

Csaba Osztrogonác <ossy>

Component:

Tools / Tests

Assignee:

Tor Arne Vestbø <vestbo>

Status:

RESOLVED FIXED

Severity:

Blocker

CC:

abecsi, hausmann, noam, ossy, vestbo, webkit.review.bot

Priority:

Keywords:

Qt, QtTriaged

Version:

528+ (Nightly build)

Hardware:

All

OS:

All

Attachments:

Description	Flags
build log from the bot	none
build log from the bot - same revision, second build, success	none
diff between WebKitBuild directories	none
Patch	ossy: review+

Description Csaba Osztrogonác 2012-08-13 07:38:05 PDT

Created attachment 157995 [details]
build log from the bot

There are an incremental build bug on the Qt bots intermittently.
(~ once a week) Unfortunately it's impossible to reproduce it.

But now I have a build log for it and I saved the full 
WebKitBuild directory to help to find the root of the bug.

Comment 1 Csaba Osztrogonác 2012-08-13 07:49:33 PDT

Created attachment 157999 [details]
build log from the bot - same revision, second build, success

Comment 2 Csaba Osztrogonác 2012-08-13 08:00:17 PDT

Created attachment 158002 [details]
diff between WebKitBuild directories

 left side (-): broken build
right side (+): second and working build

Comment 3 Csaba Osztrogonác 2012-08-22 08:39:54 PDT

I found many build failures which can be related to this bug:
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20NRWT/builds/15427/steps/compile-webkit/logs/stdio/text
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20NRWT/builds/17506/steps/compile-webkit/logs/stdio/text
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20NRWT/builds/17673/steps/compile-webkit/logs/stdio/text
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20NRWT/builds/17795/steps/compile-webkit/logs/stdio/text
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20NRWT/builds/17978/steps/compile-webkit/logs/stdio/text

It's strange that all of these build failures are on our "x86-32 Linux Qt Release NRWT" bot, which builds on 4 threads. But I can't find any build
failure on our "x86-64 Linux Qt Release" bot which builds on 8 threads.

Comment 4 Csaba Osztrogonác 2012-08-22 09:11:32 PDT

More build failures from the "x86-32 Linux Qt Release - Qt5-WebKit1" bot:
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20-%20Qt5-WebKit1/builds/9610/steps/compile-webkit/logs/stdio
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20-%20Qt5-WebKit1/builds/9888/steps/compile-webkit/logs/stdio
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20-%20Qt5-WebKit1/builds/10019/steps/compile-webkit/logs/stdio
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20-%20Qt5-WebKit1/builds/10131/steps/compile-webkit/logs/stdio
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20-%20Qt5-WebKit1/builds/10417/steps/compile-webkit/logs/stdio
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20-%20Qt5-WebKit1/builds/10548/steps/compile-webkit/logs/stdio
- http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20-%20Qt5-WebKit1/builds/10578/steps/compile-webkit/logs/stdio

This bot builds with -j4 too.

Comment 5 Csaba Osztrogonác 2012-08-22 09:12:33 PDT

Do we need more build logs from other bots?

Comment 6 Csaba Osztrogonác 2012-08-27 06:28:27 PDT

One more occurance:
http://build.webkit.sed.hu/builders/ARMv7%20Linux%20Qt5%20Release%20%28Build%29/builds/10491/steps/compile-webkit/logs/stdio

Comment 7 Csaba Osztrogonác 2012-08-27 23:57:04 PDT

One more different strange build fail: http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20NRWT/builds/18644/steps/compile-webkit/logs/stdio

Comment 8 Csaba Osztrogonác 2012-08-29 01:44:05 PDT

Two more occurances of this bug:
- http://build.webkit.sed.hu/builders/ARMv7%20Linux%20Qt5%20Release%20%28Build%29/builds/10561/steps/compile-webkit/logs/stdio
- http://build.webkit.sed.hu/builders/ARMv7%20Linux%20Qt5%20Release%20%28Build%29/builds/10564/steps/compile-webkit/logs/stdio

Comment 9 Csaba Osztrogonác 2012-08-31 06:34:57 PDT

One more fail, now on the EWS bot:
- bug: https://bugs.webkit.org/show_bug.cgi?id=95382#c5
- build log: http://queues.webkit.org/results/13689448

It would be great to fix this crazy bug as soon as possible ...
Or else nobody will trust in Qt buildbots and EWS bot. :-(

Comment 10 Simon Hausmann 2012-08-31 06:50:06 PDT

The "make[4]: *** No targets.  Stop." makes me wonder if this is perhaps the result of a corrupted make file due to two qmake instances writing the same file. That would also explain why the likely-hood of this is somewhat related to the number of parallel jobs.

Comment 11 Csaba Osztrogonác 2012-08-31 07:14:08 PDT

Maybe it is related to https://trac.webkit.org/changeset/121966

Comment 12 Csaba Osztrogonác 2012-09-01 00:33:50 PDT

One more fail: http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20NRWT/builds/18968/steps/compile-webkit/logs/stdio

Comment 13 Tor Arne Vestbø 2012-09-04 10:10:19 PDT

Taking

Comment 14 Tor Arne Vestbø 2012-09-05 05:48:33 PDT

Created attachment 162231 [details]
Patch

Comment 15 Csaba Osztrogonác 2012-09-05 05:53:34 PDT

Comment on attachment 162231 [details]
Patch

r=me, thanks for the fix.

Comment 16 Tor Arne Vestbø 2012-09-05 06:11:31 PDT

Committed r127583: <http://trac.webkit.org/changeset/127583>