Bug 72498 – NRWT not running webkit_gpu_tests properly for Leopard (Mac OS 10.5)

• Component: Tools / Tests
• Status: RESOLVED FIXED

Split off of https://bugs.webkit.org/show_bug.cgi?id=72275 ('check-webkit-style broken by r99773: "Could not determine the port"') ...

As of this point, webkit_gpu_tests was green for Leopard:
http://build.chromium.org/p/chromium.webkit/builders/Webkit%20Mac10.5%20%28CG%29/builds/1197 
webkit r100350
chromium r110212
[from webkit_gpu_tests stdio] Found:  254 tests

The next run had a failure while parsing expectations:
http://build.chromium.org/p/chromium.webkit/builders/Webkit%20Mac10.5%20%28CG%29/builds/1198
webkit r100373
chromium r110235

That continued to be the case until this run:
http://build.chromium.org/p/chromium.webkit/builders/Webkit%20Mac10.5%20%28CG%29/builds/1202
webkit r100398
chromium r110256
At this point, it was able to parse the expectations but was now trying to run 27185 tests instead of 254.  (NRWT is apparently interpreting the chromium-gpu-cg-mac platform as a CPU platform, not a GPU platform)
[from webkit_gpu_tests stdio] Found:  27185 tests

As of webkit r100439 , I can confirm that it is still trying to run 27185 tests.

Based on the evidence above, I believe this is the result of a webkit change within the range r100350:r100398 .

I think these changes are the most likely suspects.  CCing the authors:
http://trac.webkit.org/changeset/100350
http://trac.webkit.org/changeset/100352
http://trac.webkit.org/changeset/100355
http://trac.webkit.org/changeset/100356
http://trac.webkit.org/changeset/100360
http://trac.webkit.org/changeset/100368
http://trac.webkit.org/changeset/100374

[Tony Chang]

It looks like this is due to r100355.

Now that we moved the tests into the regular layout test run, on chromium-gpu-cg-mac-leopard, paths is empty, so test_files.find() returns all tests.

It seems like we just want to remove the webkit_gpu_tests step from the Leopard bots.

James or Dirk, can you confirm that this is the desired behavior?

[Dirk Pranke]

(In reply to comment #2)
> It looks like this is due to r100355.
> 
> Now that we moved the tests into the regular layout test run, on chromium-gpu-cg-mac-leopard, paths is empty, so test_files.find() returns all tests.
> 
> It seems like we just want to remove the webkit_gpu_tests step from the Leopard bots.
> 
> James or Dirk, can you confirm that this is the desired behavior?

Ah, whoops, I forgot about that default behavior, and had assumed that there would still be some tests to run, but I didn't think it through. Obviously, we don't want to run all 27k tests on Leopard ;).

Yeah, we can either remove the webkit_gpu_tests step from Leopard, or give them some dummy list of paths to run just to be a no-op (e.g., one test or a small directory or something) until we can remove webkit_gpu_tests everywhere.

(In reply to comment #3)
> (In reply to comment #2)
> > It looks like this is due to r100355.
> > 
> > Now that we moved the tests into the regular layout test run, on chromium-gpu-cg-mac-leopard, paths is empty, so test_files.find() returns all tests.
> > 
> > It seems like we just want to remove the webkit_gpu_tests step from the Leopard bots.
> > 
> > James or Dirk, can you confirm that this is the desired behavior?
> 
> Ah, whoops, I forgot about that default behavior, and had assumed that there would still be some tests to run, but I didn't think it through. Obviously, we don't want to run all 27k tests on Leopard ;).
> 
> Yeah, we can either remove the webkit_gpu_tests step from Leopard, or give them some dummy list of paths to run just to be a no-op (e.g., one test or a small directory or something) until we can remove webkit_gpu_tests everywhere.

How is it that Leopard and SnowLeopard are being treated differently?  Shouldn't they both be treated essentially the same?

[Dirk Pranke]

(In reply to comment #4)
> How is it that Leopard and SnowLeopard are being treated differently?  Shouldn't they both be treated essentially the same?

IIRC, we don't support accelerated video compositing on Leopard, but we do on SL.

[Dirk Pranke]

Okay, per conversation w/ jamesr just now, it looks like it might make the most sense to just stop running webkit_gpu_tests on Leopard CG.

[Dirk Pranke]

http://codereview.chromium.org/8575012/

I think this'll require a master restart, though. In the meantime, it's not great that we're running 27k tests, so I'll upload a hack to run *something*.

[Dirk Pranke]

Created attachment 115423 [details]
Patch

[James Robinson]

Comment on attachment 115423 [details]
Patch

R=me

[James Robinson]

Sorry about that, the intent was to run zero tests on 10.5 webkit_gpu_tests.  We don't ship accelerated compositing on that platform and afaik never intend to, so there aren't any tests that are useful to run on that platform in the gpu configuration.

[Dirk Pranke]

Committed r100491: <http://trac.webkit.org/changeset/100491>

[WebKit Review Bot]

Comment on attachment 115423 [details]
Patch

Clearing flags on attachment: 115423

Committed r100509: <http://trac.webkit.org/changeset/100509>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.

[Dirk Pranke]

This was closed incorrectly (and landed twice) because I forgot to clear the CQ+ flag; I mean to leave it open until the buildbot picks up the other change and I can revert the hack to chromium_gpu.py.

[Dirk Pranke]

Ok, the CG bots are gone. We're still running fast/canvas and canvas/philip on the Leopard bot (now a Skia bot), and although they are not accelerated, the tests are passing.

Given that I'm trying to remove the webkit_gpu_tests step completely, I'm going to ignore this for now and close this bug.