Bug 47653 – Introduce the ChromiumXVFBPort for running commit-queue on EC2

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Adam Barth]

Introduce the ChromiumXVFBPort for running commit-queue on EC2

[Adam Barth]

Created attachment 70709 [details]
Patch

[Eric Seidel (no email)]

Comment on attachment 70709 [details]
Patch

Seems like this needs a FIXME about finding a better way to do this...

[Adam Barth]

Committed r69736: <http://trac.webkit.org/changeset/69736>

[Adam Langley]

Wait, we're adding a new type of chromium-linux target? :(

I don't think WebKit can handle this. We have an extensive script to setup Ubuntu machines for running layout tests in the Chromium tree. That should remove differences which arise from different installed fonts. test_shell also fixes the fontconfig config in order to remove differences due to that. (If this is pure WebKit, then does that mean that you're using DRT? Does DRT do the same thing?)

I haven't done any WebKit work in a long time (mostly because of the horror of layout tests) and I wouldn't want things to get worse. One should try very hard to eliminate the differences before going this route. (Maybe you have, and failed?)

[Evan Martin]

All the X-based ports (Qt, Gtk) should use Xvfb to run on EC2.

[Adam Barth]

> I don't think WebKit can handle this. We have an extensive script to setup Ubuntu machines for running layout tests in the Chromium tree. That should remove differences which arise from different installed fonts. test_shell also fixes the fontconfig config in order to remove differences due to that. (If this is pure WebKit, then does that mean that you're using DRT? Does DRT do the same thing?)

Is that script something that makes sense to move to webkit.org?  With my current setup (just following the instructions), we've got about a dozen or so failing tests.  A bunch of those tests are failing because of something wrong with filesystem, so they seem unrelated to fonts.

> I haven't done any WebKit work in a long time (mostly because of the horror of layout tests) and I wouldn't want things to get worse. One should try very hard to eliminate the differences before going this route. (Maybe you have, and failed?)

What differences are you referring to?  This patch just lets the bots run the tests under xvfb.

> All the X-based ports (Qt, Gtk) should use Xvfb to run on EC2.

Yep.  AFAIK, none of these ports have tried to run tests using the webkitpy infrastructure.  Presumably they have something like this in buildbot-land.

[Tony Chang]

I think what agl is asking is why is this a separate port as opposed to just a flag (and as a flag, gtk and qt would be able to use it too).  Having it be a separate port makes it sound like we're going to check in separate results.

Also, where's xvfb-run live?

I think it's a good idea to add xvfb support directly to NRWT, but there are some implementation details that can make it better than how chromium uses xvfb (e.g., we should start one xvfb per test thread?).

I think GTK+ is very close to using NRWT on their buildbots (see the related work in adding http locking), so they would probably prefer the ability to use this.

Finally, DRT on Chromium Linux doesn't need an X connection (although that may change since we haven't added plugin support yet).

[Adam Barth]

> I think what agl is asking is why is this a separate port as opposed to just a flag (and as a flag, gtk and qt would be able to use it too).  Having it be a separate port makes it sound like we're going to check in separate results.

Well, it's really something about the platform, but our port system here doesn't have any concept of platform.  In fact, this code doesn't even know that Mac Chromium is any different from Linux Chromium.  This doesn't have anything to do with test results.

> Also, where's xvfb-run live?

I'm not sure I understand the question.  It's in the path somewhere.

> I think it's a good idea to add xvfb support directly to NRWT, but there are some implementation details that can make it better than how chromium uses xvfb (e.g., we should start one xvfb per test thread?).

That sounds like a reasonable idea.  I don't really know anything about xvfb.

> Finally, DRT on Chromium Linux doesn't need an X connection (although that may change since we haven't added plugin support yet).

Oh, I thought I tried running it without xvfb.  I can try again.

[Tony Chang]

(In reply to comment #8)
> Well, it's really something about the platform, but our port system here doesn't have any concept of platform.  In fact, this code doesn't even know that Mac Chromium is any different from Linux Chromium.  This doesn't have anything to do with test results.

Oh, this has to do with ORWT.  I don't know about the port abstraction in ORWT, but in NRWT, the port would be the wrong layer.  How hard is it to add platform to the port system in ORWT?  Seems like it would be trivial?

> > Also, where's xvfb-run live?
> 
> I'm not sure I understand the question.  It's in the path somewhere.

I mean, where's the code for it.  Do you have to worry about conflicting display ids from running two ORWTs on the same machine?  What about cleaning up Xvfb processes if ORWT crashes?

[Adam Barth]

> Oh, this has to do with ORWT.

It actually has to do with the commit-queue, which doesn't care whether its using old or new run-webkit-tests.

> > > Also, where's xvfb-run live?
> > 
> > I'm not sure I understand the question.  It's in the path somewhere.
> 
> I mean, where's the code for it.  Do you have to worry about conflicting display ids from running two ORWTs on the same machine?  What about cleaning up Xvfb processes if ORWT crashes?

"apt-get install xvfb" installs it.  I don't know how well it handles error conditions.

[Tony Chang]

(In reply to comment #10)
> > I mean, where's the code for it.  Do you have to worry about conflicting display ids from running two ORWTs on the same machine?  What about cleaning up Xvfb processes if ORWT crashes?
> 
> "apt-get install xvfb" installs it.  I don't know how well it handles error conditions.

Ah, I see why I'm confused.  I didn't know about xvfb-run, which is itself a wrapper script around Xvfb.

To answer my own question, yes, it picks a random display id and cleans up after itself in case of a crash.