Bug 213927 – General update of WPT tests (2020-07)

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Carlos Alberto Lopez Perez]

The last time we updated the whole set of imported WPT tests was around 2018.
Let's update it again!

[Carlos Alberto Lopez Perez]

Created attachment 403466 [details]
test patch for EWS, ignore

test patch for EWS, ignore

[Carlos Alberto Lopez Perez]

Created attachment 403666 [details]
test patch for EWS, ignore

[Carlos Alberto Lopez Perez]

Created attachment 403680 [details]
test patch for EWS, ignore

[Chris Dumez]

I have been working on doing this as well. However, I have been doing it folder by folder because I don't think a 51MB patch is reviewable. I wish we could do a full-resync like this and trust the result but in my experience, it is important to sanity check the results.

[Carlos Alberto Lopez Perez]

(In reply to Chris Dumez from comment #4)
> I have been working on doing this as well. However, I have been doing it
> folder by folder because I don't think a 51MB patch is reviewable. I wish we
> could do a full-resync like this and trust the result but in my experience,
> it is important to sanity check the results.

Right..

I'm also finding quite a lot of troubles with the tooling (svn-apply is really slow or goes nuts with such big patches see bug 212766 or bug 214006).

I also found that doing a general update of all the wpt tests with the w3c test importer causes it to import lot of unwanted things (see bug 214006). The patch currently attached here used that modified version of the importer to update everything. 

On top of that I also found that we have several imported tests that were never exported to WPT (which I kept on this patch)
 
In any case you are totally right that reviewing a 51MB patch is a nightmare. I think it makes sense to split it.

[Carlos Alberto Lopez Perez]

(In reply to Chris Dumez from comment #4)
> I have been working on doing this as well. However, I have been doing it
> folder by folder because I don't think a 51MB patch is reviewable. I wish we
> could do a full-resync like this and trust the result but in my experience,
> it is important to sanity check the results.

Right..

I'm also finding quite a lot of troubles with the tooling (svn-apply is really slow or goes nuts with such big patches see bug 212766 or bug 214022).

I also found that doing a general update of all the wpt tests with the w3c test importer causes it to import lot of unwanted things (see bug 214006). The patch currently attached here used that modified version of the importer to update everything. 

On top of that I also found that we have several imported tests that were never exported to WPT (which I kept on this patch)
 
In any case you are totally right that reviewing a 51MB patch is a nightmare. I think it makes sense to split it.

[Chris Dumez]

(In reply to Carlos Alberto Lopez Perez from comment #6)
> (In reply to Chris Dumez from comment #4)
> > I have been working on doing this as well. However, I have been doing it
> > folder by folder because I don't think a 51MB patch is reviewable. I wish we
> > could do a full-resync like this and trust the result but in my experience,
> > it is important to sanity check the results.
> 
> Right..
> 
> I'm also finding quite a lot of troubles with the tooling (svn-apply is
> really slow or goes nuts with such big patches see bug 212766 or bug 214022).
> 
> I also found that doing a general update of all the wpt tests with the w3c
> test importer causes it to import lot of unwanted things (see bug 214006).
> The patch currently attached here used that modified version of the importer
> to update everything. 
> 
> On top of that I also found that we have several imported tests that were
> never exported to WPT (which I kept on this patch)
>  
> In any case you are totally right that reviewing a 51MB patch is a
> nightmare. I think it makes sense to split it.

And by the way, I do appreciate help on this. I am currently maintaining a Frankenstein version of testharness.js that is able to run both new tests and old tests that have not been resync'd yet. I am very much looking forward to having all WPT tests resync'd so that we can use a clean version of testharness.js.

[Carlos Alberto Lopez Perez]

(In reply to Chris Dumez from comment #7)
> (In reply to Carlos Alberto Lopez Perez from comment #6)
> > (In reply to Chris Dumez from comment #4)
> > > I have been working on doing this as well. However, I have been doing it
> > > folder by folder because I don't think a 51MB patch is reviewable. I wish we
> > > could do a full-resync like this and trust the result but in my experience,
> > > it is important to sanity check the results.
> > 
> > Right..
> > 
> > I'm also finding quite a lot of troubles with the tooling (svn-apply is
> > really slow or goes nuts with such big patches see bug 212766 or bug 214022).
> > 
> > I also found that doing a general update of all the wpt tests with the w3c
> > test importer causes it to import lot of unwanted things (see bug 214006).
> > The patch currently attached here used that modified version of the importer
> > to update everything. 
> > 
> > On top of that I also found that we have several imported tests that were
> > never exported to WPT (which I kept on this patch)
> >  
> > In any case you are totally right that reviewing a 51MB patch is a
> > nightmare. I think it makes sense to split it.
> 
> And by the way, I do appreciate help on this. I am currently maintaining a
> Frankenstein version of testharness.js that is able to run both new tests
> and old tests that have not been resync'd yet. I am very much looking
> forward to having all WPT tests resync'd so that we can use a clean version
> of testharness.js.


I will be trying to update individual folders as time allows, but trying to keep patches reviewable (~2MB maximum if possible).

Let's hope that we manage to update everything this year :)

I will be using this as a master bug and adding the other bugs for updates as blocking this.

[Carlos Alberto Lopez Perez]

Closing this.

All WPT tests were updated (see bugs depending on this one) and the version we are running of WPT tests now is at least from 2020-07