In WebKit, reftests are NAME.html NAME-expected.html Moreover, the <link rel="match"> points to the non-existing NAME-ref.html file. When export-w3c-test-changes runs the lint tool, it complains about 1) NAME-expected.html being a support file in the wrong directory. 2) NAME.html pointing to non-existent ref file. The support script should rename NAME-expected.html to NAME-ref.html before exporting in order to fix these errors.
Created attachment 341663 [details] Patch
The issue with ref tests is that one ref file in WPT might be used for several test files, so I am not sure we can always update correctly all ref files. In the simple one test/one ref case, this should work so maybe we could add a warning or add a check for this multi case
So you mean that when we have a single ref for multiple reftests, the import script will generate multiple -expected.html? I guess it's a problem with the import / way WebKit handles reftest then. I saw a FIXME in the import script about that.
I suppose for new reftests, it's safe to just rename -expected => -ref. For existing reftests using a single ref for multiple files or when upstream names are not of the form *-ref.html or similar (e.g. ./css/css-fill-stroke/reference/paint-order-001-ref.tentative.html) I guess the patch will just fail to apply.
Cancelling review since it seems we don't have a clear agreement on how we will handle the mismatch between WebKit / WPT formats...
Is it Tools/Scripts/import-w3c-tests that renames references to -expected.html?
(In reply to Philip Jägenstedt from comment #6) > Is it Tools/Scripts/import-w3c-tests that renames references to > -expected.html? Yes it is.
That'll definitely make it challenging to work effectively with reftests, and it looks like plenty of WebKit-specific failures in WPT are reftests. Is there any possible fix here short of getting rid of the renaming, with all that would entail?
<rdar://problem/105269974>
Not sure the name of this Radar is completely correct. You can export reftests so long as you have both -expected.html and -ref.html files in your WebKit repository. It's far from ideal though.
(In reply to Antoine Quint from comment #10) > Not sure the name of this Radar is completely correct. You can export > reftests so long as you have both -expected.html and -ref.html files in your > WebKit repository. It's far from ideal though. It wasn't possible before because we renamed the -ref.html file and deleted the original -ref.html. Now we have duplicates, which allows exporting, but is far from ideal.