Using BUG/BUGWK in test_expectations is error prone, should use BUGCR/BUGWK
Created attachment 72833 [details]
FYI, script to update the expectations files was
for f in `find LayoutTests -name '*_expectations.txt'`; do
sed -E -i '' "s/BUG([0-9]+)/BUGCR\1/g" "$f"
Comment on attachment 72833 [details]
Please send out and email to chromium-dev announcing this is about to change. Then feel free to commit. You'll need to make sure you're synced as this file changes frequently.
Created attachment 75967 [details]
Created attachment 75969 [details]
Script used to create the test_expectations.txt changes
Created attachment 75970 [details]
Output from running rename.sh
Ok, finally revisited this (i.e. synced + re-ran script -- which takes forever).
Just an update, I noticed that there were more than a few uses of BUG to indicate a webkit bug (BUGWK) rather than a chromium bug (BUGCR). Rather than sorting through hundreds of them myself, rename.sh attempts to determine (via wget + grep) which it should be.
Sometimes the script fails. Last time I ran it I actually did resolve these by hands, but didn't this time. I attached the output of running the script (rename.out) just for the record.
Comment on attachment 75967 [details]
View in context: https://bugs.webkit.org/attachment.cgi?id=75967&action=review
This looks great!
> + For all test_expectations.txt files, change all occurrences of BUG to
> + either BUGCR or BUGWK.
Indentation is off here. Use spaces, not tabs.
> + Add presubmit check that BUG isn't used, either BUGCR or BUGWK.
ditto re: spaces
> + self._add_error(lineno, 'Bug must be either BUGCR or BUGWK for test: %s' %
Nit: now there is also BUGV8_XXX
Ojan: Benjamin is not a committer. Should this be marked cq+? I expect he's going to need to upload a new patch in which case we should probably mark it r-.
(In reply to comment #9)
> Ojan: Benjamin is not a committer. Should this be marked cq+? I expect he's going to need to upload a new patch in which case we should probably mark it r-.
Every time I upload this patch it already seems to be obsolete i.e. needs a non-trivial merge. It touches ~half the lines in each expectations file so that's not surprising.
I was thinking about submitting it 500 lines at a time, or at a time when everybody is asleep (i.e. in my (Sydney) time zone) -- though unfortunately that also means that most committers are also asleep. Maybe I can contact somebody in Tokyo.
There are definitely some changes which get much easier once you're a committer. :) Seems you need about 4 more patches before you can be nominated though.
(In reply to comment #11)
> There are definitely some changes which get much easier once you're a committer. :) Seems you need about 4 more patches before you can be nominated though.
Yeah this change can presumably wait until then.
(However, if this is submitted then I need one less... hmm.)
Created attachment 76590 [details]
Comment on attachment 76590 [details]
Will commit this shortly.
Comment on attachment 76590 [details]
Clearing flags on attachment: 76590
Committed r74070: <http://trac.webkit.org/changeset/74070>
All reviewed patches have been landed. Closing bug.
I believe this broke the python tests.
Yep(In reply to comment #17)
> I believe this broke the python tests.
http://trac.webkit.org/changeset/74070 might have broken SnowLeopard Intel Release (Tests)
The following tests are not passing:
(In reply to comment #18)
> Yep(In reply to comment #17)
> > I believe this broke the python tests.
Just need to change the occurrences of BUG1234 with BUGCR1234 in WebKitTools/Scripts/webkitpy/style/checkers/test_expectations_unittest.py
(In reply to comment #19)
> http://trac.webkit.org/changeset/74070 might have broken SnowLeopard Intel Release (Tests)
> The following tests are not passing:
I don't think so.
For what it's worth, this patch should've had a test added to webkitpy/layout_tests/layout_package/test_expectations.py .
And, there's no need to be duplicating all of this unit testing logic in two different files. The style checker file should just be a trivial wrapper that creates a TestExpectations object and logs any errors that occur.
I stumbled across this duplication (yet again) while testing the changes for bug 51222. I wouldn't be surprised if some of the style checking testing predated some of the unit tests, and prior to the cleanup in 51222, figuring out if things linted correctly was a little harder than it should be. I'll submit a separate patch to de-dup the code.