Using BUG/BUGWK in test_expectations is error prone, should use BUGCR/BUGWK
Created attachment 72833 [details] Patch
FYI, script to update the expectations files was for f in `find LayoutTests -name '*_expectations.txt'`; do sed -E -i '' "s/BUG([0-9]+)/BUGCR\1/g" "$f" done
Comment on attachment 72833 [details] Patch Please send out and email to chromium-dev announcing this is about to change. Then feel free to commit. You'll need to make sure you're synced as this file changes frequently.
Created attachment 75967 [details] Patch
Created attachment 75969 [details] Script used to create the test_expectations.txt changes
Created attachment 75970 [details] Output from running rename.sh
Ok, finally revisited this (i.e. synced + re-ran script -- which takes forever). Just an update, I noticed that there were more than a few uses of BUG to indicate a webkit bug (BUGWK) rather than a chromium bug (BUGCR). Rather than sorting through hundreds of them myself, rename.sh attempts to determine (via wget + grep) which it should be. Sometimes the script fails. Last time I ran it I actually did resolve these by hands, but didn't this time. I attached the output of running the script (rename.out) just for the record.
Comment on attachment 75967 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=75967&action=review This looks great! > LayoutTests/ChangeLog:9 > + For all test_expectations.txt files, change all occurrences of BUG to > + either BUGCR or BUGWK. Indentation is off here. Use spaces, not tabs. > WebKitTools/ChangeLog:9 > + Add presubmit check that BUG isn't used, either BUGCR or BUGWK. ditto re: spaces > WebKitTools/Scripts/webkitpy/layout_tests/layout_package/test_expectations.py:600 > + self._add_error(lineno, 'Bug must be either BUGCR or BUGWK for test: %s' % Nit: now there is also BUGV8_XXX
Ojan: Benjamin is not a committer. Should this be marked cq+? I expect he's going to need to upload a new patch in which case we should probably mark it r-.
(In reply to comment #9) > Ojan: Benjamin is not a committer. Should this be marked cq+? I expect he's going to need to upload a new patch in which case we should probably mark it r-. Every time I upload this patch it already seems to be obsolete i.e. needs a non-trivial merge. It touches ~half the lines in each expectations file so that's not surprising. I was thinking about submitting it 500 lines at a time, or at a time when everybody is asleep (i.e. in my (Sydney) time zone) -- though unfortunately that also means that most committers are also asleep. Maybe I can contact somebody in Tokyo.
There are definitely some changes which get much easier once you're a committer. :) Seems you need about 4 more patches before you can be nominated though.
(In reply to comment #11) > There are definitely some changes which get much easier once you're a committer. :) Seems you need about 4 more patches before you can be nominated though. Yeah this change can presumably wait until then. (However, if this is submitted then I need one less... hmm.)
Created attachment 76590 [details] Patch
Comment on attachment 76590 [details] Patch Will commit this shortly.
Comment on attachment 76590 [details] Patch Clearing flags on attachment: 76590 Committed r74070: <http://trac.webkit.org/changeset/74070>
All reviewed patches have been landed. Closing bug.
I believe this broke the python tests.
Yep(In reply to comment #17) > I believe this broke the python tests. Yep.
http://trac.webkit.org/changeset/74070 might have broken SnowLeopard Intel Release (Tests) The following tests are not passing: fast/css/focus-ring-detached.html fast/css/focus-ring-multiline.html
(In reply to comment #18) > Yep(In reply to comment #17) > > I believe this broke the python tests. > > Yep. Just need to change the occurrences of BUG1234 with BUGCR1234 in WebKitTools/Scripts/webkitpy/style/checkers/test_expectations_unittest.py
(In reply to comment #19) > http://trac.webkit.org/changeset/74070 might have broken SnowLeopard Intel Release (Tests) > The following tests are not passing: > fast/css/focus-ring-detached.html > fast/css/focus-ring-multiline.html I don't think so.
For what it's worth, this patch should've had a test added to webkitpy/layout_tests/layout_package/test_expectations.py . And, there's no need to be duplicating all of this unit testing logic in two different files. The style checker file should just be a trivial wrapper that creates a TestExpectations object and logs any errors that occur. I stumbled across this duplication (yet again) while testing the changes for bug 51222. I wouldn't be surprised if some of the style checking testing predated some of the unit tests, and prior to the cleanup in 51222, figuring out if things linted correctly was a little harder than it should be. I'll submit a separate patch to de-dup the code.