104850 2012-12-12 15:07:11 -0800 Fix detection of ref test files in old and new run webkit tests 2012-12-12 18:44:26 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) PC Windows 7 RESOLVED FIXED P2 Normal --- 1 roger_fong webkit-unassigned abarth dpranke esprehn roger_fong thorton webkit.review.bot oldest_to_newest 789960 0 roger_fong 2012-12-12 15:07:11 -0800 Previously skipped ref tests here: http://trac.webkit.org/changeset/125604 But only where results used the -expected tag. Some ref tests now use the -ref tag instead of -expected. Changing old-run-webkit-tests to reflect this. 790040 1 179147 roger_fong 2012-12-12 16:23:04 -0800 Created attachment 179147 patch 790141 2 179169 roger_fong 2012-12-12 17:56:21 -0800 Created attachment 179169 patch 790152 3 179169 thorton 2012-12-12 18:08:06 -0800 Comment on attachment 179169 patch View in context: https://bugs.webkit.org/attachment.cgi?id=179169&action=review > Tools/Scripts/old-run-webkit-tests:2638 > + my $prefixExtensionExpression = "^(($refTag|$notrefTag)-)"; Don't need the outer parens, I think. 790157 4 roger_fong 2012-12-12 18:13:20 -0800 http://trac.webkit.org/changeset/137553 790162 5 179169 dpranke 2012-12-12 18:16:34 -0800 Comment on attachment 179169 patch Looks like I was mistaken, and the W3C does not use ref- or noref- as prefixes for the references, just -ref and -notref instead of our -expected and -expected-mismatch. What they *do* use ref- for is for actual common reference files, which would live in a references/ directory (e.g., css2.1/reference/ref-filled-green-100px-square.htm ). However, in this case, we'd probably want to actually treat such tests as pixel tests, if we ran them at all. Note that we haven't actually imported any such references yet. See http://wiki.csswg.org/test/reftest and https://bugs.webkit.org/show_bug.cgi?id=83048 . 790177 6 esprehn 2012-12-12 18:34:05 -0800 This broke run-webkit-tests for me: Using port 'chromium-linux-x86_64' Test configuration: <lucid, x86_64, debug> Placing test results in /media/Checkouts/chrome1/src/webkit/Debug/layout-test-results Baseline search path: chromium-linux -> chromium-win -> chromium -> generic Using Debug build Pixel tests enabled Regular timeout: 12000, slow test timeout: 60000 Command line: /media/Checkouts/chrome1/src/out/Debug/DumpRenderTree - Collecting tests ... AttributeError raised: 'str' object has no attribute 'startsWith' Traceback (most recent call last): File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 407, in main run_details = run(port, options, args, stderr) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 105, in run run_details = manager.run(args) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 346, in run paths, test_names = self._collect_tests(args) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 265, in _collect_tests return self._finder.find_tests(self._options, args) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py", line 50, in find_tests test_files = self._port.tests(paths) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base.py", line 580, in tests tests = self._real_tests(expanded_paths) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base.py", line 600, in _real_tests files = find_files.find(self._filesystem, self.layout_tests_dir(), paths, skipped_directories, Port._is_test_file, self.test_key) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/common/find_files.py", line 61, in find return _normalized_find(filesystem, _normalize(filesystem, base_dir, paths), skipped_directories, file_filter, directory_sort_key) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/common/find_files.py", line 83, in _normalized_find all_files = itertools.chain(*(sort_by_directory_key(filesystem.files_under(path, skipped_directories, file_filter)) for path in paths_to_walk)) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/common/find_files.py", line 83, in <genexpr> all_files = itertools.chain(*(sort_by_directory_key(filesystem.files_under(path, skipped_directories, file_filter)) for path in paths_to_walk)) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/filesystem.py", line 109, in files_under if file_filter(self, dirpath, filename): File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base.py", line 626, in _is_test_file return Port._has_supported_extension(filesystem, filename) and not Port.is_reference_html_file(filesystem, dirname, filename) File "/media/Checkouts/chrome1/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base.py", line 610, in is_reference_html_file if filename.startswith('ref-') or filename.startsWith('notref-'): AttributeError: 'str' object has no attribute 'startsWith' Failed to execute Tools/Scripts/new-run-webkit-tests at ./Tools/Scripts/run-webkit-tests line 126. 790179 7 esprehn 2012-12-12 18:35:36 -0800 (In reply to comment #6) > This broke run-webkit-tests for me: > Note this is when I do ./Tools/Scripts/run-webkit-tests --debug --chromium LayoutTests/ 790181 8 179169 esprehn 2012-12-12 18:37:27 -0800 Comment on attachment 179169 patch View in context: https://bugs.webkit.org/attachment.cgi?id=179169&action=review > Tools/Scripts/webkitpy/layout_tests/port/base.py:610 > + if filename.startswith('ref-') or filename.startsWith('notref-'): This is a typo, the method in python is all lower case. 790187 9 roger_fong 2012-12-12 18:44:26 -0800 Sorry fixed it r137558 179147 2012-12-12 16:23:04 -0800 2012-12-12 17:56:28 -0800 patch ref_patch.txt text/plain 1929 roger_fong SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDEzNzUxNykKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDEyIEBACisyMDEyLTEyLTEyICBSb2dlciBGb25nICA8cm9nZXJfZm9uZ0BhcHBsZS5jb20+ CisKKyAgICAgICAgTWFrZSBvbGQtcnVuLXdlYmtpdC10ZXN0cyBza2lwIHJlZiB0ZXN0cyB3aGVy ZSByZXN1bHRzIGFyZSBtYXJrZWQgd2l0aCAtcmVmIG9yIC1ub3JlZiB0YWcuCisgICAgICAgIGh0 dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0xMDQ4NTAuCisKKyAgICAgICAg UmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgKiBTY3JpcHRzL29sZC1ydW4t d2Via2l0LXRlc3RzOgorCiAyMDEyLTEyLTEyICBEaXJrIFByYW5rZSAgPGRwcmFua2VAY2hyb21p dW0ub3JnPgogCiAgICAgICAgIG5yd3Q6IHJlbW92ZSBkZXByZWNhdGVkIE9SV1QgY29tcGF0aWJp bGl0eSBjb21tYW5kIGxpbmUgZmxhZ3MKSW5kZXg6IFRvb2xzL1NjcmlwdHMvb2xkLXJ1bi13ZWJr aXQtdGVzdHMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvU2NyaXB0cy9vbGQtcnVuLXdlYmtpdC10ZXN0 cwkocmV2aXNpb24gMTM3NTAyKQorKysgVG9vbHMvU2NyaXB0cy9vbGQtcnVuLXdlYmtpdC10ZXN0 cwkod29ya2luZyBjb3B5KQpAQCAtMjAwLDYgKzIwMCw3IEBAICR0ZXN0SFRUUCA9IDAgaWYgKGlz V3goKSk7CiBteSAkcGVybEludGVycHJldGVyID0gInBlcmwiOwogCiBteSAkZXhwZWN0ZWRUYWcg PSAiZXhwZWN0ZWQiOworbXkgJHJlZlRhZyA9ICJyZWYiOwogbXkgJG1pc21hdGNoVGFnID0gIm1p c21hdGNoIjsKIG15ICRhY3R1YWxUYWcgPSAiYWN0dWFsIjsKIG15ICRwcmV0dHlEaWZmVGFnID0g InByZXR0eS1kaWZmIjsKQEAgLTI2MzIsNyArMjYzMyw3IEBAIHN1YiBpc1VzZWRJblJlZnRlc3Qo JCkKICAgICBteSAkZmlsZW5hbWUgPSAkX1swXTsKICAgICBteSBAZXh0ZW5zaW9ucyA9ICgnaHRt bCcsJ3NodG1sJywneG1sJywneGh0bWwnLCdodG0nLCdwaHAnLCdzdmcnLCdtaHQnLCdwbCcpOwog ICAgIG15ICRleHRlbnNpb25zSm9pbmVkID0gam9pbigifCIsIEBleHRlbnNpb25zKTsKLSAgICBt eSAkZXh0ZW5zaW9uRXhwcmVzc2lvbiA9ICItJGV4cGVjdGVkVGFnKC0kbWlzbWF0Y2hUYWcpP1xc LigiLiRleHRlbnNpb25zSm9pbmVkLiIpXCQiOworICAgIG15ICRleHRlbnNpb25FeHByZXNzaW9u ID0gIi0oJGV4cGVjdGVkVGFnfCRyZWZUYWcpKC0kbWlzbWF0Y2hUYWcpP1xcLigiLiRleHRlbnNp b25zSm9pbmVkLiIpXCQiOwogICAgIAogICAgIGlmICgkZmlsZW5hbWUgPX4gLyRleHRlbnNpb25F eHByZXNzaW9uLykgewogICAgICAgICByZXR1cm4gMTsKQEAgLTI2NDAsNyArMjY0MSw3IEBAIHN1 YiBpc1VzZWRJblJlZnRlc3QoJCkKICAgICBteSAkYmFzZSA9IHN0cmlwRXh0ZW5zaW9uKCRmaWxl bmFtZSk7CiAgICAgCiAgICAgZm9yZWFjaCBteSAkZXh0ZW5zaW9uIChAZXh0ZW5zaW9ucykgewot ICAgICAgICBpZiAoLWYgIiRiYXNlLSRleHBlY3RlZFRhZy4kZXh0ZW5zaW9uIiB8fCAtZiAiJGJh c2UtJGV4cGVjdGVkVGFnLSRtaXNtYXRjaFRhZy4kZXh0ZW5zaW9uIikgeworICAgICAgICBpZiAo LWYgIiRiYXNlLSRleHBlY3RlZFRhZy4kZXh0ZW5zaW9uIiB8fCAtZiAiJGJhc2UtJHJlZlRhZy4k ZXh0ZW5zaW9uIiB8fC1mICIkYmFzZS0kZXhwZWN0ZWRUYWctJG1pc21hdGNoVGFnLiRleHRlbnNp b24iKSB7CiAgICAgICAgICAgICByZXR1cm4gMTsKICAgICAgICAgfQogICAgIH0K 179169 2012-12-12 17:56:21 -0800 2012-12-12 18:37:27 -0800 patch refpatch.patch text/plain 3345 roger_fong SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDEzNzU0MCkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE4IEBACisyMDEyLTEyLTEyICBSb2dlciBGb25nICA8cm9nZXJfZm9uZ0BhcHBsZS5jb20+ CisKKyAgICAgICAgRml4IGRldGVjdGlvbiBvZiByZWYgdGVzdCBmaWxlcyBpbiBvbGQgYW5kIG5l dyBydW4gd2Via2l0IHRlc3RzLgorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93 X2J1Zy5jZ2k/aWQ9MTA0ODUwLgorCisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEp LgorCisgICAgICAgICogU2NyaXB0cy9vbGQtcnVuLXdlYmtpdC10ZXN0czoKKyAgICAgICAgKGlz VXNlZEluUmVmdGVzdCk6IAorICAgICAgICBBZGQgbm90cmVmIGFuZCByZWYgcHJlZml4IGFuZCBz dWZmaXggY2FzZXMgdG8gaXNVc2VkaW5SZWZ0ZXN0LgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0 cHkvbGF5b3V0X3Rlc3RzL3BvcnQvYmFzZS5weToKKyAgICAgICAgKFBvcnQpOgorICAgICAgICAo UG9ydC5pc19yZWZlcmVuY2VfaHRtbF9maWxlKTogCisgICAgICAgIFR5cG8gaW4gTlJXVC4KKwog MjAxMi0xMi0xMiAgRXJpYyBTZWlkZWwgIDxlcmljQHdlYmtpdC5vcmc+CiAKICAgICAgICAgQWRk IHN1cHBvcnQgZm9yIHJ1bi1wZXJmLXRlc3RzIC0tY2hyb21pdW0tYW5kcm9pZCAtLXByb2ZpbGUK SW5kZXg6IFRvb2xzL1NjcmlwdHMvb2xkLXJ1bi13ZWJraXQtdGVzdHMKPT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0g VG9vbHMvU2NyaXB0cy9vbGQtcnVuLXdlYmtpdC10ZXN0cwkocmV2aXNpb24gMTM3NTAyKQorKysg VG9vbHMvU2NyaXB0cy9vbGQtcnVuLXdlYmtpdC10ZXN0cwkod29ya2luZyBjb3B5KQpAQCAtMjAx LDYgKzIwMSw4IEBAIG15ICRwZXJsSW50ZXJwcmV0ZXIgPSAicGVybCI7CiAKIG15ICRleHBlY3Rl ZFRhZyA9ICJleHBlY3RlZCI7CiBteSAkbWlzbWF0Y2hUYWcgPSAibWlzbWF0Y2giOworbXkgJHJl ZlRhZyA9ICJyZWYiOworbXkgJG5vdHJlZlRhZyA9ICJub3RyZWYiOwogbXkgJGFjdHVhbFRhZyA9 ICJhY3R1YWwiOwogbXkgJHByZXR0eURpZmZUYWcgPSAicHJldHR5LWRpZmYiOwogbXkgJGRpZmZz VGFnID0gImRpZmZzIjsKQEAgLTI2MzIsMTUgKzI2MzQsMTggQEAgc3ViIGlzVXNlZEluUmVmdGVz dCgkKQogICAgIG15ICRmaWxlbmFtZSA9ICRfWzBdOwogICAgIG15IEBleHRlbnNpb25zID0gKCdo dG1sJywnc2h0bWwnLCd4bWwnLCd4aHRtbCcsJ2h0bScsJ3BocCcsJ3N2ZycsJ21odCcsJ3BsJyk7 CiAgICAgbXkgJGV4dGVuc2lvbnNKb2luZWQgPSBqb2luKCJ8IiwgQGV4dGVuc2lvbnMpOwotICAg IG15ICRleHRlbnNpb25FeHByZXNzaW9uID0gIi0kZXhwZWN0ZWRUYWcoLSRtaXNtYXRjaFRhZyk/ XFwuKCIuJGV4dGVuc2lvbnNKb2luZWQuIilcJCI7Ci0gICAgCi0gICAgaWYgKCRmaWxlbmFtZSA9 fiAvJGV4dGVuc2lvbkV4cHJlc3Npb24vKSB7CisgICAgbXkgJHN1ZmZpeEV4dGVuc2lvbkV4cHJl c3Npb24gPSAiLSgkZXhwZWN0ZWRUYWd8JHJlZlRhZ3wkbm90cmVmVGFnKSgtJG1pc21hdGNoVGFn KT9cXC4oIi4kZXh0ZW5zaW9uc0pvaW5lZC4iKVwkIjsKKyAgICBteSAkcHJlZml4RXh0ZW5zaW9u RXhwcmVzc2lvbiA9ICJeKCgkcmVmVGFnfCRub3RyZWZUYWcpLSkiOworICAgIGlmICgkZmlsZW5h bWUgPX4gLyRzdWZmaXhFeHRlbnNpb25FeHByZXNzaW9uLyB8fCAkZmlsZW5hbWUgPX4gLyRwcmVm aXhFeHRlbnNpb25FeHByZXNzaW9uLykgewogICAgICAgICByZXR1cm4gMTsKICAgICB9CiAgICAg bXkgJGJhc2UgPSBzdHJpcEV4dGVuc2lvbigkZmlsZW5hbWUpOwotICAgIAorCiAgICAgZm9yZWFj aCBteSAkZXh0ZW5zaW9uIChAZXh0ZW5zaW9ucykgewotICAgICAgICBpZiAoLWYgIiRiYXNlLSRl eHBlY3RlZFRhZy4kZXh0ZW5zaW9uIiB8fCAtZiAiJGJhc2UtJGV4cGVjdGVkVGFnLSRtaXNtYXRj aFRhZy4kZXh0ZW5zaW9uIikgeworICAgICAgICBpZiAoLWYgIiRiYXNlLSRleHBlY3RlZFRhZy4k ZXh0ZW5zaW9uIiB8fCAKKyAgICAgICAgICAgIC1mICIkYmFzZS0kcmVmVGFnLiRleHRlbnNpb24i IHx8IC1mICIkYmFzZS0kbm90cmVmVGFnLiRleHRlbnNpb24iIHx8CisgICAgICAgICAgICAtZiAi JGJhc2UtJGV4cGVjdGVkVGFnLSRtaXNtYXRjaFRhZy4kZXh0ZW5zaW9uIiB8fAorICAgICAgICAg ICAgLWYgIiRyZWZUYWctJGJhc2UuJGV4dGVuc2lvbiIgfHwgLWYgIiRub3RyZWZUYWctJGJhc2Uu JGV4dGVuc2lvbiIpIHsKICAgICAgICAgICAgIHJldHVybiAxOwogICAgICAgICB9CiAgICAgfQpJ bmRleDogVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC9iYXNlLnB5Cj09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT0KLS0tIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvYmFz ZS5weQkocmV2aXNpb24gMTM3NTAyKQorKysgVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRf dGVzdHMvcG9ydC9iYXNlLnB5CSh3b3JraW5nIGNvcHkpCkBAIC02MDUsOCArNjA1LDkgQEAgY2xh c3MgUG9ydChvYmplY3QpOgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAn Lmh0bScsICcucGhwJywgJy5zdmcnLCAnLm1odCddKQogCiAgICAgQHN0YXRpY21ldGhvZAorICAg ICMgSWYgYW55IGNoYW5nZXMgYXJlIG1hZGUgaGVyZSBiZSBzdXJlIHRvIHVwZGF0ZSB0aGUgaXNV c2VkSW5SZWZ0ZXN0IG1ldGhvZCBpbiBvbGQtcnVuLXdlYmtpdC10ZXN0cyBhcyB3ZWxsLgogICAg IGRlZiBpc19yZWZlcmVuY2VfaHRtbF9maWxlKGZpbGVzeXN0ZW0sIGRpcm5hbWUsIGZpbGVuYW1l KToKLSAgICAgICAgaWYgZmlsZW5hbWUuc3RhcnRzd2l0aCgncmVmLScpIG9yIGZpbGVuYW1lLmVu ZHN3aXRoKCdub3RyZWYtJyk6CisgICAgICAgIGlmIGZpbGVuYW1lLnN0YXJ0c3dpdGgoJ3JlZi0n KSBvciBmaWxlbmFtZS5zdGFydHNXaXRoKCdub3RyZWYtJyk6CiAgICAgICAgICAgICByZXR1cm4g VHJ1ZQogICAgICAgICBmaWxlbmFtZV93aWhvdXRfZXh0LCB1bnVzZWQgPSBmaWxlc3lzdGVtLnNw bGl0ZXh0KGZpbGVuYW1lKQogICAgICAgICBmb3Igc3VmZml4IGluIFsnLWV4cGVjdGVkJywgJy1l eHBlY3RlZC1taXNtYXRjaCcsICctcmVmJywgJy1ub3RyZWYnXToK