215054 2020-08-01 00:44:21 -0700 Prevent multiple invocations of generate-message-receiver.py 2020-08-03 09:30:40 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=215064 https://bugs.webkit.org/show_bug.cgi?id=211112 InRadar P2 Normal --- 1 krollin krollin darin dino graouts simon.fraser webkit-bug-importer oldest_to_newest 1677095 0 krollin 2020-08-01 00:44:21 -0700 WebKit/DerivedSources.make is written such that generate-message-receiver.py can be invoked N times, where N is the value passed to -j. This results in the N invocations of generate-message-receiver.py trying trying to generate the same set of files at the same time, possibly leading to their being corrupted. We have reports of generated files starting with a 4K block of NULs, and this contention may be causing it. Address this by using a "pattern rule" in the makefile, which is documented in GNU make's documentation as the approach to take when there is one build rule that creates multiple outputs, as is the case here. See also Bug 166814 and <http://trac.webkit.org/changeset/210507>. 1677096 1 krollin 2020-08-01 00:44:26 -0700 rdar://problem/63595589 1677098 2 405775 krollin 2020-08-01 00:47:10 -0700 Created attachment 405775 Patch 1677133 3 ews-feeder 2020-08-01 10:54:49 -0700 Committed r265178: <https://trac.webkit.org/changeset/265178> All reviewed patches have been landed. Closing bug and clearing flags on attachment 405775. 1677147 4 405775 darin 2020-08-01 13:40:25 -0700 Comment on attachment 405775 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=405775&action=review > Source/WebKit/DerivedSources.make:257 > +MESSAGES_FILES := $(addsufix Messages.h,$(notdir $(MESSAGE_RECEIVERS))) The "addsuffix" here was missing an "f" so this didn’t quite work. Also, I did some further research, and I think a better solution is a "grouped target" rather than a pattern rule. 1677152 5 darin 2020-08-01 14:08:10 -0700 I put a fix for those things into bug 215064. 1677241 6 krollin 2020-08-02 18:09:24 -0700 I commented in <rdar://problem/63595589> that "grouped targets" won't work. 1677242 7 krollin 2020-08-02 18:10:32 -0700 That missing "f" was from the original code. I wonder how long it had been broken that way. 1677243 8 krollin 2020-08-02 18:12:32 -0700 Issue was introduced May 6 with r261254. The first report of the generated file corruption was on May 25. 1677248 9 darin 2020-08-02 19:30:16 -0700 That revision replaced the existing pattern rule with a non-pattern rule! I think we found the root cause. 405775 2020-08-01 00:47:10 -0700 2020-08-01 10:54:49 -0700 Patch bug-215054-20200801004709.patch text/plain 3752 krollin U3VidmVyc2lvbiBSZXZpc2lvbjogMjY1MTYwCmRpZmYgLS1naXQgYS9Tb3VyY2UvV2ViS2l0L0No YW5nZUxvZyBiL1NvdXJjZS9XZWJLaXQvQ2hhbmdlTG9nCmluZGV4IDZmYzRlZDY5ZjljOTlhNmE0 MTY0ZmFlNTViMzdmZmY0OTIzOTVjZDAuLjcyYjFhZDk4ODg1N2M0YWRhOWYwOWYzZjQ1YzJlYjZl N2I0YWMxNzIgMTAwNjQ0Ci0tLSBhL1NvdXJjZS9XZWJLaXQvQ2hhbmdlTG9nCisrKyBiL1NvdXJj ZS9XZWJLaXQvQ2hhbmdlTG9nCkBAIC0xLDMgKzEsMjggQEAKKzIwMjAtMDgtMDEgIEtlaXRoIFJv bGxpbiAgPGtyb2xsaW5AYXBwbGUuY29tPgorCisgICAgICAgIFByZXZlbnQgbXVsdGlwbGUgaW52 b2NhdGlvbnMgb2YgZ2VuZXJhdGUtbWVzc2FnZS1yZWNlaXZlci5weQorICAgICAgICBodHRwczov L2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MjE1MDU0CisKKyAgICAgICAgUmV2aWV3 ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgV2ViS2l0L0Rlcml2ZWRTb3VyY2VzLm1h a2UgaXMgd3JpdHRlbiBzdWNoIHRoYXQKKyAgICAgICAgZ2VuZXJhdGUtbWVzc2FnZS1yZWNlaXZl ci5weSBjYW4gYmUgaW52b2tlZCBOIHRpbWVzLCB3aGVyZSBOIGlzIHRoZQorICAgICAgICB2YWx1 ZSBwYXNzZWQgdG8gLWouIFRoaXMgcmVzdWx0cyBpbiB0aGUgTiBpbnZvY2F0aW9ucyBvZgorICAg ICAgICBnZW5lcmF0ZS1tZXNzYWdlLXJlY2VpdmVyLnB5IHRyeWluZyB0cnlpbmcgdG8gZ2VuZXJh dGUgdGhlIHNhbWUgc2V0IG9mCisgICAgICAgIGZpbGVzIGF0IHRoZSBzYW1lIHRpbWUsIHBvc3Np Ymx5IGxlYWRpbmcgdG8gdGhlaXIgYmVpbmcgY29ycnVwdGVkLiBXZQorICAgICAgICBoYXZlIHJl cG9ydHMgb2YgZ2VuZXJhdGVkIGZpbGVzIHN0YXJ0aW5nIHdpdGggYSA0SyBibG9jayBvZiBOVUxz LCBhbmQKKyAgICAgICAgdGhpcyBjb250ZW50aW9uIG1heSBiZSBjYXVzaW5nIGl0LiBBZGRyZXNz IHRoaXMgYnkgdXNpbmcgYSAicGF0dGVybgorICAgICAgICBydWxlIiBpbiB0aGUgbWFrZWZpbGUs IHdoaWNoIGlzIGRvY3VtZW50ZWQgaW4gR05VIG1ha2UncyBkb2N1bWVudGF0aW9uCisgICAgICAg IGFzIHRoZSBhcHByb2FjaCB0byB0YWtlIHdoZW4gdGhlcmUgaXMgb25lIGJ1aWxkIHJ1bGUgdGhh dCBjcmVhdGVzCisgICAgICAgIG11bHRpcGxlIG91dHB1dHMsIGFzIGlzIHRoZSBjYXNlIGhlcmUu CisKKyAgICAgICAgU2VlIGFsc28gPGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNn aT9pZD0xNjY4MTQ+IGFuZAorICAgICAgICA8aHR0cDovL3RyYWMud2Via2l0Lm9yZy9jaGFuZ2Vz ZXQvMjEwNTA3Pi4KKworICAgICAgICBObyBuZXcgdGVzdHMgLS0gbm8gbmV3IG9yIGNoYW5nZWQg ZnVuY3Rpb25hbGl0eS4KKworICAgICAgICAqIERlcml2ZWRTb3VyY2VzLm1ha2U6CisKIDIwMjAt MDctMzEgIEppZXdlbiBUYW4gIDxqaWV3ZW5fdGFuQGFwcGxlLmNvbT4KIAogICAgICAgICBTT0F1 dGhvcml6YXRpb25TZXNzaW9uOjpkaXNtaXNzVmlld0NvbnRyb2xsZXIgY291bGQgY3Jhc2ggb24g Y2FsbGluZyBXZWJQYWdlUHJveHk6OnBsYXRmb3JtV2luZG93CmRpZmYgLS1naXQgYS9Tb3VyY2Uv V2ViS2l0L0Rlcml2ZWRTb3VyY2VzLm1ha2UgYi9Tb3VyY2UvV2ViS2l0L0Rlcml2ZWRTb3VyY2Vz Lm1ha2UKaW5kZXggZGZhN2MyYjVlNTMxMmVjNTFkZDZmZTY5MjI0MzM5YmU4MTdjYTdiYy4uYmU4 NzRkMTViZWNmY2I0MTI0ODE2NjFkMGNmNzE4NGUwNDBkM2NlZCAxMDA2NDQKLS0tIGEvU291cmNl L1dlYktpdC9EZXJpdmVkU291cmNlcy5tYWtlCisrKyBiL1NvdXJjZS9XZWJLaXQvRGVyaXZlZFNv dXJjZXMubWFrZQpAQCAtMjUzLDE2ICsyNTMsMjYgQEAgU0NSSVBUUyA9IFwKIEZSQU1FV09SS19G TEFHUyA6PSAkKHNoZWxsIGVjaG8gJChCVUlMVF9QUk9EVUNUU19ESVIpICQoRlJBTUVXT1JLX1NF QVJDSF9QQVRIUykgJChTWVNURU1fRlJBTUVXT1JLX1NFQVJDSF9QQVRIUykgfCBwZXJsIC1lICdw cmludCAiLUYgIiAuIGpvaW4oIiAtRiAiLCBzcGxpdCgiICIsIDw+KSk7JykKIEhFQURFUl9GTEFH UyA6PSAkKHNoZWxsIGVjaG8gJChCVUlMVF9QUk9EVUNUU19ESVIpICQoSEVBREVSX1NFQVJDSF9Q QVRIUykgJChTWVNURU1fSEVBREVSX1NFQVJDSF9QQVRIUykgfCBwZXJsIC1lICdwcmludCAiLUki IC4gam9pbigiIC1JIiwgc3BsaXQoIiAiLCA8PikpOycpCiAKK01FU1NBR0VfUkVDRUlWRVJfRklM RVMgOj0gJChhZGRzdWZmaXggTWVzc2FnZVJlY2VpdmVyLmNwcCwkKG5vdGRpciAkKE1FU1NBR0Vf UkVDRUlWRVJTKSkpCitNRVNTQUdFU19GSUxFUyA6PSAkKGFkZHN1Zml4IE1lc3NhZ2VzLmgsJChu b3RkaXIgJChNRVNTQUdFX1JFQ0VJVkVSUykpKQorTUVTU0FHRV9SRVBMSUVTX0ZJTEVTIDo9ICQo YWRkc3VmZml4IE1lc3NhZ2VzUmVwbGllcy5oLCQobm90ZGlyICQoTUVTU0FHRV9SRUNFSVZFUlMp KSkKK01FU1NBR0VTX0lOX0ZJTEVTIDo9ICQoYWRkc3VmZml4IC5tZXNzYWdlcy5pbiwkKE1FU1NB R0VfUkVDRUlWRVJTKSkKKworTUVTU0FHRV9SRUNFSVZFUl9QQVRURVJOUyA6PSAkKHN1YnN0IC4s JSwkKE1FU1NBR0VfUkVDRUlWRVJfRklMRVMpKQorTUVTU0FHRVNfUEFUVEVSTlMgOj0gJChzdWJz dCAuLCUsJChNRVNTQUdFU19GSUxFUykpCitNRVNTQUdFX1JFUExJRVNfUEFUVEVSTlMgOj0gJChz dWJzdCAuLCUsJChNRVNTQUdFX1JFUExJRVNfRklMRVMpKQorCiAuUEhPTlkgOiBhbGwKIAotYWxs IDogTWVzc2FnZU5hbWVzLmggTWVzc2FnZU5hbWVzLmNwcCBcCi0gICAgJChhZGRzdWZmaXggTWVz c2FnZVJlY2VpdmVyLmNwcCwkKG5vdGRpciAkKE1FU1NBR0VfUkVDRUlWRVJTKSkpIFwKLSAgICAk KGFkZHN1ZmZpeCBNZXNzYWdlcy5oLCQobm90ZGlyICQoTUVTU0FHRV9SRUNFSVZFUlMpKSkgXAot ICAgICQoYWRkc3VmZml4IE1lc3NhZ2VzUmVwbGllcy5oLCQobm90ZGlyICQoTUVTU0FHRV9SRUNF SVZFUlMpKSkgXAorYWxsIDogXAorICAgICQoTUVTU0FHRV9SRUNFSVZFUl9GSUxFUykgXAorICAg ICQoTUVTU0FHRVNfRklMRVMpIFwKKyAgICAkKE1FU1NBR0VfUkVQTElFU19GSUxFUykgXAorICAg IE1lc3NhZ2VOYW1lcy5oIE1lc3NhZ2VOYW1lcy5jcHAgXAogIwogCi0kKGFkZHN1ZmZpeCBNZXNz YWdlUmVjZWl2ZXIuY3BwLCQobm90ZGlyICQoTUVTU0FHRV9SRUNFSVZFUlMpKSkgJChhZGRzdWZp eCBNZXNzYWdlcy5oLCQobm90ZGlyICQoTUVTU0FHRV9SRUNFSVZFUlMpKSkgJChhZGRzdWZmaXgg TWVzc2FnZXNSZXBsaWVzLmgsJChub3RkaXIgJChNRVNTQUdFX1JFQ0VJVkVSUykpKSBNZXNzYWdl TmFtZXMuaCBNZXNzYWdlTmFtZXMuY3BwOiAkKGFkZHN1ZmZpeCAubWVzc2FnZXMuaW4sJChNRVNT QUdFX1JFQ0VJVkVSUykpICQoU0NSSVBUUykKLQlAcHl0aG9uICQoV2ViS2l0MikvU2NyaXB0cy9n ZW5lcmF0ZS1tZXNzYWdlLXJlY2VpdmVyLnB5ICQoV2ViS2l0MikgJChNRVNTQUdFX1JFQ0VJVkVS UykKKyQoTUVTU0FHRV9SRUNFSVZFUl9QQVRURVJOUykgJChNRVNTQUdFU19QQVRURVJOUykgJChN RVNTQUdFX1JFUExJRVNfUEFUVEVSTlMpIE1lc3NhZ2VOYW1lcyVoIE1lc3NhZ2VOYW1lcyVjcHAg OiAkKE1FU1NBR0VTX0lOX0ZJTEVTKSAkKFNDUklQVFMpCisJcHl0aG9uICQoV2ViS2l0MikvU2Ny aXB0cy9nZW5lcmF0ZS1tZXNzYWdlLXJlY2VpdmVyLnB5ICQoV2ViS2l0MikgJChNRVNTQUdFX1JF Q0VJVkVSUykKIAogVEVYVF9QUkVQUk9DRVNTT1JfRkxBR1M9LUUgLVAgLXcKIAo=