134804 2014-07-10 09:56:35 -0700 [GTK] Do not include files that are not in git in the tarball 2014-07-31 01:41:58 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED Gtk P2 Normal --- 1 cgarcia webkit-unassigned gustavo svillar oldest_to_newest 1021641 0 cgarcia 2014-07-10 09:56:35 -0700 With the current approach of listing the source directory and add all files except the ones excluded by the manifest, we might end up with undesired files in the tarball, like new files of work in progress patches, or files you worked on once and forgot to remove. It could be quite embarrassing. I think it would be safer to skip any files not in git, except for directories added from the builddir (the documentation) 1021644 1 234708 cgarcia 2014-07-10 10:00:34 -0700 Created attachment 234708 Patch 1021652 2 234708 mrobinson 2014-07-10 10:30:44 -0700 Comment on attachment 234708 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=234708&action=review > Tools/gtk/make-dist.py:94 > + self.files_in_cms = self.list_cms_files() I'd prefer files_ins_vcs or files_in_version_control > Tools/gtk/make-dist.py:104 > + cmd = ['git', 'ls-tree', '-r', '--name-only', 'HEAD', self.source_root] > + p = subprocess.Popen(cmd, stdout=subprocess.PIPE) This could be one line. This doesn't really support SVN. Is there a script in the WebKit scripts that can do this for us? > Tools/gtk/make-dist.py:129 > - if not passes_all_rules(file): > + if not passes_all_rules(file) or self.should_skip_file(file): Is this going to do the right thing for files from the build directory? 1021788 3 cgarcia 2014-07-10 23:13:30 -0700 (In reply to comment #2) > (From update of attachment 234708 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=234708&action=review > > > Tools/gtk/make-dist.py:94 > > + self.files_in_cms = self.list_cms_files() > > I'd prefer files_ins_vcs or files_in_version_control I actually meant scm :-P which is what webkitpy uses, but I don't really mind either vcs or version_control > > Tools/gtk/make-dist.py:104 > > + cmd = ['git', 'ls-tree', '-r', '--name-only', 'HEAD', self.source_root] > > + p = subprocess.Popen(cmd, stdout=subprocess.PIPE) > > This could be one line. This doesn't really support SVN. Is there a script in the WebKit scripts that can do this for us? No, there isn't. It's very unlikely that someone will make a release without a git mirror, so I don't think it's worth it. I can leave a FIXME just in case someone want to add support for it (I won't) > > Tools/gtk/make-dist.py:129 > > - if not passes_all_rules(file): > > + if not passes_all_rules(file) or self.should_skip_file(file): > > Is this going to do the right thing for files from the build directory? Yes, it actually allows individual files when the source root is not in git (this is the case of files in buildir), assuming that in those cases they have been explicitly added by the manifest. We can try to be more restrictive here, and allow only files that are not in git when they are in the build dir. 1021803 4 234750 cgarcia 2014-07-11 02:33:29 -0700 Created attachment 234750 Patch Updated patch. Addressed review comments, and made the rule to skip files more restrictive to only allow files not in git when they are added from build dir. 1025923 5 234750 pnormand 2014-07-31 01:24:41 -0700 Comment on attachment 234750 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=234750&action=review > Tools/ChangeLog:9 > + Skip all files in the source tree that are not under the control > + version, except for files add from the build dir like the documentation. Nit: "not under version control" and "files added from..." 1025927 6 cgarcia 2014-07-31 01:41:58 -0700 Committed r171845: <http://trac.webkit.org/changeset/171845> 234708 2014-07-10 10:00:34 -0700 2014-07-11 02:33:29 -0700 Patch wk-distcheck-git.diff text/plain 2678 cgarcia ZGlmZiAtLWdpdCBhL1Rvb2xzL0NoYW5nZUxvZyBiL1Rvb2xzL0NoYW5nZUxvZwppbmRleCA5ZjJh NjM0Li5hNzcyYWQxIDEwMDY0NAotLS0gYS9Ub29scy9DaGFuZ2VMb2cKKysrIGIvVG9vbHMvQ2hh bmdlTG9nCkBAIC0xLDUgKzEsMjQgQEAKIDIwMTQtMDctMTAgIENhcmxvcyBHYXJjaWEgQ2FtcG9z ICA8Y2dhcmNpYUBpZ2FsaWEuY29tPgogCisgICAgICAgIFtHVEtdIERvIG5vdCBpbmNsdWRlIGZp bGVzIHRoYXQgYXJlIG5vdCBpbiBnaXQgaW4gdGhlIHRhcmJhbGwKKyAgICAgICAgaHR0cHM6Ly9i dWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTEzNDgwNAorCisgICAgICAgIFJldmlld2Vk IGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIFNraXAgYWxsIGZpbGVzIGluIHRoZSBzb3Vy Y2UgdHJlZSB0aGF0IGFyZSBub3QgdW5kZXIgdGhlIGNvbnRyb2wKKyAgICAgICAgdmVyc2lvbiwg ZXhjZXB0IGZvciBmaWxlcyBhZGQgZnJvbSB0aGUgYnVpbGQgZGlyIGxpa2UgdGhlIGRvY3VtZW50 YXRpb24uCisKKyAgICAgICAgKiBndGsvbWFrZS1kaXN0LnB5OgorICAgICAgICAoRGlyZWN0b3J5 Ll9faW5pdF9fKTogSW5pdGlhbGl6ZSB0aGUgbGlzdCBvZiBmaWxlcyB1bmRlciBjb250cm9sIHZl cnNpb24uCisgICAgICAgIChEaXJlY3RvcnkubGlzdF9jbXNfZmlsZXMpOiBIZWxwZXIgZnVuY3Rp b24gdG8gbGlzdCBhbGwgZmlsZXMKKyAgICAgICAgdW5kZXIgdGhlIGNvbnRyb2wgdmVyc2lvbiBm b3IgdGhlIGN1cnJlbnQgc291cmNlIHJvb3QuCisgICAgICAgIChEaXJlY3Rvcnkuc2hvdWxkX3Nr aXBfZmlsZSk6IFJldHVybiBUcnVlIGZvciBmaWxlcyB0aGF0IGFyZSBub3QKKyAgICAgICAgdW5k ZXIgY29udHJvbCB2ZXJzaW9uIG9ubHkgd2hlbiB0aGUgc291cmNlIHJvb3QgaXMgYWxzbyB1bmRl cgorICAgICAgICBjb250cm9sIHZlcnNpb24uCisgICAgICAgIChEaXJlY3RvcnkuZ2V0X2ZpbGVz KTogRmlsdGVyIHRoZSBmaWxlcyBhbHNvIHVzaW5nIHNob3VsZF9za2lwX2ZpbGUoKS4KKworMjAx NC0wNy0xMCAgQ2FybG9zIEdhcmNpYSBDYW1wb3MgIDxjZ2FyY2lhQGlnYWxpYS5jb20+CisKICAg ICAgICAgW0dUS10gUmVkdWNlIHRoZSBzaXplIG9mIHRoZSB0YXJiYWxsIGdlbmVyYXRlZCBieSBk aXN0Y2hlY2sKICAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lk PTEzNDgwMgogCmRpZmYgLS1naXQgYS9Ub29scy9ndGsvbWFrZS1kaXN0LnB5IGIvVG9vbHMvZ3Rr L21ha2UtZGlzdC5weQppbmRleCAwMjBlZGNiLi5hZGY5NmM5IDEwMDc1NQotLS0gYS9Ub29scy9n dGsvbWFrZS1kaXN0LnB5CisrKyBiL1Rvb2xzL2d0ay9tYWtlLWRpc3QucHkKQEAgLTkxLDEyICs5 MSwyOSBAQCBjbGFzcyBEaXJlY3Rvcnkob2JqZWN0KToKICAgICAgICAgc2VsZi50YXJiYWxsX3Jv b3QgPSB0YXJiYWxsX3Jvb3QKICAgICAgICAgc2VsZi5ydWxlcyA9IFJ1bGVzZXQoKQogCisgICAg ICAgIHNlbGYuZmlsZXNfaW5fY21zID0gc2VsZi5saXN0X2Ntc19maWxlcygpCisKICAgICBkZWYg YWRkX3J1bGUoc2VsZiwgcnVsZSk6CiAgICAgICAgIHNlbGYucnVsZXMuYWRkX3J1bGUocnVsZSkK IAogICAgIGRlZiBnZXRfdGFyYmFsbF9wYXRoKHNlbGYsIGZpbGVuYW1lKToKICAgICAgICAgcmV0 dXJuIGZpbGVuYW1lLnJlcGxhY2Uoc2VsZi5zb3VyY2Vfcm9vdCwgc2VsZi50YXJiYWxsX3Jvb3Qs IDEpCiAKKyAgICBkZWYgbGlzdF9jbXNfZmlsZXMoc2VsZik6CisgICAgICAgIGNtZCA9IFsnZ2l0 JywgJ2xzLXRyZWUnLCAnLXInLCAnLS1uYW1lLW9ubHknLCAnSEVBRCcsIHNlbGYuc291cmNlX3Jv b3RdCisgICAgICAgIHAgPSBzdWJwcm9jZXNzLlBvcGVuKGNtZCwgc3Rkb3V0PXN1YnByb2Nlc3Mu UElQRSkKKyAgICAgICAgb3V0ID0gcC5jb21tdW5pY2F0ZSgpWzBdCisgICAgICAgIGlmIG5vdCBv dXQ6CisgICAgICAgICAgICByZXR1cm4gW10KKyAgICAgICAgcmV0dXJuIG91dC5yc3RyaXAoJ1xu Jykuc3BsaXQoJ1xuJykKKworICAgIGRlZiBzaG91bGRfc2tpcF9maWxlKHNlbGYsIHBhdGgpOgor ICAgICAgICBpZiBub3Qgc2VsZi5maWxlc19pbl9jbXM6CisgICAgICAgICAgICAjIElmIHRoZSBz b3VyY2Ugcm9vdCBpcyBub3QgaW4gQ01TIHdlIGRvbid0IHdhbnQgdG8gc2tpcCB0aGlzIGZpbGUu CisgICAgICAgICAgICByZXR1cm4gRmFsc2UKKworICAgICAgICByZXR1cm4gcGF0aCBub3QgaW4g c2VsZi5maWxlc19pbl9jbXMKKwogICAgIGRlZiBnZXRfZmlsZXMoc2VsZik6CiAgICAgICAgIGZv ciByb290LCBkaXJzLCBmaWxlcyBpbiBvcy53YWxrKHNlbGYuc291cmNlX3Jvb3QpOgogCkBAIC0x MDksNyArMTI2LDcgQEAgY2xhc3MgRGlyZWN0b3J5KG9iamVjdCk6CiAKICAgICAgICAgICAgIGZv ciBmaWxlIGluIGZpbGVzOgogICAgICAgICAgICAgICAgIGZpbGUgPSBvcy5wYXRoLmpvaW4ocm9v dCwgZmlsZSkKLSAgICAgICAgICAgICAgICBpZiBub3QgcGFzc2VzX2FsbF9ydWxlcyhmaWxlKToK KyAgICAgICAgICAgICAgICBpZiBub3QgcGFzc2VzX2FsbF9ydWxlcyhmaWxlKSBvciBzZWxmLnNo b3VsZF9za2lwX2ZpbGUoZmlsZSk6CiAgICAgICAgICAgICAgICAgICAgIGNvbnRpbnVlCiAgICAg ICAgICAgICAgICAgeWllbGQgKGZpbGUsIHNlbGYuZ2V0X3RhcmJhbGxfcGF0aChmaWxlKSkKIAo= 234750 2014-07-11 02:33:29 -0700 2014-07-31 01:24:41 -0700 Patch wk-tarball-git.diff text/plain 3410 cgarcia ZGlmZiAtLWdpdCBhL1Rvb2xzL0NoYW5nZUxvZyBiL1Rvb2xzL0NoYW5nZUxvZwppbmRleCA5ZjJh NjM0Li5hNzcyYWQxIDEwMDY0NAotLS0gYS9Ub29scy9DaGFuZ2VMb2cKKysrIGIvVG9vbHMvQ2hh bmdlTG9nCkBAIC0xLDUgKzEsMjQgQEAKIDIwMTQtMDctMTAgIENhcmxvcyBHYXJjaWEgQ2FtcG9z ICA8Y2dhcmNpYUBpZ2FsaWEuY29tPgogCisgICAgICAgIFtHVEtdIERvIG5vdCBpbmNsdWRlIGZp bGVzIHRoYXQgYXJlIG5vdCBpbiBnaXQgaW4gdGhlIHRhcmJhbGwKKyAgICAgICAgaHR0cHM6Ly9i dWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTEzNDgwNAorCisgICAgICAgIFJldmlld2Vk IGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIFNraXAgYWxsIGZpbGVzIGluIHRoZSBzb3Vy Y2UgdHJlZSB0aGF0IGFyZSBub3QgdW5kZXIgdGhlIGNvbnRyb2wKKyAgICAgICAgdmVyc2lvbiwg ZXhjZXB0IGZvciBmaWxlcyBhZGQgZnJvbSB0aGUgYnVpbGQgZGlyIGxpa2UgdGhlIGRvY3VtZW50 YXRpb24uCisKKyAgICAgICAgKiBndGsvbWFrZS1kaXN0LnB5OgorICAgICAgICAoRGlyZWN0b3J5 Ll9faW5pdF9fKTogSW5pdGlhbGl6ZSB0aGUgbGlzdCBvZiBmaWxlcyB1bmRlciBjb250cm9sIHZl cnNpb24uCisgICAgICAgIChEaXJlY3RvcnkubGlzdF9jbXNfZmlsZXMpOiBIZWxwZXIgZnVuY3Rp b24gdG8gbGlzdCBhbGwgZmlsZXMKKyAgICAgICAgdW5kZXIgdGhlIGNvbnRyb2wgdmVyc2lvbiBm b3IgdGhlIGN1cnJlbnQgc291cmNlIHJvb3QuCisgICAgICAgIChEaXJlY3Rvcnkuc2hvdWxkX3Nr aXBfZmlsZSk6IFJldHVybiBUcnVlIGZvciBmaWxlcyB0aGF0IGFyZSBub3QKKyAgICAgICAgdW5k ZXIgY29udHJvbCB2ZXJzaW9uIG9ubHkgd2hlbiB0aGUgc291cmNlIHJvb3QgaXMgYWxzbyB1bmRl cgorICAgICAgICBjb250cm9sIHZlcnNpb24uCisgICAgICAgIChEaXJlY3RvcnkuZ2V0X2ZpbGVz KTogRmlsdGVyIHRoZSBmaWxlcyBhbHNvIHVzaW5nIHNob3VsZF9za2lwX2ZpbGUoKS4KKworMjAx NC0wNy0xMCAgQ2FybG9zIEdhcmNpYSBDYW1wb3MgIDxjZ2FyY2lhQGlnYWxpYS5jb20+CisKICAg ICAgICAgW0dUS10gUmVkdWNlIHRoZSBzaXplIG9mIHRoZSB0YXJiYWxsIGdlbmVyYXRlZCBieSBk aXN0Y2hlY2sKICAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lk PTEzNDgwMgogCmRpZmYgLS1naXQgYS9Ub29scy9ndGsvbWFrZS1kaXN0LnB5IGIvVG9vbHMvZ3Rr L21ha2UtZGlzdC5weQppbmRleCAwMjBlZGNiLi5iYmYyMmMyIDEwMDc1NQotLS0gYS9Ub29scy9n dGsvbWFrZS1kaXN0LnB5CisrKyBiL1Rvb2xzL2d0ay9tYWtlLWRpc3QucHkKQEAgLTgxLDYgKzgx LDEwIEBAIGNsYXNzIEZpbGUob2JqZWN0KToKICAgICAgICAgc2VsZi5zb3VyY2Vfcm9vdCA9IHNv dXJjZV9yb290CiAgICAgICAgIHNlbGYudGFyYmFsbF9yb290ID0gdGFyYmFsbF9yb290CiAKKyAg ICBkZWYgc2hvdWxkX3NraXBfZmlsZShzZWxmLCBwYXRoKToKKyAgICAgICAgIyBEbyBub3Qgc2tp cCBmaWxlcyBleHBsaWNpdGx5IGFkZGVkIGZyb20gdGhlIG1hbmlmZXN0LgorICAgICAgICByZXR1 cm4gRmFsc2UKKwogICAgIGRlZiBnZXRfZmlsZXMoc2VsZik6CiAgICAgICAgIHlpZWxkIChzZWxm LnNvdXJjZV9yb290LCBzZWxmLnRhcmJhbGxfcm9vdCkKIApAQCAtOTEsMTIgKzk1LDI1IEBAIGNs YXNzIERpcmVjdG9yeShvYmplY3QpOgogICAgICAgICBzZWxmLnRhcmJhbGxfcm9vdCA9IHRhcmJh bGxfcm9vdAogICAgICAgICBzZWxmLnJ1bGVzID0gUnVsZXNldCgpCiAKKyAgICAgICAgc2VsZi5m aWxlc19pbl92ZXJzaW9uX2NvbnRyb2wgPSBzZWxmLmxpc3RfZmlsZXNfaW5fdmVyc2lvbl9jb250 cm9sKCkKKwogICAgIGRlZiBhZGRfcnVsZShzZWxmLCBydWxlKToKICAgICAgICAgc2VsZi5ydWxl cy5hZGRfcnVsZShydWxlKQogCiAgICAgZGVmIGdldF90YXJiYWxsX3BhdGgoc2VsZiwgZmlsZW5h bWUpOgogICAgICAgICByZXR1cm4gZmlsZW5hbWUucmVwbGFjZShzZWxmLnNvdXJjZV9yb290LCBz ZWxmLnRhcmJhbGxfcm9vdCwgMSkKIAorICAgIGRlZiBsaXN0X2ZpbGVzX2luX3ZlcnNpb25fY29u dHJvbChzZWxmKToKKyAgICAgICAgIyBGSVhNRTogT25seSBnaXQgaXMgc3VwcG9ydGVkIGZvciBu b3cuCisgICAgICAgIHAgPSBzdWJwcm9jZXNzLlBvcGVuKFsnZ2l0JywgJ2xzLXRyZWUnLCAnLXIn LCAnLS1uYW1lLW9ubHknLCAnSEVBRCcsIHNlbGYuc291cmNlX3Jvb3RdLCBzdGRvdXQ9c3VicHJv Y2Vzcy5QSVBFKQorICAgICAgICBvdXQgPSBwLmNvbW11bmljYXRlKClbMF0KKyAgICAgICAgaWYg bm90IG91dDoKKyAgICAgICAgICAgIHJldHVybiBbXQorICAgICAgICByZXR1cm4gb3V0LnJzdHJp cCgnXG4nKS5zcGxpdCgnXG4nKQorCisgICAgZGVmIHNob3VsZF9za2lwX2ZpbGUoc2VsZiwgcGF0 aCk6CisgICAgICAgIHJldHVybiBwYXRoIG5vdCBpbiBzZWxmLmZpbGVzX2luX3ZlcnNpb25fY29u dHJvbAorCiAgICAgZGVmIGdldF9maWxlcyhzZWxmKToKICAgICAgICAgZm9yIHJvb3QsIGRpcnMs IGZpbGVzIGluIG9zLndhbGsoc2VsZi5zb3VyY2Vfcm9vdCk6CiAKQEAgLTE4OSw5ICsyMDYsMTgg QEAgY2xhc3MgTWFuaWZlc3Qob2JqZWN0KToKICAgICAgICAgZWxpZiBwYXJ0c1swXSA9PSAiaW5j bHVkZSIgYW5kIGxlbihwYXJ0cykgPiAxOgogICAgICAgICAgICAgc2VsZi5hZGRfcnVsZShSdWxl KFJ1bGUuUmVzdWx0LklOQ0xVREUsIHNlbGYucmVzb2x2ZV92YXJpYWJsZXMocGFydHNbMV0pKSkK IAorICAgIGRlZiBzaG91bGRfc2tpcF9maWxlKHNlbGYsIGRpcmVjdG9yeSwgZmlsZW5hbWUpOgor ICAgICAgICAjIE9ubHkgYWxsb3cgZmlsZXMgdGhhdCBhcmUgbm90IGluIHZlcnNpb24gY29udHJv bCB3aGVuIHRoZXkgYXJlIGV4cGxpY2l0bHkgaW5jbHVkZWQgaW4gdGhlIG1hbmlmZXN0IGZyb20g dGhlIGJ1aWxkIGRpci4KKyAgICAgICAgaWYgZmlsZW5hbWUuc3RhcnRzd2l0aChzZWxmLmJ1aWxk X3Jvb3QpOgorICAgICAgICAgICAgcmV0dXJuIEZhbHNlCisKKyAgICAgICAgcmV0dXJuIGRpcmVj dG9yeS5zaG91bGRfc2tpcF9maWxlKGZpbGVuYW1lKQorCiAgICAgZGVmIGdldF9maWxlcyhzZWxm KToKICAgICAgICAgZm9yIGRpcmVjdG9yeSBpbiBzZWxmLmRpcmVjdG9yaWVzOgogICAgICAgICAg ICAgZm9yIGZpbGVfdHVwbGUgaW4gZGlyZWN0b3J5LmdldF9maWxlcygpOgorICAgICAgICAgICAg ICAgIGlmIHNlbGYuc2hvdWxkX3NraXBfZmlsZShkaXJlY3RvcnksIGZpbGVfdHVwbGVbMF0pOgor ICAgICAgICAgICAgICAgICAgICBjb250aW51ZQogICAgICAgICAgICAgICAgIHlpZWxkIGZpbGVf dHVwbGUKIAogICAgIGRlZiBjcmVhdGVfdGFyZmlsZShzZWxmLCBvdXRwdXQpOgo=