38003 2010-04-22 13:25:28 -0700 We should add FTS3 support to WebKit 2010-05-25 15:42:23 -0700 1 1 1 Unclassified WebKit New Bugs 528+ (Nightly build) All All RESOLVED FIXED P2 Normal --- 1 dumi dumi abarth aroben beidson dglazkov eric fishd michaeln mjs webkit.review.bot oldest_to_newest 216050 0 dumi 2010-04-22 13:25:28 -0700 Gears supported FTS2, so I think we should add FTS3 support to WebKit, to make it easier for web developers who used Gears port their apps to WebSQLDatabases. 216312 1 fishd 2010-04-23 00:35:10 -0700 Can you comment on why FTS3 should be added instead of FTS2? Are there compatibility concerns? 216503 2 dumi 2010-04-23 12:11:29 -0700 (In reply to comment #1) > Can you comment on why FTS3 should be added instead of FTS2? Are there > compatibility concerns? FTS2 has a design problem: calling VACUUM on FTS2 databases could corrupt them. According to Scott, FTS3 is pretty much FTS2 with this problem fixed. It is unclear if FTS2 databases are compatible with FTS3. We believe they should be, but I haven't tested this. However, WebKit doesn't have FTS2 support, and Chromium's FTS2 support is incomplete at the moment because we don't allow the MATCH operator (bug 37049). So I think it should be OK to treat Chromium's FTS2 support as experimental, and say that full-text search will be officially supported starting with FTS3. 216524 3 michaeln 2010-04-23 13:03:41 -0700 (In reply to comment #1) > Can you comment on why FTS3 should be added instead of FTS2? Are there > compatibility concerns? FTS3 is being actively supported by SQLite, FTS2 is not. FTS2 and has a known bug (around vacuum) that is fixed in FTS3. As I understand it, FTS3 was the fix for that known bug. There is a data format change, FTS2 code can't read FTS3 data and vice versa. (Just passing along what I've learned from shess) 216692 4 fishd 2010-04-23 21:52:21 -0700 OK, I agree that it would be valuable for FTS3 to be enabled across all ports of WebKit. /cc @mjs for his opinion. 216705 5 beidson 2010-04-24 00:03:08 -0700 I've been discussing this in emails with some of our SQLite gurus at Apple. I think they think this is fine, as long as we still restrict some things with the authorizer. fts3_tokenizer has come up by name, there may be others. 216707 6 beidson 2010-04-24 00:03:56 -0700 (In reply to comment #5) > I've been discussing this in emails with some of our SQLite gurus at Apple. I > think they think this is fine, as long as we still restrict some things with > the authorizer. fts3_tokenizer has come up by name, there may be others. Sorry - also meant to mention that I'll have all the info they mean to give me by Monday. 216714 7 dumi 2010-04-24 03:21:18 -0700 (In reply to comment #6) > (In reply to comment #5) > > I've been discussing this in emails with some of our SQLite gurus at Apple. I > > think they think this is fine, as long as we still restrict some things with > > the authorizer. fts3_tokenizer has come up by name, there may be others. > > Sorry - also meant to mention that I'll have all the info they mean to give me > by Monday. Great! fts2_tokenizer() is currently disabled in DatabaseAuthorizer, because our security experts didn't like it too, and I'm pretty sure they agree that fts3_tokenizer() should be disabled too. 220865 8 dumi 2010-05-04 16:33:00 -0700 Brady, any updates? 226042 9 56118 dumi 2010-05-14 16:08:52 -0700 Created attachment 56118 enable FTS3 230416 10 dumi 2010-05-25 13:33:20 -0700 Ping. 230428 11 56118 beidson 2010-05-25 13:51:30 -0700 Comment on attachment 56118 enable FTS3 It really sucks that we're fragmenting capabilities like this, but I guess we already did it with fts2. That said, no one here sees an inherent problem with fts3. 230481 12 dumi 2010-05-25 15:06:00 -0700 Landed as r60180. 230506 13 webkit.review.bot 2010-05-25 15:42:23 -0700 http://trac.webkit.org/changeset/60188 might have broken GTK Linux 32-bit Release 56118 2010-05-14 16:08:52 -0700 2010-05-25 13:51:29 -0700 enable FTS3 patch text/plain 1493 dumi SW5kZXg6IFdlYkNvcmUvQ2hhbmdlTG9nCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFdlYkNvcmUvQ2hhbmdlTG9n CShyZXZpc2lvbiA1OTQ5OSkKKysrIFdlYkNvcmUvQ2hhbmdlTG9nCSh3b3JraW5nIGNvcHkpCkBA IC0xLDMgKzEsMTQgQEAKKzIwMTAtMDUtMTQgIER1bWl0cnUgRGFuaWxpdWMgIDxkdW1pQGNocm9t aXVtLm9yZz4KKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAg ICBBbGxvdyBGVFMzIGZ1bmN0aW9ucy4KKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcv c2hvd19idWcuY2dpP2lkPTM4MDAzCisKKyAgICAgICAgKiBzdG9yYWdlL0RhdGFiYXNlQXV0aG9y aXplci5jcHA6CisgICAgICAgIChXZWJDb3JlOjpEYXRhYmFzZUF1dGhvcml6ZXI6OmFkZFdoaXRl bGlzdGVkRnVuY3Rpb25zKToKKyAgICAgICAgKFdlYkNvcmU6OkRhdGFiYXNlQXV0aG9yaXplcjo6 Y3JlYXRlVlRhYmxlKToKKwogMjAxMC0wNS0xNCAgS2VubmV0aCBSdXNzZWxsICA8a2JyQGdvb2ds ZS5jb20+CiAKICAgICAgICAgUmV2aWV3ZWQgYnkgRGFyaW4gQWRsZXIuCkluZGV4OiBXZWJDb3Jl L3N0b3JhZ2UvRGF0YWJhc2VBdXRob3JpemVyLmNwcAo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBXZWJDb3JlL3N0 b3JhZ2UvRGF0YWJhc2VBdXRob3JpemVyLmNwcAkocmV2aXNpb24gNTk0OTkpCisrKyBXZWJDb3Jl L3N0b3JhZ2UvRGF0YWJhc2VBdXRob3JpemVyLmNwcAkod29ya2luZyBjb3B5KQpAQCAtMTAyLDYg KzEwMiw3IEBAIHZvaWQgRGF0YWJhc2VBdXRob3JpemVyOjphZGRXaGl0ZWxpc3RlZEYKICAgICBt X3doaXRlbGlzdGVkRnVuY3Rpb25zLmFkZCgidG90YWwiKTsKIAogICAgIC8vIFNRTGl0ZSBGVFMg ZnVuY3Rpb25zCisgICAgbV93aGl0ZWxpc3RlZEZ1bmN0aW9ucy5hZGQoIm1hdGNoIik7CiAgICAg bV93aGl0ZWxpc3RlZEZ1bmN0aW9ucy5hZGQoInNuaXBwZXQiKTsKICAgICBtX3doaXRlbGlzdGVk RnVuY3Rpb25zLmFkZCgib2Zmc2V0cyIpOwogICAgIG1fd2hpdGVsaXN0ZWRGdW5jdGlvbnMuYWRk KCJvcHRpbWl6ZSIpOwpAQCAtMjY4LDggKzI2OSw4IEBAIGludCBEYXRhYmFzZUF1dGhvcml6ZXI6 OmNyZWF0ZVZUYWJsZShjb24KICAgICBpZiAobV9yZWFkT25seSAmJiBtX3NlY3VyaXR5RW5hYmxl ZCkKICAgICAgICAgcmV0dXJuIFNRTEF1dGhEZW55OwogCi0gICAgLy8gZnRzMiBpcyB1c2VkIGlu IENocm9taXVtCi0gICAgaWYgKG1vZHVsZU5hbWUgIT0gImZ0czIiKQorICAgIC8vIEFsbG93IG9u bHkgdGhlIEZUUzMgZXh0ZW5zaW9uCisgICAgaWYgKG1vZHVsZU5hbWUgIT0gImZ0czMiKQogICAg ICAgICByZXR1cm4gU1FMQXV0aERlbnk7CiAKICAgICBtX2xhc3RBY3Rpb25DaGFuZ2VkRGF0YWJh c2UgPSB0cnVlOwo=