100274 2012-10-24 11:32:19 -0700 [Linux] SQLite databases are slow on filesystems with write barriers enabled 2016-02-02 06:30:53 -0800 1 1 1 Unclassified WebKit New Bugs 528+ (Nightly build) Unspecified Unspecified NEW P2 Normal --- 103890 1 zan zan allan.jensen beidson j.isorce jturcotte kbalazs michael oldest_to_newest 749865 0 zan 2012-10-24 11:32:19 -0700 [Linux] SQLite databases are slow on filesystems with write barriers enabled 749867 1 170440 zan 2012-10-24 11:34:31 -0700 Created attachment 170440 Patch 749869 2 zan 2012-10-24 11:35:02 -0700 (In reply to comment #0) > [Linux] SQLite databases are slow on filesystems with write barriers enabled Explained here: https://bugs.webkit.org/show_bug.cgi?id=96862#c9 https://bugs.webkit.org/show_bug.cgi?id=96862#c11 766673 3 kbalazs 2012-11-14 02:37:07 -0800 *** Bug 101614 has been marked as a duplicate of this bug. *** 766676 4 kbalazs 2012-11-14 02:38:32 -0800 This will solve an annoying issue with WTR as well: bug 101614. Somebody please put an r+ on this! :) 766703 5 jturcotte 2012-11-14 03:08:30 -0800 This makes sense for testing but I don't think this is good for end users. Wouldn't it be better to let the WTR injected bundle control this setting instead? 766732 6 kbalazs 2012-11-14 04:13:31 -0800 (In reply to comment #5) > This makes sense for testing but I don't think this is good for end users. > > Wouldn't it be better to let the WTR injected bundle control this setting instead? In CookiJarQt we use this setting for end users as well. Synchronized mode is very slow and also it seems unstable so I would vote for turning it off in for users as well, not because it's easier to implement but because it is better for them. 766743 7 jturcotte 2012-11-14 04:28:37 -0800 (In reply to comment #6) > In CookiJarQt we use this setting for end users as well. Synchronized mode is very slow and also it seems unstable so I would vote for turning it off in for users as well, not because it's easier to implement but because it is better for them. I don't think it's right either for the cookie jar, we have no code dealing with the possibility of a corrupted databases and it _will_ happen out there somewhere. Synchronized is set to normal by default for a reason, because it's better in most cases. No, it's not unstable. For a network cache DB I understand that it could make sense to disable it (no real data is lost), but for user data IMO it's wrong. Tell my why it's better for them to risk having their data corrupted? In this case the user will most likely not even notice the delay added by fsync. 766786 8 kbalazs 2012-11-14 05:34:31 -0800 By unstability I was thinking about bug 101614. If the fsync can crash than it adds risk not reduce it. Actually we have code to deal with data corruption, see IconDatabase::checkIntegrity. Btw IconDatabase is also a cache so IMHO data loss is not fatal as long as we can handle it. 766804 9 jturcotte 2012-11-14 06:02:16 -0800 (In reply to comment #8) > By unstability I was thinking about bug 101614. If the fsync can crash than it adds risk not reduce it. With the data given on that bug, I have difficulties believing that this is a bug in SQLite. Could you reproduce it on any other machine than yours? The real problem that we have to fix is that the tests take longer to run than they should. > Actually we have code to deal with data corruption, see IconDatabase::checkIntegrity. Btw IconDatabase is also a cache so IMHO data loss is not fatal as long as we can handle it. Right now we don't, not for CookiJarQt, and neither for other uses of SQLiteDatabase. If we handle it it's fine, but having users asking on forums and getting told that the solution to their obscure problem is to delete some ~/.local/someDatabase.db is what I would like to avoid. 766954 10 beidson 2012-11-14 09:01:53 -0800 (In reply to comment #8) > Btw IconDatabase is also a cache so IMHO data loss is not fatal as long as we can handle it. I can assure you that users don't see the IconDatabase as a cache, and we will reject any attempt to treat it as such on our platform. 766955 11 beidson 2012-11-14 09:03:01 -0800 (In reply to comment #10) > (In reply to comment #8) > > Btw IconDatabase is also a cache so IMHO data loss is not fatal as long as we can handle it. > > I can assure you that users don't see the IconDatabase as a cache, and we will reject any attempt to treat it as such on our platform. (I know this bug is only about Linux, I'm just warning anyone who - in the future - might think this type of change is solid cross platform) 781283 12 zan 2012-12-03 03:14:11 -0800 (Based on the loosely formed consensus that the 'problematic' behavior should only be avoided in testing.) I was thinking of how to implement this and am currently holding an opinion that it would require complex paths taken that wouldn't justify the trade-off: - a setting in an injected bundle that's then handled in WebKit2, - a WebKit2-specific setting that is then communicated to WebCore, - a new setting in WebCore::Settings interface that is not even reachable directly from SQLiteDatabase. I'm at the moment in favor of putting information on wiki about disabling the write barriers in fstab. Opinions? 781406 13 allan.jensen 2012-12-03 06:43:23 -0800 (In reply to comment #12) > (Based on the loosely formed consensus that the 'problematic' behavior should only be avoided in testing.) > > I was thinking of how to implement this and am currently holding an opinion that it would require complex paths taken that wouldn't justify the trade-off: > - a setting in an injected bundle that's then handled in WebKit2, > - a WebKit2-specific setting that is then communicated to WebCore, > - a new setting in WebCore::Settings interface that is not even reachable directly from SQLiteDatabase. > > I'm at the moment in favor of putting information on wiki about disabling the write barriers in fstab. > > Opinions? I believe the Cookie-database would be better as an injected bundle, or fetched over IPC. Having it hard-coded as an SQLite database has always been an too inflexible choice when we want QtWebKit to be a toolkit for a browser and not a browser in itself. 814043 14 zan 2013-01-23 02:32:48 -0800 (In reply to comment #13) > (In reply to comment #12) > > (Based on the loosely formed consensus that the 'problematic' behavior should only be avoided in testing.) > > > > I was thinking of how to implement this and am currently holding an opinion that it would require complex paths taken that wouldn't justify the trade-off: > > - a setting in an injected bundle that's then handled in WebKit2, > > - a WebKit2-specific setting that is then communicated to WebCore, > > - a new setting in WebCore::Settings interface that is not even reachable directly from SQLiteDatabase. > > > > I'm at the moment in favor of putting information on wiki about disabling the write barriers in fstab. > > > > Opinions? > > I believe the Cookie-database would be better as an injected bundle, or fetched over IPC. Having it hard-coded as an SQLite database has always been an too inflexible choice when we want QtWebKit to be a toolkit for a browser and not a browser in itself. This bug is not really about how and where the cookie database should operate. Rather than that it tries to address the problem of write barriers being enabled on some filesystems and how to work around that. 170440 2012-10-24 11:34:31 -0700 2012-12-03 05:49:14 -0800 Patch bug-100274-20121024113306.patch text/plain 1729 zan U3VidmVyc2lvbiBSZXZpc2lvbjogMTMyMzE2CmRpZmYgLS1naXQgYS9Tb3VyY2UvV2ViQ29yZS9D aGFuZ2VMb2cgYi9Tb3VyY2UvV2ViQ29yZS9DaGFuZ2VMb2cKaW5kZXggZTcxYjYwNzZiOTA0ZGMy YTQ2NDQ4ZDk2MTFmNzM0ZGYyNGU5MzQwYy4uN2U0ZTU4Y2M4MGFjNmE2YWJkYmUwZGI4NTY1MTM5 OTdhN2VmMGVjNCAxMDA2NDQKLS0tIGEvU291cmNlL1dlYkNvcmUvQ2hhbmdlTG9nCisrKyBiL1Nv dXJjZS9XZWJDb3JlL0NoYW5nZUxvZwpAQCAtMSwzICsxLDE5IEBACisyMDEyLTEwLTI0ICBaYW4g RG9iZXJzZWsgIDx6YW5kb2JlcnNla0BnbWFpbC5jb20+CisKKyAgICAgICAgW0xpbnV4XSBTUUxp dGUgZGF0YWJhc2VzIGFyZSBzbG93IG9uIGZpbGVzeXN0ZW1zIHdpdGggd3JpdGUgYmFycmllcnMg ZW5hYmxlZAorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9 MTAwMjc0CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAg U2V0IHRoZSBzeW5jaHJvbm91cyBwcmFnbWEgdG8gb2ZmIGZvciBTUUxpdGUgZGF0YWJhc2VzIG9u IExpbnV4IE9TLiBUaGlzCisgICAgICAgIGF2b2lkcyBwZXJmb3JtYW5jZSBpc3N1ZXMgd2hlbiB3 cml0aW5nIHRvIGEgZGF0YWJhc2Ugb24gYSBmaWxlc3lzdGVtIHdpdGgKKyAgICAgICAgd3JpdGUg YmFycmllcnMgZW5hYmxlZC4KKworICAgICAgICBObyBuZXcgdGVzdHMgLSBubyBuZXcgdGVzdGFi bGUgZnVuY3Rpb25hbGl0eS4KKworICAgICAgICAqIHBsYXRmb3JtL3NxbC9TUUxpdGVEYXRhYmFz ZS5jcHA6CisgICAgICAgIChXZWJDb3JlOjpTUUxpdGVEYXRhYmFzZTo6b3Blbik6CisKIDIwMTIt MTAtMjMgIFl1cnkgU2VtaWtoYXRza3kgIDx5dXJ5c0BjaHJvbWl1bS5vcmc+CiAKICAgICAgICAg TWVtb3J5IGluc3RydW1lbnRhdGlvbjogZG9uJ3QgY291bnQgYWdlbnQtc3BlY2lmaWMgZnJvbnQt ZW5kcyBzZXBhcmF0ZWx5CmRpZmYgLS1naXQgYS9Tb3VyY2UvV2ViQ29yZS9wbGF0Zm9ybS9zcWwv U1FMaXRlRGF0YWJhc2UuY3BwIGIvU291cmNlL1dlYkNvcmUvcGxhdGZvcm0vc3FsL1NRTGl0ZURh dGFiYXNlLmNwcAppbmRleCAyNDE4ZmMxMmM0NTg1NDRhNzllYTU4ODhlMmUwMGYwYTZkM2MwMzNj Li5iNDg3N2FkYTdlZmNkYzA2MmE2YWRkNzVhZjU0YWRmNmRiYWUyZjYwIDEwMDY0NAotLS0gYS9T b3VyY2UvV2ViQ29yZS9wbGF0Zm9ybS9zcWwvU1FMaXRlRGF0YWJhc2UuY3BwCisrKyBiL1NvdXJj ZS9XZWJDb3JlL3BsYXRmb3JtL3NxbC9TUUxpdGVEYXRhYmFzZS5jcHAKQEAgLTk4LDYgKzk4LDEx IEBAIGJvb2wgU1FMaXRlRGF0YWJhc2U6Om9wZW4oY29uc3QgU3RyaW5nJiBmaWxlbmFtZSwgYm9v bCBmb3JXZWJTUUxEYXRhYmFzZSkKICAgICBpZiAoIVNRTGl0ZVN0YXRlbWVudCgqdGhpcywgQVND SUlMaXRlcmFsKCJQUkFHTUEgdGVtcF9zdG9yZSA9IE1FTU9SWTsiKSkuZXhlY3V0ZUNvbW1hbmQo KSkKICAgICAgICAgTE9HX0VSUk9SKCJTUUxpdGUgZGF0YWJhc2UgY291bGQgbm90IHNldCB0ZW1w X3N0b3JlIHRvIG1lbW9yeSIpOwogCisjaWYgT1MoTElOVVgpCisgICAgLy8gQnlwYXNzZXMgcGVy Zm9ybWFuY2UgaXNzdWVzIG9uIExpbnV4IGZpbGVzeXN0ZW1zIHdpdGggd3JpdGUgYmFycmllcnMg ZW5hYmxlZC4KKyAgICBzZXRTeW5jaHJvbm91cyhTeW5jT2ZmKTsKKyNlbmRpZgorCiAgICAgcmV0 dXJuIGlzT3BlbigpOwogfQogCg==