DRT doesn't seem to clear the memory cache between tests. I was trying to make a test that has window.internals.settings.setImageSubsamplingEnabled(true); and a ref test that does not, but the CachedImage was created by the test then just re-used by the reference, giving me a bogus pass.
Antti says: I think that is a too big of a hammer. You can use internals.clearMemoryCache() if your test relies on empty memory cache.
I am personally in favor of this change. I will help reduce flakiness. I had to add internals.clearMemoryCache() calls at the beginning of 24 tests recently to make them not flaky, I don't think this is very scalable. We already disable PageCache during testing, I think it makes sense to make the MemoryCache opt-in as well.
(In reply to comment #2) > I am personally in favor of this change. I will help reduce flakiness. I had > to add internals.clearMemoryCache() calls at the beginning of 24 tests > recently to make them not flaky, I don't think this is very scalable. We > already disable PageCache during testing, I think it makes sense to make the > MemoryCache opt-in as well. I would also add that I wasted engineering time investigating the source of the flakiness until I was able to determine the reason was the memory cache. This WILL happen again and more people WILL waste time on this.
Reopening.
One concern if that *may* make the tests substantially slower. We'll need to measure performance if we do this.
internals.clearMemoryCache is not to meant to be flakiness fix but used when a test actually relies on having a clear cache. Doing it everywhere would slow down tests (no idea if enough to matter) and reduce implicit test coverage, moving testing further from real browser behavior in general.