A follow-up to https://bugs.webkit.org/show_bug.cgi?id=80806 There are still some minor flashing/sorting artifacts that occur that are most likely to to this issue.
So here's the situation: I was able to whip up a quick fix for this, and it did fix the flashing sorting artifacts that we were seeing: - css space cube demo - mozilla cardboard box + paper sheep demo However, the fix as I've hacked it currently, uses only a sub-portion of the clipped geometry to do layer sorting. (so that I can still use FloatQuad instead of the clipped polygon). Of course, this causes other potential layer sorting bugs, for example, it regresses acko.net to have flaky sorting again. There are two options I can think of to fix this: (1) make layer sorter work with general clipped polygons of up to 8 vertices. We can create a quick data structure, ClippedQuad, that internally holds up to 8 vertices without using the allocator. (2) make layer sorter support multiple quads per layer and accumulate all topological dependencies for all quads. Any preferences? At the moment it seems like #2 will be easier to implement.
(In reply to comment #1) > Of course, this causes other potential layer sorting bugs, for example, it regresses acko.net to have flaky sorting again. > Actually, those acko.net sorting bugs already existed, even before the previous perspective clipping patch. So my fix should be OK without forcing more complexity.
Created attachment 136096 [details] Working fix, without unit test yet
Created attachment 136184 [details] Patch with unit test Shuffled around some CCMathUtil code at the same time, for cleanliness
Comment on attachment 136184 [details] Patch with unit test R=me.
Comment on attachment 136184 [details] Patch with unit test Thanks!
Comment on attachment 136184 [details] Patch with unit test Clearing flags on attachment: 136184 Committed r113606: <http://trac.webkit.org/changeset/113606>
All reviewed patches have been landed. Closing bug.