Bug 135252 – Paint-on captions are sometimes displayed "out-of-order"

• Component: Media
• Status: RESOLVED INVALID

[Brent Fulgham]

"Paint-on" style closed captions (608 captions) sometimes are drawn out of order, so that the second part of a phrase is drawn above the first part of a phrase. This makes it difficult to follow the caption content, and reduces usability.

[Brent Fulgham]

<rdar://problem/15317278>

[Brent Fulgham]

Created attachment 235454 [details]
Patch

[Brent Fulgham]

It looks like this changes the results in "media/track/track-cue-rendering-snap-to-lines-not-set.html" so that the "Random Cue 3" is now lower on the page than it was before.

[Build Bot]

Comment on attachment 235454 [details]
Patch

Attachment 235454 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/4963829842706432

New failing tests:
media/track/track-cue-rendering-snap-to-lines-not-set.html

[Build Bot]

Created attachment 235460 [details]
Archive of layout-test-results from webkit-ews-05 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-05  Port: mac-mountainlion  Platform: Mac OS X 10.8.5

[Brent Fulgham]

It looks like this changes the results in "media/track/track-cue-rendering-snap-to-lines-not-set.html" so that the "Random Cue 3" is now lower on the page than it was before.

[Brent Fulgham]

Created attachment 235462 [details]
Patch

[Jer Noble]

Comment on attachment 235454 [details]
Patch

So this approach works, but it goes against the "boxes in boxes" rules in repositionCueSnapToLinesNotSet(). (I.e., "If there are multiple such positions that are equidistant from their current position, use the highest one amongst them."  

The idea of that algorithm is that cues are added in order of age, from oldest to newest.  So perhaps instead of reversing direction when we hit a cue which is "newer" (so to speak) than the current cue, we may want to sort the incoming cues by time and then by line number before adding them to the cue area.  In theory, this should ensure that cues with higher line numbers are placed below cues with lower ones.

[Brent Fulgham]

(In reply to comment #8)
> (From update of attachment 235454 [details])
> The idea of that algorithm is that cues are added in order of age, from oldest to newest.  So perhaps instead of reversing direction when we hit a cue which is "newer" (so to speak) than the current cue, we may want to sort the incoming cues by time and then by line number before adding them to the cue area.  In theory, this should ensure that cues with higher line numbers are placed below cues with lower ones.

The logging shows that the cues are already in the correct order within the container. The following sets of two captions are always ordered such that the older caption is in position 0, and the newer in position 1:

7/23/14 6:06:10.090 PM com.apple.WebKit.WebContent.Development[51770]: InbandTextTrackPrivateAVF::processCue(0 of 2) - position (12.00, 61.00), "EAST SIDE OF THE PARK. "
7/23/14 6:06:10.090 PM com.apple.WebKit.WebContent.Development[51770]: InbandTextTrackPrivateAVF::processCue(1 of 2) - position (12.00, 66.00), "NO ANIMAL"

7/23/14 6:06:10.327 PM com.apple.WebKit.WebContent.Development[51770]: InbandTextTrackPrivateAVF::processCue(0 of 2) - position (12.00, 61.00), "EAST SIDE OF THE PARK. "
7/23/14 6:06:10.327 PM com.apple.WebKit.WebContent.Development[51770]: InbandTextTrackPrivateAVF::processCue(1 of 2) - position (12.00, 66.00), "NO ANIMALS, GUESTS"

7/23/14 6:06:10.427 PM com.apple.WebKit.WebContent.Development[51770]: InbandTextTrackPrivateAVF::processCue(0 of 2) - position (12.00, 61.00), "EAST SIDE OF THE PARK. "
7/23/14 6:06:10.427 PM com.apple.WebKit.WebContent.Development[51770]: InbandTextTrackPrivateAVF::processCue(1 of 2) - position (12.00, 66.00), "NO ANIMALS, GUESTS OR "

7/23/14 6:06:10.994 PM com.apple.WebKit.WebContent.Development[51770]: InbandTextTrackPrivateAVF::processCue(0 of 2) - position (12.00, 61.00), "EAST SIDE OF THE PARK. "
7/23/14 6:06:10.994 PM com.apple.WebKit.WebContent.Development[51770]: InbandTextTrackPrivateAVF::processCue(1 of 2) - position (12.00, 66.00), "NO ANIMALS, GUESTS OR STAFF "

However, the repositioning logic caused weird ordering like the following:
"EAST SIDE OF THE PARK. "
"NO ANIMAL"

"NO ANIMALS, GUESTS"
"EAST SIDE OF THE PARK. "

"EAST SIDE OF THE PARK. "
"NO ANIMALS, GUESTS, OR "

"EAST SIDE OF THE PARK. "
"NO ANIMALS, GUESTS, OR STAFF"

So it seems like the positioning logic is sometimes violating the "age" of the cue.

[Build Bot]

Comment on attachment 235462 [details]
Patch

Attachment 235462 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.appspot.com/results/5597148540305408

New failing tests:
media/track/add-and-remove-track.html

[Build Bot]

Created attachment 235467 [details]
Archive of layout-test-results from webkit-ews-16 for mac-mountainlion-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: webkit-ews-16  Port: mac-mountainlion-wk2  Platform: Mac OS X 10.8.5

[Brent Fulgham]

Closing this bug. I'm doing the work under Bug 135332.