Bug 261064 - Text recognition results appear on wrong position when the parents elements containing <video> element is brought fullscreen
Summary: Text recognition results appear on wrong position when the parents elements c...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: Safari 16
Hardware: Mac (Apple Silicon) macOS 13
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-09-02 09:43 PDT by Ethan Wong
Modified: 2023-09-09 09:44 PDT (History)
2 users (show)

See Also:

Attachments
A page that produces this issue (544 bytes, text/html)
2023-09-02 09:43 PDT, Ethan Wong 		no flags Details
A screen recording of this issue (3.50 MB, video/mp4)
2023-09-02 09:46 PDT, Ethan Wong 		no flags Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Ethan Wong 2023-09-02 09:43:55 PDT 
Created attachment 467525 [details]
A page that produces this issue

Text recognition results may appear on wrong position when the parent element containing <video> elements is brought fullscreen.

See the attached example and screen recording for this issue.
Comment 1 Ethan Wong 2023-09-02 09:46:48 PDT 
Created attachment 467526 [details]
A screen recording of this issue
Comment 2 Wenson Hsieh 2023-09-03 15:01:18 PDT 
Oh, I think the issue is that we don't correctly account for margin when positioning the native VisionKit overlay view, in the case of fullscreen video.
Comment 3 Wenson Hsieh 2023-09-03 16:54:29 PDT 
Part of a fix so far involves extracting `inlineVideoFrame()` out of `VideoFullscreenManager.mm`, and using it in `WebPage::beginTextRecognitionForVideoInElementFullScreen` when determining the video rect in root view coordinates.

Unfortunately, even after this, the live text button ends up appearing over the built-in video controls... needs further investigation to figure out how to avoid this subsequent issue.
Comment 4 Radar WebKit Bug Importer 2023-09-09 09:44:15 PDT 
<rdar://problem/115220914>