Bug 181898 – [WK2] [macOS] Implement a mechanism to test drag and drop

• Status: RESOLVED
• Resolution: FIXED
• Component: HTML Editing

[Wenson Hsieh]

We're currently only able to write macOS drag and drop tests for WebKit1.

This means that any features that specifically target the macOS WebKit2 port, like <https://bugs.webkit.org/show_bug.cgi?id=181294>, or bugs that specifically affect WebKit2 macOS drag and drop, like <https://bugs.webkit.org/show_bug.cgi?id=181896>, can't be tested.

[Wenson Hsieh]

<rdar://problem/39181698>

[Wenson Hsieh]

Created attachment 346946 [details]
First pass

[Wenson Hsieh]

Created attachment 346951 [details]
Fix macOS 10.12 and WPE builds

[EWS Watchlist]

Thanks for the patch. If this patch contains new public API please make sure it follows the guidelines for new WebKit2 GTK+ API. See http://trac.webkit.org/wiki/WebKitGTK/AddingNewWebKit2API

[Wenson Hsieh]

Created attachment 346956 [details]
Try to fix macOS 10.12 build

[Wenson Hsieh]

Created attachment 346957 [details]
Fix 32-bit macOS build

[Daniel Bates]

Is it not possible to expose this functionality in WebKitTestRunner?

[Wenson Hsieh]

(In reply to Daniel Bates from comment #7)
> Is it not possible to expose this functionality in WebKitTestRunner?

Great question! I think it's possible, but I have some concerns/comments:

- In order to enable the same cross-platform test coverage as we have in our API tests (currently, DragAndDropTestsIOS.mm) this would require us to either mirror tons of platform API in UI-script-side bindings, or have a large number of specific, one-off utility functions per test. This is because there are a myriad of ways to both write and read pasteboard data on macOS and iOS (using NSItemProviders on iOS and NSPasteboard on macOS). On iOS, one can write data by creating an NSItemProvider and manually adding NSData for a specific type identifier, or one can write data by registering an NSItemProvider with an NSObject that conforms to NSItemProviderWriting. One can also write multiple objects when dragging, and additionally influence the type precedence of data in each item provider by changing the order in which data or objects are registered. Our existing drag and drop API tests on iOS exercise all of these cases in one form or another, and providing the same flexibility in WebKitTestRunner would require plumbing a large amount of platform-specific API through UIScriptController or the test runner.

- There's also a nontrivial amount of actual WebKit API (well technically, SPI) that is currently tested via drag and drop API tests. These mainly exist in WKUIDelegatePrivate and WKWebViewPrivate, and are used heavily by Mail and Safari on iOS. WKAttachmentTests.mm contains many examples where I used DragAndDropSimulator to ensure test coverage of the WebKit attachment SPI when using drag and drop (both on the ObjC-side and JS bindings-side). As I continue working on drag and drop support on macOS WebKit2 in support of Mail compose, I'll need to add many macOS tests here as well to test how the WebKit attachment API works with drag and drop. Additionally, a large number of tests in DragAndDropSimulatorIOS also exercise Objective-C SPI that we provide to observe, intercept, and adjust the behavior of drag and drop (used mostly by Mail and Safari). If these were in WebKitTestRunner, we'd have to do tons of plumbing (similar to the aforementioned case) to enable the same level of test coverage as we have in API tests.

All this being said, there are some very neat benefits to exposing this functionality in WebKitTestRunner: some of our existing drag and drop API tests really don't exercise platform API/SPI (both from WebKit and UIKit/AppKit) in ways that are interesting. For such types of API tests, we could convert them into cross-platform layout tests without running into my concerns above, which would allow us to get EWS coverage for them, which API tests (at least, currently) lack.

Ultimately, I think we should make it possible to write drag and drop tests as either API tests or layout tests (whichever one is more relevant to the scenario being tested). For tests that exercise the nooks and crannies in platform ObjC API/SPI (of which there are many), I think we need the flexibility that API tests offer; for tests that are inherently cross-platform and don't rely on specific API, layout tests seem like a better fit. Perhaps what we should do is add some way to add testing support logic that is available in both API tests and layout tests. This would be similar to the Tools/TestRunnerShared folder, which is currently used for code that is common to both DumpRenderTree and WebKitTestRunner, only this would be code that is applicable to all three test harnesses (TestWebKitAPI and both test runners, WK1 and WK2). Or, alternately, we could rename TestRunnerShared to something like TestingShared, and have it serve TestWebKitAPI/DumpRenderTree/WebKitTestRunner — then, DragAndDropSimulator could be moved there, and we'd be able to write drag and drop tests that run in any of our relevant test harnesses.

What do you think?

[Simon Fraser (smfr)]

Comment on attachment 346957 [details]
Fix 32-bit macOS build

View in context: https://bugs.webkit.org/attachment.cgi?id=346957&action=review

> Tools/TestWebKitAPI/Tests/WebKitCocoa/DragAndDropTests.h:26
> +#pragma once

Is this file needed at all?

> Tools/TestWebKitAPI/mac/DragAndDropSimulatorMac.mm:47
> +    WeakObjCPtr<DragAndDropSimulator> _simulator;

I don't really like just "simulator": confused with iOS simulator. Maybe _dragAndDropSimulator

[Wenson Hsieh]

Comment on attachment 346957 [details]
Fix 32-bit macOS build

View in context: https://bugs.webkit.org/attachment.cgi?id=346957&action=review

>> Tools/TestWebKitAPI/Tests/WebKitCocoa/DragAndDropTests.h:26
>> +#pragma once
> 
> Is this file needed at all?

Nope — good catch! Removed this file.

>> Tools/TestWebKitAPI/mac/DragAndDropSimulatorMac.mm:47
>> +    WeakObjCPtr<DragAndDropSimulator> _simulator;
> 
> I don't really like just "simulator": confused with iOS simulator. Maybe _dragAndDropSimulator

Sounds good — renamed to _dragAndDropSimulator.

[Daniel Bates]

(In reply to Wenson Hsieh from comment #8)
> (In reply to Daniel Bates from comment #7)
> > Is it not possible to expose this functionality in WebKitTestRunner?
> 
> Great question! I think it's possible, but I have some concerns/comments:
> 
> - In order to enable the same cross-platform test coverage as we have in our
> API tests (currently, DragAndDropTestsIOS.mm) this would require us to
> either mirror tons of platform API in UI-script-side bindings, or have a
> large number of specific, one-off utility functions per test. This is
> because there are a myriad of ways to both write and read pasteboard data on
> macOS and iOS (using NSItemProviders on iOS and NSPasteboard on macOS). On
> iOS, one can write data by creating an NSItemProvider and manually adding
> NSData for a specific type identifier, or one can write data by registering
> an NSItemProvider with an NSObject that conforms to NSItemProviderWriting.
> One can also write multiple objects when dragging, and additionally
> influence the type precedence of data in each item provider by changing the
> order in which data or objects are registered. Our existing drag and drop
> API tests on iOS exercise all of these cases in one form or another, and
> providing the same flexibility in WebKitTestRunner would require plumbing a
> large amount of platform-specific API through UIScriptController or the test
> runner.
> 
> - There's also a nontrivial amount of actual WebKit API (well technically,
> SPI) that is currently tested via drag and drop API tests. These mainly
> exist in WKUIDelegatePrivate and WKWebViewPrivate, and are used heavily by
> Mail and Safari on iOS. WKAttachmentTests.mm contains many examples where I
> used DragAndDropSimulator to ensure test coverage of the WebKit attachment
> SPI when using drag and drop (both on the ObjC-side and JS bindings-side).
> As I continue working on drag and drop support on macOS WebKit2 in support
> of Mail compose, I'll need to add many macOS tests here as well to test how
> the WebKit attachment API works with drag and drop. Additionally, a large
> number of tests in DragAndDropSimulatorIOS also exercise Objective-C SPI
> that we provide to observe, intercept, and adjust the behavior of drag and
> drop (used mostly by Mail and Safari). If these were in WebKitTestRunner,
> we'd have to do tons of plumbing (similar to the aforementioned case) to
> enable the same level of test coverage as we have in API tests.
> 
> All this being said, there are some very neat benefits to exposing this
> functionality in WebKitTestRunner: some of our existing drag and drop API
> tests really don't exercise platform API/SPI (both from WebKit and
> UIKit/AppKit) in ways that are interesting. For such types of API tests, we
> could convert them into cross-platform layout tests without running into my
> concerns above, which would allow us to get EWS coverage for them, which API
> tests (at least, currently) lack.
> 
> Ultimately, I think we should make it possible to write drag and drop tests
> as either API tests or layout tests (whichever one is more relevant to the
> scenario being tested). For tests that exercise the nooks and crannies in
> platform ObjC API/SPI (of which there are many), I think we need the
> flexibility that API tests offer; for tests that are inherently
> cross-platform and don't rely on specific API, layout tests seem like a
> better fit. Perhaps what we should do is add some way to add testing support
> logic that is available in both API tests and layout tests. This would be
> similar to the Tools/TestRunnerShared folder, which is currently used for
> code that is common to both DumpRenderTree and WebKitTestRunner, only this
> would be code that is applicable to all three test harnesses (TestWebKitAPI
> and both test runners, WK1 and WK2). Or, alternately, we could rename
> TestRunnerShared to something like TestingShared, and have it serve
> TestWebKitAPI/DumpRenderTree/WebKitTestRunner — then, DragAndDropSimulator
> could be moved there, and we'd be able to write drag and drop tests that run
> in any of our relevant test harnesses.
> 
> What do you think?

Wenson and I talked about this in person today (08/13). Wenson explained to me that it is not straightforward to expose window.eventSender-like API in UIScriptController to support writing drag-and-drop layouts test that work on both Mac and iOS. In particular, Wenson explained that iOS does not allow us to simulate low-level HID dragging and dropping. We would need to provide our own mocks to achieve comparable testing and/or expose platform-specific UIScriptController functions to setup system state to be able to make use of the systems abstractions that we do have on iOS. We will likely need a way to test drag and drop interactions with respect to SPI callbacks  and our own conformance to the AppKit/UIKit drag-and-drop API that will necessitate API unit tests. For now, it seems sufficient to continue building up own drag-and-drop unit test capabilities. Maybe one day we can expose more of this testing machinery to layout tests.

[Wenson Hsieh]

Created attachment 347033 [details]
Patch for landing

[WebKit Commit Bot]

Comment on attachment 347033 [details]
Patch for landing

Clearing flags on attachment: 347033

Committed r234816: <https://trac.webkit.org/changeset/234816>