Bug 220995 - [LayoutTests] Convert xmlhttprequest php to Python
Summary: [LayoutTests] Convert xmlhttprequest php to Python
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-01-26 10:58 PST by Jonathan Bedard
Modified: 2021-02-09 13:51 PST (History)
10 users (show)

See Also:


Attachments
Patch (86.57 KB, patch)
2021-01-26 13:21 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (139.15 KB, patch)
2021-01-27 11:06 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (219.26 KB, patch)
2021-01-27 16:47 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (342.66 KB, patch)
2021-01-29 12:57 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (346.02 KB, patch)
2021-02-01 09:52 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (375.14 KB, patch)
2021-02-01 14:39 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (374.91 KB, patch)
2021-02-02 09:44 PST, Jonathan Bedard
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (375.83 KB, patch)
2021-02-03 19:45 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (375.48 KB, patch)
2021-02-05 12:10 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (375.70 KB, patch)
2021-02-05 13:16 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (377.75 KB, patch)
2021-02-08 11:03 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (4.55 KB, patch)
2021-02-09 12:55 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Bedard 2021-01-26 10:58:47 PST
Converting all php in xmlhttprequest to Python to gauge the difficulty of doing this across all layout tests.
Comment 1 Radar WebKit Bug Importer 2021-01-26 13:18:17 PST
<rdar://problem/73630008>
Comment 2 Jonathan Bedard 2021-01-26 13:21:55 PST
Created attachment 418472 [details]
Patch
Comment 3 Jonathan Bedard 2021-01-27 11:06:57 PST
Created attachment 418562 [details]
Patch
Comment 4 Jonathan Bedard 2021-01-27 16:47:45 PST
Created attachment 418599 [details]
Patch
Comment 5 Jonathan Bedard 2021-01-27 20:12:25 PST
(In reply to Jonathan Bedard from comment #4)
> Created attachment 418599 [details]
> Patch

The test failures caused by this change are because Mojave does not have Python 3. We’re about to update our Mac EWS bots, though.
Comment 6 Jonathan Bedard 2021-01-29 12:57:25 PST
Created attachment 418761 [details]
Patch
Comment 7 Jonathan Bedard 2021-02-01 09:52:06 PST
Created attachment 418889 [details]
Patch
Comment 8 Jonathan Bedard 2021-02-01 14:39:01 PST
Created attachment 418926 [details]
Patch
Comment 9 Jonathan Bedard 2021-02-02 09:44:15 PST
Created attachment 419015 [details]
Patch
Comment 10 Jonathan Bedard 2021-02-03 19:45:41 PST
Created attachment 419220 [details]
Patch
Comment 11 Sam Sneddon [:gsnedders] 2021-02-05 07:49:44 PST
Comment on attachment 419220 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=419220&action=review

Non-reviewer review:

I'll point out that we have Python-using copies of most of these in wpt/xhr today, as both are ultimately derived from the same written-in-PHP-from-Opera-Software test suite. So starting with XHR might not be the most valuable place to start! That said, this mostly looks good.

> LayoutTests/http/tests/xmlhttprequest/null-auth.py:9
> +sys.stdout.write('Content-Type: text/html')
> +sys.stdout.write('\r\n\r\n')
> +
> +print('<p>Test that null values in XHR login/password parameters are treated correctly.</p>')
> +print('<p>No auth tokens should be sent with this request.</p>')

I find the combination of sys.stdout.write and print somewhat surprising, even though I understand _why_ it's doing it (to ensure headers are terminated with CRLF, and not caring about the body of the response). Equally, it feels like in many cases the print could be of a triple-quoted multi-line string, rather than like this. Not sure how difficult/worthwhile making such changes is, though?

> LayoutTests/http/tests/xmlhttprequest/resources/multipart-post-echo.py:16
> +for key in sorted(form.keys()):

That this is sorting them is a notable change from the original PHP, and causes changed expectations. I can't recall quite how precisely this is specified, but it would be nice to avoid changing the order? (Especially when CPython 3.6+ preserves order with dicts, so the iteration order isn't at the mercy of the dict construction any more.)

> LayoutTests/http/tests/local/formdata/send-form-data-constructed-from-form-using-open-panel-expected.txt:5
> -Response: submitted=true&checkbox=value2&radio=value3&text=value4&file=test.txt:text/plain:Hello
> +Response: checkbox=value2&radio=value3&submitted=true&text=value4&file=test.txt:text/plain:Hello

e.g., this change
Comment 12 Jonathan Bedard 2021-02-05 12:08:31 PST
Comment on attachment 419220 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=419220&action=review

>> LayoutTests/http/tests/xmlhttprequest/null-auth.py:9
>> +print('<p>No auth tokens should be sent with this request.</p>')
> 
> I find the combination of sys.stdout.write and print somewhat surprising, even though I understand _why_ it's doing it (to ensure headers are terminated with CRLF, and not caring about the body of the response). Equally, it feels like in many cases the print could be of a triple-quoted multi-line string, rather than like this. Not sure how difficult/worthwhile making such changes is, though?

I'll upload a change that makes an attempt to standardize this...I think it's really going to be a question about style preferences.

>> LayoutTests/http/tests/xmlhttprequest/resources/multipart-post-echo.py:16
>> +for key in sorted(form.keys()):
> 
> That this is sorting them is a notable change from the original PHP, and causes changed expectations. I can't recall quite how precisely this is specified, but it would be nice to avoid changing the order? (Especially when CPython 3.6+ preserves order with dicts, so the iteration order isn't at the mercy of the dict construction any more.)

Turns out the cgi module pretty much always sorts this (ie, key in sorted(form.keys()), key in form.keys(), key in form are all sorted)

I'd sort of like to keep it sorted(form.keys()), so future contributors will immediately know how this is organized.
Comment 13 Jonathan Bedard 2021-02-05 12:10:00 PST
Created attachment 419442 [details]
Patch
Comment 14 Jonathan Bedard 2021-02-05 13:16:48 PST
Created attachment 419453 [details]
Patch
Comment 15 Jonathan Bedard 2021-02-05 13:25:19 PST
(In reply to Jonathan Bedard from comment #14)
> Created attachment 419453 [details]
> Patch

Not a big fan of how the multi-line strings end up looking....
Comment 16 Alex Christensen 2021-02-05 14:11:59 PST
Comment on attachment 419442 [details]
Patch

Why not perl?
r=me
Comment 17 Jonathan Bedard 2021-02-05 14:22:12 PST
(In reply to Alex Christensen from comment #16)
> Comment on attachment 419442 [details]
> Patch
> 
> Why not perl?
> r=me

Because there are many folks who want to replace our perl with Python :)
Comment 18 Jonathan Bedard 2021-02-08 11:03:15 PST
Created attachment 419608 [details]
Patch
Comment 19 EWS 2021-02-08 14:04:17 PST
Committed r272548: <https://commits.webkit.org/r272548>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 419608 [details].
Comment 20 Jonathan Bedard 2021-02-09 12:55:14 PST
Reopening to attach new patch.
Comment 21 Jonathan Bedard 2021-02-09 12:55:16 PST
Created attachment 419752 [details]
Patch
Comment 22 EWS 2021-02-09 13:51:48 PST
Committed r272609: <https://commits.webkit.org/r272609>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 419752 [details].