Forbidden to access WinCairo Buildbot sub-directories of layout test results as "403 Forbidden"

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Fujii Hironori]

Forbidden to access WinCairo Buildbot sub-directories of layout test results as "403 Forbidden"

Slack discussions: 

fujihiro
@Aakash Jain: Why does WinCairo test result sub-directories say "403 Forbidden"? For example, https://build.webkit.org/results/WinCairo-64-bit-WKL-Release-Tests/r273718%20%28288%29/canvas/

fujihiro
Other builders can show the sub-directories. Only WinCairo-64-bit-WKL-Debug-Tests and WinCairo-64-bit-WKL-Release-Tests can't show. https://build.webkit.org/results

Aakash Jain
I'll check. (edited) 

Aakash Jain
@fujihiro Please check if your bots have umask = 022 in their buildbot.tac file. All our bots use that. It is for setting the file permissions.

Stephan Szabo
It does appear to have that on the two bots above.
...
usepty = 0
umask = 022
maxdelay = 300
...

Aakash Jain
not sure then.
Maybe you can download the zip files from https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/, unzip them locally and see file permissions, and then do same thing for some other queue and compare them.

Stephan Szabo
Downloading the latest GTK release tests zip and the WinCairo release tests zip to my mac, the base permissions on say "canvas" are the same drwxr-xr-x, but the GTK one has the "@" for extended attributes.

Stephan Szabo
Asking ls -@, it seems like it's a com.apple.quarantine on the directories in the GTK one.

Stephan Szabo
I'll continue poking at it, but for @fujihiro since it's not going to be first priority, for now, you'll probably need to grab the zip for the results and uncompress it locally.

fujihiro
Thank you, @Aakash Jain and @Stephan Szabo . But, I don't see such @ on my Mac. Why?
fujii@fujiis-Mac-mini $ ls -dl@ gtk/canvas wincairo/canvas
drwxr-xr-x  3 fujii  staff  96 Mar  2 11:13 gtk/canvas/
drwxr-xr-x  3 fujii  staff  96 Mar  2 12:07 wincairo/canvas/

Stephan Szabo
Hmm, interesting - I wonder if it depends on how it was uncompressed. But, you are also seeing rwxr-xr-x for both, so at least that's the same.

[Fujii Hironori]

WinCairo is using 'zipfile' Pytnon module to create the zip files, while GTK, WPE and AppleWin are using 'zip' command.
https://github.com/WebKit/WebKit/blob/main/Tools/CISupport/test-result-archive#L86

[Fujii Hironori]

Excerpts of the result of 'zipdetails' command

WinCairo:

100079 LOCAL HEADER #24C     04034B50
10007D Extract Zip Spec      14 '2.0'
10007E Extract OS            00 'MS-DOS'
10007F General Purpose Flag  0000
       [Bits 1-2]            0 'Normal Compression'
100081 Compression Method    0008 'Deflated'
100083 Last Mod Time         526F3AFD 'Mon Mar 15 07:23:58 2021'
100087 CRC                   FAF8203F
10008B Compressed Length     00000057
10008F Uncompressed Length   00000061
100093 Filename Length       003B
100095 Extra Length          0000
100097 Filename              'canvas/philip/tests/2d.composite.operat
                             ion.clear-actual.txt'
1000D2 PAYLOAD               5.1..0..w_.....:..>@.b..P..d..-
                             ...q..L7.*.%...-
                             1..y...&....'...X....B&.CMR....*...^.t.^


GTK:

03705D0 LOCAL HEADER #612     04034B50
03705D4 Extract Zip Spec      0A '1.0'
03705D5 Extract OS            00 'MS-DOS'
03705D6 General Purpose Flag  0000
03705D8 Compression Method    0000 'Stored'
03705DA Last Mod Time         526F6B4F 'Mon Mar 15 13:26:30 2021'
03705DE CRC                   00000000
03705E2 Compressed Length     00000000
03705E6 Uncompressed Length   00000000
03705EA Filename Length       0007
03705EC Extra Length          001C
03705EE Filename              'canvas/'
03705F5 Extra ID #0001        5455 'UT: Extended Timestamp'
03705F7   Length              0009
03705F9   Flags               '03 mod access'
03705FA   Mod Time            604FC2F6 'Tue Mar 16 05:26:30 2021'
03705FE   Access Time         604FC963 'Tue Mar 16 05:53:55 2021'
0370602 Extra ID #0002        7875 'ux: Unix Extra Type 3'
0370604   Length              000B
0370606   Version             01
0370607   UID Size            04
0370608   UID                 00000077
037060C   GID Size            04
037060D   GID                 00000084

03706A7 LOCAL HEADER #615     04034B50
03706AB Extract Zip Spec      14 '2.0'
03706AC Extract OS            00 'MS-DOS'
03706AD General Purpose Flag  0004
        [Bits 1-2]            0 'Normal Compression'
03706AF Compression Method    0008 'Deflated'
03706B1 Last Mod Time         526F6B62 'Mon Mar 15 13:27:04 2021'
03706B5 CRC                   07E13F75
03706B9 Compressed Length     00000055
03706BD Uncompressed Length   00000148
03706C1 Filename Length       003C
03706C3 Extra Length          001C
03706C5 Filename              'canvas/philip/tests/2d.text.draw.baseli
                              ne.hanging-actual.txt'
0370701 Extra ID #0001        5455 'UT: Extended Timestamp'
0370703   Length              0009
0370705   Flags               '03 mod access'
0370706   Mod Time            604FC317 'Tue Mar 16 05:27:03 2021'
037070A   Access Time         604FC963 'Tue Mar 16 05:53:55 2021'
037070E Extra ID #0002        7875 'ux: Unix Extra Type 3'
0370710   Length              000B
0370712   Version             01
0370713   UID Size            04
0370714   UID                 00000077
0370718   GID Size            04
0370719   GID                 00000084
037071D PAYLOAD               sK..IMQH,.N-*....RH./Q(..H.Q.625.1.B
                              ...X..a.c....ZQ..\......)...U0.r#.@K..h.
                              cB}7.6..


GTK' zip file has entries of directories and a extra metadata 'ux: Unix Extra Type 3'.

[Fujii Hironori]

Created attachment 423248 [details]
zipdetails of WinCiaro

I created small test-result-archive.zip files by invoking test-result-archive script on my Windows and Linux.
And, invoked zipdetails command for both zip files.

python .\Tools\CISupport\test-result-archive archive --platform=wincairo --release

python Tools/CISupport/test-result-archive archive --platform=gtk --release

[Fujii Hironori]

Created attachment 423249 [details]
zipdetails of GTK

[Fujii Hironori]

I unzipped zip files of GTK and WinCairo with 022 and 077 umasks.

$ umask 022
$ unzip gtk.zip -d gtk-022
$ unzip wincairo.zip -d wincairo-022
$ umask 077
$ unzip gtk.zip -d gtk-077
$ unzip wincairo.zip -d wincairo-077
$ ls -dl {gtk,wincairo}-{022,077}/canvas
drwxr-xr-x 1 fujii fujii 512 Mar 16 05:26 gtk-022/canvas/
drwxr-xr-x 1 fujii fujii 512 Mar 16 05:26 gtk-077/canvas/
drwxr-xr-x 1 fujii fujii 512 Mar 16 08:53 wincairo-022/canvas/
drwx------ 1 fujii fujii 512 Mar 16 08:54 wincairo-077/canvas/

WinCairo's sub-directories are affected by umask.

[Fujii Hironori]

Hi, Aakash.
Because WinCairo's zip files don't contain Unix permission information, the sub-directories are affected by umask of Buildbot controller.
Could you address this issue in the server side?

[[e-mail]]

I added "nginx" to group ACL for the WinCairo-64-bit-WKL-Release-Tests directory on build.webkit.org so nginx would be able to read these files without POSIX permission. So we don't have to worry about umask or lacking of UNIX permission for Windows generated files.

[Fujii Hironori]

Thank you very much.

[Fujii Hironori]

*** Bug 227928 has been marked as a duplicate of this bug. ***

[Fujii Hironori]

Hi Lingho,

We are having the same problem again after introducing WinCairo WK2 testers.
The following directories report "403 Forbidden". Could you take a look again?

https://build.webkit.org/results/WinCairo-64-bit-Debug-Tests/260833%40main%20%2810670%29/fast/
https://build.webkit.org/results/WinCairo-64-bit-Release-Tests/260835%40main%20%28136%29/fast/

[[e-mail]]

Hello Fujii,

I have applied the same ACLs on WinCairo-64-bit-Release-Tests and WinCairo-64-bit-Debug-Tests. Could you please try again?

...
ling

[Fujii Hironori]

It works. Thank you, Ling.