256330 2023-05-04 12:41:28 -0700 Unexpected "Extra bytes past the end" error when decompressing data with DecompressionStream 2023-05-17 13:35:17 -0700 1 1 1 Unclassified WebKit WebKit Misc. Safari 16 Unspecified macOS 13 RESOLVED DUPLICATE 254021 InRadar P2 Normal --- 1 gildas.lormeau webkit-unassigned 32jojo32 achristensen ap ashley bfulgham brandonstewart webkit-bug-importer youennf oldest_to_newest 1952979 0 gildas.lormeau 2023-05-04 12:41:28 -0700 I'm the author of zip.js, see https://github.com/gildas-lormeau/zip.js/. A user has reported a bug that is specific to Safari. This bug is a regression introduced in the version 16.4. An unexpected error "TypeError: Extra bytes past the end" is thrown when unzipping a file. When disabling the usage of the `DecompressionStream` API in zip.js (via the option `useCompressionStream`), the bug disappears. Unfortunately, I cannot provide this zip file. You can find more information here: https://github.com/gildas-lormeau/zip.js/issues/412 1953046 1 ap 2023-05-04 16:55:50 -0700 Thank you for the report! Unfortunately, this seems unlikely to be actionable without an example archive. > This bug is a regression introduced in the version 16.4. Could you please elaborate on this? Safari 16.4 is the first version to support DecompressionStream, so it could never work differently before. Are you saying that something used to work as expected using a polyfill, and doesn't work with native implementation? 1953055 2 gildas.lormeau 2023-05-04 17:45:27 -0700 Indeed, I was meaning it's a regression in zip.js that has been introduced by Safari 16.4. In Safari 16.3 (and Firefox), zip.js uses its own implementation. Note that zip.js does not contain user-agent sniffing code. Therefore, I am very reluctant to circumvent it. 1953056 3 gildas.lormeau 2023-05-04 17:48:18 -0700 By the way, what is the "TypeError: Extra bytes past the end" error? It does not seem to be specified here: https://wicg.github.io/compression/#compression-stream. 1953061 4 gildas.lormeau 2023-05-04 18:15:39 -0700 Actually, the error seems to be thrown when reading 128 bytes at the offset 1664 of a compressed stream weighting 247,454 bytes. It looks like the error would be thrown probably when creating a `Uint8Array()` of 128 bytes containing the decompressed data. 1953063 5 gildas.lormeau 2023-05-04 18:18:07 -0700 Sorry, I was meaning "128 bytes containing the **compressed** data." in the last sentence of my last comment. 1953188 6 ap 2023-05-05 08:55:40 -0700 It's raised here: https://github.com/WebKit/WebKit/blob/main/Source/WebCore/Modules/compression/DecompressionStreamDecoder.cpp#L138. I am not familiar with this API or code, but from quickly glancing at the spec, it does specify raising a TypeError for decoding errors. 1953189 7 webkit-bug-importer 2023-05-05 08:55:55 -0700 <rdar://problem/108950984> 1953213 8 gildas.lormeau 2023-05-05 11:01:56 -0700 Thank you for the additional info. It looks like this issue is very similar to these ones in zip.js: https://github.com/gildas-lormeau/zip.js/issues/44, https://github.com/gildas-lormeau/zip.js/issues/101. I'm trying to find the zip file that was attached. It also looks like it was possible to reproduce the issue by using .NET's DeflateStream in 2013 (see first link). 1953271 9 gildas.lormeau 2023-05-05 13:11:43 -0700 I was not able to find the zip file unfortunately. 1953281 10 gildas.lormeau 2023-05-05 13:50:36 -0700 I was able to find an example of code to reproduce the issue inspired from this thread on SO: https://stackoverflow.com/questions/67526864/how-to-inflate-a-gzip-with-a-really-old-zlib. Steps to reproduce the issue: - Open a new tab in Safari - Open the developer tools - Run the code below ` (async () => { const result = new TransformStream(); (new Blob([new Uint8Array([0xec, 0x9a, 0x7f, 0x54, 0x1c, 0x55, 0x96, 0xc7, 0x6f, 0x75, 0x37, 0xd0, 0xfc, 0x70])])).stream().pipeThrough(new DecompressionStream("deflate-raw")).pipeTo(result.writable); console.log(await new Response(result.readable).blob()); })(); ` Expected result: An error similar to "TypeError: The input is ended without reaching the stream end" (Firefox) or "TypeError: Compressed input was truncated." (Chrome) should be thrown. Result in Safari 16.4: The error "TypeError: Extra bytes past the end." is thrown instead. The code run as expected in: - Firefox nightly: yes - Chrome: yes 1953338 11 gildas.lormeau 2023-05-05 17:46:50 -0700 Sorry, forget my last comment. I think my test is wrong. 1955622 12 32jojo32 2023-05-16 03:28:53 -0700 I believe I might have a better example. The example I share tries to compress text and then decompress. Chrome handles it correctly. Unfortunately text is pretty long as I was not able to replicate it on smaller substring. To replicate the bug open the file and you should see one error log and one success log. Chrome shows two success logs. https://gist.github.com/Joozty/3bd03a81d4d6540eaea683d9c6389703 1955684 13 brandonstewart 2023-05-16 09:35:48 -0700 @Jozef Thanks for the test case; I just tried your example and in Safari and STP I was seeing the issue. When I tried a build off of mainline though I am seeing 2 success messages in the console (using MiniBrower). I integrated a fix a few days ago that handled an edge case during the flush step. https://github.com/WebKit/WebKit/pull/13741 1955881 14 ashley 2023-05-17 05:04:43 -0700 So did Safari 16.4 ship broken CompressionStreams when using large amounts of content? This in turn would break the zip.js library in many cases, and would explain why we've been getting a bunch of customer support queries about errors opening zip-based content in our web app... 1955993 15 bfulgham 2023-05-17 13:02:06 -0700 (In reply to Ashley Gullen from comment #14) > So did Safari 16.4 ship broken CompressionStreams when using large amounts > of content? This in turn would break the zip.js library in many cases, and > would explain why we've been getting a bunch of customer support queries > about errors opening zip-based content in our web app... Compression Streams first shipped in Safari 16.4, so yes -- these bugs would be new with that release. We think this issue is a duplicate of Bug 254021, which has a test case that we were able to use to confirm and fix. 1956005 16 bfulgham 2023-05-17 13:35:17 -0700 *** This bug has been marked as a duplicate of bug 254021 ***