Fix reading slightly incorrect Zip files - #68106

Merged
danmoseley merged 7 commits into
dotnet:mainfrom
danmoseley:zip64
Apr 20, 2022
Merged

Fix reading slightly incorrect Zip files#68106
danmoseley merged 7 commits into
dotnet:mainfrom
danmoseley:zip64

Conversation

@danmoseley

@danmoseleydanmoseley commented Apr 16, 2022

Copy link
Copy Markdown
Contributor

@ghostghost assigned danmoseleyApr 16, 2022
@ghost

ghost commented Apr 16, 2022

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

Issue Details

Fix #49580

Author:danmoseley
Assignees:danmoseley
Labels:

area-System.IO.Compression

Milestone:-

@danmoseley

danmoseley commented Apr 16, 2022

Copy link
Copy Markdown
ContributorAuthor

Also fix new SC test that will only pass elevated. cc @deeprobin -- our PR validation runs elevated, I believe.

danmoseley
danmoseley commented Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
gfoidl
gfoidl reviewed Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated
danmoseley
danmoseley commented Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
@danmoseley

danmoseley commented Apr 17, 2022

Copy link
Copy Markdown
ContributorAuthor

Updated with full annotations added for the test zip, making clear what's going on.

@danmoseley

danmoseley commented Apr 18, 2022

Copy link
Copy Markdown
ContributorAuthor

failures are #67349

@danmoseley

danmoseley commented Apr 18, 2022

Copy link
Copy Markdown
ContributorAuthor

BTW @dotnet/area-system-io-compression are we aware of any public corpus of test zip files? Clearly, there is some variation in spec interpretations. If there was such a corpus, I could imagine we write a script to unzip them all with "main" and also with 6.0 and verify the same outputs/errors. That would reduce the risk of the #1094 experience we had.

deeprobin
deeprobin approved these changes Apr 19, 2022
View reviewed changes

@deeprobindeeprobin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits.

Looks overall good to me

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated
Dan Moseley@deeprobin
Apply suggestions from code review
2c788e2
Co-authored-by: Robin Lindner <robin@deeprobin.de>
adamsitnik
adamsitnik approved these changes Apr 20, 2022
View reviewed changes

@adamsitnikadamsitnik left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I've left some comments, but all of them are just nits.

@danmoseley thanks a lot!

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated
Assert.Equal(6, entry.CompressedLength); // it should have used 32-bit size
}

private static readonly byte[] s_tinyZip64 =

@adamsitnikadamsitnikApr 20, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: based on the description provided for the tests above, I think that "slightlyIncorrect" would be a better name than "tiny" here

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
danmoseley
danmoseley commented Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated
danmoseley
danmoseley commented Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated
danmoseley
danmoseley commented Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated
Dan Moseley@adamsitnik
Apply suggestions from code review
8e33f06
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@danmoseley

danmoseley commented Apr 20, 2022

Copy link
Copy Markdown
ContributorAuthor

@dotnet/dnceng seems an issue in dotnet-install.sh?

2022-04-20T16:20:22.1681480Z ##[section]Starting: Restore and Build Product
2022-04-20T16:20:22.1693590Z ==============================================================================
2022-04-20T16:20:22.1694320Z Task : Command line
2022-04-20T16:20:22.1694920Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-20T16:20:22.1695490Z Version : 2.201.1
2022-04-20T16:20:22.1695890Z Author : Microsoft Corporation
2022-04-20T16:20:22.1696510Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-20T16:20:22.1697200Z ==============================================================================
2022-04-20T16:20:22.5454730Z Generating script.
2022-04-20T16:20:22.5776320Z Script contents:
2022-04-20T16:20:22.5837500Z ./build.sh -subset libs+libs.tests -configuration Debug -ci -arch x64 -framework net7.0 -testscope innerloop /p:ArchiveTests=true
2022-04-20T16:20:22.5927300Z ========================== Starting Command Output ===========================
2022-04-20T16:20:22.6362330Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/7198096b-97d2-4e9e-b9f3-8d34e7d82013.sh
2022-04-20T16:20:24.1231070Z __DistroRid: osx-x64
2022-04-20T16:20:24.5786000Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-04-20T16:21:23.4178630Z Attempting to install dotnet from public_location.
2022-04-20T16:21:23.4381070Z /Users/runner/work/1/s//.dotnet/dotnet-install.sh: line 2271: syntax error: unexpected end of file

@ChadNedzlek

ChadNedzlek commented Apr 20, 2022

Copy link
Copy Markdown
Contributor

@danmoseley. It does look like there is a bug with it, but that tool isn't written/maintained by dnceng. If you click the "report infrastructure issue" we could try and get it assigned to the right people.

@danmoseley
danmoseley merged commit 9a4cdce into dotnet:mainApr 20, 2022
@danmoseley
danmoseley deleted the zip64 branch April 20, 2022 19:07
@danmoseley

danmoseley commented Apr 20, 2022

Copy link
Copy Markdown
ContributorAuthor

My bad, thanks @ChadNedzlek and @missymessa

@missymessa

missymessa commented Apr 20, 2022

Copy link
Copy Markdown
Member

My bad, thanks @ChadNedzlek and @missymessa

No worries, thanks for using the new features :)

directhex pushed a commit to directhex/runtime that referenced this pull request Apr 21, 2022
Dan Moseley@deeprobin
@adamsitnikJo Shields
Fix reading slightly incorrect Zip files (dotnet#68106)
55d0cb6
* Add test that fails
* fix
* Fix test failing unelevated
* annotate
* overflow
* Apply suggestions from code review
Co-authored-by: Robin Lindner <robin@deeprobin.de>
* Apply suggestions from code review
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
Co-authored-by: Robin Lindner <robin@deeprobin.de>
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@ghostghost locked as resolved and limited conversation to collaborators May 21, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@adamsitnikadamsitnikadamsitnik approved these changes
@carlossanlopcarlossanlopAwaiting requested review from carlossanlop
+2 more reviewers
@gfoidlgfoidlgfoidl left review comments
@deeprobindeeprobindeeprobin approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@danmoseleydanmoseley

Labels

area-System.IO.Compression

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

"A local file header is corrupt" error occurs while unpacking the ZIP archive

6 participants

@danmoseley@ChadNedzlek@missymessa@gfoidl@adamsitnik@deeprobin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Fix reading slightly incorrect Zip files - #68106
Merged
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Apr 20, 2022
Merged
Fix reading slightly incorrect Zip files#68106
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Conversation
@danmoseley
@danmoseleydanmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

Contributor
@ghostghost
assigned danmoseleyApr 16, 2022
@ghost
ghost
commented
Apr 16, 2022

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-io-compression

See info in area-owners.md if you want to be subscribed.
Issue Details
Fix #49580
Author:danmoseley
Assignees:danmoseley
Labels:
area-System.IO.Compression
Milestone:-
@danmoseley
danmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

ContributorAuthor
Also fix new SC test that will only pass elevated. cc @deeprobin -- our PR validation runs elevated, I believe.
danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
gfoidl
gfoidl
reviewed
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
@danmoseley
danmoseley
commented
Apr 17, 2022

Copy link


Copy Markdown

ContributorAuthor
Updated with full annotations added for the test zip, making clear what's going on.
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
failures are #67349
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
BTW @dotnet/area-system-io-compression are we aware of any public corpus of test zip files? Clearly, there is some variation in spec interpretations. If there was such a corpus, I could imagine we write a script to unzip them all with "main" and also with 6.0 and verify the same outputs/errors. That would reduce the risk of the #1094 experience we had.
deeprobin
deeprobin
approved these changes
Apr 19, 2022
View reviewed changes
@deeprobindeeprobin
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits.
Looks overall good to me
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@deeprobin
Apply suggestions from code review
2c788e2
Co-authored-by: Robin Lindner <robin@deeprobin.de>
adamsitnik
adamsitnik
approved these changes
Apr 20, 2022
View reviewed changes
@adamsitnikadamsitnik
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I've left some comments, but all of them are just nits.
@danmoseley thanks a lot!
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

 Assert.Equal(6, entry.CompressedLength); // it should have used 32-bit size
 }

 private static readonly byte[] s_tinyZip64 =
@adamsitnikadamsitnikApr 20, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: based on the description provided for the tests above, I think that "slightlyIncorrect" would be a better name than "tiny" here
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@adamsitnik
Apply suggestions from code review
8e33f06
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@dotnet/dnceng seems an issue in dotnet-install.sh?
2022-04-20T16:20:22.1681480Z ##[section]Starting: Restore and Build Product
2022-04-20T16:20:22.1693590Z ==============================================================================
2022-04-20T16:20:22.1694320Z Task : Command line
2022-04-20T16:20:22.1694920Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-20T16:20:22.1695490Z Version : 2.201.1
2022-04-20T16:20:22.1695890Z Author : Microsoft Corporation
2022-04-20T16:20:22.1696510Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-20T16:20:22.1697200Z ==============================================================================
2022-04-20T16:20:22.5454730Z Generating script.
2022-04-20T16:20:22.5776320Z Script contents:
2022-04-20T16:20:22.5837500Z ./build.sh -subset libs+libs.tests -configuration Debug -ci -arch x64 -framework net7.0 -testscope innerloop /p:ArchiveTests=true
2022-04-20T16:20:22.5927300Z ========================== Starting Command Output ===========================
2022-04-20T16:20:22.6362330Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/7198096b-97d2-4e9e-b9f3-8d34e7d82013.sh
2022-04-20T16:20:24.1231070Z __DistroRid: osx-x64
2022-04-20T16:20:24.5786000Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-04-20T16:21:23.4178630Z Attempting to install dotnet from public_location.
2022-04-20T16:21:23.4381070Z /Users/runner/work/1/s//.dotnet/dotnet-install.sh: line 2271: syntax error: unexpected end of file

@ChadNedzlek
ChadNedzlek
commented
Apr 20, 2022

Copy link


Copy Markdown

Contributor
@danmoseley. It does look like there is a bug with it, but that tool isn't written/maintained by dnceng. If you click the "report infrastructure issue" we could try and get it assigned to the right people.
@danmoseley
danmoseley
merged commit 9a4cdce
into
dotnet:mainApr 20, 2022
@danmoseley
danmoseley
deleted the

zip64

branch
April 20, 2022 19:07
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
My bad, thanks @ChadNedzlek and @missymessa
@missymessa
missymessa
commented
Apr 20, 2022

Copy link


Copy Markdown

Member
My bad, thanks @ChadNedzlek and @missymessa
No worries, thanks for using the new features :)
directhex pushed a commit
to directhex/runtime
that referenced
this pull request
Apr 21, 2022
Dan Moseley@deeprobin
@adamsitnikJo Shields
Fix reading slightly incorrect Zip files (dotnet#68106)
55d0cb6
* Add test that fails
* fix
* Fix test failing unelevated
* annotate
* overflow
* Apply suggestions from code review
Co-authored-by: Robin Lindner <robin@deeprobin.de>
* Apply suggestions from code review
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
Co-authored-by: Robin Lindner <robin@deeprobin.de>
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
May 21, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@adamsitnikadamsitnikadamsitnik approved these changes
@carlossanlopcarlossanlopAwaiting requested review from carlossanlop

+2 more reviewers

@gfoidlgfoidlgfoidl left review comments
@deeprobindeeprobindeeprobin approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@danmoseleydanmoseley

Labels

area-System.IO.Compression

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

"A local file header is corrupt" error occurs while unpacking the ZIP archive


6 participants

@danmoseley@ChadNedzlek@missymessa@gfoidl@adamsitnik@deeprobin

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Fix reading slightly incorrect Zip files - #68106
Merged
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Apr 20, 2022
Merged
Fix reading slightly incorrect Zip files#68106
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Conversation
@danmoseley
@danmoseleydanmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

Contributor
@ghostghost
assigned danmoseleyApr 16, 2022
@ghost
ghost
commented
Apr 16, 2022

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-io-compression

See info in area-owners.md if you want to be subscribed.
Issue Details
Fix #49580
Author:danmoseley
Assignees:danmoseley
Labels:
area-System.IO.Compression
Milestone:-
@danmoseley
danmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

ContributorAuthor
Also fix new SC test that will only pass elevated. cc @deeprobin -- our PR validation runs elevated, I believe.
danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
gfoidl
gfoidl
reviewed
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
@danmoseley
danmoseley
commented
Apr 17, 2022

Copy link


Copy Markdown

ContributorAuthor
Updated with full annotations added for the test zip, making clear what's going on.
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
failures are #67349
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
BTW @dotnet/area-system-io-compression are we aware of any public corpus of test zip files? Clearly, there is some variation in spec interpretations. If there was such a corpus, I could imagine we write a script to unzip them all with "main" and also with 6.0 and verify the same outputs/errors. That would reduce the risk of the #1094 experience we had.
deeprobin
deeprobin
approved these changes
Apr 19, 2022
View reviewed changes
@deeprobindeeprobin
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits.
Looks overall good to me
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@deeprobin
Apply suggestions from code review
2c788e2
Co-authored-by: Robin Lindner <robin@deeprobin.de>
adamsitnik
adamsitnik
approved these changes
Apr 20, 2022
View reviewed changes
@adamsitnikadamsitnik
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I've left some comments, but all of them are just nits.
@danmoseley thanks a lot!
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

 Assert.Equal(6, entry.CompressedLength); // it should have used 32-bit size
 }

 private static readonly byte[] s_tinyZip64 =
@adamsitnikadamsitnikApr 20, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: based on the description provided for the tests above, I think that "slightlyIncorrect" would be a better name than "tiny" here
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@adamsitnik
Apply suggestions from code review
8e33f06
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@dotnet/dnceng seems an issue in dotnet-install.sh?
2022-04-20T16:20:22.1681480Z ##[section]Starting: Restore and Build Product
2022-04-20T16:20:22.1693590Z ==============================================================================
2022-04-20T16:20:22.1694320Z Task : Command line
2022-04-20T16:20:22.1694920Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-20T16:20:22.1695490Z Version : 2.201.1
2022-04-20T16:20:22.1695890Z Author : Microsoft Corporation
2022-04-20T16:20:22.1696510Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-20T16:20:22.1697200Z ==============================================================================
2022-04-20T16:20:22.5454730Z Generating script.
2022-04-20T16:20:22.5776320Z Script contents:
2022-04-20T16:20:22.5837500Z ./build.sh -subset libs+libs.tests -configuration Debug -ci -arch x64 -framework net7.0 -testscope innerloop /p:ArchiveTests=true
2022-04-20T16:20:22.5927300Z ========================== Starting Command Output ===========================
2022-04-20T16:20:22.6362330Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/7198096b-97d2-4e9e-b9f3-8d34e7d82013.sh
2022-04-20T16:20:24.1231070Z __DistroRid: osx-x64
2022-04-20T16:20:24.5786000Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-04-20T16:21:23.4178630Z Attempting to install dotnet from public_location.
2022-04-20T16:21:23.4381070Z /Users/runner/work/1/s//.dotnet/dotnet-install.sh: line 2271: syntax error: unexpected end of file

@ChadNedzlek
ChadNedzlek
commented
Apr 20, 2022

Copy link


Copy Markdown

Contributor
@danmoseley. It does look like there is a bug with it, but that tool isn't written/maintained by dnceng. If you click the "report infrastructure issue" we could try and get it assigned to the right people.
@danmoseley
danmoseley
merged commit 9a4cdce
into
dotnet:mainApr 20, 2022
@danmoseley
danmoseley
deleted the

zip64

branch
April 20, 2022 19:07
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
My bad, thanks @ChadNedzlek and @missymessa
@missymessa
missymessa
commented
Apr 20, 2022

Copy link


Copy Markdown

Member
My bad, thanks @ChadNedzlek and @missymessa
No worries, thanks for using the new features :)
directhex pushed a commit
to directhex/runtime
that referenced
this pull request
Apr 21, 2022
Dan Moseley@deeprobin
@adamsitnikJo Shields
Fix reading slightly incorrect Zip files (dotnet#68106)
55d0cb6
* Add test that fails
* fix
* Fix test failing unelevated
* annotate
* overflow
* Apply suggestions from code review
Co-authored-by: Robin Lindner <robin@deeprobin.de>
* Apply suggestions from code review
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
Co-authored-by: Robin Lindner <robin@deeprobin.de>
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
May 21, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@adamsitnikadamsitnikadamsitnik approved these changes
@carlossanlopcarlossanlopAwaiting requested review from carlossanlop

+2 more reviewers

@gfoidlgfoidlgfoidl left review comments
@deeprobindeeprobindeeprobin approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@danmoseleydanmoseley

Labels

area-System.IO.Compression

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

"A local file header is corrupt" error occurs while unpacking the ZIP archive


6 participants

@danmoseley@ChadNedzlek@missymessa@gfoidl@adamsitnik@deeprobin

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Fix reading slightly incorrect Zip files - #68106
Merged
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Apr 20, 2022
Merged
Fix reading slightly incorrect Zip files#68106
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Conversation
@danmoseley
@danmoseleydanmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

Contributor
@ghostghost
assigned danmoseleyApr 16, 2022
@ghost
ghost
commented
Apr 16, 2022

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-io-compression

See info in area-owners.md if you want to be subscribed.
Issue Details
Fix #49580
Author:danmoseley
Assignees:danmoseley
Labels:
area-System.IO.Compression
Milestone:-
@danmoseley
danmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

ContributorAuthor
Also fix new SC test that will only pass elevated. cc @deeprobin -- our PR validation runs elevated, I believe.
danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
gfoidl
gfoidl
reviewed
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
@danmoseley
danmoseley
commented
Apr 17, 2022

Copy link


Copy Markdown

ContributorAuthor
Updated with full annotations added for the test zip, making clear what's going on.
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
failures are #67349
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
BTW @dotnet/area-system-io-compression are we aware of any public corpus of test zip files? Clearly, there is some variation in spec interpretations. If there was such a corpus, I could imagine we write a script to unzip them all with "main" and also with 6.0 and verify the same outputs/errors. That would reduce the risk of the #1094 experience we had.
deeprobin
deeprobin
approved these changes
Apr 19, 2022
View reviewed changes
@deeprobindeeprobin
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits.
Looks overall good to me
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@deeprobin
Apply suggestions from code review
2c788e2
Co-authored-by: Robin Lindner <robin@deeprobin.de>
adamsitnik
adamsitnik
approved these changes
Apr 20, 2022
View reviewed changes
@adamsitnikadamsitnik
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I've left some comments, but all of them are just nits.
@danmoseley thanks a lot!
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

 Assert.Equal(6, entry.CompressedLength); // it should have used 32-bit size
 }

 private static readonly byte[] s_tinyZip64 =
@adamsitnikadamsitnikApr 20, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: based on the description provided for the tests above, I think that "slightlyIncorrect" would be a better name than "tiny" here
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@adamsitnik
Apply suggestions from code review
8e33f06
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@dotnet/dnceng seems an issue in dotnet-install.sh?
2022-04-20T16:20:22.1681480Z ##[section]Starting: Restore and Build Product
2022-04-20T16:20:22.1693590Z ==============================================================================
2022-04-20T16:20:22.1694320Z Task : Command line
2022-04-20T16:20:22.1694920Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-20T16:20:22.1695490Z Version : 2.201.1
2022-04-20T16:20:22.1695890Z Author : Microsoft Corporation
2022-04-20T16:20:22.1696510Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-20T16:20:22.1697200Z ==============================================================================
2022-04-20T16:20:22.5454730Z Generating script.
2022-04-20T16:20:22.5776320Z Script contents:
2022-04-20T16:20:22.5837500Z ./build.sh -subset libs+libs.tests -configuration Debug -ci -arch x64 -framework net7.0 -testscope innerloop /p:ArchiveTests=true
2022-04-20T16:20:22.5927300Z ========================== Starting Command Output ===========================
2022-04-20T16:20:22.6362330Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/7198096b-97d2-4e9e-b9f3-8d34e7d82013.sh
2022-04-20T16:20:24.1231070Z __DistroRid: osx-x64
2022-04-20T16:20:24.5786000Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-04-20T16:21:23.4178630Z Attempting to install dotnet from public_location.
2022-04-20T16:21:23.4381070Z /Users/runner/work/1/s//.dotnet/dotnet-install.sh: line 2271: syntax error: unexpected end of file

@ChadNedzlek
ChadNedzlek
commented
Apr 20, 2022

Copy link


Copy Markdown

Contributor
@danmoseley. It does look like there is a bug with it, but that tool isn't written/maintained by dnceng. If you click the "report infrastructure issue" we could try and get it assigned to the right people.
@danmoseley
danmoseley
merged commit 9a4cdce
into
dotnet:mainApr 20, 2022
@danmoseley
danmoseley
deleted the

zip64

branch
April 20, 2022 19:07
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
My bad, thanks @ChadNedzlek and @missymessa
@missymessa
missymessa
commented
Apr 20, 2022

Copy link


Copy Markdown

Member
My bad, thanks @ChadNedzlek and @missymessa
No worries, thanks for using the new features :)
directhex pushed a commit
to directhex/runtime
that referenced
this pull request
Apr 21, 2022
Dan Moseley@deeprobin
@adamsitnikJo Shields
Fix reading slightly incorrect Zip files (dotnet#68106)
55d0cb6
* Add test that fails
* fix
* Fix test failing unelevated
* annotate
* overflow
* Apply suggestions from code review
Co-authored-by: Robin Lindner <robin@deeprobin.de>
* Apply suggestions from code review
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
Co-authored-by: Robin Lindner <robin@deeprobin.de>
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
May 21, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@adamsitnikadamsitnikadamsitnik approved these changes
@carlossanlopcarlossanlopAwaiting requested review from carlossanlop

+2 more reviewers

@gfoidlgfoidlgfoidl left review comments
@deeprobindeeprobindeeprobin approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@danmoseleydanmoseley

Labels

area-System.IO.Compression

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

"A local file header is corrupt" error occurs while unpacking the ZIP archive


6 participants

@danmoseley@ChadNedzlek@missymessa@gfoidl@adamsitnik@deeprobin

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Fix reading slightly incorrect Zip files - #68106
Merged
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Apr 20, 2022
Merged
Fix reading slightly incorrect Zip files#68106
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Conversation
@danmoseley
@danmoseleydanmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

Contributor
@ghostghost
assigned danmoseleyApr 16, 2022
@ghost
ghost
commented
Apr 16, 2022

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-io-compression

See info in area-owners.md if you want to be subscribed.
Issue Details
Fix #49580
Author:danmoseley
Assignees:danmoseley
Labels:
area-System.IO.Compression
Milestone:-
@danmoseley
danmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

ContributorAuthor
Also fix new SC test that will only pass elevated. cc @deeprobin -- our PR validation runs elevated, I believe.
danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
gfoidl
gfoidl
reviewed
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
@danmoseley
danmoseley
commented
Apr 17, 2022

Copy link


Copy Markdown

ContributorAuthor
Updated with full annotations added for the test zip, making clear what's going on.
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
failures are #67349
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
BTW @dotnet/area-system-io-compression are we aware of any public corpus of test zip files? Clearly, there is some variation in spec interpretations. If there was such a corpus, I could imagine we write a script to unzip them all with "main" and also with 6.0 and verify the same outputs/errors. That would reduce the risk of the #1094 experience we had.
deeprobin
deeprobin
approved these changes
Apr 19, 2022
View reviewed changes
@deeprobindeeprobin
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits.
Looks overall good to me
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@deeprobin
Apply suggestions from code review
2c788e2
Co-authored-by: Robin Lindner <robin@deeprobin.de>
adamsitnik
adamsitnik
approved these changes
Apr 20, 2022
View reviewed changes
@adamsitnikadamsitnik
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I've left some comments, but all of them are just nits.
@danmoseley thanks a lot!
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

 Assert.Equal(6, entry.CompressedLength); // it should have used 32-bit size
 }

 private static readonly byte[] s_tinyZip64 =
@adamsitnikadamsitnikApr 20, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: based on the description provided for the tests above, I think that "slightlyIncorrect" would be a better name than "tiny" here
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@adamsitnik
Apply suggestions from code review
8e33f06
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@dotnet/dnceng seems an issue in dotnet-install.sh?
2022-04-20T16:20:22.1681480Z ##[section]Starting: Restore and Build Product
2022-04-20T16:20:22.1693590Z ==============================================================================
2022-04-20T16:20:22.1694320Z Task : Command line
2022-04-20T16:20:22.1694920Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-20T16:20:22.1695490Z Version : 2.201.1
2022-04-20T16:20:22.1695890Z Author : Microsoft Corporation
2022-04-20T16:20:22.1696510Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-20T16:20:22.1697200Z ==============================================================================
2022-04-20T16:20:22.5454730Z Generating script.
2022-04-20T16:20:22.5776320Z Script contents:
2022-04-20T16:20:22.5837500Z ./build.sh -subset libs+libs.tests -configuration Debug -ci -arch x64 -framework net7.0 -testscope innerloop /p:ArchiveTests=true
2022-04-20T16:20:22.5927300Z ========================== Starting Command Output ===========================
2022-04-20T16:20:22.6362330Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/7198096b-97d2-4e9e-b9f3-8d34e7d82013.sh
2022-04-20T16:20:24.1231070Z __DistroRid: osx-x64
2022-04-20T16:20:24.5786000Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-04-20T16:21:23.4178630Z Attempting to install dotnet from public_location.
2022-04-20T16:21:23.4381070Z /Users/runner/work/1/s//.dotnet/dotnet-install.sh: line 2271: syntax error: unexpected end of file

@ChadNedzlek
ChadNedzlek
commented
Apr 20, 2022

Copy link


Copy Markdown

Contributor
@danmoseley. It does look like there is a bug with it, but that tool isn't written/maintained by dnceng. If you click the "report infrastructure issue" we could try and get it assigned to the right people.
@danmoseley
danmoseley
merged commit 9a4cdce
into
dotnet:mainApr 20, 2022
@danmoseley
danmoseley
deleted the

zip64

branch
April 20, 2022 19:07
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
My bad, thanks @ChadNedzlek and @missymessa
@missymessa
missymessa
commented
Apr 20, 2022

Copy link


Copy Markdown

Member
My bad, thanks @ChadNedzlek and @missymessa
No worries, thanks for using the new features :)
directhex pushed a commit
to directhex/runtime
that referenced
this pull request
Apr 21, 2022
Dan Moseley@deeprobin
@adamsitnikJo Shields
Fix reading slightly incorrect Zip files (dotnet#68106)
55d0cb6
* Add test that fails
* fix
* Fix test failing unelevated
* annotate
* overflow
* Apply suggestions from code review
Co-authored-by: Robin Lindner <robin@deeprobin.de>
* Apply suggestions from code review
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
Co-authored-by: Robin Lindner <robin@deeprobin.de>
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
May 21, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@adamsitnikadamsitnikadamsitnik approved these changes
@carlossanlopcarlossanlopAwaiting requested review from carlossanlop

+2 more reviewers

@gfoidlgfoidlgfoidl left review comments
@deeprobindeeprobindeeprobin approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@danmoseleydanmoseley

Labels

area-System.IO.Compression

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

"A local file header is corrupt" error occurs while unpacking the ZIP archive


6 participants

@danmoseley@ChadNedzlek@missymessa@gfoidl@adamsitnik@deeprobin

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Fix reading slightly incorrect Zip files - #68106
Merged
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Apr 20, 2022
Merged
Fix reading slightly incorrect Zip files#68106
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Conversation
@danmoseley
@danmoseleydanmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

Contributor
@ghostghost
assigned danmoseleyApr 16, 2022
@ghost
ghost
commented
Apr 16, 2022

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-io-compression

See info in area-owners.md if you want to be subscribed.
Issue Details
Fix #49580
Author:danmoseley
Assignees:danmoseley
Labels:
area-System.IO.Compression
Milestone:-
@danmoseley
danmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

ContributorAuthor
Also fix new SC test that will only pass elevated. cc @deeprobin -- our PR validation runs elevated, I believe.
danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
gfoidl
gfoidl
reviewed
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
@danmoseley
danmoseley
commented
Apr 17, 2022

Copy link


Copy Markdown

ContributorAuthor
Updated with full annotations added for the test zip, making clear what's going on.
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
failures are #67349
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
BTW @dotnet/area-system-io-compression are we aware of any public corpus of test zip files? Clearly, there is some variation in spec interpretations. If there was such a corpus, I could imagine we write a script to unzip them all with "main" and also with 6.0 and verify the same outputs/errors. That would reduce the risk of the #1094 experience we had.
deeprobin
deeprobin
approved these changes
Apr 19, 2022
View reviewed changes
@deeprobindeeprobin
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits.
Looks overall good to me
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@deeprobin
Apply suggestions from code review
2c788e2
Co-authored-by: Robin Lindner <robin@deeprobin.de>
adamsitnik
adamsitnik
approved these changes
Apr 20, 2022
View reviewed changes
@adamsitnikadamsitnik
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I've left some comments, but all of them are just nits.
@danmoseley thanks a lot!
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

 Assert.Equal(6, entry.CompressedLength); // it should have used 32-bit size
 }

 private static readonly byte[] s_tinyZip64 =
@adamsitnikadamsitnikApr 20, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: based on the description provided for the tests above, I think that "slightlyIncorrect" would be a better name than "tiny" here
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@adamsitnik
Apply suggestions from code review
8e33f06
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@dotnet/dnceng seems an issue in dotnet-install.sh?
2022-04-20T16:20:22.1681480Z ##[section]Starting: Restore and Build Product
2022-04-20T16:20:22.1693590Z ==============================================================================
2022-04-20T16:20:22.1694320Z Task : Command line
2022-04-20T16:20:22.1694920Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-20T16:20:22.1695490Z Version : 2.201.1
2022-04-20T16:20:22.1695890Z Author : Microsoft Corporation
2022-04-20T16:20:22.1696510Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-20T16:20:22.1697200Z ==============================================================================
2022-04-20T16:20:22.5454730Z Generating script.
2022-04-20T16:20:22.5776320Z Script contents:
2022-04-20T16:20:22.5837500Z ./build.sh -subset libs+libs.tests -configuration Debug -ci -arch x64 -framework net7.0 -testscope innerloop /p:ArchiveTests=true
2022-04-20T16:20:22.5927300Z ========================== Starting Command Output ===========================
2022-04-20T16:20:22.6362330Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/7198096b-97d2-4e9e-b9f3-8d34e7d82013.sh
2022-04-20T16:20:24.1231070Z __DistroRid: osx-x64
2022-04-20T16:20:24.5786000Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-04-20T16:21:23.4178630Z Attempting to install dotnet from public_location.
2022-04-20T16:21:23.4381070Z /Users/runner/work/1/s//.dotnet/dotnet-install.sh: line 2271: syntax error: unexpected end of file

@ChadNedzlek
ChadNedzlek
commented
Apr 20, 2022

Copy link


Copy Markdown

Contributor
@danmoseley. It does look like there is a bug with it, but that tool isn't written/maintained by dnceng. If you click the "report infrastructure issue" we could try and get it assigned to the right people.
@danmoseley
danmoseley
merged commit 9a4cdce
into
dotnet:mainApr 20, 2022
@danmoseley
danmoseley
deleted the

zip64

branch
April 20, 2022 19:07
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
My bad, thanks @ChadNedzlek and @missymessa
@missymessa
missymessa
commented
Apr 20, 2022

Copy link


Copy Markdown

Member
My bad, thanks @ChadNedzlek and @missymessa
No worries, thanks for using the new features :)
directhex pushed a commit
to directhex/runtime
that referenced
this pull request
Apr 21, 2022
Dan Moseley@deeprobin
@adamsitnikJo Shields
Fix reading slightly incorrect Zip files (dotnet#68106)
55d0cb6
* Add test that fails
* fix
* Fix test failing unelevated
* annotate
* overflow
* Apply suggestions from code review
Co-authored-by: Robin Lindner <robin@deeprobin.de>
* Apply suggestions from code review
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
Co-authored-by: Robin Lindner <robin@deeprobin.de>
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
May 21, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@adamsitnikadamsitnikadamsitnik approved these changes
@carlossanlopcarlossanlopAwaiting requested review from carlossanlop

+2 more reviewers

@gfoidlgfoidlgfoidl left review comments
@deeprobindeeprobindeeprobin approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@danmoseleydanmoseley

Labels

area-System.IO.Compression

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

"A local file header is corrupt" error occurs while unpacking the ZIP archive


6 participants

@danmoseley@ChadNedzlek@missymessa@gfoidl@adamsitnik@deeprobin

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Fix reading slightly incorrect Zip files - #68106
Merged
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Apr 20, 2022
Merged
Fix reading slightly incorrect Zip files#68106
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Conversation
@danmoseley
@danmoseleydanmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

Contributor
@ghostghost
assigned danmoseleyApr 16, 2022
@ghost
ghost
commented
Apr 16, 2022

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-io-compression

See info in area-owners.md if you want to be subscribed.
Issue Details
Fix #49580
Author:danmoseley
Assignees:danmoseley
Labels:
area-System.IO.Compression
Milestone:-
@danmoseley
danmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

ContributorAuthor
Also fix new SC test that will only pass elevated. cc @deeprobin -- our PR validation runs elevated, I believe.
danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
gfoidl
gfoidl
reviewed
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
@danmoseley
danmoseley
commented
Apr 17, 2022

Copy link


Copy Markdown

ContributorAuthor
Updated with full annotations added for the test zip, making clear what's going on.
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
failures are #67349
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
BTW @dotnet/area-system-io-compression are we aware of any public corpus of test zip files? Clearly, there is some variation in spec interpretations. If there was such a corpus, I could imagine we write a script to unzip them all with "main" and also with 6.0 and verify the same outputs/errors. That would reduce the risk of the #1094 experience we had.
deeprobin
deeprobin
approved these changes
Apr 19, 2022
View reviewed changes
@deeprobindeeprobin
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits.
Looks overall good to me
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@deeprobin
Apply suggestions from code review
2c788e2
Co-authored-by: Robin Lindner <robin@deeprobin.de>
adamsitnik
adamsitnik
approved these changes
Apr 20, 2022
View reviewed changes
@adamsitnikadamsitnik
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I've left some comments, but all of them are just nits.
@danmoseley thanks a lot!
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

 Assert.Equal(6, entry.CompressedLength); // it should have used 32-bit size
 }

 private static readonly byte[] s_tinyZip64 =
@adamsitnikadamsitnikApr 20, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: based on the description provided for the tests above, I think that "slightlyIncorrect" would be a better name than "tiny" here
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@adamsitnik
Apply suggestions from code review
8e33f06
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@dotnet/dnceng seems an issue in dotnet-install.sh?
2022-04-20T16:20:22.1681480Z ##[section]Starting: Restore and Build Product
2022-04-20T16:20:22.1693590Z ==============================================================================
2022-04-20T16:20:22.1694320Z Task : Command line
2022-04-20T16:20:22.1694920Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-20T16:20:22.1695490Z Version : 2.201.1
2022-04-20T16:20:22.1695890Z Author : Microsoft Corporation
2022-04-20T16:20:22.1696510Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-20T16:20:22.1697200Z ==============================================================================
2022-04-20T16:20:22.5454730Z Generating script.
2022-04-20T16:20:22.5776320Z Script contents:
2022-04-20T16:20:22.5837500Z ./build.sh -subset libs+libs.tests -configuration Debug -ci -arch x64 -framework net7.0 -testscope innerloop /p:ArchiveTests=true
2022-04-20T16:20:22.5927300Z ========================== Starting Command Output ===========================
2022-04-20T16:20:22.6362330Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/7198096b-97d2-4e9e-b9f3-8d34e7d82013.sh
2022-04-20T16:20:24.1231070Z __DistroRid: osx-x64
2022-04-20T16:20:24.5786000Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-04-20T16:21:23.4178630Z Attempting to install dotnet from public_location.
2022-04-20T16:21:23.4381070Z /Users/runner/work/1/s//.dotnet/dotnet-install.sh: line 2271: syntax error: unexpected end of file

@ChadNedzlek
ChadNedzlek
commented
Apr 20, 2022

Copy link


Copy Markdown

Contributor
@danmoseley. It does look like there is a bug with it, but that tool isn't written/maintained by dnceng. If you click the "report infrastructure issue" we could try and get it assigned to the right people.
@danmoseley
danmoseley
merged commit 9a4cdce
into
dotnet:mainApr 20, 2022
@danmoseley
danmoseley
deleted the

zip64

branch
April 20, 2022 19:07
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
My bad, thanks @ChadNedzlek and @missymessa
@missymessa
missymessa
commented
Apr 20, 2022

Copy link


Copy Markdown

Member
My bad, thanks @ChadNedzlek and @missymessa
No worries, thanks for using the new features :)
directhex pushed a commit
to directhex/runtime
that referenced
this pull request
Apr 21, 2022
Dan Moseley@deeprobin
@adamsitnikJo Shields
Fix reading slightly incorrect Zip files (dotnet#68106)
55d0cb6
* Add test that fails
* fix
* Fix test failing unelevated
* annotate
* overflow
* Apply suggestions from code review
Co-authored-by: Robin Lindner <robin@deeprobin.de>
* Apply suggestions from code review
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
Co-authored-by: Robin Lindner <robin@deeprobin.de>
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
May 21, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@adamsitnikadamsitnikadamsitnik approved these changes
@carlossanlopcarlossanlopAwaiting requested review from carlossanlop

+2 more reviewers

@gfoidlgfoidlgfoidl left review comments
@deeprobindeeprobindeeprobin approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@danmoseleydanmoseley

Labels

area-System.IO.Compression

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

"A local file header is corrupt" error occurs while unpacking the ZIP archive


6 participants

@danmoseley@ChadNedzlek@missymessa@gfoidl@adamsitnik@deeprobin

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Fix reading slightly incorrect Zip files - #68106
Merged
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Apr 20, 2022
Merged
Fix reading slightly incorrect Zip files#68106
danmoseley merged 7 commits into
dotnet:mainfrom 
danmoseley:zip64
Conversation
@danmoseley
@danmoseleydanmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

Contributor
@ghostghost
assigned danmoseleyApr 16, 2022
@ghost
ghost
commented
Apr 16, 2022

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-io-compression

See info in area-owners.md if you want to be subscribed.
Issue Details
Fix #49580
Author:danmoseley
Assignees:danmoseley
Labels:
area-System.IO.Compression
Milestone:-
@danmoseley
danmoseley
commented
Apr 16, 2022

Copy link


Copy Markdown

ContributorAuthor
Also fix new SC test that will only pass elevated. cc @deeprobin -- our PR validation runs elevated, I believe.
danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
gfoidl
gfoidl
reviewed
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 16, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
@danmoseley
danmoseley
commented
Apr 17, 2022

Copy link


Copy Markdown

ContributorAuthor
Updated with full annotations added for the test zip, making clear what's going on.
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
failures are #67349
@danmoseley
danmoseley
commented
Apr 18, 2022

Copy link


Copy Markdown

ContributorAuthor
BTW @dotnet/area-system-io-compression are we aware of any public corpus of test zip files? Clearly, there is some variation in spec interpretations. If there was such a corpus, I could imagine we write a script to unzip them all with "main" and also with 6.0 and verify the same outputs/errors. That would reduce the risk of the #1094 experience we had.
deeprobin
deeprobin
approved these changes
Apr 19, 2022
View reviewed changes
@deeprobindeeprobin
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits.
Looks overall good to me
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@deeprobin
Apply suggestions from code review
2c788e2
Co-authored-by: Robin Lindner <robin@deeprobin.de>
adamsitnik
adamsitnik
approved these changes
Apr 20, 2022
View reviewed changes
@adamsitnikadamsitnik
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I've left some comments, but all of them are just nits.
@danmoseley thanks a lot!
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
Outdated

 Assert.Equal(6, entry.CompressedLength); // it should have used 32-bit size
 }

 private static readonly byte[] s_tinyZip64 =
@adamsitnikadamsitnikApr 20, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: based on the description provided for the tests above, I think that "slightlyIncorrect" would be a better name than "tiny" here
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Comment threadsrc/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs
danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

danmoseley
danmoseley
commented
Apr 20, 2022
View reviewed changes
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs
Outdated

Dan Moseley@adamsitnik
Apply suggestions from code review
8e33f06
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@dotnet/dnceng seems an issue in dotnet-install.sh?
2022-04-20T16:20:22.1681480Z ##[section]Starting: Restore and Build Product
2022-04-20T16:20:22.1693590Z ==============================================================================
2022-04-20T16:20:22.1694320Z Task : Command line
2022-04-20T16:20:22.1694920Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-20T16:20:22.1695490Z Version : 2.201.1
2022-04-20T16:20:22.1695890Z Author : Microsoft Corporation
2022-04-20T16:20:22.1696510Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-20T16:20:22.1697200Z ==============================================================================
2022-04-20T16:20:22.5454730Z Generating script.
2022-04-20T16:20:22.5776320Z Script contents:
2022-04-20T16:20:22.5837500Z ./build.sh -subset libs+libs.tests -configuration Debug -ci -arch x64 -framework net7.0 -testscope innerloop /p:ArchiveTests=true
2022-04-20T16:20:22.5927300Z ========================== Starting Command Output ===========================
2022-04-20T16:20:22.6362330Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/7198096b-97d2-4e9e-b9f3-8d34e7d82013.sh
2022-04-20T16:20:24.1231070Z __DistroRid: osx-x64
2022-04-20T16:20:24.5786000Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-04-20T16:21:23.4178630Z Attempting to install dotnet from public_location.
2022-04-20T16:21:23.4381070Z /Users/runner/work/1/s//.dotnet/dotnet-install.sh: line 2271: syntax error: unexpected end of file

@ChadNedzlek
ChadNedzlek
commented
Apr 20, 2022

Copy link


Copy Markdown

Contributor
@danmoseley. It does look like there is a bug with it, but that tool isn't written/maintained by dnceng. If you click the "report infrastructure issue" we could try and get it assigned to the right people.
@danmoseley
danmoseley
merged commit 9a4cdce
into
dotnet:mainApr 20, 2022
@danmoseley
danmoseley
deleted the

zip64

branch
April 20, 2022 19:07
@danmoseley
danmoseley
commented
Apr 20, 2022

Copy link


Copy Markdown

ContributorAuthor
My bad, thanks @ChadNedzlek and @missymessa
@missymessa
missymessa
commented
Apr 20, 2022

Copy link


Copy Markdown

Member
My bad, thanks @ChadNedzlek and @missymessa
No worries, thanks for using the new features :)
directhex pushed a commit
to directhex/runtime
that referenced
this pull request
Apr 21, 2022
Dan Moseley@deeprobin
@adamsitnikJo Shields
Fix reading slightly incorrect Zip files (dotnet#68106)
55d0cb6
* Add test that fails
* fix
* Fix test failing unelevated
* annotate
* overflow
* Apply suggestions from code review
Co-authored-by: Robin Lindner <robin@deeprobin.de>
* Apply suggestions from code review
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
Co-authored-by: Robin Lindner <robin@deeprobin.de>
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
May 21, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@adamsitnikadamsitnikadamsitnik approved these changes
@carlossanlopcarlossanlopAwaiting requested review from carlossanlop

+2 more reviewers

@gfoidlgfoidlgfoidl left review comments
@deeprobindeeprobindeeprobin approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@danmoseleydanmoseley

Labels

area-System.IO.Compression

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

"A local file header is corrupt" error occurs while unpacking the ZIP archive


6 participants

@danmoseley@ChadNedzlek@missymessa@gfoidl@adamsitnik@deeprobin