Feature/zip archive forward read - #126646

Draft
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read
Draft

Feature/zip archive forward read#126646
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read

Conversation

@alinpahontu2912

Copy link
Copy Markdown
Member

Fixes#1550

Experiment with new Forward only reading mode for ziparchvie, that relies on the local headers of entries instead of the central directory, avoiding loading everything into memory at once

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an experimental forward-only streaming read mode for System.IO.Compression.ZipArchive to support sequential iteration over ZIP entries via local file headers (instead of loading the central directory up-front), including async iteration and new unit tests.

Changes:

  • Add ZipArchiveMode.ForwardRead plus public ZipArchive.GetNextEntry() / GetNextEntryAsync(...) APIs for forward-only entry iteration.
  • Implement forward-read parsing and stream wrappers (ReadAheadStream, BoundedReadOnlyStream) to enable non-seekable streaming scenarios.
  • Add new unit tests validating basic ForwardRead behavior and unsupported operations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
FileDescription
src/libraries/System.IO.Compression/tests/ZipArchive/zip_ForwardReadTests.csAdds new test coverage for ForwardRead iteration, sync/async behavior, and unsupported APIs.
src/libraries/System.IO.Compression/tests/System.IO.Compression.Tests.csprojIncludes the new ForwardRead test file in the test project.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipCustomStreams.csAdds BoundedReadOnlyStream and ReadAheadStream used by ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveMode.csIntroduces ZipArchiveMode.ForwardRead enum value with documentation.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.csAdds ForwardRead entry initialization and Open() support for ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.csImplements ForwardRead iteration, local header parsing, data-descriptor handling, and non-seekable wrapping.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.Async.csEnables ForwardRead setup for async factory and disposal paths.
src/libraries/System.IO.Compression/src/Resources/Strings.resxAdds new SR strings for ForwardRead error messages.
src/libraries/System.IO.Compression/ref/System.IO.Compression.csUpdates public surface area (new mode + new ZipArchive methods).

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +695 to +698
// Known size, not encrypted
Stream bounded = new BoundedReadOnlyStream(_archiveStream, compressedSize);
Stream decompressor = CreateForwardReadDecompressor(bounded, compressionMethod, uncompressedSize, leaveOpen: false);
dataStream = new CrcValidatingReadStream(decompressor, crc32, uncompressedSize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should postpone DataStream creation and do it lazily like we do for the other entries.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can defer that for the on data descriptor entries

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:475

  • The XML documentation for Open() (and Open(FileAccess)) was not updated to describe ZipArchiveMode.ForwardRead. The summary still says "If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, ... If Update mode, ..." with no mention of ForwardRead, even though this PR introduces a new code path in the switch. Similarly, the <list> in Open(FileAccess) doesn't include the ForwardRead bullet, and the exception list omits the new NotSupportedException/InvalidOperationException cases (encrypted, no data stream, stream already opened). Please extend the doc comments to cover the new mode.
 /// <summary>
/// Opens the entry. If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, the returned stream will be writable and not seekable. If Update mode, the returned stream will be readable, writable, seekable, and support SetLength.
/// </summary>
/// <returns>A Stream that represents the contents of the entry.</returns>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>
public Stream Open()
{
ThrowIfInvalidArchive();
switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}
}
/// <summary>
/// Opens the entry with the specified access mode. This allows for more granular control over the returned stream's capabilities.
/// </summary>
/// <param name="access">The file access mode for the returned stream.</param>
/// <returns>A <see cref="Stream"/> that represents the contents of the entry with the specified access capabilities.</returns>
/// <remarks>
/// <para>The allowed <paramref name="access"/> values depend on the <see cref="ZipArchiveMode"/>:</para>
/// <list type="bullet">
/// <item><description><see cref="ZipArchiveMode.Read"/>: Only <see cref="FileAccess.Read"/> is allowed.</description></item>
/// <item><description><see cref="ZipArchiveMode.Create"/>: <see cref="FileAccess.Write"/> and <see cref="FileAccess.ReadWrite"/> are allowed (both write-only).</description></item>
/// <item><description><see cref="ZipArchiveMode.Update"/>: All values are allowed. <see cref="FileAccess.Read"/> reads directly from the archive. <see cref="FileAccess.Write"/> discards existing content and provides an empty writable stream. <see cref="FileAccess.ReadWrite"/> loads existing content into memory (equivalent to <see cref="Open()"/>).</description></item>
/// </list>
/// </remarks>
/// <exception cref="ArgumentOutOfRangeException"><paramref name="access"/> is not a valid <see cref="FileAccess"/> value.</exception>
/// <exception cref="InvalidOperationException">The requested access is not compatible with the archive's open mode.</exception>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +22 to +112
// ── Core reading scenarios ──────────────────────────────────────────

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task NonSeekableStream_ConsumeSkipConsume_ReadsCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
using WrappedStream nonSeekable = new(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(nonSeekable, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);
using (Stream ds = first.Open())
{
Assert.Equal(expected[0], await ReadStreamFully(ds, async));
}

// Skip second entry without opening
ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

ZipArchiveEntry? third = await GetNextEntry(archive, async);
Assert.NotNull(third);
using (Stream ds = third.Open())
{
Assert.Equal(expected[2], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[InlineData(true, true)]
[InlineData(true, false)]
[InlineData(false, true)]
[InlineData(false, false)]
public async Task StoredEntries_SeekableAndNonSeekable_ReadCorrectly(bool async, bool readSeekable)
{
// Always created on seekable stream → known sizes, no data descriptors
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.NoCompression, seekable: true);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
Stream readStream = readSeekable
? archiveStream
: new WrappedStream(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(readStream, ZipArchiveMode.ForwardRead);

for (int i = 0; i < expected.Length; i++)
{
ZipArchiveEntry? entry = await GetNextEntry(archive, async);
Assert.NotNull(entry);
Assert.Equal(ZipCompressionMethod.Stored, entry.CompressionMethod);

using Stream ds = entry.Open();
Assert.Equal(expected[i], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task PartialRead_ThenAdvance_ReadsNextEntryCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);

using MemoryStream archiveStream = new(zipBytes);
using ZipArchive archive = new(archiveStream, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);

using (Stream ds = first.Open())
{
byte[] partial = new byte[3];
await ReadStream(ds, partial, async);
}

ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

using Stream ds2 = second.Open();
Assert.Equal(s_mediumContent, await ReadStreamFully(ds2, async));
}

alinpahontu2912and others added 9 commits June 26, 2026 14:09
Adds a new ForwardRead mode to ZipArchive that enables forward-only
sequential reading of ZIP entries from non-seekable streams, using
local file headers instead of the central directory.
Changes:
- ZipArchiveMode.cs: Add ForwardRead = 3 enum value
- ZipCustomStreams.cs: Add BoundedReadOnlyStream and ReadAheadStream
helper stream classes
- ZipArchive.cs: Add GetNextEntry()/GetNextEntryAsync() methods,
ForwardRead constructor case, ValidateMode/DecideArchiveStream
support, data descriptor parsing, and property guards
- ZipArchive.Async.cs: Add ForwardRead cases to CreateAsync and
DisposeAsyncCore
- ZipArchiveEntry.cs: Add forward-read constructor, ForwardReadDataStream
property, UpdateFromDataDescriptor method, OpenInForwardReadMode,
and property setter guards
- Strings.resx: Add ForwardRead error message strings
- ref/System.IO.Compression.cs: Add public API surface
- Tests: Add comprehensive zip_ForwardReadTests covering deflate,
stored, data descriptors, non-seekable streams, empty archives,
partial reads, error cases, and async operations
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alinpahontu2912
alinpahontu2912force-pushed the feature/zip-archive-forward-read branch from 2ae6843 to 6bb976aCompareJune 26, 2026 11:09
CopilotAI review requested due to automatic review settings June 26, 2026 11:09

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:456

  • ForwardRead mode is handled in the synchronous Open()/Open(FileAccess) path, but ZipArchiveEntry.OpenAsync(...) and OpenAsync(FileAccess, ...) (in ZipArchiveEntry.Async.cs) do not have a ForwardRead case and will fall into the Update/default path in release builds. That will produce incorrect behavior (and could attempt Update-mode semantics on a ForwardRead archive). ForwardRead should be explicitly handled in the async Open* methods as well.

[!NOTE] This comment was created by GitHub Copilot.

 switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}

Comment on lines 152 to +154
public System.IO.Compression.ZipArchiveEntry? GetEntry(string entryName) { throw null; }
public System.IO.Compression.ZipArchiveEntry? GetNextEntry() { throw null; }
public System.Threading.Tasks.ValueTask<System.IO.Compression.ZipArchiveEntry?> GetNextEntryAsync(System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
Comment on lines +834 to +848
// Read enough for the 32-bit layout: CRC(4) + CompSize(4) + UncompSize(4),
// plus CRC(4) again if the signature consumed firstWord.
int smallSize = (hasSignature ? 4 : 0) + 8;
Span<byte> buf = stackalloc byte[20];
stream.ReadExactly(buf[..smallSize]);

var small = ParseDataDescriptor(buf[..smallSize], hasSignature, isZip64: false, firstWord);
if (small.Crc32 == knownCrc32 && small.UncompressedSize == knownUncompressedSize)
return small;

// 32-bit interpretation didn't match — read 8 more bytes for the Zip64 layout.
stream.ReadExactly(buf.Slice(smallSize, 8));
int fullSize = smallSize + 8;

return ParseDataDescriptor(buf[..fullSize], hasSignature, isZip64: true, firstWord);

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task TruncatedStoredEntry_ThrowsOnRead(bool async)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Forward-only API for System.IO.Compression

3 participants

@alinpahontu2912@rzikm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Feature/zip archive forward read - #126646

Draft
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read
Draft

Feature/zip archive forward read#126646
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read

Conversation

@alinpahontu2912

Copy link
Copy Markdown
Member

Fixes#1550

Experiment with new Forward only reading mode for ziparchvie, that relies on the local headers of entries instead of the central directory, avoiding loading everything into memory at once

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an experimental forward-only streaming read mode for System.IO.Compression.ZipArchive to support sequential iteration over ZIP entries via local file headers (instead of loading the central directory up-front), including async iteration and new unit tests.

Changes:

  • Add ZipArchiveMode.ForwardRead plus public ZipArchive.GetNextEntry() / GetNextEntryAsync(...) APIs for forward-only entry iteration.
  • Implement forward-read parsing and stream wrappers (ReadAheadStream, BoundedReadOnlyStream) to enable non-seekable streaming scenarios.
  • Add new unit tests validating basic ForwardRead behavior and unsupported operations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
FileDescription
src/libraries/System.IO.Compression/tests/ZipArchive/zip_ForwardReadTests.csAdds new test coverage for ForwardRead iteration, sync/async behavior, and unsupported APIs.
src/libraries/System.IO.Compression/tests/System.IO.Compression.Tests.csprojIncludes the new ForwardRead test file in the test project.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipCustomStreams.csAdds BoundedReadOnlyStream and ReadAheadStream used by ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveMode.csIntroduces ZipArchiveMode.ForwardRead enum value with documentation.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.csAdds ForwardRead entry initialization and Open() support for ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.csImplements ForwardRead iteration, local header parsing, data-descriptor handling, and non-seekable wrapping.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.Async.csEnables ForwardRead setup for async factory and disposal paths.
src/libraries/System.IO.Compression/src/Resources/Strings.resxAdds new SR strings for ForwardRead error messages.
src/libraries/System.IO.Compression/ref/System.IO.Compression.csUpdates public surface area (new mode + new ZipArchive methods).

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +695 to +698
// Known size, not encrypted
Stream bounded = new BoundedReadOnlyStream(_archiveStream, compressedSize);
Stream decompressor = CreateForwardReadDecompressor(bounded, compressionMethod, uncompressedSize, leaveOpen: false);
dataStream = new CrcValidatingReadStream(decompressor, crc32, uncompressedSize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should postpone DataStream creation and do it lazily like we do for the other entries.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can defer that for the on data descriptor entries

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:475

  • The XML documentation for Open() (and Open(FileAccess)) was not updated to describe ZipArchiveMode.ForwardRead. The summary still says "If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, ... If Update mode, ..." with no mention of ForwardRead, even though this PR introduces a new code path in the switch. Similarly, the <list> in Open(FileAccess) doesn't include the ForwardRead bullet, and the exception list omits the new NotSupportedException/InvalidOperationException cases (encrypted, no data stream, stream already opened). Please extend the doc comments to cover the new mode.
 /// <summary>
/// Opens the entry. If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, the returned stream will be writable and not seekable. If Update mode, the returned stream will be readable, writable, seekable, and support SetLength.
/// </summary>
/// <returns>A Stream that represents the contents of the entry.</returns>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>
public Stream Open()
{
ThrowIfInvalidArchive();
switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}
}
/// <summary>
/// Opens the entry with the specified access mode. This allows for more granular control over the returned stream's capabilities.
/// </summary>
/// <param name="access">The file access mode for the returned stream.</param>
/// <returns>A <see cref="Stream"/> that represents the contents of the entry with the specified access capabilities.</returns>
/// <remarks>
/// <para>The allowed <paramref name="access"/> values depend on the <see cref="ZipArchiveMode"/>:</para>
/// <list type="bullet">
/// <item><description><see cref="ZipArchiveMode.Read"/>: Only <see cref="FileAccess.Read"/> is allowed.</description></item>
/// <item><description><see cref="ZipArchiveMode.Create"/>: <see cref="FileAccess.Write"/> and <see cref="FileAccess.ReadWrite"/> are allowed (both write-only).</description></item>
/// <item><description><see cref="ZipArchiveMode.Update"/>: All values are allowed. <see cref="FileAccess.Read"/> reads directly from the archive. <see cref="FileAccess.Write"/> discards existing content and provides an empty writable stream. <see cref="FileAccess.ReadWrite"/> loads existing content into memory (equivalent to <see cref="Open()"/>).</description></item>
/// </list>
/// </remarks>
/// <exception cref="ArgumentOutOfRangeException"><paramref name="access"/> is not a valid <see cref="FileAccess"/> value.</exception>
/// <exception cref="InvalidOperationException">The requested access is not compatible with the archive's open mode.</exception>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +22 to +112
// ── Core reading scenarios ──────────────────────────────────────────

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task NonSeekableStream_ConsumeSkipConsume_ReadsCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
using WrappedStream nonSeekable = new(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(nonSeekable, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);
using (Stream ds = first.Open())
{
Assert.Equal(expected[0], await ReadStreamFully(ds, async));
}

// Skip second entry without opening
ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

ZipArchiveEntry? third = await GetNextEntry(archive, async);
Assert.NotNull(third);
using (Stream ds = third.Open())
{
Assert.Equal(expected[2], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[InlineData(true, true)]
[InlineData(true, false)]
[InlineData(false, true)]
[InlineData(false, false)]
public async Task StoredEntries_SeekableAndNonSeekable_ReadCorrectly(bool async, bool readSeekable)
{
// Always created on seekable stream → known sizes, no data descriptors
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.NoCompression, seekable: true);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
Stream readStream = readSeekable
? archiveStream
: new WrappedStream(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(readStream, ZipArchiveMode.ForwardRead);

for (int i = 0; i < expected.Length; i++)
{
ZipArchiveEntry? entry = await GetNextEntry(archive, async);
Assert.NotNull(entry);
Assert.Equal(ZipCompressionMethod.Stored, entry.CompressionMethod);

using Stream ds = entry.Open();
Assert.Equal(expected[i], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task PartialRead_ThenAdvance_ReadsNextEntryCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);

using MemoryStream archiveStream = new(zipBytes);
using ZipArchive archive = new(archiveStream, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);

using (Stream ds = first.Open())
{
byte[] partial = new byte[3];
await ReadStream(ds, partial, async);
}

ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

using Stream ds2 = second.Open();
Assert.Equal(s_mediumContent, await ReadStreamFully(ds2, async));
}

alinpahontu2912and others added 9 commits June 26, 2026 14:09
Adds a new ForwardRead mode to ZipArchive that enables forward-only
sequential reading of ZIP entries from non-seekable streams, using
local file headers instead of the central directory.
Changes:
- ZipArchiveMode.cs: Add ForwardRead = 3 enum value
- ZipCustomStreams.cs: Add BoundedReadOnlyStream and ReadAheadStream
helper stream classes
- ZipArchive.cs: Add GetNextEntry()/GetNextEntryAsync() methods,
ForwardRead constructor case, ValidateMode/DecideArchiveStream
support, data descriptor parsing, and property guards
- ZipArchive.Async.cs: Add ForwardRead cases to CreateAsync and
DisposeAsyncCore
- ZipArchiveEntry.cs: Add forward-read constructor, ForwardReadDataStream
property, UpdateFromDataDescriptor method, OpenInForwardReadMode,
and property setter guards
- Strings.resx: Add ForwardRead error message strings
- ref/System.IO.Compression.cs: Add public API surface
- Tests: Add comprehensive zip_ForwardReadTests covering deflate,
stored, data descriptors, non-seekable streams, empty archives,
partial reads, error cases, and async operations
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alinpahontu2912
alinpahontu2912force-pushed the feature/zip-archive-forward-read branch from 2ae6843 to 6bb976aCompareJune 26, 2026 11:09
CopilotAI review requested due to automatic review settings June 26, 2026 11:09

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:456

  • ForwardRead mode is handled in the synchronous Open()/Open(FileAccess) path, but ZipArchiveEntry.OpenAsync(...) and OpenAsync(FileAccess, ...) (in ZipArchiveEntry.Async.cs) do not have a ForwardRead case and will fall into the Update/default path in release builds. That will produce incorrect behavior (and could attempt Update-mode semantics on a ForwardRead archive). ForwardRead should be explicitly handled in the async Open* methods as well.

[!NOTE] This comment was created by GitHub Copilot.

 switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}

Comment on lines 152 to +154
public System.IO.Compression.ZipArchiveEntry? GetEntry(string entryName) { throw null; }
public System.IO.Compression.ZipArchiveEntry? GetNextEntry() { throw null; }
public System.Threading.Tasks.ValueTask<System.IO.Compression.ZipArchiveEntry?> GetNextEntryAsync(System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
Comment on lines +834 to +848
// Read enough for the 32-bit layout: CRC(4) + CompSize(4) + UncompSize(4),
// plus CRC(4) again if the signature consumed firstWord.
int smallSize = (hasSignature ? 4 : 0) + 8;
Span<byte> buf = stackalloc byte[20];
stream.ReadExactly(buf[..smallSize]);

var small = ParseDataDescriptor(buf[..smallSize], hasSignature, isZip64: false, firstWord);
if (small.Crc32 == knownCrc32 && small.UncompressedSize == knownUncompressedSize)
return small;

// 32-bit interpretation didn't match — read 8 more bytes for the Zip64 layout.
stream.ReadExactly(buf.Slice(smallSize, 8));
int fullSize = smallSize + 8;

return ParseDataDescriptor(buf[..fullSize], hasSignature, isZip64: true, firstWord);

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task TruncatedStoredEntry_ThrowsOnRead(bool async)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Forward-only API for System.IO.Compression

3 participants

@alinpahontu2912@rzikm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Feature/zip archive forward read - #126646

Draft
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read
Draft

Feature/zip archive forward read#126646
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read

Conversation

@alinpahontu2912

Copy link
Copy Markdown
Member

Fixes#1550

Experiment with new Forward only reading mode for ziparchvie, that relies on the local headers of entries instead of the central directory, avoiding loading everything into memory at once

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an experimental forward-only streaming read mode for System.IO.Compression.ZipArchive to support sequential iteration over ZIP entries via local file headers (instead of loading the central directory up-front), including async iteration and new unit tests.

Changes:

  • Add ZipArchiveMode.ForwardRead plus public ZipArchive.GetNextEntry() / GetNextEntryAsync(...) APIs for forward-only entry iteration.
  • Implement forward-read parsing and stream wrappers (ReadAheadStream, BoundedReadOnlyStream) to enable non-seekable streaming scenarios.
  • Add new unit tests validating basic ForwardRead behavior and unsupported operations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
FileDescription
src/libraries/System.IO.Compression/tests/ZipArchive/zip_ForwardReadTests.csAdds new test coverage for ForwardRead iteration, sync/async behavior, and unsupported APIs.
src/libraries/System.IO.Compression/tests/System.IO.Compression.Tests.csprojIncludes the new ForwardRead test file in the test project.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipCustomStreams.csAdds BoundedReadOnlyStream and ReadAheadStream used by ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveMode.csIntroduces ZipArchiveMode.ForwardRead enum value with documentation.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.csAdds ForwardRead entry initialization and Open() support for ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.csImplements ForwardRead iteration, local header parsing, data-descriptor handling, and non-seekable wrapping.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.Async.csEnables ForwardRead setup for async factory and disposal paths.
src/libraries/System.IO.Compression/src/Resources/Strings.resxAdds new SR strings for ForwardRead error messages.
src/libraries/System.IO.Compression/ref/System.IO.Compression.csUpdates public surface area (new mode + new ZipArchive methods).

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +695 to +698
// Known size, not encrypted
Stream bounded = new BoundedReadOnlyStream(_archiveStream, compressedSize);
Stream decompressor = CreateForwardReadDecompressor(bounded, compressionMethod, uncompressedSize, leaveOpen: false);
dataStream = new CrcValidatingReadStream(decompressor, crc32, uncompressedSize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should postpone DataStream creation and do it lazily like we do for the other entries.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can defer that for the on data descriptor entries

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:475

  • The XML documentation for Open() (and Open(FileAccess)) was not updated to describe ZipArchiveMode.ForwardRead. The summary still says "If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, ... If Update mode, ..." with no mention of ForwardRead, even though this PR introduces a new code path in the switch. Similarly, the <list> in Open(FileAccess) doesn't include the ForwardRead bullet, and the exception list omits the new NotSupportedException/InvalidOperationException cases (encrypted, no data stream, stream already opened). Please extend the doc comments to cover the new mode.
 /// <summary>
/// Opens the entry. If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, the returned stream will be writable and not seekable. If Update mode, the returned stream will be readable, writable, seekable, and support SetLength.
/// </summary>
/// <returns>A Stream that represents the contents of the entry.</returns>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>
public Stream Open()
{
ThrowIfInvalidArchive();
switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}
}
/// <summary>
/// Opens the entry with the specified access mode. This allows for more granular control over the returned stream's capabilities.
/// </summary>
/// <param name="access">The file access mode for the returned stream.</param>
/// <returns>A <see cref="Stream"/> that represents the contents of the entry with the specified access capabilities.</returns>
/// <remarks>
/// <para>The allowed <paramref name="access"/> values depend on the <see cref="ZipArchiveMode"/>:</para>
/// <list type="bullet">
/// <item><description><see cref="ZipArchiveMode.Read"/>: Only <see cref="FileAccess.Read"/> is allowed.</description></item>
/// <item><description><see cref="ZipArchiveMode.Create"/>: <see cref="FileAccess.Write"/> and <see cref="FileAccess.ReadWrite"/> are allowed (both write-only).</description></item>
/// <item><description><see cref="ZipArchiveMode.Update"/>: All values are allowed. <see cref="FileAccess.Read"/> reads directly from the archive. <see cref="FileAccess.Write"/> discards existing content and provides an empty writable stream. <see cref="FileAccess.ReadWrite"/> loads existing content into memory (equivalent to <see cref="Open()"/>).</description></item>
/// </list>
/// </remarks>
/// <exception cref="ArgumentOutOfRangeException"><paramref name="access"/> is not a valid <see cref="FileAccess"/> value.</exception>
/// <exception cref="InvalidOperationException">The requested access is not compatible with the archive's open mode.</exception>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +22 to +112
// ── Core reading scenarios ──────────────────────────────────────────

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task NonSeekableStream_ConsumeSkipConsume_ReadsCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
using WrappedStream nonSeekable = new(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(nonSeekable, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);
using (Stream ds = first.Open())
{
Assert.Equal(expected[0], await ReadStreamFully(ds, async));
}

// Skip second entry without opening
ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

ZipArchiveEntry? third = await GetNextEntry(archive, async);
Assert.NotNull(third);
using (Stream ds = third.Open())
{
Assert.Equal(expected[2], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[InlineData(true, true)]
[InlineData(true, false)]
[InlineData(false, true)]
[InlineData(false, false)]
public async Task StoredEntries_SeekableAndNonSeekable_ReadCorrectly(bool async, bool readSeekable)
{
// Always created on seekable stream → known sizes, no data descriptors
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.NoCompression, seekable: true);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
Stream readStream = readSeekable
? archiveStream
: new WrappedStream(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(readStream, ZipArchiveMode.ForwardRead);

for (int i = 0; i < expected.Length; i++)
{
ZipArchiveEntry? entry = await GetNextEntry(archive, async);
Assert.NotNull(entry);
Assert.Equal(ZipCompressionMethod.Stored, entry.CompressionMethod);

using Stream ds = entry.Open();
Assert.Equal(expected[i], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task PartialRead_ThenAdvance_ReadsNextEntryCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);

using MemoryStream archiveStream = new(zipBytes);
using ZipArchive archive = new(archiveStream, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);

using (Stream ds = first.Open())
{
byte[] partial = new byte[3];
await ReadStream(ds, partial, async);
}

ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

using Stream ds2 = second.Open();
Assert.Equal(s_mediumContent, await ReadStreamFully(ds2, async));
}

alinpahontu2912and others added 9 commits June 26, 2026 14:09
Adds a new ForwardRead mode to ZipArchive that enables forward-only
sequential reading of ZIP entries from non-seekable streams, using
local file headers instead of the central directory.
Changes:
- ZipArchiveMode.cs: Add ForwardRead = 3 enum value
- ZipCustomStreams.cs: Add BoundedReadOnlyStream and ReadAheadStream
helper stream classes
- ZipArchive.cs: Add GetNextEntry()/GetNextEntryAsync() methods,
ForwardRead constructor case, ValidateMode/DecideArchiveStream
support, data descriptor parsing, and property guards
- ZipArchive.Async.cs: Add ForwardRead cases to CreateAsync and
DisposeAsyncCore
- ZipArchiveEntry.cs: Add forward-read constructor, ForwardReadDataStream
property, UpdateFromDataDescriptor method, OpenInForwardReadMode,
and property setter guards
- Strings.resx: Add ForwardRead error message strings
- ref/System.IO.Compression.cs: Add public API surface
- Tests: Add comprehensive zip_ForwardReadTests covering deflate,
stored, data descriptors, non-seekable streams, empty archives,
partial reads, error cases, and async operations
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alinpahontu2912
alinpahontu2912force-pushed the feature/zip-archive-forward-read branch from 2ae6843 to 6bb976aCompareJune 26, 2026 11:09
CopilotAI review requested due to automatic review settings June 26, 2026 11:09

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:456

  • ForwardRead mode is handled in the synchronous Open()/Open(FileAccess) path, but ZipArchiveEntry.OpenAsync(...) and OpenAsync(FileAccess, ...) (in ZipArchiveEntry.Async.cs) do not have a ForwardRead case and will fall into the Update/default path in release builds. That will produce incorrect behavior (and could attempt Update-mode semantics on a ForwardRead archive). ForwardRead should be explicitly handled in the async Open* methods as well.

[!NOTE] This comment was created by GitHub Copilot.

 switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}

Comment on lines 152 to +154
public System.IO.Compression.ZipArchiveEntry? GetEntry(string entryName) { throw null; }
public System.IO.Compression.ZipArchiveEntry? GetNextEntry() { throw null; }
public System.Threading.Tasks.ValueTask<System.IO.Compression.ZipArchiveEntry?> GetNextEntryAsync(System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
Comment on lines +834 to +848
// Read enough for the 32-bit layout: CRC(4) + CompSize(4) + UncompSize(4),
// plus CRC(4) again if the signature consumed firstWord.
int smallSize = (hasSignature ? 4 : 0) + 8;
Span<byte> buf = stackalloc byte[20];
stream.ReadExactly(buf[..smallSize]);

var small = ParseDataDescriptor(buf[..smallSize], hasSignature, isZip64: false, firstWord);
if (small.Crc32 == knownCrc32 && small.UncompressedSize == knownUncompressedSize)
return small;

// 32-bit interpretation didn't match — read 8 more bytes for the Zip64 layout.
stream.ReadExactly(buf.Slice(smallSize, 8));
int fullSize = smallSize + 8;

return ParseDataDescriptor(buf[..fullSize], hasSignature, isZip64: true, firstWord);

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task TruncatedStoredEntry_ThrowsOnRead(bool async)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Forward-only API for System.IO.Compression

3 participants

@alinpahontu2912@rzikm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Feature/zip archive forward read - #126646

Draft
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read
Draft

Feature/zip archive forward read#126646
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read

Conversation

@alinpahontu2912

Copy link
Copy Markdown
Member

Fixes#1550

Experiment with new Forward only reading mode for ziparchvie, that relies on the local headers of entries instead of the central directory, avoiding loading everything into memory at once

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an experimental forward-only streaming read mode for System.IO.Compression.ZipArchive to support sequential iteration over ZIP entries via local file headers (instead of loading the central directory up-front), including async iteration and new unit tests.

Changes:

  • Add ZipArchiveMode.ForwardRead plus public ZipArchive.GetNextEntry() / GetNextEntryAsync(...) APIs for forward-only entry iteration.
  • Implement forward-read parsing and stream wrappers (ReadAheadStream, BoundedReadOnlyStream) to enable non-seekable streaming scenarios.
  • Add new unit tests validating basic ForwardRead behavior and unsupported operations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
FileDescription
src/libraries/System.IO.Compression/tests/ZipArchive/zip_ForwardReadTests.csAdds new test coverage for ForwardRead iteration, sync/async behavior, and unsupported APIs.
src/libraries/System.IO.Compression/tests/System.IO.Compression.Tests.csprojIncludes the new ForwardRead test file in the test project.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipCustomStreams.csAdds BoundedReadOnlyStream and ReadAheadStream used by ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveMode.csIntroduces ZipArchiveMode.ForwardRead enum value with documentation.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.csAdds ForwardRead entry initialization and Open() support for ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.csImplements ForwardRead iteration, local header parsing, data-descriptor handling, and non-seekable wrapping.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.Async.csEnables ForwardRead setup for async factory and disposal paths.
src/libraries/System.IO.Compression/src/Resources/Strings.resxAdds new SR strings for ForwardRead error messages.
src/libraries/System.IO.Compression/ref/System.IO.Compression.csUpdates public surface area (new mode + new ZipArchive methods).

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +695 to +698
// Known size, not encrypted
Stream bounded = new BoundedReadOnlyStream(_archiveStream, compressedSize);
Stream decompressor = CreateForwardReadDecompressor(bounded, compressionMethod, uncompressedSize, leaveOpen: false);
dataStream = new CrcValidatingReadStream(decompressor, crc32, uncompressedSize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should postpone DataStream creation and do it lazily like we do for the other entries.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can defer that for the on data descriptor entries

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:475

  • The XML documentation for Open() (and Open(FileAccess)) was not updated to describe ZipArchiveMode.ForwardRead. The summary still says "If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, ... If Update mode, ..." with no mention of ForwardRead, even though this PR introduces a new code path in the switch. Similarly, the <list> in Open(FileAccess) doesn't include the ForwardRead bullet, and the exception list omits the new NotSupportedException/InvalidOperationException cases (encrypted, no data stream, stream already opened). Please extend the doc comments to cover the new mode.
 /// <summary>
/// Opens the entry. If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, the returned stream will be writable and not seekable. If Update mode, the returned stream will be readable, writable, seekable, and support SetLength.
/// </summary>
/// <returns>A Stream that represents the contents of the entry.</returns>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>
public Stream Open()
{
ThrowIfInvalidArchive();
switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}
}
/// <summary>
/// Opens the entry with the specified access mode. This allows for more granular control over the returned stream's capabilities.
/// </summary>
/// <param name="access">The file access mode for the returned stream.</param>
/// <returns>A <see cref="Stream"/> that represents the contents of the entry with the specified access capabilities.</returns>
/// <remarks>
/// <para>The allowed <paramref name="access"/> values depend on the <see cref="ZipArchiveMode"/>:</para>
/// <list type="bullet">
/// <item><description><see cref="ZipArchiveMode.Read"/>: Only <see cref="FileAccess.Read"/> is allowed.</description></item>
/// <item><description><see cref="ZipArchiveMode.Create"/>: <see cref="FileAccess.Write"/> and <see cref="FileAccess.ReadWrite"/> are allowed (both write-only).</description></item>
/// <item><description><see cref="ZipArchiveMode.Update"/>: All values are allowed. <see cref="FileAccess.Read"/> reads directly from the archive. <see cref="FileAccess.Write"/> discards existing content and provides an empty writable stream. <see cref="FileAccess.ReadWrite"/> loads existing content into memory (equivalent to <see cref="Open()"/>).</description></item>
/// </list>
/// </remarks>
/// <exception cref="ArgumentOutOfRangeException"><paramref name="access"/> is not a valid <see cref="FileAccess"/> value.</exception>
/// <exception cref="InvalidOperationException">The requested access is not compatible with the archive's open mode.</exception>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +22 to +112
// ── Core reading scenarios ──────────────────────────────────────────

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task NonSeekableStream_ConsumeSkipConsume_ReadsCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
using WrappedStream nonSeekable = new(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(nonSeekable, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);
using (Stream ds = first.Open())
{
Assert.Equal(expected[0], await ReadStreamFully(ds, async));
}

// Skip second entry without opening
ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

ZipArchiveEntry? third = await GetNextEntry(archive, async);
Assert.NotNull(third);
using (Stream ds = third.Open())
{
Assert.Equal(expected[2], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[InlineData(true, true)]
[InlineData(true, false)]
[InlineData(false, true)]
[InlineData(false, false)]
public async Task StoredEntries_SeekableAndNonSeekable_ReadCorrectly(bool async, bool readSeekable)
{
// Always created on seekable stream → known sizes, no data descriptors
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.NoCompression, seekable: true);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
Stream readStream = readSeekable
? archiveStream
: new WrappedStream(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(readStream, ZipArchiveMode.ForwardRead);

for (int i = 0; i < expected.Length; i++)
{
ZipArchiveEntry? entry = await GetNextEntry(archive, async);
Assert.NotNull(entry);
Assert.Equal(ZipCompressionMethod.Stored, entry.CompressionMethod);

using Stream ds = entry.Open();
Assert.Equal(expected[i], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task PartialRead_ThenAdvance_ReadsNextEntryCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);

using MemoryStream archiveStream = new(zipBytes);
using ZipArchive archive = new(archiveStream, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);

using (Stream ds = first.Open())
{
byte[] partial = new byte[3];
await ReadStream(ds, partial, async);
}

ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

using Stream ds2 = second.Open();
Assert.Equal(s_mediumContent, await ReadStreamFully(ds2, async));
}

alinpahontu2912and others added 9 commits June 26, 2026 14:09
Adds a new ForwardRead mode to ZipArchive that enables forward-only
sequential reading of ZIP entries from non-seekable streams, using
local file headers instead of the central directory.
Changes:
- ZipArchiveMode.cs: Add ForwardRead = 3 enum value
- ZipCustomStreams.cs: Add BoundedReadOnlyStream and ReadAheadStream
helper stream classes
- ZipArchive.cs: Add GetNextEntry()/GetNextEntryAsync() methods,
ForwardRead constructor case, ValidateMode/DecideArchiveStream
support, data descriptor parsing, and property guards
- ZipArchive.Async.cs: Add ForwardRead cases to CreateAsync and
DisposeAsyncCore
- ZipArchiveEntry.cs: Add forward-read constructor, ForwardReadDataStream
property, UpdateFromDataDescriptor method, OpenInForwardReadMode,
and property setter guards
- Strings.resx: Add ForwardRead error message strings
- ref/System.IO.Compression.cs: Add public API surface
- Tests: Add comprehensive zip_ForwardReadTests covering deflate,
stored, data descriptors, non-seekable streams, empty archives,
partial reads, error cases, and async operations
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alinpahontu2912
alinpahontu2912force-pushed the feature/zip-archive-forward-read branch from 2ae6843 to 6bb976aCompareJune 26, 2026 11:09
CopilotAI review requested due to automatic review settings June 26, 2026 11:09

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:456

  • ForwardRead mode is handled in the synchronous Open()/Open(FileAccess) path, but ZipArchiveEntry.OpenAsync(...) and OpenAsync(FileAccess, ...) (in ZipArchiveEntry.Async.cs) do not have a ForwardRead case and will fall into the Update/default path in release builds. That will produce incorrect behavior (and could attempt Update-mode semantics on a ForwardRead archive). ForwardRead should be explicitly handled in the async Open* methods as well.

[!NOTE] This comment was created by GitHub Copilot.

 switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}

Comment on lines 152 to +154
public System.IO.Compression.ZipArchiveEntry? GetEntry(string entryName) { throw null; }
public System.IO.Compression.ZipArchiveEntry? GetNextEntry() { throw null; }
public System.Threading.Tasks.ValueTask<System.IO.Compression.ZipArchiveEntry?> GetNextEntryAsync(System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
Comment on lines +834 to +848
// Read enough for the 32-bit layout: CRC(4) + CompSize(4) + UncompSize(4),
// plus CRC(4) again if the signature consumed firstWord.
int smallSize = (hasSignature ? 4 : 0) + 8;
Span<byte> buf = stackalloc byte[20];
stream.ReadExactly(buf[..smallSize]);

var small = ParseDataDescriptor(buf[..smallSize], hasSignature, isZip64: false, firstWord);
if (small.Crc32 == knownCrc32 && small.UncompressedSize == knownUncompressedSize)
return small;

// 32-bit interpretation didn't match — read 8 more bytes for the Zip64 layout.
stream.ReadExactly(buf.Slice(smallSize, 8));
int fullSize = smallSize + 8;

return ParseDataDescriptor(buf[..fullSize], hasSignature, isZip64: true, firstWord);

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task TruncatedStoredEntry_ThrowsOnRead(bool async)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Forward-only API for System.IO.Compression

3 participants

@alinpahontu2912@rzikm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Feature/zip archive forward read - #126646

Draft
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read
Draft

Feature/zip archive forward read#126646
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read

Conversation

@alinpahontu2912

Copy link
Copy Markdown
Member

Fixes#1550

Experiment with new Forward only reading mode for ziparchvie, that relies on the local headers of entries instead of the central directory, avoiding loading everything into memory at once

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an experimental forward-only streaming read mode for System.IO.Compression.ZipArchive to support sequential iteration over ZIP entries via local file headers (instead of loading the central directory up-front), including async iteration and new unit tests.

Changes:

  • Add ZipArchiveMode.ForwardRead plus public ZipArchive.GetNextEntry() / GetNextEntryAsync(...) APIs for forward-only entry iteration.
  • Implement forward-read parsing and stream wrappers (ReadAheadStream, BoundedReadOnlyStream) to enable non-seekable streaming scenarios.
  • Add new unit tests validating basic ForwardRead behavior and unsupported operations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
FileDescription
src/libraries/System.IO.Compression/tests/ZipArchive/zip_ForwardReadTests.csAdds new test coverage for ForwardRead iteration, sync/async behavior, and unsupported APIs.
src/libraries/System.IO.Compression/tests/System.IO.Compression.Tests.csprojIncludes the new ForwardRead test file in the test project.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipCustomStreams.csAdds BoundedReadOnlyStream and ReadAheadStream used by ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveMode.csIntroduces ZipArchiveMode.ForwardRead enum value with documentation.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.csAdds ForwardRead entry initialization and Open() support for ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.csImplements ForwardRead iteration, local header parsing, data-descriptor handling, and non-seekable wrapping.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.Async.csEnables ForwardRead setup for async factory and disposal paths.
src/libraries/System.IO.Compression/src/Resources/Strings.resxAdds new SR strings for ForwardRead error messages.
src/libraries/System.IO.Compression/ref/System.IO.Compression.csUpdates public surface area (new mode + new ZipArchive methods).

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +695 to +698
// Known size, not encrypted
Stream bounded = new BoundedReadOnlyStream(_archiveStream, compressedSize);
Stream decompressor = CreateForwardReadDecompressor(bounded, compressionMethod, uncompressedSize, leaveOpen: false);
dataStream = new CrcValidatingReadStream(decompressor, crc32, uncompressedSize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should postpone DataStream creation and do it lazily like we do for the other entries.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can defer that for the on data descriptor entries

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:475

  • The XML documentation for Open() (and Open(FileAccess)) was not updated to describe ZipArchiveMode.ForwardRead. The summary still says "If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, ... If Update mode, ..." with no mention of ForwardRead, even though this PR introduces a new code path in the switch. Similarly, the <list> in Open(FileAccess) doesn't include the ForwardRead bullet, and the exception list omits the new NotSupportedException/InvalidOperationException cases (encrypted, no data stream, stream already opened). Please extend the doc comments to cover the new mode.
 /// <summary>
/// Opens the entry. If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, the returned stream will be writable and not seekable. If Update mode, the returned stream will be readable, writable, seekable, and support SetLength.
/// </summary>
/// <returns>A Stream that represents the contents of the entry.</returns>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>
public Stream Open()
{
ThrowIfInvalidArchive();
switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}
}
/// <summary>
/// Opens the entry with the specified access mode. This allows for more granular control over the returned stream's capabilities.
/// </summary>
/// <param name="access">The file access mode for the returned stream.</param>
/// <returns>A <see cref="Stream"/> that represents the contents of the entry with the specified access capabilities.</returns>
/// <remarks>
/// <para>The allowed <paramref name="access"/> values depend on the <see cref="ZipArchiveMode"/>:</para>
/// <list type="bullet">
/// <item><description><see cref="ZipArchiveMode.Read"/>: Only <see cref="FileAccess.Read"/> is allowed.</description></item>
/// <item><description><see cref="ZipArchiveMode.Create"/>: <see cref="FileAccess.Write"/> and <see cref="FileAccess.ReadWrite"/> are allowed (both write-only).</description></item>
/// <item><description><see cref="ZipArchiveMode.Update"/>: All values are allowed. <see cref="FileAccess.Read"/> reads directly from the archive. <see cref="FileAccess.Write"/> discards existing content and provides an empty writable stream. <see cref="FileAccess.ReadWrite"/> loads existing content into memory (equivalent to <see cref="Open()"/>).</description></item>
/// </list>
/// </remarks>
/// <exception cref="ArgumentOutOfRangeException"><paramref name="access"/> is not a valid <see cref="FileAccess"/> value.</exception>
/// <exception cref="InvalidOperationException">The requested access is not compatible with the archive's open mode.</exception>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +22 to +112
// ── Core reading scenarios ──────────────────────────────────────────

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task NonSeekableStream_ConsumeSkipConsume_ReadsCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
using WrappedStream nonSeekable = new(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(nonSeekable, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);
using (Stream ds = first.Open())
{
Assert.Equal(expected[0], await ReadStreamFully(ds, async));
}

// Skip second entry without opening
ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

ZipArchiveEntry? third = await GetNextEntry(archive, async);
Assert.NotNull(third);
using (Stream ds = third.Open())
{
Assert.Equal(expected[2], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[InlineData(true, true)]
[InlineData(true, false)]
[InlineData(false, true)]
[InlineData(false, false)]
public async Task StoredEntries_SeekableAndNonSeekable_ReadCorrectly(bool async, bool readSeekable)
{
// Always created on seekable stream → known sizes, no data descriptors
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.NoCompression, seekable: true);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
Stream readStream = readSeekable
? archiveStream
: new WrappedStream(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(readStream, ZipArchiveMode.ForwardRead);

for (int i = 0; i < expected.Length; i++)
{
ZipArchiveEntry? entry = await GetNextEntry(archive, async);
Assert.NotNull(entry);
Assert.Equal(ZipCompressionMethod.Stored, entry.CompressionMethod);

using Stream ds = entry.Open();
Assert.Equal(expected[i], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task PartialRead_ThenAdvance_ReadsNextEntryCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);

using MemoryStream archiveStream = new(zipBytes);
using ZipArchive archive = new(archiveStream, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);

using (Stream ds = first.Open())
{
byte[] partial = new byte[3];
await ReadStream(ds, partial, async);
}

ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

using Stream ds2 = second.Open();
Assert.Equal(s_mediumContent, await ReadStreamFully(ds2, async));
}

alinpahontu2912and others added 9 commits June 26, 2026 14:09
Adds a new ForwardRead mode to ZipArchive that enables forward-only
sequential reading of ZIP entries from non-seekable streams, using
local file headers instead of the central directory.
Changes:
- ZipArchiveMode.cs: Add ForwardRead = 3 enum value
- ZipCustomStreams.cs: Add BoundedReadOnlyStream and ReadAheadStream
helper stream classes
- ZipArchive.cs: Add GetNextEntry()/GetNextEntryAsync() methods,
ForwardRead constructor case, ValidateMode/DecideArchiveStream
support, data descriptor parsing, and property guards
- ZipArchive.Async.cs: Add ForwardRead cases to CreateAsync and
DisposeAsyncCore
- ZipArchiveEntry.cs: Add forward-read constructor, ForwardReadDataStream
property, UpdateFromDataDescriptor method, OpenInForwardReadMode,
and property setter guards
- Strings.resx: Add ForwardRead error message strings
- ref/System.IO.Compression.cs: Add public API surface
- Tests: Add comprehensive zip_ForwardReadTests covering deflate,
stored, data descriptors, non-seekable streams, empty archives,
partial reads, error cases, and async operations
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alinpahontu2912
alinpahontu2912force-pushed the feature/zip-archive-forward-read branch from 2ae6843 to 6bb976aCompareJune 26, 2026 11:09
CopilotAI review requested due to automatic review settings June 26, 2026 11:09

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:456

  • ForwardRead mode is handled in the synchronous Open()/Open(FileAccess) path, but ZipArchiveEntry.OpenAsync(...) and OpenAsync(FileAccess, ...) (in ZipArchiveEntry.Async.cs) do not have a ForwardRead case and will fall into the Update/default path in release builds. That will produce incorrect behavior (and could attempt Update-mode semantics on a ForwardRead archive). ForwardRead should be explicitly handled in the async Open* methods as well.

[!NOTE] This comment was created by GitHub Copilot.

 switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}

Comment on lines 152 to +154
public System.IO.Compression.ZipArchiveEntry? GetEntry(string entryName) { throw null; }
public System.IO.Compression.ZipArchiveEntry? GetNextEntry() { throw null; }
public System.Threading.Tasks.ValueTask<System.IO.Compression.ZipArchiveEntry?> GetNextEntryAsync(System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
Comment on lines +834 to +848
// Read enough for the 32-bit layout: CRC(4) + CompSize(4) + UncompSize(4),
// plus CRC(4) again if the signature consumed firstWord.
int smallSize = (hasSignature ? 4 : 0) + 8;
Span<byte> buf = stackalloc byte[20];
stream.ReadExactly(buf[..smallSize]);

var small = ParseDataDescriptor(buf[..smallSize], hasSignature, isZip64: false, firstWord);
if (small.Crc32 == knownCrc32 && small.UncompressedSize == knownUncompressedSize)
return small;

// 32-bit interpretation didn't match — read 8 more bytes for the Zip64 layout.
stream.ReadExactly(buf.Slice(smallSize, 8));
int fullSize = smallSize + 8;

return ParseDataDescriptor(buf[..fullSize], hasSignature, isZip64: true, firstWord);

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task TruncatedStoredEntry_ThrowsOnRead(bool async)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Forward-only API for System.IO.Compression

3 participants

@alinpahontu2912@rzikm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Feature/zip archive forward read - #126646

Draft
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read
Draft

Feature/zip archive forward read#126646
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read

Conversation

@alinpahontu2912

Copy link
Copy Markdown
Member

Fixes#1550

Experiment with new Forward only reading mode for ziparchvie, that relies on the local headers of entries instead of the central directory, avoiding loading everything into memory at once

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an experimental forward-only streaming read mode for System.IO.Compression.ZipArchive to support sequential iteration over ZIP entries via local file headers (instead of loading the central directory up-front), including async iteration and new unit tests.

Changes:

  • Add ZipArchiveMode.ForwardRead plus public ZipArchive.GetNextEntry() / GetNextEntryAsync(...) APIs for forward-only entry iteration.
  • Implement forward-read parsing and stream wrappers (ReadAheadStream, BoundedReadOnlyStream) to enable non-seekable streaming scenarios.
  • Add new unit tests validating basic ForwardRead behavior and unsupported operations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
FileDescription
src/libraries/System.IO.Compression/tests/ZipArchive/zip_ForwardReadTests.csAdds new test coverage for ForwardRead iteration, sync/async behavior, and unsupported APIs.
src/libraries/System.IO.Compression/tests/System.IO.Compression.Tests.csprojIncludes the new ForwardRead test file in the test project.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipCustomStreams.csAdds BoundedReadOnlyStream and ReadAheadStream used by ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveMode.csIntroduces ZipArchiveMode.ForwardRead enum value with documentation.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.csAdds ForwardRead entry initialization and Open() support for ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.csImplements ForwardRead iteration, local header parsing, data-descriptor handling, and non-seekable wrapping.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.Async.csEnables ForwardRead setup for async factory and disposal paths.
src/libraries/System.IO.Compression/src/Resources/Strings.resxAdds new SR strings for ForwardRead error messages.
src/libraries/System.IO.Compression/ref/System.IO.Compression.csUpdates public surface area (new mode + new ZipArchive methods).

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +695 to +698
// Known size, not encrypted
Stream bounded = new BoundedReadOnlyStream(_archiveStream, compressedSize);
Stream decompressor = CreateForwardReadDecompressor(bounded, compressionMethod, uncompressedSize, leaveOpen: false);
dataStream = new CrcValidatingReadStream(decompressor, crc32, uncompressedSize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should postpone DataStream creation and do it lazily like we do for the other entries.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can defer that for the on data descriptor entries

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:475

  • The XML documentation for Open() (and Open(FileAccess)) was not updated to describe ZipArchiveMode.ForwardRead. The summary still says "If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, ... If Update mode, ..." with no mention of ForwardRead, even though this PR introduces a new code path in the switch. Similarly, the <list> in Open(FileAccess) doesn't include the ForwardRead bullet, and the exception list omits the new NotSupportedException/InvalidOperationException cases (encrypted, no data stream, stream already opened). Please extend the doc comments to cover the new mode.
 /// <summary>
/// Opens the entry. If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, the returned stream will be writable and not seekable. If Update mode, the returned stream will be readable, writable, seekable, and support SetLength.
/// </summary>
/// <returns>A Stream that represents the contents of the entry.</returns>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>
public Stream Open()
{
ThrowIfInvalidArchive();
switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}
}
/// <summary>
/// Opens the entry with the specified access mode. This allows for more granular control over the returned stream's capabilities.
/// </summary>
/// <param name="access">The file access mode for the returned stream.</param>
/// <returns>A <see cref="Stream"/> that represents the contents of the entry with the specified access capabilities.</returns>
/// <remarks>
/// <para>The allowed <paramref name="access"/> values depend on the <see cref="ZipArchiveMode"/>:</para>
/// <list type="bullet">
/// <item><description><see cref="ZipArchiveMode.Read"/>: Only <see cref="FileAccess.Read"/> is allowed.</description></item>
/// <item><description><see cref="ZipArchiveMode.Create"/>: <see cref="FileAccess.Write"/> and <see cref="FileAccess.ReadWrite"/> are allowed (both write-only).</description></item>
/// <item><description><see cref="ZipArchiveMode.Update"/>: All values are allowed. <see cref="FileAccess.Read"/> reads directly from the archive. <see cref="FileAccess.Write"/> discards existing content and provides an empty writable stream. <see cref="FileAccess.ReadWrite"/> loads existing content into memory (equivalent to <see cref="Open()"/>).</description></item>
/// </list>
/// </remarks>
/// <exception cref="ArgumentOutOfRangeException"><paramref name="access"/> is not a valid <see cref="FileAccess"/> value.</exception>
/// <exception cref="InvalidOperationException">The requested access is not compatible with the archive's open mode.</exception>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +22 to +112
// ── Core reading scenarios ──────────────────────────────────────────

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task NonSeekableStream_ConsumeSkipConsume_ReadsCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
using WrappedStream nonSeekable = new(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(nonSeekable, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);
using (Stream ds = first.Open())
{
Assert.Equal(expected[0], await ReadStreamFully(ds, async));
}

// Skip second entry without opening
ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

ZipArchiveEntry? third = await GetNextEntry(archive, async);
Assert.NotNull(third);
using (Stream ds = third.Open())
{
Assert.Equal(expected[2], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[InlineData(true, true)]
[InlineData(true, false)]
[InlineData(false, true)]
[InlineData(false, false)]
public async Task StoredEntries_SeekableAndNonSeekable_ReadCorrectly(bool async, bool readSeekable)
{
// Always created on seekable stream → known sizes, no data descriptors
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.NoCompression, seekable: true);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
Stream readStream = readSeekable
? archiveStream
: new WrappedStream(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(readStream, ZipArchiveMode.ForwardRead);

for (int i = 0; i < expected.Length; i++)
{
ZipArchiveEntry? entry = await GetNextEntry(archive, async);
Assert.NotNull(entry);
Assert.Equal(ZipCompressionMethod.Stored, entry.CompressionMethod);

using Stream ds = entry.Open();
Assert.Equal(expected[i], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task PartialRead_ThenAdvance_ReadsNextEntryCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);

using MemoryStream archiveStream = new(zipBytes);
using ZipArchive archive = new(archiveStream, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);

using (Stream ds = first.Open())
{
byte[] partial = new byte[3];
await ReadStream(ds, partial, async);
}

ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

using Stream ds2 = second.Open();
Assert.Equal(s_mediumContent, await ReadStreamFully(ds2, async));
}

alinpahontu2912and others added 9 commits June 26, 2026 14:09
Adds a new ForwardRead mode to ZipArchive that enables forward-only
sequential reading of ZIP entries from non-seekable streams, using
local file headers instead of the central directory.
Changes:
- ZipArchiveMode.cs: Add ForwardRead = 3 enum value
- ZipCustomStreams.cs: Add BoundedReadOnlyStream and ReadAheadStream
helper stream classes
- ZipArchive.cs: Add GetNextEntry()/GetNextEntryAsync() methods,
ForwardRead constructor case, ValidateMode/DecideArchiveStream
support, data descriptor parsing, and property guards
- ZipArchive.Async.cs: Add ForwardRead cases to CreateAsync and
DisposeAsyncCore
- ZipArchiveEntry.cs: Add forward-read constructor, ForwardReadDataStream
property, UpdateFromDataDescriptor method, OpenInForwardReadMode,
and property setter guards
- Strings.resx: Add ForwardRead error message strings
- ref/System.IO.Compression.cs: Add public API surface
- Tests: Add comprehensive zip_ForwardReadTests covering deflate,
stored, data descriptors, non-seekable streams, empty archives,
partial reads, error cases, and async operations
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alinpahontu2912
alinpahontu2912force-pushed the feature/zip-archive-forward-read branch from 2ae6843 to 6bb976aCompareJune 26, 2026 11:09
CopilotAI review requested due to automatic review settings June 26, 2026 11:09

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:456

  • ForwardRead mode is handled in the synchronous Open()/Open(FileAccess) path, but ZipArchiveEntry.OpenAsync(...) and OpenAsync(FileAccess, ...) (in ZipArchiveEntry.Async.cs) do not have a ForwardRead case and will fall into the Update/default path in release builds. That will produce incorrect behavior (and could attempt Update-mode semantics on a ForwardRead archive). ForwardRead should be explicitly handled in the async Open* methods as well.

[!NOTE] This comment was created by GitHub Copilot.

 switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}

Comment on lines 152 to +154
public System.IO.Compression.ZipArchiveEntry? GetEntry(string entryName) { throw null; }
public System.IO.Compression.ZipArchiveEntry? GetNextEntry() { throw null; }
public System.Threading.Tasks.ValueTask<System.IO.Compression.ZipArchiveEntry?> GetNextEntryAsync(System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
Comment on lines +834 to +848
// Read enough for the 32-bit layout: CRC(4) + CompSize(4) + UncompSize(4),
// plus CRC(4) again if the signature consumed firstWord.
int smallSize = (hasSignature ? 4 : 0) + 8;
Span<byte> buf = stackalloc byte[20];
stream.ReadExactly(buf[..smallSize]);

var small = ParseDataDescriptor(buf[..smallSize], hasSignature, isZip64: false, firstWord);
if (small.Crc32 == knownCrc32 && small.UncompressedSize == knownUncompressedSize)
return small;

// 32-bit interpretation didn't match — read 8 more bytes for the Zip64 layout.
stream.ReadExactly(buf.Slice(smallSize, 8));
int fullSize = smallSize + 8;

return ParseDataDescriptor(buf[..fullSize], hasSignature, isZip64: true, firstWord);

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task TruncatedStoredEntry_ThrowsOnRead(bool async)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Forward-only API for System.IO.Compression

3 participants

@alinpahontu2912@rzikm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Feature/zip archive forward read - #126646

Draft
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read
Draft

Feature/zip archive forward read#126646
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read

Conversation

@alinpahontu2912

Copy link
Copy Markdown
Member

Fixes#1550

Experiment with new Forward only reading mode for ziparchvie, that relies on the local headers of entries instead of the central directory, avoiding loading everything into memory at once

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an experimental forward-only streaming read mode for System.IO.Compression.ZipArchive to support sequential iteration over ZIP entries via local file headers (instead of loading the central directory up-front), including async iteration and new unit tests.

Changes:

  • Add ZipArchiveMode.ForwardRead plus public ZipArchive.GetNextEntry() / GetNextEntryAsync(...) APIs for forward-only entry iteration.
  • Implement forward-read parsing and stream wrappers (ReadAheadStream, BoundedReadOnlyStream) to enable non-seekable streaming scenarios.
  • Add new unit tests validating basic ForwardRead behavior and unsupported operations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
FileDescription
src/libraries/System.IO.Compression/tests/ZipArchive/zip_ForwardReadTests.csAdds new test coverage for ForwardRead iteration, sync/async behavior, and unsupported APIs.
src/libraries/System.IO.Compression/tests/System.IO.Compression.Tests.csprojIncludes the new ForwardRead test file in the test project.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipCustomStreams.csAdds BoundedReadOnlyStream and ReadAheadStream used by ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveMode.csIntroduces ZipArchiveMode.ForwardRead enum value with documentation.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.csAdds ForwardRead entry initialization and Open() support for ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.csImplements ForwardRead iteration, local header parsing, data-descriptor handling, and non-seekable wrapping.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.Async.csEnables ForwardRead setup for async factory and disposal paths.
src/libraries/System.IO.Compression/src/Resources/Strings.resxAdds new SR strings for ForwardRead error messages.
src/libraries/System.IO.Compression/ref/System.IO.Compression.csUpdates public surface area (new mode + new ZipArchive methods).

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +695 to +698
// Known size, not encrypted
Stream bounded = new BoundedReadOnlyStream(_archiveStream, compressedSize);
Stream decompressor = CreateForwardReadDecompressor(bounded, compressionMethod, uncompressedSize, leaveOpen: false);
dataStream = new CrcValidatingReadStream(decompressor, crc32, uncompressedSize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should postpone DataStream creation and do it lazily like we do for the other entries.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can defer that for the on data descriptor entries

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:475

  • The XML documentation for Open() (and Open(FileAccess)) was not updated to describe ZipArchiveMode.ForwardRead. The summary still says "If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, ... If Update mode, ..." with no mention of ForwardRead, even though this PR introduces a new code path in the switch. Similarly, the <list> in Open(FileAccess) doesn't include the ForwardRead bullet, and the exception list omits the new NotSupportedException/InvalidOperationException cases (encrypted, no data stream, stream already opened). Please extend the doc comments to cover the new mode.
 /// <summary>
/// Opens the entry. If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, the returned stream will be writable and not seekable. If Update mode, the returned stream will be readable, writable, seekable, and support SetLength.
/// </summary>
/// <returns>A Stream that represents the contents of the entry.</returns>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>
public Stream Open()
{
ThrowIfInvalidArchive();
switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}
}
/// <summary>
/// Opens the entry with the specified access mode. This allows for more granular control over the returned stream's capabilities.
/// </summary>
/// <param name="access">The file access mode for the returned stream.</param>
/// <returns>A <see cref="Stream"/> that represents the contents of the entry with the specified access capabilities.</returns>
/// <remarks>
/// <para>The allowed <paramref name="access"/> values depend on the <see cref="ZipArchiveMode"/>:</para>
/// <list type="bullet">
/// <item><description><see cref="ZipArchiveMode.Read"/>: Only <see cref="FileAccess.Read"/> is allowed.</description></item>
/// <item><description><see cref="ZipArchiveMode.Create"/>: <see cref="FileAccess.Write"/> and <see cref="FileAccess.ReadWrite"/> are allowed (both write-only).</description></item>
/// <item><description><see cref="ZipArchiveMode.Update"/>: All values are allowed. <see cref="FileAccess.Read"/> reads directly from the archive. <see cref="FileAccess.Write"/> discards existing content and provides an empty writable stream. <see cref="FileAccess.ReadWrite"/> loads existing content into memory (equivalent to <see cref="Open()"/>).</description></item>
/// </list>
/// </remarks>
/// <exception cref="ArgumentOutOfRangeException"><paramref name="access"/> is not a valid <see cref="FileAccess"/> value.</exception>
/// <exception cref="InvalidOperationException">The requested access is not compatible with the archive's open mode.</exception>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +22 to +112
// ── Core reading scenarios ──────────────────────────────────────────

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task NonSeekableStream_ConsumeSkipConsume_ReadsCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
using WrappedStream nonSeekable = new(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(nonSeekable, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);
using (Stream ds = first.Open())
{
Assert.Equal(expected[0], await ReadStreamFully(ds, async));
}

// Skip second entry without opening
ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

ZipArchiveEntry? third = await GetNextEntry(archive, async);
Assert.NotNull(third);
using (Stream ds = third.Open())
{
Assert.Equal(expected[2], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[InlineData(true, true)]
[InlineData(true, false)]
[InlineData(false, true)]
[InlineData(false, false)]
public async Task StoredEntries_SeekableAndNonSeekable_ReadCorrectly(bool async, bool readSeekable)
{
// Always created on seekable stream → known sizes, no data descriptors
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.NoCompression, seekable: true);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
Stream readStream = readSeekable
? archiveStream
: new WrappedStream(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(readStream, ZipArchiveMode.ForwardRead);

for (int i = 0; i < expected.Length; i++)
{
ZipArchiveEntry? entry = await GetNextEntry(archive, async);
Assert.NotNull(entry);
Assert.Equal(ZipCompressionMethod.Stored, entry.CompressionMethod);

using Stream ds = entry.Open();
Assert.Equal(expected[i], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task PartialRead_ThenAdvance_ReadsNextEntryCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);

using MemoryStream archiveStream = new(zipBytes);
using ZipArchive archive = new(archiveStream, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);

using (Stream ds = first.Open())
{
byte[] partial = new byte[3];
await ReadStream(ds, partial, async);
}

ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

using Stream ds2 = second.Open();
Assert.Equal(s_mediumContent, await ReadStreamFully(ds2, async));
}

alinpahontu2912and others added 9 commits June 26, 2026 14:09
Adds a new ForwardRead mode to ZipArchive that enables forward-only
sequential reading of ZIP entries from non-seekable streams, using
local file headers instead of the central directory.
Changes:
- ZipArchiveMode.cs: Add ForwardRead = 3 enum value
- ZipCustomStreams.cs: Add BoundedReadOnlyStream and ReadAheadStream
helper stream classes
- ZipArchive.cs: Add GetNextEntry()/GetNextEntryAsync() methods,
ForwardRead constructor case, ValidateMode/DecideArchiveStream
support, data descriptor parsing, and property guards
- ZipArchive.Async.cs: Add ForwardRead cases to CreateAsync and
DisposeAsyncCore
- ZipArchiveEntry.cs: Add forward-read constructor, ForwardReadDataStream
property, UpdateFromDataDescriptor method, OpenInForwardReadMode,
and property setter guards
- Strings.resx: Add ForwardRead error message strings
- ref/System.IO.Compression.cs: Add public API surface
- Tests: Add comprehensive zip_ForwardReadTests covering deflate,
stored, data descriptors, non-seekable streams, empty archives,
partial reads, error cases, and async operations
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alinpahontu2912
alinpahontu2912force-pushed the feature/zip-archive-forward-read branch from 2ae6843 to 6bb976aCompareJune 26, 2026 11:09
CopilotAI review requested due to automatic review settings June 26, 2026 11:09

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:456

  • ForwardRead mode is handled in the synchronous Open()/Open(FileAccess) path, but ZipArchiveEntry.OpenAsync(...) and OpenAsync(FileAccess, ...) (in ZipArchiveEntry.Async.cs) do not have a ForwardRead case and will fall into the Update/default path in release builds. That will produce incorrect behavior (and could attempt Update-mode semantics on a ForwardRead archive). ForwardRead should be explicitly handled in the async Open* methods as well.

[!NOTE] This comment was created by GitHub Copilot.

 switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}

Comment on lines 152 to +154
public System.IO.Compression.ZipArchiveEntry? GetEntry(string entryName) { throw null; }
public System.IO.Compression.ZipArchiveEntry? GetNextEntry() { throw null; }
public System.Threading.Tasks.ValueTask<System.IO.Compression.ZipArchiveEntry?> GetNextEntryAsync(System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
Comment on lines +834 to +848
// Read enough for the 32-bit layout: CRC(4) + CompSize(4) + UncompSize(4),
// plus CRC(4) again if the signature consumed firstWord.
int smallSize = (hasSignature ? 4 : 0) + 8;
Span<byte> buf = stackalloc byte[20];
stream.ReadExactly(buf[..smallSize]);

var small = ParseDataDescriptor(buf[..smallSize], hasSignature, isZip64: false, firstWord);
if (small.Crc32 == knownCrc32 && small.UncompressedSize == knownUncompressedSize)
return small;

// 32-bit interpretation didn't match — read 8 more bytes for the Zip64 layout.
stream.ReadExactly(buf.Slice(smallSize, 8));
int fullSize = smallSize + 8;

return ParseDataDescriptor(buf[..fullSize], hasSignature, isZip64: true, firstWord);

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task TruncatedStoredEntry_ThrowsOnRead(bool async)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Forward-only API for System.IO.Compression

3 participants

@alinpahontu2912@rzikm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Feature/zip archive forward read - #126646

Draft
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read
Draft

Feature/zip archive forward read#126646
alinpahontu2912 wants to merge 9 commits into
dotnet:mainfrom
alinpahontu2912:feature/zip-archive-forward-read

Conversation

@alinpahontu2912

Copy link
Copy Markdown
Member

Fixes#1550

Experiment with new Forward only reading mode for ziparchvie, that relies on the local headers of entries instead of the central directory, avoiding loading everything into memory at once

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an experimental forward-only streaming read mode for System.IO.Compression.ZipArchive to support sequential iteration over ZIP entries via local file headers (instead of loading the central directory up-front), including async iteration and new unit tests.

Changes:

  • Add ZipArchiveMode.ForwardRead plus public ZipArchive.GetNextEntry() / GetNextEntryAsync(...) APIs for forward-only entry iteration.
  • Implement forward-read parsing and stream wrappers (ReadAheadStream, BoundedReadOnlyStream) to enable non-seekable streaming scenarios.
  • Add new unit tests validating basic ForwardRead behavior and unsupported operations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
FileDescription
src/libraries/System.IO.Compression/tests/ZipArchive/zip_ForwardReadTests.csAdds new test coverage for ForwardRead iteration, sync/async behavior, and unsupported APIs.
src/libraries/System.IO.Compression/tests/System.IO.Compression.Tests.csprojIncludes the new ForwardRead test file in the test project.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipCustomStreams.csAdds BoundedReadOnlyStream and ReadAheadStream used by ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveMode.csIntroduces ZipArchiveMode.ForwardRead enum value with documentation.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.csAdds ForwardRead entry initialization and Open() support for ForwardRead mode.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.csImplements ForwardRead iteration, local header parsing, data-descriptor handling, and non-seekable wrapping.
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.Async.csEnables ForwardRead setup for async factory and disposal paths.
src/libraries/System.IO.Compression/src/Resources/Strings.resxAdds new SR strings for ForwardRead error messages.
src/libraries/System.IO.Compression/ref/System.IO.Compression.csUpdates public surface area (new mode + new ZipArchive methods).

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +695 to +698
// Known size, not encrypted
Stream bounded = new BoundedReadOnlyStream(_archiveStream, compressedSize);
Stream decompressor = CreateForwardReadDecompressor(bounded, compressionMethod, uncompressedSize, leaveOpen: false);
dataStream = new CrcValidatingReadStream(decompressor, crc32, uncompressedSize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should postpone DataStream creation and do it lazily like we do for the other entries.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can defer that for the on data descriptor entries

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:475

  • The XML documentation for Open() (and Open(FileAccess)) was not updated to describe ZipArchiveMode.ForwardRead. The summary still says "If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, ... If Update mode, ..." with no mention of ForwardRead, even though this PR introduces a new code path in the switch. Similarly, the <list> in Open(FileAccess) doesn't include the ForwardRead bullet, and the exception list omits the new NotSupportedException/InvalidOperationException cases (encrypted, no data stream, stream already opened). Please extend the doc comments to cover the new mode.
 /// <summary>
/// Opens the entry. If the archive that the entry belongs to was opened in Read mode, the returned stream will be readable, and it may or may not be seekable. If Create mode, the returned stream will be writable and not seekable. If Update mode, the returned stream will be readable, writable, seekable, and support SetLength.
/// </summary>
/// <returns>A Stream that represents the contents of the entry.</returns>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>
public Stream Open()
{
ThrowIfInvalidArchive();
switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}
}
/// <summary>
/// Opens the entry with the specified access mode. This allows for more granular control over the returned stream's capabilities.
/// </summary>
/// <param name="access">The file access mode for the returned stream.</param>
/// <returns>A <see cref="Stream"/> that represents the contents of the entry with the specified access capabilities.</returns>
/// <remarks>
/// <para>The allowed <paramref name="access"/> values depend on the <see cref="ZipArchiveMode"/>:</para>
/// <list type="bullet">
/// <item><description><see cref="ZipArchiveMode.Read"/>: Only <see cref="FileAccess.Read"/> is allowed.</description></item>
/// <item><description><see cref="ZipArchiveMode.Create"/>: <see cref="FileAccess.Write"/> and <see cref="FileAccess.ReadWrite"/> are allowed (both write-only).</description></item>
/// <item><description><see cref="ZipArchiveMode.Update"/>: All values are allowed. <see cref="FileAccess.Read"/> reads directly from the archive. <see cref="FileAccess.Write"/> discards existing content and provides an empty writable stream. <see cref="FileAccess.ReadWrite"/> loads existing content into memory (equivalent to <see cref="Open()"/>).</description></item>
/// </list>
/// </remarks>
/// <exception cref="ArgumentOutOfRangeException"><paramref name="access"/> is not a valid <see cref="FileAccess"/> value.</exception>
/// <exception cref="InvalidOperationException">The requested access is not compatible with the archive's open mode.</exception>
/// <exception cref="IOException">The entry is already currently open for writing. -or- The entry has been deleted from the archive. -or- The archive that this entry belongs to was opened in ZipArchiveMode.Create, and this entry has already been written to once.</exception>
/// <exception cref="InvalidDataException">The entry is missing from the archive or is corrupt and cannot be read. -or- The entry has been compressed using a compression method that is not supported.</exception>
/// <exception cref="ObjectDisposedException">The ZipArchive that this entry belongs to has been disposed.</exception>

Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment threadsrc/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchive.cs Outdated
Comment on lines +22 to +112
// ── Core reading scenarios ──────────────────────────────────────────

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task NonSeekableStream_ConsumeSkipConsume_ReadsCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
using WrappedStream nonSeekable = new(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(nonSeekable, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);
using (Stream ds = first.Open())
{
Assert.Equal(expected[0], await ReadStreamFully(ds, async));
}

// Skip second entry without opening
ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

ZipArchiveEntry? third = await GetNextEntry(archive, async);
Assert.NotNull(third);
using (Stream ds = third.Open())
{
Assert.Equal(expected[2], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[InlineData(true, true)]
[InlineData(true, false)]
[InlineData(false, true)]
[InlineData(false, false)]
public async Task StoredEntries_SeekableAndNonSeekable_ReadCorrectly(bool async, bool readSeekable)
{
// Always created on seekable stream → known sizes, no data descriptors
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.NoCompression, seekable: true);
byte[][] expected = [s_smallContent, s_mediumContent, s_largeContent];

using MemoryStream archiveStream = new(zipBytes);
Stream readStream = readSeekable
? archiveStream
: new WrappedStream(archiveStream, canRead: true, canWrite: false, canSeek: false, null);
using ZipArchive archive = new(readStream, ZipArchiveMode.ForwardRead);

for (int i = 0; i < expected.Length; i++)
{
ZipArchiveEntry? entry = await GetNextEntry(archive, async);
Assert.NotNull(entry);
Assert.Equal(ZipCompressionMethod.Stored, entry.CompressionMethod);

using Stream ds = entry.Open();
Assert.Equal(expected[i], await ReadStreamFully(ds, async));
}

Assert.Null(await GetNextEntry(archive, async));
}

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task PartialRead_ThenAdvance_ReadsNextEntryCorrectly(bool async)
{
byte[] zipBytes = CreateZipWithEntries(CompressionLevel.Optimal, seekable: false);

using MemoryStream archiveStream = new(zipBytes);
using ZipArchive archive = new(archiveStream, ZipArchiveMode.ForwardRead);

ZipArchiveEntry? first = await GetNextEntry(archive, async);
Assert.NotNull(first);

using (Stream ds = first.Open())
{
byte[] partial = new byte[3];
await ReadStream(ds, partial, async);
}

ZipArchiveEntry? second = await GetNextEntry(archive, async);
Assert.NotNull(second);
Assert.Equal("medium.bin", second.FullName);

using Stream ds2 = second.Open();
Assert.Equal(s_mediumContent, await ReadStreamFully(ds2, async));
}

alinpahontu2912and others added 9 commits June 26, 2026 14:09
Adds a new ForwardRead mode to ZipArchive that enables forward-only
sequential reading of ZIP entries from non-seekable streams, using
local file headers instead of the central directory.
Changes:
- ZipArchiveMode.cs: Add ForwardRead = 3 enum value
- ZipCustomStreams.cs: Add BoundedReadOnlyStream and ReadAheadStream
helper stream classes
- ZipArchive.cs: Add GetNextEntry()/GetNextEntryAsync() methods,
ForwardRead constructor case, ValidateMode/DecideArchiveStream
support, data descriptor parsing, and property guards
- ZipArchive.Async.cs: Add ForwardRead cases to CreateAsync and
DisposeAsyncCore
- ZipArchiveEntry.cs: Add forward-read constructor, ForwardReadDataStream
property, UpdateFromDataDescriptor method, OpenInForwardReadMode,
and property setter guards
- Strings.resx: Add ForwardRead error message strings
- ref/System.IO.Compression.cs: Add public API surface
- Tests: Add comprehensive zip_ForwardReadTests covering deflate,
stored, data descriptors, non-seekable streams, empty archives,
partial reads, error cases, and async operations
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alinpahontu2912
alinpahontu2912force-pushed the feature/zip-archive-forward-read branch from 2ae6843 to 6bb976aCompareJune 26, 2026 11:09
CopilotAI review requested due to automatic review settings June 26, 2026 11:09

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs:456

  • ForwardRead mode is handled in the synchronous Open()/Open(FileAccess) path, but ZipArchiveEntry.OpenAsync(...) and OpenAsync(FileAccess, ...) (in ZipArchiveEntry.Async.cs) do not have a ForwardRead case and will fall into the Update/default path in release builds. That will produce incorrect behavior (and could attempt Update-mode semantics on a ForwardRead archive). ForwardRead should be explicitly handled in the async Open* methods as well.

[!NOTE] This comment was created by GitHub Copilot.

 switch (_archive.Mode)
{
case ZipArchiveMode.Read:
return OpenInReadMode(checkOpenable: true);
case ZipArchiveMode.Create:
return OpenInWriteMode();
case ZipArchiveMode.ForwardRead:
return OpenInForwardReadMode();
case ZipArchiveMode.Update:
default:
Debug.Assert(_archive.Mode == ZipArchiveMode.Update);
return OpenInUpdateMode();
}

Comment on lines 152 to +154
public System.IO.Compression.ZipArchiveEntry? GetEntry(string entryName) { throw null; }
public System.IO.Compression.ZipArchiveEntry? GetNextEntry() { throw null; }
public System.Threading.Tasks.ValueTask<System.IO.Compression.ZipArchiveEntry?> GetNextEntryAsync(System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
Comment on lines +834 to +848
// Read enough for the 32-bit layout: CRC(4) + CompSize(4) + UncompSize(4),
// plus CRC(4) again if the signature consumed firstWord.
int smallSize = (hasSignature ? 4 : 0) + 8;
Span<byte> buf = stackalloc byte[20];
stream.ReadExactly(buf[..smallSize]);

var small = ParseDataDescriptor(buf[..smallSize], hasSignature, isZip64: false, firstWord);
if (small.Crc32 == knownCrc32 && small.UncompressedSize == knownUncompressedSize)
return small;

// 32-bit interpretation didn't match — read 8 more bytes for the Zip64 layout.
stream.ReadExactly(buf.Slice(smallSize, 8));
int fullSize = smallSize + 8;

return ParseDataDescriptor(buf[..fullSize], hasSignature, isZip64: true, firstWord);

[Theory]
[MemberData(nameof(Get_Booleans_Data))]
public async Task TruncatedStoredEntry_ThrowsOnRead(bool async)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Forward-only API for System.IO.Compression

3 participants

@alinpahontu2912@rzikm