Add Base64url encoding/decoding by buyaa-n · Pull Request #102364 · dotnet/runtime

buyaa-n · 2024-05-17T04:27:12Z

Base64Url encoding doesn't append padding, therefore encoded byte count differs from Base64 encoding:
Source byte count Base64 encoded byte count Base64Url encoded byte count
1 4 2
2 4 3
3 4 4
4 8 6
5 8 7
6 8 8
Base64Url decoding ignore whitespace and padding, therefore decodable byte count differs from Base64 decoding, the exact decoding result also depend on isFinalBlock and if padding involved
Source byte count Base64 decoded byte count Base64Url decoded byte count
1 max 0 max 0
2 max 0 max 1
3 max 0 max 2
4 max 3 max 3
5 max 3 max 3
6 max 3 max 4
7 max 3 max 5
8 max 6 max 6

Approved API shape:

namespaceSystem.Buffers.Text;publicstaticclassBase64Url{publicstaticintGetMaxDecodedLength(intbase64Length);publicstaticintGetEncodedLength(intbytesLength);publicstaticOperationStatusEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintcharsWritten);publicstaticbyte[]EncodeToUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintbytesConsumed,outintcharsWritten,boolisFinalBlock=true);publicstaticintEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination);publicstaticboolTryEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintcharsWritten);publicstaticchar[]EncodeToChars(ReadOnlySpan<byte>source);publicstaticstringEncodeToString(ReadOnlySpan<byte>source);publicstaticboolTryEncodeToUtf8InPlace(Span<byte>buffer,intdataLength,outintbytesWritten);publicstaticOperationStatusDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintcharsConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination);publicstaticboolTryDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromChars(ReadOnlySpan<char>source);publicstaticintDecodeFromUtf8InPlace(Span<byte>buffer);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText,outintdecodedLength);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText,outintdecodedLength);}

Draft PR until below list is completed, for now mainly for checking perf, run tests on different CI legs, and get early feedback.

Handle '%' in decoding as a valid padding character for Base64Url
Doc needs improvement
A few APIs missing unit tests
Fix failures on ARM CI legs

Fixes#1658

ghost · 2024-05-17T04:27:17Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

gfoidl

Just skimmed over it...
I saw your comment, but for checking perf some of my comments may help (a bit).

Co-authored-by: Günther Foidl <gue@korporal.at>

buyaa-n · 2024-05-17T18:20:40Z

/benchmark

buyaa-n · 2024-05-17T18:28:38Z

Just skimmed over it... I saw your comment, but for checking perf some of my comments may help (a bit).

Thank you, sure perf related feedbacks appreciated, I updated the description, any feedbacks are welcome.

pr-benchmarks · 2024-05-17T18:33:47Z

Crank Pull Request Bot

/benchmark <benchmark[,...]> <profile[,...]> <component,[...]> <arguments>

Benchmarks:

micro: .NET Performance micro benchmarks (set filter by adding --variable filter=...; by default filter=*LinqBenchmarks*)
plaintext: TechEmpower Plaintext Scenario - ASP.NET Platform implementation
json: TechEmpower JSON Scenario - ASP.NET Platform implementation
fortunes: TechEmpower Fortunes Scenario - ASP.NET Platform implementation
fortunes_ef: TechEmpower Fortunes Scenario with EF Core - ASP.NET Platform implementation
httpclient: HttpClient Benchmark (change HTTP version by adding e.g. --variable httpVersion=3.0; change response size by adding e.g. --variable responseSize=256; default: HTTP/1.1 GET 8K)

Profiles:

aspnet-citrine-lin: Intel/Linux 28 Cores
aspnet-perf-lin: Intel/Linux 12 Cores
aspnet-citrine-amd: Amd/Linux 48 Cores

Components:

runtime
libs

Arguments: any additional arguments to pass through to crank, e.g. --variable name=value

Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

stephentoub · 2024-06-12T01:14:57Z

+ padding++;
+ }
+
+ if (TBase64Decoder.IsValidPadding(Unsafe.Subtract(ref ptrToLastElement, 1)))


What guarantees this is in bounds?

This is only called with a buffer that has size of 4
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Lines 498 to 499 in e617b8f
constintBlockSize=4;
Span<byte>buffer=stackallocbyte[BlockSize];

runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Line 545 in e617b8f
intpaddingCount=GetPaddingCount<TBase64Decoder>(refbuffer[^1]);

stephentoub · 2024-06-12T01:19:49Z

 }

- end = srcMax - 64;
+ end = srcMax - 32;


Why did this change from 64 to 32?

It was 32 originally, but unintentionally changed with this PR
The method comment also mentions it require 32 byte
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Encoder.cs
Lines 316 to 322 in e617b8f
privatestaticunsafevoidAvx2Encode<TBase64Encoder,T>(refbyte*srcBytes,refT*destBytes,byte*srcEnd,intsourceLength,intdestLength,byte*srcStart,T*destStart)
whereTBase64Encoder:IBase64Encoder<T>
whereT: unmanaged
{
// If we have AVX2 support, pick off 24 bytes at a time for as long as we can.
// But because we read 32 bytes at a time, ensure we have enough room to do a
// full 32-byte read without segfaulting.

stephentoub

Thanks!

MihaZupan · 2024-06-12T21:11:33Z

+ where TBase64Decoder : IBase64Decoder<byte>
 {
-const int BlockSize = 4;
+ int BlockSize = Math.Min(source.Length - (int)sourceIndex, 4);


Would anything bad happen if this were left at a const 4?
This not being const would penalize the non-url path as well a bit.

For Base64Url when the source is not multiple of 4 we need to adjust the buffer size accordingly, the BlockSize value used to fill the whitespace with padding (row 653) so that it could decoded correctly (in case remaining bytes were decodable or valid)

MihaZupan · 2024-06-12T21:19:43Z

@MihuBot fuzz Base64

lewing · 2024-06-19T21:21:39Z

also showing regressions in dotnet/perf-autofiling-issues#36512

and improvements dotnet/perf-autofiling-issues#36643

buyaa-n · 2024-05-17T04:27:12Z

Base64Url encoding doesn't append padding, therefore encoded byte count differs from Base64 encoding:
Source byte count Base64 encoded byte count Base64Url encoded byte count
1 4 2
2 4 3
3 4 4
4 8 6
5 8 7
6 8 8
Base64Url decoding ignore whitespace and padding, therefore decodable byte count differs from Base64 decoding, the exact decoding result also depend on isFinalBlock and if padding involved
Source byte count Base64 decoded byte count Base64Url decoded byte count
1 max 0 max 0
2 max 0 max 1
3 max 0 max 2
4 max 3 max 3
5 max 3 max 3
6 max 3 max 4
7 max 3 max 5
8 max 6 max 6

Approved API shape:

namespaceSystem.Buffers.Text;publicstaticclassBase64Url{publicstaticintGetMaxDecodedLength(intbase64Length);publicstaticintGetEncodedLength(intbytesLength);publicstaticOperationStatusEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintcharsWritten);publicstaticbyte[]EncodeToUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintbytesConsumed,outintcharsWritten,boolisFinalBlock=true);publicstaticintEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination);publicstaticboolTryEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintcharsWritten);publicstaticchar[]EncodeToChars(ReadOnlySpan<byte>source);publicstaticstringEncodeToString(ReadOnlySpan<byte>source);publicstaticboolTryEncodeToUtf8InPlace(Span<byte>buffer,intdataLength,outintbytesWritten);publicstaticOperationStatusDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintcharsConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination);publicstaticboolTryDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromChars(ReadOnlySpan<char>source);publicstaticintDecodeFromUtf8InPlace(Span<byte>buffer);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText,outintdecodedLength);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText,outintdecodedLength);}

Draft PR until below list is completed, for now mainly for checking perf, run tests on different CI legs, and get early feedback.

Handle '%' in decoding as a valid padding character for Base64Url
Doc needs improvement
A few APIs missing unit tests
Fix failures on ARM CI legs

Fixes#1658

ghost · 2024-05-17T04:27:17Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

gfoidl

Just skimmed over it...
I saw your comment, but for checking perf some of my comments may help (a bit).

Co-authored-by: Günther Foidl <gue@korporal.at>

buyaa-n · 2024-05-17T18:20:40Z

/benchmark

buyaa-n · 2024-05-17T18:28:38Z

Just skimmed over it... I saw your comment, but for checking perf some of my comments may help (a bit).

Thank you, sure perf related feedbacks appreciated, I updated the description, any feedbacks are welcome.

pr-benchmarks · 2024-05-17T18:33:47Z

Crank Pull Request Bot

/benchmark <benchmark[,...]> <profile[,...]> <component,[...]> <arguments>

Benchmarks:

micro: .NET Performance micro benchmarks (set filter by adding --variable filter=...; by default filter=*LinqBenchmarks*)
plaintext: TechEmpower Plaintext Scenario - ASP.NET Platform implementation
json: TechEmpower JSON Scenario - ASP.NET Platform implementation
fortunes: TechEmpower Fortunes Scenario - ASP.NET Platform implementation
fortunes_ef: TechEmpower Fortunes Scenario with EF Core - ASP.NET Platform implementation
httpclient: HttpClient Benchmark (change HTTP version by adding e.g. --variable httpVersion=3.0; change response size by adding e.g. --variable responseSize=256; default: HTTP/1.1 GET 8K)

Profiles:

aspnet-citrine-lin: Intel/Linux 28 Cores
aspnet-perf-lin: Intel/Linux 12 Cores
aspnet-citrine-amd: Amd/Linux 48 Cores

Components:

runtime
libs

Arguments: any additional arguments to pass through to crank, e.g. --variable name=value

Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

stephentoub · 2024-06-12T01:14:57Z

+ padding++;
+ }
+
+ if (TBase64Decoder.IsValidPadding(Unsafe.Subtract(ref ptrToLastElement, 1)))


What guarantees this is in bounds?

This is only called with a buffer that has size of 4
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Lines 498 to 499 in e617b8f
constintBlockSize=4;
Span<byte>buffer=stackallocbyte[BlockSize];

runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Line 545 in e617b8f
intpaddingCount=GetPaddingCount<TBase64Decoder>(refbuffer[^1]);

stephentoub · 2024-06-12T01:19:49Z

 }

- end = srcMax - 64;
+ end = srcMax - 32;


Why did this change from 64 to 32?

It was 32 originally, but unintentionally changed with this PR
The method comment also mentions it require 32 byte
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Encoder.cs
Lines 316 to 322 in e617b8f
privatestaticunsafevoidAvx2Encode<TBase64Encoder,T>(refbyte*srcBytes,refT*destBytes,byte*srcEnd,intsourceLength,intdestLength,byte*srcStart,T*destStart)
whereTBase64Encoder:IBase64Encoder<T>
whereT: unmanaged
{
// If we have AVX2 support, pick off 24 bytes at a time for as long as we can.
// But because we read 32 bytes at a time, ensure we have enough room to do a
// full 32-byte read without segfaulting.

stephentoub

Thanks!

MihaZupan · 2024-06-12T21:11:33Z

+ where TBase64Decoder : IBase64Decoder<byte>
 {
-const int BlockSize = 4;
+ int BlockSize = Math.Min(source.Length - (int)sourceIndex, 4);


Would anything bad happen if this were left at a const 4?
This not being const would penalize the non-url path as well a bit.

For Base64Url when the source is not multiple of 4 we need to adjust the buffer size accordingly, the BlockSize value used to fill the whitespace with padding (row 653) so that it could decoded correctly (in case remaining bytes were decodable or valid)

MihaZupan · 2024-06-12T21:19:43Z

@MihuBot fuzz Base64

lewing · 2024-06-19T21:21:39Z

also showing regressions in dotnet/perf-autofiling-issues#36512

and improvements dotnet/perf-autofiling-issues#36643

buyaa-n · 2024-05-17T04:27:12Z

Base64Url encoding doesn't append padding, therefore encoded byte count differs from Base64 encoding:
Source byte count Base64 encoded byte count Base64Url encoded byte count
1 4 2
2 4 3
3 4 4
4 8 6
5 8 7
6 8 8
Base64Url decoding ignore whitespace and padding, therefore decodable byte count differs from Base64 decoding, the exact decoding result also depend on isFinalBlock and if padding involved
Source byte count Base64 decoded byte count Base64Url decoded byte count
1 max 0 max 0
2 max 0 max 1
3 max 0 max 2
4 max 3 max 3
5 max 3 max 3
6 max 3 max 4
7 max 3 max 5
8 max 6 max 6

Approved API shape:

namespaceSystem.Buffers.Text;publicstaticclassBase64Url{publicstaticintGetMaxDecodedLength(intbase64Length);publicstaticintGetEncodedLength(intbytesLength);publicstaticOperationStatusEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintcharsWritten);publicstaticbyte[]EncodeToUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintbytesConsumed,outintcharsWritten,boolisFinalBlock=true);publicstaticintEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination);publicstaticboolTryEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintcharsWritten);publicstaticchar[]EncodeToChars(ReadOnlySpan<byte>source);publicstaticstringEncodeToString(ReadOnlySpan<byte>source);publicstaticboolTryEncodeToUtf8InPlace(Span<byte>buffer,intdataLength,outintbytesWritten);publicstaticOperationStatusDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintcharsConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination);publicstaticboolTryDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromChars(ReadOnlySpan<char>source);publicstaticintDecodeFromUtf8InPlace(Span<byte>buffer);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText,outintdecodedLength);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText,outintdecodedLength);}

Draft PR until below list is completed, for now mainly for checking perf, run tests on different CI legs, and get early feedback.

Handle '%' in decoding as a valid padding character for Base64Url
Doc needs improvement
A few APIs missing unit tests
Fix failures on ARM CI legs

Fixes#1658

ghost · 2024-05-17T04:27:17Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

gfoidl

Just skimmed over it...
I saw your comment, but for checking perf some of my comments may help (a bit).

Co-authored-by: Günther Foidl <gue@korporal.at>

buyaa-n · 2024-05-17T18:20:40Z

/benchmark

buyaa-n · 2024-05-17T18:28:38Z

Just skimmed over it... I saw your comment, but for checking perf some of my comments may help (a bit).

Thank you, sure perf related feedbacks appreciated, I updated the description, any feedbacks are welcome.

pr-benchmarks · 2024-05-17T18:33:47Z

Crank Pull Request Bot

/benchmark <benchmark[,...]> <profile[,...]> <component,[...]> <arguments>

Benchmarks:

micro: .NET Performance micro benchmarks (set filter by adding --variable filter=...; by default filter=*LinqBenchmarks*)
plaintext: TechEmpower Plaintext Scenario - ASP.NET Platform implementation
json: TechEmpower JSON Scenario - ASP.NET Platform implementation
fortunes: TechEmpower Fortunes Scenario - ASP.NET Platform implementation
fortunes_ef: TechEmpower Fortunes Scenario with EF Core - ASP.NET Platform implementation
httpclient: HttpClient Benchmark (change HTTP version by adding e.g. --variable httpVersion=3.0; change response size by adding e.g. --variable responseSize=256; default: HTTP/1.1 GET 8K)

Profiles:

aspnet-citrine-lin: Intel/Linux 28 Cores
aspnet-perf-lin: Intel/Linux 12 Cores
aspnet-citrine-amd: Amd/Linux 48 Cores

Components:

runtime
libs

Arguments: any additional arguments to pass through to crank, e.g. --variable name=value

Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

stephentoub · 2024-06-12T01:14:57Z

+ padding++;
+ }
+
+ if (TBase64Decoder.IsValidPadding(Unsafe.Subtract(ref ptrToLastElement, 1)))


What guarantees this is in bounds?

This is only called with a buffer that has size of 4
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Lines 498 to 499 in e617b8f
constintBlockSize=4;
Span<byte>buffer=stackallocbyte[BlockSize];

runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Line 545 in e617b8f
intpaddingCount=GetPaddingCount<TBase64Decoder>(refbuffer[^1]);

stephentoub · 2024-06-12T01:19:49Z

 }

- end = srcMax - 64;
+ end = srcMax - 32;


Why did this change from 64 to 32?

It was 32 originally, but unintentionally changed with this PR
The method comment also mentions it require 32 byte
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Encoder.cs
Lines 316 to 322 in e617b8f
privatestaticunsafevoidAvx2Encode<TBase64Encoder,T>(refbyte*srcBytes,refT*destBytes,byte*srcEnd,intsourceLength,intdestLength,byte*srcStart,T*destStart)
whereTBase64Encoder:IBase64Encoder<T>
whereT: unmanaged
{
// If we have AVX2 support, pick off 24 bytes at a time for as long as we can.
// But because we read 32 bytes at a time, ensure we have enough room to do a
// full 32-byte read without segfaulting.

stephentoub

Thanks!

MihaZupan · 2024-06-12T21:11:33Z

+ where TBase64Decoder : IBase64Decoder<byte>
 {
-const int BlockSize = 4;
+ int BlockSize = Math.Min(source.Length - (int)sourceIndex, 4);


Would anything bad happen if this were left at a const 4?
This not being const would penalize the non-url path as well a bit.

For Base64Url when the source is not multiple of 4 we need to adjust the buffer size accordingly, the BlockSize value used to fill the whitespace with padding (row 653) so that it could decoded correctly (in case remaining bytes were decodable or valid)

MihaZupan · 2024-06-12T21:19:43Z

@MihuBot fuzz Base64

lewing · 2024-06-19T21:21:39Z

also showing regressions in dotnet/perf-autofiling-issues#36512

and improvements dotnet/perf-autofiling-issues#36643

buyaa-n · 2024-05-17T04:27:12Z

Base64Url encoding doesn't append padding, therefore encoded byte count differs from Base64 encoding:
Source byte count Base64 encoded byte count Base64Url encoded byte count
1 4 2
2 4 3
3 4 4
4 8 6
5 8 7
6 8 8
Base64Url decoding ignore whitespace and padding, therefore decodable byte count differs from Base64 decoding, the exact decoding result also depend on isFinalBlock and if padding involved
Source byte count Base64 decoded byte count Base64Url decoded byte count
1 max 0 max 0
2 max 0 max 1
3 max 0 max 2
4 max 3 max 3
5 max 3 max 3
6 max 3 max 4
7 max 3 max 5
8 max 6 max 6

Approved API shape:

namespaceSystem.Buffers.Text;publicstaticclassBase64Url{publicstaticintGetMaxDecodedLength(intbase64Length);publicstaticintGetEncodedLength(intbytesLength);publicstaticOperationStatusEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintcharsWritten);publicstaticbyte[]EncodeToUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintbytesConsumed,outintcharsWritten,boolisFinalBlock=true);publicstaticintEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination);publicstaticboolTryEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintcharsWritten);publicstaticchar[]EncodeToChars(ReadOnlySpan<byte>source);publicstaticstringEncodeToString(ReadOnlySpan<byte>source);publicstaticboolTryEncodeToUtf8InPlace(Span<byte>buffer,intdataLength,outintbytesWritten);publicstaticOperationStatusDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintcharsConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination);publicstaticboolTryDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromChars(ReadOnlySpan<char>source);publicstaticintDecodeFromUtf8InPlace(Span<byte>buffer);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText,outintdecodedLength);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText,outintdecodedLength);}

Draft PR until below list is completed, for now mainly for checking perf, run tests on different CI legs, and get early feedback.

Handle '%' in decoding as a valid padding character for Base64Url
Doc needs improvement
A few APIs missing unit tests
Fix failures on ARM CI legs

Fixes#1658

ghost · 2024-05-17T04:27:17Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

gfoidl

Just skimmed over it...
I saw your comment, but for checking perf some of my comments may help (a bit).

Co-authored-by: Günther Foidl <gue@korporal.at>

buyaa-n · 2024-05-17T18:20:40Z

/benchmark

buyaa-n · 2024-05-17T18:28:38Z

Just skimmed over it... I saw your comment, but for checking perf some of my comments may help (a bit).

Thank you, sure perf related feedbacks appreciated, I updated the description, any feedbacks are welcome.

pr-benchmarks · 2024-05-17T18:33:47Z

Crank Pull Request Bot

/benchmark <benchmark[,...]> <profile[,...]> <component,[...]> <arguments>

Benchmarks:

micro: .NET Performance micro benchmarks (set filter by adding --variable filter=...; by default filter=*LinqBenchmarks*)
plaintext: TechEmpower Plaintext Scenario - ASP.NET Platform implementation
json: TechEmpower JSON Scenario - ASP.NET Platform implementation
fortunes: TechEmpower Fortunes Scenario - ASP.NET Platform implementation
fortunes_ef: TechEmpower Fortunes Scenario with EF Core - ASP.NET Platform implementation
httpclient: HttpClient Benchmark (change HTTP version by adding e.g. --variable httpVersion=3.0; change response size by adding e.g. --variable responseSize=256; default: HTTP/1.1 GET 8K)

Profiles:

aspnet-citrine-lin: Intel/Linux 28 Cores
aspnet-perf-lin: Intel/Linux 12 Cores
aspnet-citrine-amd: Amd/Linux 48 Cores

Components:

runtime
libs

Arguments: any additional arguments to pass through to crank, e.g. --variable name=value

Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

stephentoub · 2024-06-12T01:14:57Z

+ padding++;
+ }
+
+ if (TBase64Decoder.IsValidPadding(Unsafe.Subtract(ref ptrToLastElement, 1)))


What guarantees this is in bounds?

This is only called with a buffer that has size of 4
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Lines 498 to 499 in e617b8f
constintBlockSize=4;
Span<byte>buffer=stackallocbyte[BlockSize];

runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Line 545 in e617b8f
intpaddingCount=GetPaddingCount<TBase64Decoder>(refbuffer[^1]);

stephentoub · 2024-06-12T01:19:49Z

 }

- end = srcMax - 64;
+ end = srcMax - 32;


Why did this change from 64 to 32?

It was 32 originally, but unintentionally changed with this PR
The method comment also mentions it require 32 byte
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Encoder.cs
Lines 316 to 322 in e617b8f
privatestaticunsafevoidAvx2Encode<TBase64Encoder,T>(refbyte*srcBytes,refT*destBytes,byte*srcEnd,intsourceLength,intdestLength,byte*srcStart,T*destStart)
whereTBase64Encoder:IBase64Encoder<T>
whereT: unmanaged
{
// If we have AVX2 support, pick off 24 bytes at a time for as long as we can.
// But because we read 32 bytes at a time, ensure we have enough room to do a
// full 32-byte read without segfaulting.

stephentoub

Thanks!

MihaZupan · 2024-06-12T21:11:33Z

+ where TBase64Decoder : IBase64Decoder<byte>
 {
-const int BlockSize = 4;
+ int BlockSize = Math.Min(source.Length - (int)sourceIndex, 4);


Would anything bad happen if this were left at a const 4?
This not being const would penalize the non-url path as well a bit.

For Base64Url when the source is not multiple of 4 we need to adjust the buffer size accordingly, the BlockSize value used to fill the whitespace with padding (row 653) so that it could decoded correctly (in case remaining bytes were decodable or valid)

MihaZupan · 2024-06-12T21:19:43Z

@MihuBot fuzz Base64

lewing · 2024-06-19T21:21:39Z

also showing regressions in dotnet/perf-autofiling-issues#36512

and improvements dotnet/perf-autofiling-issues#36643

buyaa-n · 2024-05-17T04:27:12Z

Base64Url encoding doesn't append padding, therefore encoded byte count differs from Base64 encoding:
Source byte count Base64 encoded byte count Base64Url encoded byte count
1 4 2
2 4 3
3 4 4
4 8 6
5 8 7
6 8 8
Base64Url decoding ignore whitespace and padding, therefore decodable byte count differs from Base64 decoding, the exact decoding result also depend on isFinalBlock and if padding involved
Source byte count Base64 decoded byte count Base64Url decoded byte count
1 max 0 max 0
2 max 0 max 1
3 max 0 max 2
4 max 3 max 3
5 max 3 max 3
6 max 3 max 4
7 max 3 max 5
8 max 6 max 6

Approved API shape:

namespaceSystem.Buffers.Text;publicstaticclassBase64Url{publicstaticintGetMaxDecodedLength(intbase64Length);publicstaticintGetEncodedLength(intbytesLength);publicstaticOperationStatusEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintcharsWritten);publicstaticbyte[]EncodeToUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintbytesConsumed,outintcharsWritten,boolisFinalBlock=true);publicstaticintEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination);publicstaticboolTryEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintcharsWritten);publicstaticchar[]EncodeToChars(ReadOnlySpan<byte>source);publicstaticstringEncodeToString(ReadOnlySpan<byte>source);publicstaticboolTryEncodeToUtf8InPlace(Span<byte>buffer,intdataLength,outintbytesWritten);publicstaticOperationStatusDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintcharsConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination);publicstaticboolTryDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromChars(ReadOnlySpan<char>source);publicstaticintDecodeFromUtf8InPlace(Span<byte>buffer);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText,outintdecodedLength);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText,outintdecodedLength);}

Draft PR until below list is completed, for now mainly for checking perf, run tests on different CI legs, and get early feedback.

Handle '%' in decoding as a valid padding character for Base64Url
Doc needs improvement
A few APIs missing unit tests
Fix failures on ARM CI legs

Fixes#1658

ghost · 2024-05-17T04:27:17Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

gfoidl

Just skimmed over it...
I saw your comment, but for checking perf some of my comments may help (a bit).

Co-authored-by: Günther Foidl <gue@korporal.at>

buyaa-n · 2024-05-17T18:20:40Z

/benchmark

buyaa-n · 2024-05-17T18:28:38Z

Just skimmed over it... I saw your comment, but for checking perf some of my comments may help (a bit).

Thank you, sure perf related feedbacks appreciated, I updated the description, any feedbacks are welcome.

pr-benchmarks · 2024-05-17T18:33:47Z

Crank Pull Request Bot

/benchmark <benchmark[,...]> <profile[,...]> <component,[...]> <arguments>

Benchmarks:

micro: .NET Performance micro benchmarks (set filter by adding --variable filter=...; by default filter=*LinqBenchmarks*)
plaintext: TechEmpower Plaintext Scenario - ASP.NET Platform implementation
json: TechEmpower JSON Scenario - ASP.NET Platform implementation
fortunes: TechEmpower Fortunes Scenario - ASP.NET Platform implementation
fortunes_ef: TechEmpower Fortunes Scenario with EF Core - ASP.NET Platform implementation
httpclient: HttpClient Benchmark (change HTTP version by adding e.g. --variable httpVersion=3.0; change response size by adding e.g. --variable responseSize=256; default: HTTP/1.1 GET 8K)

Profiles:

aspnet-citrine-lin: Intel/Linux 28 Cores
aspnet-perf-lin: Intel/Linux 12 Cores
aspnet-citrine-amd: Amd/Linux 48 Cores

Components:

runtime
libs

Arguments: any additional arguments to pass through to crank, e.g. --variable name=value

Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

stephentoub · 2024-06-12T01:14:57Z

+ padding++;
+ }
+
+ if (TBase64Decoder.IsValidPadding(Unsafe.Subtract(ref ptrToLastElement, 1)))


What guarantees this is in bounds?

This is only called with a buffer that has size of 4
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Lines 498 to 499 in e617b8f
constintBlockSize=4;
Span<byte>buffer=stackallocbyte[BlockSize];

runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Line 545 in e617b8f
intpaddingCount=GetPaddingCount<TBase64Decoder>(refbuffer[^1]);

stephentoub · 2024-06-12T01:19:49Z

 }

- end = srcMax - 64;
+ end = srcMax - 32;


Why did this change from 64 to 32?

It was 32 originally, but unintentionally changed with this PR
The method comment also mentions it require 32 byte
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Encoder.cs
Lines 316 to 322 in e617b8f
privatestaticunsafevoidAvx2Encode<TBase64Encoder,T>(refbyte*srcBytes,refT*destBytes,byte*srcEnd,intsourceLength,intdestLength,byte*srcStart,T*destStart)
whereTBase64Encoder:IBase64Encoder<T>
whereT: unmanaged
{
// If we have AVX2 support, pick off 24 bytes at a time for as long as we can.
// But because we read 32 bytes at a time, ensure we have enough room to do a
// full 32-byte read without segfaulting.

stephentoub

Thanks!

MihaZupan · 2024-06-12T21:11:33Z

+ where TBase64Decoder : IBase64Decoder<byte>
 {
-const int BlockSize = 4;
+ int BlockSize = Math.Min(source.Length - (int)sourceIndex, 4);


Would anything bad happen if this were left at a const 4?
This not being const would penalize the non-url path as well a bit.

For Base64Url when the source is not multiple of 4 we need to adjust the buffer size accordingly, the BlockSize value used to fill the whitespace with padding (row 653) so that it could decoded correctly (in case remaining bytes were decodable or valid)

MihaZupan · 2024-06-12T21:19:43Z

@MihuBot fuzz Base64

lewing · 2024-06-19T21:21:39Z

also showing regressions in dotnet/perf-autofiling-issues#36512

and improvements dotnet/perf-autofiling-issues#36643

buyaa-n · 2024-05-17T04:27:12Z

Base64Url encoding doesn't append padding, therefore encoded byte count differs from Base64 encoding:
Source byte count Base64 encoded byte count Base64Url encoded byte count
1 4 2
2 4 3
3 4 4
4 8 6
5 8 7
6 8 8
Base64Url decoding ignore whitespace and padding, therefore decodable byte count differs from Base64 decoding, the exact decoding result also depend on isFinalBlock and if padding involved
Source byte count Base64 decoded byte count Base64Url decoded byte count
1 max 0 max 0
2 max 0 max 1
3 max 0 max 2
4 max 3 max 3
5 max 3 max 3
6 max 3 max 4
7 max 3 max 5
8 max 6 max 6

Approved API shape:

namespaceSystem.Buffers.Text;publicstaticclassBase64Url{publicstaticintGetMaxDecodedLength(intbase64Length);publicstaticintGetEncodedLength(intbytesLength);publicstaticOperationStatusEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintcharsWritten);publicstaticbyte[]EncodeToUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintbytesConsumed,outintcharsWritten,boolisFinalBlock=true);publicstaticintEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination);publicstaticboolTryEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintcharsWritten);publicstaticchar[]EncodeToChars(ReadOnlySpan<byte>source);publicstaticstringEncodeToString(ReadOnlySpan<byte>source);publicstaticboolTryEncodeToUtf8InPlace(Span<byte>buffer,intdataLength,outintbytesWritten);publicstaticOperationStatusDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintcharsConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination);publicstaticboolTryDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromChars(ReadOnlySpan<char>source);publicstaticintDecodeFromUtf8InPlace(Span<byte>buffer);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText,outintdecodedLength);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText,outintdecodedLength);}

Draft PR until below list is completed, for now mainly for checking perf, run tests on different CI legs, and get early feedback.

Handle '%' in decoding as a valid padding character for Base64Url
Doc needs improvement
A few APIs missing unit tests
Fix failures on ARM CI legs

Fixes#1658

ghost · 2024-05-17T04:27:17Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

gfoidl

Just skimmed over it...
I saw your comment, but for checking perf some of my comments may help (a bit).

Co-authored-by: Günther Foidl <gue@korporal.at>

buyaa-n · 2024-05-17T18:20:40Z

/benchmark

buyaa-n · 2024-05-17T18:28:38Z

Just skimmed over it... I saw your comment, but for checking perf some of my comments may help (a bit).

Thank you, sure perf related feedbacks appreciated, I updated the description, any feedbacks are welcome.

pr-benchmarks · 2024-05-17T18:33:47Z

Crank Pull Request Bot

/benchmark <benchmark[,...]> <profile[,...]> <component,[...]> <arguments>

Benchmarks:

micro: .NET Performance micro benchmarks (set filter by adding --variable filter=...; by default filter=*LinqBenchmarks*)
plaintext: TechEmpower Plaintext Scenario - ASP.NET Platform implementation
json: TechEmpower JSON Scenario - ASP.NET Platform implementation
fortunes: TechEmpower Fortunes Scenario - ASP.NET Platform implementation
fortunes_ef: TechEmpower Fortunes Scenario with EF Core - ASP.NET Platform implementation
httpclient: HttpClient Benchmark (change HTTP version by adding e.g. --variable httpVersion=3.0; change response size by adding e.g. --variable responseSize=256; default: HTTP/1.1 GET 8K)

Profiles:

aspnet-citrine-lin: Intel/Linux 28 Cores
aspnet-perf-lin: Intel/Linux 12 Cores
aspnet-citrine-amd: Amd/Linux 48 Cores

Components:

runtime
libs

Arguments: any additional arguments to pass through to crank, e.g. --variable name=value

Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

stephentoub · 2024-06-12T01:14:57Z

+ padding++;
+ }
+
+ if (TBase64Decoder.IsValidPadding(Unsafe.Subtract(ref ptrToLastElement, 1)))


What guarantees this is in bounds?

This is only called with a buffer that has size of 4
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Lines 498 to 499 in e617b8f
constintBlockSize=4;
Span<byte>buffer=stackallocbyte[BlockSize];

runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Line 545 in e617b8f
intpaddingCount=GetPaddingCount<TBase64Decoder>(refbuffer[^1]);

stephentoub · 2024-06-12T01:19:49Z

 }

- end = srcMax - 64;
+ end = srcMax - 32;


Why did this change from 64 to 32?

It was 32 originally, but unintentionally changed with this PR
The method comment also mentions it require 32 byte
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Encoder.cs
Lines 316 to 322 in e617b8f
privatestaticunsafevoidAvx2Encode<TBase64Encoder,T>(refbyte*srcBytes,refT*destBytes,byte*srcEnd,intsourceLength,intdestLength,byte*srcStart,T*destStart)
whereTBase64Encoder:IBase64Encoder<T>
whereT: unmanaged
{
// If we have AVX2 support, pick off 24 bytes at a time for as long as we can.
// But because we read 32 bytes at a time, ensure we have enough room to do a
// full 32-byte read without segfaulting.

stephentoub

Thanks!

MihaZupan · 2024-06-12T21:11:33Z

+ where TBase64Decoder : IBase64Decoder<byte>
 {
-const int BlockSize = 4;
+ int BlockSize = Math.Min(source.Length - (int)sourceIndex, 4);


Would anything bad happen if this were left at a const 4?
This not being const would penalize the non-url path as well a bit.

For Base64Url when the source is not multiple of 4 we need to adjust the buffer size accordingly, the BlockSize value used to fill the whitespace with padding (row 653) so that it could decoded correctly (in case remaining bytes were decodable or valid)

MihaZupan · 2024-06-12T21:19:43Z

@MihuBot fuzz Base64

lewing · 2024-06-19T21:21:39Z

also showing regressions in dotnet/perf-autofiling-issues#36512

and improvements dotnet/perf-autofiling-issues#36643

buyaa-n · 2024-05-17T04:27:12Z

Base64Url encoding doesn't append padding, therefore encoded byte count differs from Base64 encoding:
Source byte count Base64 encoded byte count Base64Url encoded byte count
1 4 2
2 4 3
3 4 4
4 8 6
5 8 7
6 8 8
Base64Url decoding ignore whitespace and padding, therefore decodable byte count differs from Base64 decoding, the exact decoding result also depend on isFinalBlock and if padding involved
Source byte count Base64 decoded byte count Base64Url decoded byte count
1 max 0 max 0
2 max 0 max 1
3 max 0 max 2
4 max 3 max 3
5 max 3 max 3
6 max 3 max 4
7 max 3 max 5
8 max 6 max 6

Approved API shape:

namespaceSystem.Buffers.Text;publicstaticclassBase64Url{publicstaticintGetMaxDecodedLength(intbase64Length);publicstaticintGetEncodedLength(intbytesLength);publicstaticOperationStatusEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintcharsWritten);publicstaticbyte[]EncodeToUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintbytesConsumed,outintcharsWritten,boolisFinalBlock=true);publicstaticintEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination);publicstaticboolTryEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintcharsWritten);publicstaticchar[]EncodeToChars(ReadOnlySpan<byte>source);publicstaticstringEncodeToString(ReadOnlySpan<byte>source);publicstaticboolTryEncodeToUtf8InPlace(Span<byte>buffer,intdataLength,outintbytesWritten);publicstaticOperationStatusDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintcharsConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination);publicstaticboolTryDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromChars(ReadOnlySpan<char>source);publicstaticintDecodeFromUtf8InPlace(Span<byte>buffer);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText,outintdecodedLength);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText,outintdecodedLength);}

Draft PR until below list is completed, for now mainly for checking perf, run tests on different CI legs, and get early feedback.

Handle '%' in decoding as a valid padding character for Base64Url
Doc needs improvement
A few APIs missing unit tests
Fix failures on ARM CI legs

Fixes#1658

ghost · 2024-05-17T04:27:17Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

gfoidl

Just skimmed over it...
I saw your comment, but for checking perf some of my comments may help (a bit).

Co-authored-by: Günther Foidl <gue@korporal.at>

buyaa-n · 2024-05-17T18:20:40Z

/benchmark

buyaa-n · 2024-05-17T18:28:38Z

Just skimmed over it... I saw your comment, but for checking perf some of my comments may help (a bit).

Thank you, sure perf related feedbacks appreciated, I updated the description, any feedbacks are welcome.

pr-benchmarks · 2024-05-17T18:33:47Z

Crank Pull Request Bot

/benchmark <benchmark[,...]> <profile[,...]> <component,[...]> <arguments>

Benchmarks:

micro: .NET Performance micro benchmarks (set filter by adding --variable filter=...; by default filter=*LinqBenchmarks*)
plaintext: TechEmpower Plaintext Scenario - ASP.NET Platform implementation
json: TechEmpower JSON Scenario - ASP.NET Platform implementation
fortunes: TechEmpower Fortunes Scenario - ASP.NET Platform implementation
fortunes_ef: TechEmpower Fortunes Scenario with EF Core - ASP.NET Platform implementation
httpclient: HttpClient Benchmark (change HTTP version by adding e.g. --variable httpVersion=3.0; change response size by adding e.g. --variable responseSize=256; default: HTTP/1.1 GET 8K)

Profiles:

aspnet-citrine-lin: Intel/Linux 28 Cores
aspnet-perf-lin: Intel/Linux 12 Cores
aspnet-citrine-amd: Amd/Linux 48 Cores

Components:

runtime
libs

Arguments: any additional arguments to pass through to crank, e.g. --variable name=value

Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

stephentoub · 2024-06-12T01:14:57Z

+ padding++;
+ }
+
+ if (TBase64Decoder.IsValidPadding(Unsafe.Subtract(ref ptrToLastElement, 1)))


What guarantees this is in bounds?

This is only called with a buffer that has size of 4
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Lines 498 to 499 in e617b8f
constintBlockSize=4;
Span<byte>buffer=stackallocbyte[BlockSize];

runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Line 545 in e617b8f
intpaddingCount=GetPaddingCount<TBase64Decoder>(refbuffer[^1]);

stephentoub · 2024-06-12T01:19:49Z

 }

- end = srcMax - 64;
+ end = srcMax - 32;


Why did this change from 64 to 32?

It was 32 originally, but unintentionally changed with this PR
The method comment also mentions it require 32 byte
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Encoder.cs
Lines 316 to 322 in e617b8f
privatestaticunsafevoidAvx2Encode<TBase64Encoder,T>(refbyte*srcBytes,refT*destBytes,byte*srcEnd,intsourceLength,intdestLength,byte*srcStart,T*destStart)
whereTBase64Encoder:IBase64Encoder<T>
whereT: unmanaged
{
// If we have AVX2 support, pick off 24 bytes at a time for as long as we can.
// But because we read 32 bytes at a time, ensure we have enough room to do a
// full 32-byte read without segfaulting.

stephentoub

Thanks!

MihaZupan · 2024-06-12T21:11:33Z

+ where TBase64Decoder : IBase64Decoder<byte>
 {
-const int BlockSize = 4;
+ int BlockSize = Math.Min(source.Length - (int)sourceIndex, 4);


Would anything bad happen if this were left at a const 4?
This not being const would penalize the non-url path as well a bit.

For Base64Url when the source is not multiple of 4 we need to adjust the buffer size accordingly, the BlockSize value used to fill the whitespace with padding (row 653) so that it could decoded correctly (in case remaining bytes were decodable or valid)

MihaZupan · 2024-06-12T21:19:43Z

@MihuBot fuzz Base64

lewing · 2024-06-19T21:21:39Z

also showing regressions in dotnet/perf-autofiling-issues#36512

and improvements dotnet/perf-autofiling-issues#36643

buyaa-n · 2024-05-17T04:27:12Z

Base64Url encoding doesn't append padding, therefore encoded byte count differs from Base64 encoding:
Source byte count Base64 encoded byte count Base64Url encoded byte count
1 4 2
2 4 3
3 4 4
4 8 6
5 8 7
6 8 8
Base64Url decoding ignore whitespace and padding, therefore decodable byte count differs from Base64 decoding, the exact decoding result also depend on isFinalBlock and if padding involved
Source byte count Base64 decoded byte count Base64Url decoded byte count
1 max 0 max 0
2 max 0 max 1
3 max 0 max 2
4 max 3 max 3
5 max 3 max 3
6 max 3 max 4
7 max 3 max 5
8 max 6 max 6

Approved API shape:

namespaceSystem.Buffers.Text;publicstaticclassBase64Url{publicstaticintGetMaxDecodedLength(intbase64Length);publicstaticintGetEncodedLength(intbytesLength);publicstaticOperationStatusEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryEncodeToUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintcharsWritten);publicstaticbyte[]EncodeToUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintbytesConsumed,outintcharsWritten,boolisFinalBlock=true);publicstaticintEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination);publicstaticboolTryEncodeToChars(ReadOnlySpan<byte>source,Span<char>destination,outintcharsWritten);publicstaticchar[]EncodeToChars(ReadOnlySpan<byte>source);publicstaticstringEncodeToString(ReadOnlySpan<byte>source);publicstaticboolTryEncodeToUtf8InPlace(Span<byte>buffer,intdataLength,outintbytesWritten);publicstaticOperationStatusDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination);publicstaticboolTryDecodeFromUtf8(ReadOnlySpan<byte>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromUtf8(ReadOnlySpan<byte>source);publicstaticOperationStatusDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintcharsConsumed,outintbytesWritten,boolisFinalBlock=true);publicstaticintDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination);publicstaticboolTryDecodeFromChars(ReadOnlySpan<char>source,Span<byte>destination,outintbytesWritten);publicstaticbyte[]DecodeFromChars(ReadOnlySpan<char>source);publicstaticintDecodeFromUtf8InPlace(Span<byte>buffer);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText);publicstaticboolIsValid(ReadOnlySpan<char>base64UrlText,outintdecodedLength);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText);publicstaticboolIsValid(ReadOnlySpan<byte>utf8Base64UrlText,outintdecodedLength);}

Draft PR until below list is completed, for now mainly for checking perf, run tests on different CI legs, and get early feedback.

Handle '%' in decoding as a valid padding character for Base64Url
Doc needs improvement
A few APIs missing unit tests
Fix failures on ARM CI legs

Fixes#1658

ghost · 2024-05-17T04:27:17Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

gfoidl

Just skimmed over it...
I saw your comment, but for checking perf some of my comments may help (a bit).

Co-authored-by: Günther Foidl <gue@korporal.at>

buyaa-n · 2024-05-17T18:20:40Z

/benchmark

buyaa-n · 2024-05-17T18:28:38Z

Just skimmed over it... I saw your comment, but for checking perf some of my comments may help (a bit).

Thank you, sure perf related feedbacks appreciated, I updated the description, any feedbacks are welcome.

pr-benchmarks · 2024-05-17T18:33:47Z

Crank Pull Request Bot

/benchmark <benchmark[,...]> <profile[,...]> <component,[...]> <arguments>

Benchmarks:

micro: .NET Performance micro benchmarks (set filter by adding --variable filter=...; by default filter=*LinqBenchmarks*)
plaintext: TechEmpower Plaintext Scenario - ASP.NET Platform implementation
json: TechEmpower JSON Scenario - ASP.NET Platform implementation
fortunes: TechEmpower Fortunes Scenario - ASP.NET Platform implementation
fortunes_ef: TechEmpower Fortunes Scenario with EF Core - ASP.NET Platform implementation
httpclient: HttpClient Benchmark (change HTTP version by adding e.g. --variable httpVersion=3.0; change response size by adding e.g. --variable responseSize=256; default: HTTP/1.1 GET 8K)

Profiles:

aspnet-citrine-lin: Intel/Linux 28 Cores
aspnet-perf-lin: Intel/Linux 12 Cores
aspnet-citrine-amd: Amd/Linux 48 Cores

Components:

runtime
libs

Arguments: any additional arguments to pass through to crank, e.g. --variable name=value

Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

stephentoub · 2024-06-12T01:14:57Z

+ padding++;
+ }
+
+ if (TBase64Decoder.IsValidPadding(Unsafe.Subtract(ref ptrToLastElement, 1)))


What guarantees this is in bounds?

This is only called with a buffer that has size of 4
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Lines 498 to 499 in e617b8f
constintBlockSize=4;
Span<byte>buffer=stackallocbyte[BlockSize];

runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs
Line 545 in e617b8f
intpaddingCount=GetPaddingCount<TBase64Decoder>(refbuffer[^1]);

stephentoub · 2024-06-12T01:19:49Z

 }

- end = srcMax - 64;
+ end = srcMax - 32;


Why did this change from 64 to 32?

It was 32 originally, but unintentionally changed with this PR
The method comment also mentions it require 32 byte
runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Encoder.cs
Lines 316 to 322 in e617b8f
privatestaticunsafevoidAvx2Encode<TBase64Encoder,T>(refbyte*srcBytes,refT*destBytes,byte*srcEnd,intsourceLength,intdestLength,byte*srcStart,T*destStart)
whereTBase64Encoder:IBase64Encoder<T>
whereT: unmanaged
{
// If we have AVX2 support, pick off 24 bytes at a time for as long as we can.
// But because we read 32 bytes at a time, ensure we have enough room to do a
// full 32-byte read without segfaulting.

stephentoub

Thanks!

MihaZupan · 2024-06-12T21:11:33Z

+ where TBase64Decoder : IBase64Decoder<byte>
 {
-const int BlockSize = 4;
+ int BlockSize = Math.Min(source.Length - (int)sourceIndex, 4);


Would anything bad happen if this were left at a const 4?
This not being const would penalize the non-url path as well a bit.

For Base64Url when the source is not multiple of 4 we need to adjust the buffer size accordingly, the BlockSize value used to fill the whitespace with padding (row 653) so that it could decoded correctly (in case remaining bytes were decodable or valid)

MihaZupan · 2024-06-12T21:19:43Z

@MihuBot fuzz Base64

lewing · 2024-06-19T21:21:39Z

also showing regressions in dotnet/perf-autofiling-issues#36512

and improvements dotnet/perf-autofiling-issues#36643

buyaa-n added 8 commits April 24, 2024 09:13

Base64Url encoding, validation impelementation
763212a

Validation related updates
ea764bb

Try fix perf regression in vectorized methods
4489fcb

Add decoder implementation and unit tests
238b763

Share code in place decoding code
f378eb5

Add span<char> oveloads with vectorization
3f6ea88

Generalize Span char/byte implementations
2fe5d49

Move ref update to runtime and other cleanup
febb4d9

ghost added area-System.Memory new-api-needs-documentation labels May 17, 2024

dotnet-policy-serviceBot assigned buyaa-n May 17, 2024

Merge branch 'main' of github.com:dotnet/runtime into base64url
d40034f

gfoidl reviewed May 17, 2024
View reviewed changes

Generalize the AdvSimd.Arm64 vectorization added recently
0994bc8

stephentoub reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Memory/ref/System.Memory.Forwards.cs Outdated

buyaa-nand others added 2 commits May 17, 2024 11:00

Apply suggestions from code review
492694b
Co-authored-by: Günther Foidl <gue@korporal.at>

Apply some feedback
c45f58c

This comment was marked as duplicate.
Sign in to view

buyaa-n requested review from MihaZupan, bartonjs, gfoidl, stephentoub and tannergooding May 17, 2024 19:02

jkotas reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems

This was referenced Jun 11, 2024
Test failure: GC\\Features\\HeapExpansion\\Finalizer\\Finalizer.cmd #102706
Closed
CI timeout in System.Data.Tests.DataTableReadXmlSchemaTest.XsdSchemaDeserializationIgnoresLocale #103062
Closed
[browser][ST] Wasm tests timeout #103246
Closed

buyaa-nand others added 2 commits June 10, 2024 21:23

Apply suggestions from code review
dea9006
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

Rename utf8 -> source/destintion
e617b8f

build-analysisBot mentioned this pull request Jun 11, 2024
The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008
Open
3 tasks

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs Outdated

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Url/Base64UrlValidator.cs Outdated

stephentoub approved these changes Jun 12, 2024
View reviewed changes

Apply feedbacks
c3ff207

MihaZupan reviewed Jun 12, 2024
View reviewed changes

MihuBot mentioned this pull request Jun 12, 2024
[Fuzzing] [buyaa-n] Add Base64url encoding/decoding MihuBot/runtime-utils#407
Closed

Apply feedback
154b400

build-analysisBot mentioned this pull request Jun 13, 2024
System.Net.Security.Tests.SslStreamTlsResumeTests failing #103354
Closed

Apply left out feedbacks
4ccdde8

build-analysisBot mentioned this pull request Jun 14, 2024
SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds failed in CI #103449
Closed

buyaa-n merged commit 48c8805 into dotnet:mainJun 14, 2024

buyaa-n deleted the base64url branch June 14, 2024 02:06

matouskozak mentioned this pull request Jun 18, 2024
[mono][Perf] Perf_Base64 Regressions on 6/14/2024 3:42:32 AM #103653
Closed

lewing mentioned this pull request Jun 19, 2024
[Perf] Linux/x64: 13 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36512
Open

lewing mentioned this pull request Jun 21, 2024
[Perf] Linux/x64: 10 Improvements on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36643
Open

This was referenced Jun 26, 2024
[Perf] Linux/x64: 16 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36646
Open
[wasm][perf] Tracking #96444
Open

tmds mentioned this pull request Jul 2, 2024
Base64url tests fail on s390x (big endian) #104283
Closed

buyaa-n mentioned this pull request Jul 2, 2024
Remove unnecessary endianness dependent logic #104332
Merged

github-actionsBot locked and limited conversation to collaborators Jul 23, 2024

buyaa-n added 8 commits April 24, 2024 09:13

Base64Url encoding, validation impelementation
763212a

Validation related updates
ea764bb

Try fix perf regression in vectorized methods
4489fcb

Add decoder implementation and unit tests
238b763

Share code in place decoding code
f378eb5

Add span<char> oveloads with vectorization
3f6ea88

Generalize Span char/byte implementations
2fe5d49

Move ref update to runtime and other cleanup
febb4d9

ghost added area-System.Memory new-api-needs-documentation labels May 17, 2024

dotnet-policy-serviceBot assigned buyaa-n May 17, 2024

Merge branch 'main' of github.com:dotnet/runtime into base64url
d40034f

gfoidl reviewed May 17, 2024
View reviewed changes

Generalize the AdvSimd.Arm64 vectorization added recently
0994bc8

stephentoub reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Memory/ref/System.Memory.Forwards.cs Outdated

buyaa-nand others added 2 commits May 17, 2024 11:00

Apply suggestions from code review
492694b
Co-authored-by: Günther Foidl <gue@korporal.at>

Apply some feedback
c45f58c

This comment was marked as duplicate.
Sign in to view

buyaa-n requested review from MihaZupan, bartonjs, gfoidl, stephentoub and tannergooding May 17, 2024 19:02

jkotas reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems

This was referenced Jun 11, 2024
Test failure: GC\\Features\\HeapExpansion\\Finalizer\\Finalizer.cmd #102706
Closed
CI timeout in System.Data.Tests.DataTableReadXmlSchemaTest.XsdSchemaDeserializationIgnoresLocale #103062
Closed
[browser][ST] Wasm tests timeout #103246
Closed

buyaa-nand others added 2 commits June 10, 2024 21:23

Apply suggestions from code review
dea9006
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

Rename utf8 -> source/destintion
e617b8f

build-analysisBot mentioned this pull request Jun 11, 2024
The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008
Open
3 tasks

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs Outdated

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Url/Base64UrlValidator.cs Outdated

stephentoub approved these changes Jun 12, 2024
View reviewed changes

Apply feedbacks
c3ff207

MihaZupan reviewed Jun 12, 2024
View reviewed changes

MihuBot mentioned this pull request Jun 12, 2024
[Fuzzing] [buyaa-n] Add Base64url encoding/decoding MihuBot/runtime-utils#407
Closed

Apply feedback
154b400

build-analysisBot mentioned this pull request Jun 13, 2024
System.Net.Security.Tests.SslStreamTlsResumeTests failing #103354
Closed

Apply left out feedbacks
4ccdde8

build-analysisBot mentioned this pull request Jun 14, 2024
SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds failed in CI #103449
Closed

buyaa-n merged commit 48c8805 into dotnet:mainJun 14, 2024

buyaa-n deleted the base64url branch June 14, 2024 02:06

matouskozak mentioned this pull request Jun 18, 2024
[mono][Perf] Perf_Base64 Regressions on 6/14/2024 3:42:32 AM #103653
Closed

lewing mentioned this pull request Jun 19, 2024
[Perf] Linux/x64: 13 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36512
Open

lewing mentioned this pull request Jun 21, 2024
[Perf] Linux/x64: 10 Improvements on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36643
Open

This was referenced Jun 26, 2024
[Perf] Linux/x64: 16 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36646
Open
[wasm][perf] Tracking #96444
Open

tmds mentioned this pull request Jul 2, 2024
Base64url tests fail on s390x (big endian) #104283
Closed

buyaa-n mentioned this pull request Jul 2, 2024
Remove unnecessary endianness dependent logic #104332
Merged

github-actionsBot locked and limited conversation to collaborators Jul 23, 2024

buyaa-n added 8 commits April 24, 2024 09:13

Base64Url encoding, validation impelementation
763212a

Validation related updates
ea764bb

Try fix perf regression in vectorized methods
4489fcb

Add decoder implementation and unit tests
238b763

Share code in place decoding code
f378eb5

Add span<char> oveloads with vectorization
3f6ea88

Generalize Span char/byte implementations
2fe5d49

Move ref update to runtime and other cleanup
febb4d9

ghost added area-System.Memory new-api-needs-documentation labels May 17, 2024

dotnet-policy-serviceBot assigned buyaa-n May 17, 2024

Merge branch 'main' of github.com:dotnet/runtime into base64url
d40034f

gfoidl reviewed May 17, 2024
View reviewed changes

Generalize the AdvSimd.Arm64 vectorization added recently
0994bc8

stephentoub reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Memory/ref/System.Memory.Forwards.cs Outdated

buyaa-nand others added 2 commits May 17, 2024 11:00

Apply suggestions from code review
492694b
Co-authored-by: Günther Foidl <gue@korporal.at>

Apply some feedback
c45f58c

This comment was marked as duplicate.
Sign in to view

buyaa-n requested review from MihaZupan, bartonjs, gfoidl, stephentoub and tannergooding May 17, 2024 19:02

jkotas reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems

This was referenced Jun 11, 2024
Test failure: GC\\Features\\HeapExpansion\\Finalizer\\Finalizer.cmd #102706
Closed
CI timeout in System.Data.Tests.DataTableReadXmlSchemaTest.XsdSchemaDeserializationIgnoresLocale #103062
Closed
[browser][ST] Wasm tests timeout #103246
Closed

buyaa-nand others added 2 commits June 10, 2024 21:23

Apply suggestions from code review
dea9006
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

Rename utf8 -> source/destintion
e617b8f

build-analysisBot mentioned this pull request Jun 11, 2024
The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008
Open
3 tasks

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs Outdated

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Url/Base64UrlValidator.cs Outdated

stephentoub approved these changes Jun 12, 2024
View reviewed changes

Apply feedbacks
c3ff207

MihaZupan reviewed Jun 12, 2024
View reviewed changes

MihuBot mentioned this pull request Jun 12, 2024
[Fuzzing] [buyaa-n] Add Base64url encoding/decoding MihuBot/runtime-utils#407
Closed

Apply feedback
154b400

build-analysisBot mentioned this pull request Jun 13, 2024
System.Net.Security.Tests.SslStreamTlsResumeTests failing #103354
Closed

Apply left out feedbacks
4ccdde8

build-analysisBot mentioned this pull request Jun 14, 2024
SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds failed in CI #103449
Closed

buyaa-n merged commit 48c8805 into dotnet:mainJun 14, 2024

buyaa-n deleted the base64url branch June 14, 2024 02:06

matouskozak mentioned this pull request Jun 18, 2024
[mono][Perf] Perf_Base64 Regressions on 6/14/2024 3:42:32 AM #103653
Closed

lewing mentioned this pull request Jun 19, 2024
[Perf] Linux/x64: 13 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36512
Open

lewing mentioned this pull request Jun 21, 2024
[Perf] Linux/x64: 10 Improvements on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36643
Open

This was referenced Jun 26, 2024
[Perf] Linux/x64: 16 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36646
Open
[wasm][perf] Tracking #96444
Open

tmds mentioned this pull request Jul 2, 2024
Base64url tests fail on s390x (big endian) #104283
Closed

buyaa-n mentioned this pull request Jul 2, 2024
Remove unnecessary endianness dependent logic #104332
Merged

github-actionsBot locked and limited conversation to collaborators Jul 23, 2024

buyaa-n added 8 commits April 24, 2024 09:13

Base64Url encoding, validation impelementation
763212a

Validation related updates
ea764bb

Try fix perf regression in vectorized methods
4489fcb

Add decoder implementation and unit tests
238b763

Share code in place decoding code
f378eb5

Add span<char> oveloads with vectorization
3f6ea88

Generalize Span char/byte implementations
2fe5d49

Move ref update to runtime and other cleanup
febb4d9

ghost added area-System.Memory new-api-needs-documentation labels May 17, 2024

dotnet-policy-serviceBot assigned buyaa-n May 17, 2024

Merge branch 'main' of github.com:dotnet/runtime into base64url
d40034f

gfoidl reviewed May 17, 2024
View reviewed changes

Generalize the AdvSimd.Arm64 vectorization added recently
0994bc8

stephentoub reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Memory/ref/System.Memory.Forwards.cs Outdated

buyaa-nand others added 2 commits May 17, 2024 11:00

Apply suggestions from code review
492694b
Co-authored-by: Günther Foidl <gue@korporal.at>

Apply some feedback
c45f58c

This comment was marked as duplicate.
Sign in to view

buyaa-n requested review from MihaZupan, bartonjs, gfoidl, stephentoub and tannergooding May 17, 2024 19:02

jkotas reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems

This was referenced Jun 11, 2024
Test failure: GC\\Features\\HeapExpansion\\Finalizer\\Finalizer.cmd #102706
Closed
CI timeout in System.Data.Tests.DataTableReadXmlSchemaTest.XsdSchemaDeserializationIgnoresLocale #103062
Closed
[browser][ST] Wasm tests timeout #103246
Closed

buyaa-nand others added 2 commits June 10, 2024 21:23

Apply suggestions from code review
dea9006
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

Rename utf8 -> source/destintion
e617b8f

build-analysisBot mentioned this pull request Jun 11, 2024
The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008
Open
3 tasks

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs Outdated

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Url/Base64UrlValidator.cs Outdated

stephentoub approved these changes Jun 12, 2024
View reviewed changes

Apply feedbacks
c3ff207

MihaZupan reviewed Jun 12, 2024
View reviewed changes

MihuBot mentioned this pull request Jun 12, 2024
[Fuzzing] [buyaa-n] Add Base64url encoding/decoding MihuBot/runtime-utils#407
Closed

Apply feedback
154b400

build-analysisBot mentioned this pull request Jun 13, 2024
System.Net.Security.Tests.SslStreamTlsResumeTests failing #103354
Closed

Apply left out feedbacks
4ccdde8

build-analysisBot mentioned this pull request Jun 14, 2024
SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds failed in CI #103449
Closed

buyaa-n merged commit 48c8805 into dotnet:mainJun 14, 2024

buyaa-n deleted the base64url branch June 14, 2024 02:06

matouskozak mentioned this pull request Jun 18, 2024
[mono][Perf] Perf_Base64 Regressions on 6/14/2024 3:42:32 AM #103653
Closed

lewing mentioned this pull request Jun 19, 2024
[Perf] Linux/x64: 13 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36512
Open

lewing mentioned this pull request Jun 21, 2024
[Perf] Linux/x64: 10 Improvements on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36643
Open

This was referenced Jun 26, 2024
[Perf] Linux/x64: 16 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36646
Open
[wasm][perf] Tracking #96444
Open

tmds mentioned this pull request Jul 2, 2024
Base64url tests fail on s390x (big endian) #104283
Closed

buyaa-n mentioned this pull request Jul 2, 2024
Remove unnecessary endianness dependent logic #104332
Merged

github-actionsBot locked and limited conversation to collaborators Jul 23, 2024

buyaa-n added 8 commits April 24, 2024 09:13

Base64Url encoding, validation impelementation
763212a

Validation related updates
ea764bb

Try fix perf regression in vectorized methods
4489fcb

Add decoder implementation and unit tests
238b763

Share code in place decoding code
f378eb5

Add span<char> oveloads with vectorization
3f6ea88

Generalize Span char/byte implementations
2fe5d49

Move ref update to runtime and other cleanup
febb4d9

ghost added area-System.Memory new-api-needs-documentation labels May 17, 2024

dotnet-policy-serviceBot assigned buyaa-n May 17, 2024

Merge branch 'main' of github.com:dotnet/runtime into base64url
d40034f

gfoidl reviewed May 17, 2024
View reviewed changes

Generalize the AdvSimd.Arm64 vectorization added recently
0994bc8

stephentoub reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Memory/ref/System.Memory.Forwards.cs Outdated

buyaa-nand others added 2 commits May 17, 2024 11:00

Apply suggestions from code review
492694b
Co-authored-by: Günther Foidl <gue@korporal.at>

Apply some feedback
c45f58c

This comment was marked as duplicate.
Sign in to view

buyaa-n requested review from MihaZupan, bartonjs, gfoidl, stephentoub and tannergooding May 17, 2024 19:02

jkotas reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems

This was referenced Jun 11, 2024
Test failure: GC\\Features\\HeapExpansion\\Finalizer\\Finalizer.cmd #102706
Closed
CI timeout in System.Data.Tests.DataTableReadXmlSchemaTest.XsdSchemaDeserializationIgnoresLocale #103062
Closed
[browser][ST] Wasm tests timeout #103246
Closed

buyaa-nand others added 2 commits June 10, 2024 21:23

Apply suggestions from code review
dea9006
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

Rename utf8 -> source/destintion
e617b8f

build-analysisBot mentioned this pull request Jun 11, 2024
The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008
Open
3 tasks

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs Outdated

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Url/Base64UrlValidator.cs Outdated

stephentoub approved these changes Jun 12, 2024
View reviewed changes

Apply feedbacks
c3ff207

MihaZupan reviewed Jun 12, 2024
View reviewed changes

MihuBot mentioned this pull request Jun 12, 2024
[Fuzzing] [buyaa-n] Add Base64url encoding/decoding MihuBot/runtime-utils#407
Closed

Apply feedback
154b400

build-analysisBot mentioned this pull request Jun 13, 2024
System.Net.Security.Tests.SslStreamTlsResumeTests failing #103354
Closed

Apply left out feedbacks
4ccdde8

build-analysisBot mentioned this pull request Jun 14, 2024
SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds failed in CI #103449
Closed

buyaa-n merged commit 48c8805 into dotnet:mainJun 14, 2024

buyaa-n deleted the base64url branch June 14, 2024 02:06

matouskozak mentioned this pull request Jun 18, 2024
[mono][Perf] Perf_Base64 Regressions on 6/14/2024 3:42:32 AM #103653
Closed

lewing mentioned this pull request Jun 19, 2024
[Perf] Linux/x64: 13 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36512
Open

lewing mentioned this pull request Jun 21, 2024
[Perf] Linux/x64: 10 Improvements on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36643
Open

This was referenced Jun 26, 2024
[Perf] Linux/x64: 16 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36646
Open
[wasm][perf] Tracking #96444
Open

tmds mentioned this pull request Jul 2, 2024
Base64url tests fail on s390x (big endian) #104283
Closed

buyaa-n mentioned this pull request Jul 2, 2024
Remove unnecessary endianness dependent logic #104332
Merged

github-actionsBot locked and limited conversation to collaborators Jul 23, 2024

buyaa-n added 8 commits April 24, 2024 09:13

Base64Url encoding, validation impelementation
763212a

Validation related updates
ea764bb

Try fix perf regression in vectorized methods
4489fcb

Add decoder implementation and unit tests
238b763

Share code in place decoding code
f378eb5

Add span<char> oveloads with vectorization
3f6ea88

Generalize Span char/byte implementations
2fe5d49

Move ref update to runtime and other cleanup
febb4d9

ghost added area-System.Memory new-api-needs-documentation labels May 17, 2024

dotnet-policy-serviceBot assigned buyaa-n May 17, 2024

Merge branch 'main' of github.com:dotnet/runtime into base64url
d40034f

gfoidl reviewed May 17, 2024
View reviewed changes

Generalize the AdvSimd.Arm64 vectorization added recently
0994bc8

stephentoub reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Memory/ref/System.Memory.Forwards.cs Outdated

buyaa-nand others added 2 commits May 17, 2024 11:00

Apply suggestions from code review
492694b
Co-authored-by: Günther Foidl <gue@korporal.at>

Apply some feedback
c45f58c

This comment was marked as duplicate.
Sign in to view

buyaa-n requested review from MihaZupan, bartonjs, gfoidl, stephentoub and tannergooding May 17, 2024 19:02

jkotas reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems

This was referenced Jun 11, 2024
Test failure: GC\\Features\\HeapExpansion\\Finalizer\\Finalizer.cmd #102706
Closed
CI timeout in System.Data.Tests.DataTableReadXmlSchemaTest.XsdSchemaDeserializationIgnoresLocale #103062
Closed
[browser][ST] Wasm tests timeout #103246
Closed

buyaa-nand others added 2 commits June 10, 2024 21:23

Apply suggestions from code review
dea9006
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

Rename utf8 -> source/destintion
e617b8f

build-analysisBot mentioned this pull request Jun 11, 2024
The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008
Open
3 tasks

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs Outdated

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Url/Base64UrlValidator.cs Outdated

stephentoub approved these changes Jun 12, 2024
View reviewed changes

Apply feedbacks
c3ff207

MihaZupan reviewed Jun 12, 2024
View reviewed changes

MihuBot mentioned this pull request Jun 12, 2024
[Fuzzing] [buyaa-n] Add Base64url encoding/decoding MihuBot/runtime-utils#407
Closed

Apply feedback
154b400

build-analysisBot mentioned this pull request Jun 13, 2024
System.Net.Security.Tests.SslStreamTlsResumeTests failing #103354
Closed

Apply left out feedbacks
4ccdde8

build-analysisBot mentioned this pull request Jun 14, 2024
SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds failed in CI #103449
Closed

buyaa-n merged commit 48c8805 into dotnet:mainJun 14, 2024

buyaa-n deleted the base64url branch June 14, 2024 02:06

matouskozak mentioned this pull request Jun 18, 2024
[mono][Perf] Perf_Base64 Regressions on 6/14/2024 3:42:32 AM #103653
Closed

lewing mentioned this pull request Jun 19, 2024
[Perf] Linux/x64: 13 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36512
Open

lewing mentioned this pull request Jun 21, 2024
[Perf] Linux/x64: 10 Improvements on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36643
Open

This was referenced Jun 26, 2024
[Perf] Linux/x64: 16 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36646
Open
[wasm][perf] Tracking #96444
Open

tmds mentioned this pull request Jul 2, 2024
Base64url tests fail on s390x (big endian) #104283
Closed

buyaa-n mentioned this pull request Jul 2, 2024
Remove unnecessary endianness dependent logic #104332
Merged

github-actionsBot locked and limited conversation to collaborators Jul 23, 2024

buyaa-n added 8 commits April 24, 2024 09:13

Base64Url encoding, validation impelementation
763212a

Validation related updates
ea764bb

Try fix perf regression in vectorized methods
4489fcb

Add decoder implementation and unit tests
238b763

Share code in place decoding code
f378eb5

Add span<char> oveloads with vectorization
3f6ea88

Generalize Span char/byte implementations
2fe5d49

Move ref update to runtime and other cleanup
febb4d9

ghost added area-System.Memory new-api-needs-documentation labels May 17, 2024

dotnet-policy-serviceBot assigned buyaa-n May 17, 2024

Merge branch 'main' of github.com:dotnet/runtime into base64url
d40034f

gfoidl reviewed May 17, 2024
View reviewed changes

Generalize the AdvSimd.Arm64 vectorization added recently
0994bc8

stephentoub reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Memory/ref/System.Memory.Forwards.cs Outdated

buyaa-nand others added 2 commits May 17, 2024 11:00

Apply suggestions from code review
492694b
Co-authored-by: Günther Foidl <gue@korporal.at>

Apply some feedback
c45f58c

This comment was marked as duplicate.
Sign in to view

buyaa-n requested review from MihaZupan, bartonjs, gfoidl, stephentoub and tannergooding May 17, 2024 19:02

jkotas reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems

This was referenced Jun 11, 2024
Test failure: GC\\Features\\HeapExpansion\\Finalizer\\Finalizer.cmd #102706
Closed
CI timeout in System.Data.Tests.DataTableReadXmlSchemaTest.XsdSchemaDeserializationIgnoresLocale #103062
Closed
[browser][ST] Wasm tests timeout #103246
Closed

buyaa-nand others added 2 commits June 10, 2024 21:23

Apply suggestions from code review
dea9006
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

Rename utf8 -> source/destintion
e617b8f

build-analysisBot mentioned this pull request Jun 11, 2024
The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008
Open
3 tasks

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs Outdated

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Url/Base64UrlValidator.cs Outdated

stephentoub approved these changes Jun 12, 2024
View reviewed changes

Apply feedbacks
c3ff207

MihaZupan reviewed Jun 12, 2024
View reviewed changes

MihuBot mentioned this pull request Jun 12, 2024
[Fuzzing] [buyaa-n] Add Base64url encoding/decoding MihuBot/runtime-utils#407
Closed

Apply feedback
154b400

build-analysisBot mentioned this pull request Jun 13, 2024
System.Net.Security.Tests.SslStreamTlsResumeTests failing #103354
Closed

Apply left out feedbacks
4ccdde8

build-analysisBot mentioned this pull request Jun 14, 2024
SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds failed in CI #103449
Closed

buyaa-n merged commit 48c8805 into dotnet:mainJun 14, 2024

buyaa-n deleted the base64url branch June 14, 2024 02:06

matouskozak mentioned this pull request Jun 18, 2024
[mono][Perf] Perf_Base64 Regressions on 6/14/2024 3:42:32 AM #103653
Closed

lewing mentioned this pull request Jun 19, 2024
[Perf] Linux/x64: 13 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36512
Open

lewing mentioned this pull request Jun 21, 2024
[Perf] Linux/x64: 10 Improvements on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36643
Open

This was referenced Jun 26, 2024
[Perf] Linux/x64: 16 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36646
Open
[wasm][perf] Tracking #96444
Open

tmds mentioned this pull request Jul 2, 2024
Base64url tests fail on s390x (big endian) #104283
Closed

buyaa-n mentioned this pull request Jul 2, 2024
Remove unnecessary endianness dependent logic #104332
Merged

github-actionsBot locked and limited conversation to collaborators Jul 23, 2024

buyaa-n added 8 commits April 24, 2024 09:13

Base64Url encoding, validation impelementation
763212a

Validation related updates
ea764bb

Try fix perf regression in vectorized methods
4489fcb

Add decoder implementation and unit tests
238b763

Share code in place decoding code
f378eb5

Add span<char> oveloads with vectorization
3f6ea88

Generalize Span char/byte implementations
2fe5d49

Move ref update to runtime and other cleanup
febb4d9

ghost added area-System.Memory new-api-needs-documentation labels May 17, 2024

dotnet-policy-serviceBot assigned buyaa-n May 17, 2024

Merge branch 'main' of github.com:dotnet/runtime into base64url
d40034f

gfoidl reviewed May 17, 2024
View reviewed changes

Generalize the AdvSimd.Arm64 vectorization added recently
0994bc8

stephentoub reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Memory/ref/System.Memory.Forwards.cs Outdated

buyaa-nand others added 2 commits May 17, 2024 11:00

Apply suggestions from code review
492694b
Co-authored-by: Günther Foidl <gue@korporal.at>

Apply some feedback
c45f58c

This comment was marked as duplicate.
Sign in to view

buyaa-n requested review from MihaZupan, bartonjs, gfoidl, stephentoub and tannergooding May 17, 2024 19:02

jkotas reviewed May 17, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems

This was referenced Jun 11, 2024
Test failure: GC\\Features\\HeapExpansion\\Finalizer\\Finalizer.cmd #102706
Closed
CI timeout in System.Data.Tests.DataTableReadXmlSchemaTest.XsdSchemaDeserializationIgnoresLocale #103062
Closed
[browser][ST] Wasm tests timeout #103246
Closed

buyaa-nand others added 2 commits June 10, 2024 21:23

Apply suggestions from code review
dea9006
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>

Rename utf8 -> source/destintion
e617b8f

build-analysisBot mentioned this pull request Jun 11, 2024
The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008
Open
3 tasks

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Decoder.cs Outdated

stephentoub reviewed Jun 12, 2024
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Url/Base64UrlValidator.cs Outdated

stephentoub approved these changes Jun 12, 2024
View reviewed changes

Apply feedbacks
c3ff207

MihaZupan reviewed Jun 12, 2024
View reviewed changes

MihuBot mentioned this pull request Jun 12, 2024
[Fuzzing] [buyaa-n] Add Base64url encoding/decoding MihuBot/runtime-utils#407
Closed

Apply feedback
154b400

build-analysisBot mentioned this pull request Jun 13, 2024
System.Net.Security.Tests.SslStreamTlsResumeTests failing #103354
Closed

Apply left out feedbacks
4ccdde8

build-analysisBot mentioned this pull request Jun 14, 2024
SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds failed in CI #103449
Closed

buyaa-n merged commit 48c8805 into dotnet:mainJun 14, 2024

buyaa-n deleted the base64url branch June 14, 2024 02:06

matouskozak mentioned this pull request Jun 18, 2024
[mono][Perf] Perf_Base64 Regressions on 6/14/2024 3:42:32 AM #103653
Closed

lewing mentioned this pull request Jun 19, 2024
[Perf] Linux/x64: 13 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36512
Open

lewing mentioned this pull request Jun 21, 2024
[Perf] Linux/x64: 10 Improvements on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36643
Open

This was referenced Jun 26, 2024
[Perf] Linux/x64: 16 Regressions on 6/14/2024 3:42:32 AM dotnet/perf-autofiling-issues#36646
Open
[wasm][perf] Tracking #96444
Open

tmds mentioned this pull request Jul 2, 2024
Base64url tests fail on s390x (big endian) #104283
Closed

buyaa-n mentioned this pull request Jul 2, 2024
Remove unnecessary endianness dependent logic #104332
Merged

github-actionsBot locked and limited conversation to collaborators Jul 23, 2024

Source byte count	Base64 decoded byte count	Base64Url decoded byte count
1	max 0	max 0
2	max 0	max 1
3	max 0	max 2
4	max 3	max 3
5	max 3	max 3
6	max 3	max 4
7	max 3	max 5
8	max 6	max 6

	constintBlockSize=4;
	Span<byte>buffer=stackallocbyte[BlockSize];

	privatestaticunsafevoidAvx2Encode<TBase64Encoder,T>(refbytesrcBytes,refTdestBytes,bytesrcEnd,intsourceLength,intdestLength,bytesrcStart,T*destStart)
	whereTBase64Encoder:IBase64Encoder<T>
	whereT: unmanaged
	{
	// If we have AVX2 support, pick off 24 bytes at a time for as long as we can.
	// But because we read 32 bytes at a time, ensure we have enough room to do a
	// full 32-byte read without segfaulting.

Uh oh!

Conversation

buyaa-n commented May 17, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented May 17, 2024

Uh oh!

gfoidl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

buyaa-n commented May 17, 2024

Uh oh!

buyaa-n commented May 17, 2024

Uh oh!

pr-benchmarksBot commented May 17, 2024

Uh oh!

This comment was marked as duplicate.

This comment was marked as duplicate.

This comment was marked as duplicate.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

stephentoub left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MihaZupan commented Jun 12, 2024

Uh oh!

lewing commented Jun 19, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Uh oh!

Conversation

buyaa-n commented May 17, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented May 17, 2024

Uh oh!

gfoidl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

buyaa-n commented May 17, 2024•
edited
Loading

lewing commented Jun 19, 2024•
edited
Loading

buyaa-n commented May 17, 2024•
edited
Loading

lewing commented Jun 19, 2024•
edited
Loading