Skip to content

Improve XmlDictionaryWriter UTF8 encoding performance - #73336

Merged
StephenMolloy merged 27 commits into
dotnet:mainfrom
Daniel-Svensson:binary_xml_text
Apr 4, 2023
Merged

Improve XmlDictionaryWriter UTF8 encoding performance#73336
StephenMolloy merged 27 commits into
dotnet:mainfrom
Daniel-Svensson:binary_xml_text

Conversation

@Daniel-Svensson

@Daniel-SvenssonDaniel-Svensson commented Aug 3, 2022

Copy link
Copy Markdown
Contributor

Summary

  • Remove allocations for writing longer (42 /85+) strings containing non-ascii characters
  • Improve performance for utf8 encoding all strings with lenght 8 or longer
    • escpecially for cases where they contain a mix of ascii and non ascii characters (for "mixed chars" variants with a few non ascii the speedup was up to ~3x by not calling into encoding.GetBytes multiple times)
  • if anyone want to run the benchmarks (original, encoding, int32, long and at least Vector256) on arm hardware I can consider making the implementation fallback to generic simd on non x86 platforms, but since i have no idea of how the unaligned loads/stores affect performance i feel it is safer to fallback to System.encoding earlier on arm.

Feedback wanted on implementation to choose

  1. For UnsafeGetUTF8Chars does it makes sense to go with the AVX version which is faster for all lenghts > 8 even if it has a bit more code than the SSE verions?
  2. For UnsafeGetUTF8Length does it make sense to go with:
    2a. the Vector256 version (in this PR)
    * and if so what cutoff to use before calling Encoding.GetByteCount (1024 , 2048 ?)
    2b. Just call Encoding.GetByteCount (it is faster always faster than current code)
    2c. Vector version (with the risk of "AVX downclocking" on older intel hardware when AVX512 support is added)
  3. After writing this I discovered the Ascii-utilities and the Narrow method in CoreLib is there any internal low overhead call path possible to use those methods from runtime libraries ?
  4. What is a reasonable cutoff for when to call into system.encoding for the non accelerated case 16, 24,32?
Original UnsafeGetUTF8Chars benchmarks

UnsafeGetUTF8Chars benchmarks

Source: https://github.com/Daniel-Svensson/ClrExperiments/blob/master/BinaryXmlBenchmarks/ConsoleApp1/Utf8Benchmarks.cs

non ascii the speedup was up to ~3x when only a few characters were by not calling into encoding.GetBytes multiple times) so thoses measurements are not show below

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
AMD Ryzen 7 5800X, 1 CPU, 16 logical and 8 physical cores
.NET SDK=7.0.100-preview.6.22352.1
[Host] : .NET 7.0.0 (7.0.22.32404), X64 RyuJIT
Job-GOOLTT : .NET 7.0.0 (7.0.22.32404), X64 RyuJIT
MaxRelativeError=0.01 IterationTime=250.0000 ms 
MethodStringLengthInCharsScenarioMeanErrorStdDevMedian
Original5AsciiOnly3.667 ns0.0079 ns0.0066 ns3.669 ns
Encoding5AsciiOnly7.491 ns0.0127 ns0.0119 ns7.490 ns
SimdSSE_v45AsciiOnly3.199 ns0.0053 ns0.0050 ns3.199 ns
SimdAVX_25AsciiOnly3.495 ns0.0064 ns0.0057 ns3.493 ns
SimdVector2565AsciiOnly3.487 ns0.0095 ns0.0153 ns3.483 ns
Original8AsciiOnly4.524 ns0.0129 ns0.0115 ns4.526 ns
Encoding8AsciiOnly7.694 ns0.0056 ns0.0044 ns7.696 ns
SimdSSE_v48AsciiOnly2.768 ns0.0051 ns0.0048 ns2.767 ns
SimdAVX_28AsciiOnly2.767 ns0.0070 ns0.0065 ns2.768 ns
SimdVector2568AsciiOnly2.777 ns0.0037 ns0.0034 ns2.777 ns
Original10AsciiOnly5.115 ns0.0189 ns0.0177 ns5.108 ns
Encoding10AsciiOnly7.916 ns0.0122 ns0.0102 ns7.914 ns
SimdSSE_v410AsciiOnly2.973 ns0.0093 ns0.0087 ns2.970 ns
SimdAVX_210AsciiOnly2.772 ns0.0053 ns0.0047 ns2.773 ns
SimdVector25610AsciiOnly3.067 ns0.0167 ns0.0148 ns3.065 ns
Original16AsciiOnly7.357 ns0.0669 ns0.0626 ns7.361 ns
Encoding16AsciiOnly8.554 ns0.0155 ns0.0137 ns8.554 ns
SimdSSE_v416AsciiOnly2.987 ns0.0098 ns0.0087 ns2.985 ns
SimdAVX_216AsciiOnly2.770 ns0.0056 ns0.0050 ns2.770 ns
SimdVector25616AsciiOnly2.802 ns0.0077 ns0.0068 ns2.801 ns
Original20AsciiOnly8.159 ns0.0708 ns0.0695 ns8.145 ns
Encoding20AsciiOnly9.612 ns0.1040 ns0.2629 ns9.582 ns
SimdSSE_v420AsciiOnly3.229 ns0.0165 ns0.0147 ns3.225 ns
SimdAVX_220AsciiOnly3.438 ns0.0081 ns0.0076 ns3.436 ns
SimdVector25620AsciiOnly3.227 ns0.0071 ns0.0063 ns3.228 ns
Original30AsciiOnly10.717 ns0.0926 ns0.0723 ns10.707 ns
SimdSSE_v430AsciiOnly3.661 ns0.0057 ns0.0051 ns3.660 ns
SimdAVX_230AsciiOnly3.411 ns0.0051 ns0.0048 ns3.410 ns
SimdVector25630AsciiOnly3.288 ns0.0134 ns0.0125 ns3.287 ns
Original32AsciiOnly11.116 ns0.0687 ns0.0609 ns11.116 ns
SimdSSE_v432AsciiOnly3.652 ns0.0176 ns0.0156 ns3.644 ns
SimdAVX_232AsciiOnly3.425 ns0.0078 ns0.0073 ns3.424 ns
SimdVector25632AsciiOnly3.203 ns0.0053 ns0.0047 ns3.204 ns
Original34AsciiOnly11.562 ns0.0515 ns0.0430 ns11.567 ns
SimdSSE_v434AsciiOnly3.901 ns0.0157 ns0.0122 ns3.898 ns
SimdAVX_234AsciiOnly3.848 ns0.0085 ns0.0071 ns3.850 ns
SimdVector25634AsciiOnly3.651 ns0.0075 ns0.0066 ns3.652 ns
Original84AsciiOnly22.269 ns0.0708 ns0.0627 ns22.277 ns
SimdSSE_v484AsciiOnly5.958 ns0.0163 ns0.0145 ns5.955 ns
SimdAVX_284AsciiOnly5.148 ns0.0097 ns0.0091 ns5.150 ns
SimdVector25684AsciiOnly5.155 ns0.0087 ns0.0081 ns5.154 ns
Original170AsciiOnly44.331 ns0.0684 ns0.0607 ns44.326 ns
SimdSSE_v4170AsciiOnly10.980 ns0.0215 ns0.0179 ns10.985 ns
SimdAVX_2170AsciiOnly7.546 ns0.0112 ns0.0105 ns7.549 ns
SimdVector256170AsciiOnly9.863 ns0.0161 ns0.0143 ns9.863 ns

The gains of AVX is somewhat less on older hardware

32bit results:

Original: UnsafeGetUTF8Length benchmarks: removed

Source: https://github.com/Daniel-Svensson/ClrExperiments/blob/master/BinaryXmlBenchmarks/ConsoleApp1/Utf8BenchmarksLength.cs

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
AMD Ryzen 7 5800X, 1 CPU, 16 logical and 8 physical cores
.NET SDK=7.0.100-preview.6.22352.1
[Host] : .NET 7.0.0 (7.0.22.32404), X64 RyuJIT
Job-QIEUWM : .NET 7.0.0 (7.0.22.32404), X64 RyuJIT
MaxRelativeError=0.01 IterationTime=250.0000 ms 
MethodStringLengthInCharsScenarioMeanErrorStdDev
Encoding42AsciiOnly5.047 ns0.0167 ns0.0148 ns
VectorLength42AsciiOnly2.087 ns0.0069 ns0.0058 ns
VectorLength_Aligned42AsciiOnly2.077 ns0.0086 ns0.0076 ns
Encoding85AsciiOnly6.113 ns0.0233 ns0.0218 ns
VectorLength85AsciiOnly2.755 ns0.0149 ns0.0139 ns
VectorLength_Aligned85AsciiOnly2.761 ns0.0144 ns0.0134 ns
Encoding256AsciiOnly9.585 ns0.0339 ns0.0317 ns
VectorLength256AsciiOnly6.839 ns0.0227 ns0.0201 ns
VectorLength_Aligned256AsciiOnly6.944 ns0.0211 ns0.0197 ns
Encoding512AsciiOnly15.959 ns0.0308 ns0.0288 ns
VectorLength512AsciiOnly11.452 ns0.0327 ns0.0290 ns
VectorLength_Aligned512AsciiOnly11.632 ns0.0165 ns0.0147 ns
Encoding2048AsciiOnly59.927 ns0.1031 ns0.0914 ns
VectorLength2048AsciiOnly36.509 ns0.0851 ns0.0754 ns
VectorLength_Aligned2048AsciiOnly44.433 ns0.4511 ns0.7154 ns

2023-03-26: Updated PR with vectorisation removed
For latest results se comment below

@ghostghost added area-Serialization community-contribution Indicates that the PR has been added by a community member labels Aug 3, 2022
@Daniel-Svensson
Daniel-Svensson marked this pull request as ready for review August 4, 2022 17:28
@Daniel-SvenssonDaniel-Svensson changed the title Feedback wanted: Improve XmlDictionaryWriter UTF8 encoding performanceImprove XmlDictionaryWriter (text and binary xml) UTF8 encoding performanceAug 5, 2022
@Daniel-SvenssonDaniel-Svensson changed the title Improve XmlDictionaryWriter (text and binary xml) UTF8 encoding performanceImprove XmlDictionaryWriter UTF8 encoding performanceAug 5, 2022
@danmoseley

Copy link
Copy Markdown
Contributor

@HongGit who is the right reviewer for this PR?

@StephenMolloy

Copy link
Copy Markdown
Member

@tannergooding and @stephentoub... can you guys take a peek at this one? This is an improvement we'd like to take if it looks good, but we wanted some more eyes on it with the use of Vector256.

@danmoseley

Copy link
Copy Markdown
Contributor

@adamsitnik might be able to help review the vector stuff also.

int numRemaining = (int)(charsMax - chars);
int numAscii = charCount - numRemaining;

return numAscii + (_encoding ?? s_UTF8Encoding).GetByteCount(chars, numRemaining);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the possible values of _encoding? Can it be something other than Utf8?

Note that it better to call Encoding.UTF8.GetBytes directly without caching the encoding locally. Encoding.UTF8.GetBytes allows devitalization optimization to kick in that eliminates the overhead of Encoding being an abstract type.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be passed by the user when creating a text XmlDictionaryWriter, but it is only set to _encoding if the codepage is the same as utf8.
So in theory it can be any encoding class even if unlikely .

for s_encoding it does not use the default constructor but passes is (false, true) so I did no dare to do that change.
If it does not change the behaviour then that can be a simple follow up fix.

Daniel-Svenssonand others added 2 commits August 12, 2022 20:04
…tem/Xml/XmlStreamNodeWriter.cs
Co-authored-by: Stephen Toub <stoub@microsoft.com>
@danmoseley

Copy link
Copy Markdown
Contributor

as an aside @Daniel-Svensson how is the coverage of this code in dotnet/performance? ie., are there scenario/s there that will show improvement, and thus protect the improvement from future regression?

@Daniel-Svensson

Copy link
Copy Markdown
ContributorAuthor

as an aside @Daniel-Svensson how is the coverage of this code in dotnet/performance? ie., are there scenario/s there that will show improvement, and thus protect the improvement from future regression?

I don't know, i have not browsed the performance repository.

As long as there is any somewhat realistic* test using Datacontractserializer and passing in a Stream or XmlDictionaryWriter (either binary or text) or directly using the later should be fine.

*: With realistic i mean that elements/texts/propertynames are not limited to lenghts less than 8. Anything inspired by Northwind, AdwentureWorks or similar should give good coverage.

@Daniel-Svensson

Daniel-Svensson commented Mar 26, 2023

Copy link
Copy Markdown
ContributorAuthor

I've removed the improved utf8 encoding logic so the 25% speedup is gone, but at least is a bit faster for inputs with mixed ascii / non ascii characters.

While it is more than 3 times slower than original proposal it is still faster than the current version for larger strings, and for all cases where the input contains one or more "non-ascii" characters.

I do not expect any large performance changes from normal "all ascii" cases since many input strings falls in the range 8-25 characters and it should hopefully not change the performance of that case.

I have moved the vectorisation code to a sepate branch where I moved the vectorization code to Utf8Encoding and the results would be somewhat better than here, and I might create a separate PR (or not) from it.

The input to the method in question will be at most 170 (512/3) in length but depending on entry point the limit can also be 42 or 85

BenchmarkDotNet=v0.13.5, OS=Windows 11 (10.0.22621.1413/22H2/2022Update/SunValley2)
AMD Ryzen 7 5800X, 1 CPU, 16 logical and 8 physical cores
.NET SDK=8.0.100-preview.1.23115.2
[Host] : .NET 8.0.0 (8.0.23.11008), X64 RyuJIT AVX2
Job-XXUKJT : .NET 8.0.0 (8.0.23.11008), X64 RyuJIT AVX2
MaxRelativeError=0.01 IterationTime=250.0000 ms 

Ascii Only Case

MethodStringLengthInCharsScenarioMeanErrorStdDevMedian
Original5AsciiOnly3.726 ns0.0475 ns0.0397 ns3.717 ns
New5AsciiOnly3.059 ns0.0156 ns0.0139 ns3.059 ns
Encoding_GetBytes5AsciiOnly6.959 ns0.0192 ns0.0179 ns6.951 ns
Vector128 (original proposal)5AsciiOnly4.007 ns0.0507 ns0.0676 ns4.006 ns
Vector256 (AVX2)5AsciiOnly4.357 ns0.0537 ns0.0851 ns4.316 ns
Original8AsciiOnly4.373 ns0.0206 ns0.0183 ns4.374 ns
New8AsciiOnly4.214 ns0.0191 ns0.0178 ns
Encoding_GetBytes8AsciiOnly7.208 ns0.0802 ns0.0751 ns7.159 ns
Vector128 (original proposal)8AsciiOnly2.138 ns0.0102 ns0.0095 ns2.138 ns
Vector256 (AVX2)8AsciiOnly2.723 ns0.0080 ns0.0067 ns2.722 ns
Original34AsciiOnly11.589 ns0.0808 ns0.0631 ns11.589 ns
New34AsciiOnly9.909 ns0.0292 ns0.0273 ns9.909 ns
Encoding_GetBytes34AsciiOnly9.087 ns0.0260 ns0.0203 ns9.091 ns
Vector128 (original proposal)34AsciiOnly3.202 ns0.0103 ns0.0092 ns3.202 ns
Vector256 (AVX2)34AsciiOnly3.675 ns0.0237 ns0.0221 ns3.673 ns
Original50AsciiOnly17.487 ns0.0404 ns0.0358 ns17.479 ns
New50AsciiOnly10.246 ns0.0301 ns0.0267 ns10.241 ns
Encoding_GetBytes50AsciiOnly9.519 ns0.0257 ns0.0214 ns9.515 ns
Vector128 (original proposal)50AsciiOnly3.953 ns0.0508 ns0.0564 ns3.953 ns
Vector256 (AVX2)50AsciiOnly4.010 ns0.0270 ns0.0239 ns4.002 ns
Original84AsciiOnly21.932 ns0.0669 ns0.0626 ns21.931 ns
New84AsciiOnly11.791 ns0.0478 ns0.0399 ns11.789 ns
Encoding_GetBytes84AsciiOnly10.306 ns0.0329 ns0.0257 ns10.306 ns
Vector128 (original proposal)84AsciiOnly5.586 ns0.0646 ns0.0863 ns5.609 ns
Vector256 (AVX2)84AsciiOnly4.795 ns0.0156 ns0.0146 ns4.790 ns
Original170AsciiOnly43.624 ns0.1264 ns0.1056 ns43.634 ns
New170AsciiOnly13.350 ns0.0441 ns0.0391 ns13.349 ns
SealedEncoding_If_Ptr170AsciiOnly10.782 ns0.1034 ns0.0916 ns10.763 ns
Vector128 (original proposal)170AsciiOnly10.608 ns0.1151 ns0.1859 ns10.627 ns
Vector256 (AVX2)170AsciiOnly7.153 ns0.0321 ns0.0251 ns7.156 ns

Mostly Ascii case

MethodStringLengthInCharsScenarioMeanErrorStdDevMedian
Original5Mixed3.668 ns0.0232 ns0.0217 ns3.661 ns
New5Mixed3.061 ns0.0112 ns0.0099 ns3.061 ns
Encoding_GetBytes5Mixed6.941 ns0.0301 ns0.0252 ns6.933 ns
Vector128 (original proposal)5Mixed3.979 ns0.0208 ns0.0174 ns3.981 ns
Vector256 (AVX2)5Mixed4.312 ns0.0533 ns0.0473 ns4.306 ns
Original8Mixed14.868 ns0.1622 ns0.4385 ns14.713 ns
New8Mixed12.059 ns0.0981 ns0.0869 ns12.049 ns
Encoding_GetBytes8Mixed10.136 ns0.0419 ns0.0392 ns10.140 ns
Vector128 (original proposal)8Mixed13.309 ns0.0433 ns0.0405 ns13.313 ns
Vector256 (AVX2)8Mixed14.186 ns0.0908 ns0.0759 ns14.168 ns
Original34Mixed32.739 ns0.3621 ns1.0563 ns32.527 ns
New34Mixed17.102 ns0.0567 ns0.0503 ns17.094 ns
SealedEncoding_If_Ptr34Mixed18.330 ns0.0560 ns0.0468 ns18.327 ns
Vector128 (original proposal)34Mixed19.595 ns0.0523 ns0.0436 ns19.607 ns
Vector256 (AVX2)34Mixed20.805 ns0.2183 ns0.3462 ns20.880 ns
Original50Mixed47.258 ns0.5794 ns1.7083 ns46.619 ns
New50Mixed20.686 ns0.0815 ns0.0722 ns20.677 ns
Encoding_GetBytes50Mixed20.020 ns0.0785 ns0.0696 ns20.011 ns
Vector128 (original proposal)50Mixed23.761 ns0.1688 ns0.1579 ns23.730 ns
Vector256 (AVX2)50Mixed24.034 ns0.0707 ns0.0662 ns24.053 ns
Original84Mixed77.078 ns0.7535 ns1.6696 ns76.631 ns
New84Mixed29.049 ns0.0639 ns0.0597 ns29.047 ns
Encoding_GetBytes84Mixed28.414 ns0.0752 ns0.0666 ns28.416 ns
Vector128 (original proposal)84Mixed31.854 ns0.3257 ns0.4235 ns31.655 ns
Vector256 (AVX2)84Mixed32.396 ns0.0955 ns0.0847 ns32.380 ns
Original170Mixed165.831 ns1.1964 ns1.1191 ns166.444 ns
New170Mixed53.783 ns0.2215 ns0.1964 ns53.753 ns
Encoding_GetBytes170Mixed52.745 ns0.1562 ns0.1461 ns52.748 ns
Vector128 (original proposal)170Mixed55.608 ns0.2007 ns0.1877 ns55.603 ns
Vector256 (AVX2)170Mixed57.323 ns0.5780 ns0.9333 ns56.901 ns

}

internal static SealedUTF8Encoding UTF8NoBom { get; } = new SealedUTF8Encoding(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: false);
internal static SealedUTF8Encoding ValidatingUTF8 { get; } = new SealedUTF8Encoding(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true);

@Daniel-SvenssonDaniel-SvenssonMar 26, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was originally a temporary part of moving vector code to Encoding class.

It does not seem to make any impact to datacontract serialisation at the moment so I can revert the changes if you want that. From the code it looks like improvements would mainly be from classes calling into XmlConverter which uses this encoding directly


while (true)
// Fast path for small strings, use Encoding.GetBytes for larger strings since it is faster when vectorization is possible
if ((uint)charCount < 25)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling into encoding actually seems to be faster from 25 characters up, but that is when we don't need to handle branch predictions so I increased it to 32 handle misspredictions without to much affect on performance.

The "microbenchmarks" showed >5% regression where a long class name was mixed with many short strings & names when calling encoding from 25 chars and upp for the (text based) DataContractSerializer. (In the same case the binary serializer was 10% faster). Now they are maybe? 1% regression and 5% improvement, but other things might be different since it is no r2r or pgo for local build)

@StephenMolloy

Copy link
Copy Markdown
Member

Test failure appears to be unrelated. #64227

Getting back to these serializer PR's... I was going to suggest that the vectorization stuff would be better handled at the encoding layer. I am sure the folks watching over the encoding classes would welcome the kind of improvement your initial testing was showing. But I see that's already been updated.

I would remove the sealed encoding classes. The calls to them generate 'callvirt's anyway, so there isn't really a performance win there as you've already noticed.

@StephenMolloy

Copy link
Copy Markdown
Member

/azp run runtime-community

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@StephenMolloy

Copy link
Copy Markdown
Member

/azp run runtime-community

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@StephenMolloy
StephenMolloy dismissed GrabYourPitchforks’s stale reviewApril 4, 2023 22:10

Vectorizing of UTF8 was removed. This is just tweaking the hand-rolling that has already existed here for years.

@StephenMolloy
StephenMolloy merged commit e0c94f8 into dotnet:mainApr 4, 2023
@Daniel-Svensson
Daniel-Svensson deleted the binary_xml_text branch April 5, 2023 17:44
@ghostghost locked as resolved and limited conversation to collaborators May 5, 2023
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Serializationcommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants

@Daniel-Svensson@danmoseley@StephenMolloy@HongGit@GrabYourPitchforks@stephentoub@jkotas@PaulusParssinen@ts-223