Uh oh!
There was an error while loading. Please reload this page.
This repository was archived by the owner on Jul 6, 2026. It is now read-only.
[JavaCallableWrappers] Parallel.ForEach per assembly - #365
Closed
jonathanpeppers wants to merge 2 commits into
Closed
[JavaCallableWrappers] Parallel.ForEach per assembly#365jonathanpeppers wants to merge 2 commits into
jonathanpeppers wants to merge 2 commits into
Conversation
jonathanpeppers
commented
Aug 30, 2018
MemberAuthor
Build logs of these changes: GenerateJavaStubs.zip |
jonpryor
reviewed
Sep 4, 2018
| var javaTypes = new BlockingCollection<TypeDefinition> (); | ||
| Parallel.ForEach (assemblies, assembly => { | ||
| var assm = resolver.GetAssembly (assembly); |
Contributor
There was a problem hiding this comment.
Nope. Nopity nopity not going to do it NOPE.
Cecil isn't thread safe. This cannot work. See also: b1667a2.
MemberAuthor
There was a problem hiding this comment.
Hmm b1667a2, looks like it is parallelizing on the TypeDefinition and this is doing a Task per AssemblyDefinition.
Let me try writing a test like you have here, and see what happens.
Since we are looking at every type in each assembly, we can parallelize this work to make things a bit faster. However, there are a few concerns when doing this: - Made sure to add a *new* method `GetJavaTypesInParallel` so the caller can opt-in to this - The `cache` in `DirectoryAssemblyResolver` needs to now be a `ConcurrentDictionary` - The list of `javaTypes` returned by `JavaTypeScanner` needs to also use a `BlockingCollection` - Downstream in `xamarin-android`, we need to make sure `GenerateJavaStubs` is now an `AsyncTask`, so the "thread safe" logger methods are used. To time the improvements, I used this project in Xamarin.Forms: https://github.com/xamarin/Xamarin.Forms/tree/master/Xamarin.Forms.ControlGallery.Android Before I did anything, the times were: 2594 ms GenerateJavaStubs 2 calls After making things parallel: 2143 ms GenerateJavaStubs 2 calls And then after a few more LINQ improvements in the `GenerateJavaStubs` MSBuild task: 2104 ms GenerateJavaStubs 2 calls So we have close to a 500ms improvement for `GenerateJavaStubs` in this project, that will build a library project + application project. To see the full changes required downstream in `xamarin-android`: dotnet/android@master...jonathanpeppers:generatejavastubs-perf
Test fails with: Expected: 6959 But was: 6958 So we should abandon this... 😢
jonathanpeppersforce-pushed
the
javatypescanner-parallel
branch
from
September 4, 2018 15:16
b012118 to
bf0b5a0Comparejonathanpeppers
commented
Sep 4, 2018
MemberAuthor
See bf0b5a0, an adventure in showing the issue with this idea... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since we are looking at every type in each assembly, we can
parallelize this work to make things a bit faster.
However, there are a few concerns when doing this:
GetJavaTypesInParallelso thecaller can opt-in to this
cacheinDirectoryAssemblyResolverneeds to now be aConcurrentDictionaryjavaTypesreturned byJavaTypeScannerneeds to alsouse a
BlockingCollectionxamarin-android, we need to make sureGenerateJavaStubsis now anAsyncTask, so the "thread safe"logger methods are used.
To time the improvements, I used this project in Xamarin.Forms:
https://github.com/xamarin/Xamarin.Forms/tree/master/Xamarin.Forms.ControlGallery.Android
Before I did anything, the times were:
After making things parallel:
And then after a few more LINQ improvements in the
GenerateJavaStubsMSBuild task:
So we have close to a 500ms improvement for
GenerateJavaStubsinthis project, that will build a library project + application project.
To see the full changes required downstream in
xamarin-android:dotnet/android@master...jonathanpeppers:generatejavastubs-perf