Uh oh!
There was an error while loading. Please reload this page.
[improvement] Add JMH benchmark project - #176
Closed
schlosna wants to merge 2 commits into
Closed
Conversation
schlosna
commented
Jun 14, 2019
ContributorAuthor
@carterkozak@wenhoujx for SA |
schlosna
commented
Jun 14, 2019
ContributorAuthor
Some initial results from running |
wenhoujx
commented
Jun 14, 2019
Contributor
i prefer this PR over mine #175 . |
schlosna
commented
Jun 14, 2019
ContributorAuthor
And OpenJDK 11: |
| options.errorprone.errorproneArgs += [ | ||
| '-Werror', | ||
| '-Xlint:deprecation', | ||
| '-Xlint:unchecked', |
| } | ||
| } | ||
| apply plugin: 'me.champeau.gradle.jmh' |
Contributor
There was a problem hiding this comment.
Can replace the combination of the buildscript block and this apply plugin line with this, since we're pulling from plugins.gradle.org:
plugins {
id "me.champeau.gradle.jmh" version "0.4.7"
}
Contributor
There was a problem hiding this comment.
Also I'd quite like to keep the convention of declaring all plugin versions in the root level build.gradle, so there's just one place for people to look to see what functionality a project uses
| @Benchmark | ||
| public static Object unsampledCallable() throws Exception { | ||
| return unsampledCallable.call(); | ||
| } |
Contributor
There was a problem hiding this comment.
These benchmarks all test starting and completing a trace, it's significantly more common to start/complete spans inside of a single trace, it would be helpful to measure creation+completion of a span inside of traces that are and aren't sampled.
In one large internal product in particular, it's not uncommon to have several thousand spans per trace.
Contributor
There was a problem hiding this comment.
Perhaps we could wrap the callable in ~100 callables of the form
() -> {
Tracer.startSpan("span");
try {
delegate.call();
} finally {
Tracer.fastCompleteSpan();
}
}
This was referenced Jun 18, 2019
bulldozer-botBot
pushed a commit
that referenced
this pull request
Jun 19, 2019
<!-- PR title should start with '[fix]', '[improvement]' or '[break]' if this PR would cause a patch, minor or major SemVer bump. Omit the prefix if this PR doesn't warrant a standalone release. --> add span benchmarks according to @carterkozak 's comments on #176 remove ## Before this PR <!-- Describe the problem you encountered with the current state of the world (or link to an issue) and why it's important to fix now. --> ## After this PR <!-- Describe at a high-level why this approach is better. --> <!-- Reference any existing GitHub issues, e.g. 'fixes #000' or 'relevant to #000' --> ``` Benchmark (observability) Mode Cnt Score Error Units TracingBenchmark.nestedSpans SAMPLE avgt 3 51266.154 ± 19461.304 ns/op TracingBenchmark.nestedSpans:·gc.alloc.rate SAMPLE avgt 3 2598.879 ± 978.792 MB/sec TracingBenchmark.nestedSpans:·gc.alloc.rate.norm SAMPLE avgt 3 40760.031 ± 1.141 B/op TracingBenchmark.nestedSpans:·gc.churn.PS_Eden_Space SAMPLE avgt 3 2707.659 ± 1382.288 MB/sec TracingBenchmark.nestedSpans:·gc.churn.PS_Eden_Space.norm SAMPLE avgt 3 42470.187 ± 19619.815 B/op TracingBenchmark.nestedSpans:·gc.churn.PS_Survivor_Space SAMPLE avgt 3 0.056 ± 0.469 MB/sec TracingBenchmark.nestedSpans:·gc.churn.PS_Survivor_Space.norm SAMPLE avgt 3 0.890 ± 7.573 B/op TracingBenchmark.nestedSpans:·gc.count SAMPLE avgt 3 22.000 counts TracingBenchmark.nestedSpans:·gc.time SAMPLE avgt 3 28.000 ms TracingBenchmark.nestedSpans DO_NOT_SAMPLE avgt 3 26749.119 ± 17733.671 ns/op TracingBenchmark.nestedSpans:·gc.alloc.rate DO_NOT_SAMPLE avgt 3 2618.135 ± 1713.586 MB/sec TracingBenchmark.nestedSpans:·gc.alloc.rate.norm DO_NOT_SAMPLE avgt 3 21416.022 ± 0.539 B/op TracingBenchmark.nestedSpans:·gc.churn.PS_Eden_Space DO_NOT_SAMPLE avgt 3 2664.780 ± 3436.299 MB/sec TracingBenchmark.nestedSpans:·gc.churn.PS_Eden_Space.norm DO_NOT_SAMPLE avgt 3 21780.010 ± 14872.970 B/op TracingBenchmark.nestedSpans:·gc.churn.PS_Survivor_Space DO_NOT_SAMPLE avgt 3 0.128 ± 1.580 MB/sec TracingBenchmark.nestedSpans:·gc.churn.PS_Survivor_Space.norm DO_NOT_SAMPLE avgt 3 1.062 ± 13.821 B/op TracingBenchmark.nestedSpans:·gc.count DO_NOT_SAMPLE avgt 3 65.000 counts TracingBenchmark.nestedSpans:·gc.time DO_NOT_SAMPLE avgt 3 78.000 ms TracingBenchmark.nestedSpans UNDECIDED avgt 3 30374.624 ± 7959.930 ns/op TracingBenchmark.nestedSpans:·gc.alloc.rate UNDECIDED avgt 3 2442.989 ± 677.177 MB/sec TracingBenchmark.nestedSpans:·gc.alloc.rate.norm UNDECIDED avgt 3 22713.026 ± 15.117 B/op TracingBenchmark.nestedSpans:·gc.churn.PS_Eden_Space UNDECIDED avgt 3 2415.657 ± 1132.967 MB/sec TracingBenchmark.nestedSpans:·gc.churn.PS_Eden_Space.norm UNDECIDED avgt 3 22458.167 ± 7539.764 B/op TracingBenchmark.nestedSpans:·gc.churn.PS_Survivor_Space UNDECIDED avgt 3 0.160 ± 1.415 MB/sec TracingBenchmark.nestedSpans:·gc.churn.PS_Survivor_Space.norm UNDECIDED avgt 3 1.498 ± 13.569 B/op TracingBenchmark.nestedSpans:·gc.count UNDECIDED avgt 3 62.000 counts TracingBenchmark.nestedSpans:·gc.time UNDECIDED avgt 3 69.000 ms ```
carterkozak
commented
Jun 19, 2019
Contributor
Closing in favor of #179 based on this implementation, thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before this PR
There were no benchmarks for measuring tracing performance overhead and implementation efficiency.
After this PR
There is now a
tracing-benchmarksmodule to house JMH benchmarks. See #167 and #168 where these benchmarks would be useful.Closes#169