[TEST] Testing changes for #7464 by smola · Pull Request #7465 · DataDog/dd-trace-java

smola · 2024-08-18T17:52:55Z

What Does This Do

pr-commenter · 2024-08-18T18:34:37Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/git-changes-test-verification
git_commit_date	1724232069	1724232595
git_commit_sha	`99cfdba`	`e420db6`
release_version	1.39.0-SNAPSHOT~99cfdba356	1.39.0-SNAPSHOT~e420db6f76

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1724234830	1724234830
ci_job_id	611816333	611816333
ci_pipeline_id	42395068	42395068
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 18 unstable metrics.

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062769
Total [baseline] (10.391 s) : 0, 10390994
Agent [candidate] (1.049 s) : 0, 1049151
Total [candidate] (10.326 s) : 0, 10326048
section appsec
Agent [baseline] (1.175 s) : 0, 1174543
Total [baseline] (10.513 s) : 0, 10513112
Agent [candidate] (1.169 s) : 0, 1169292
Total [candidate] (10.454 s) : 0, 10454407
section iast
Agent [baseline] (1.184 s) : 0, 1184177
Total [baseline] (10.821 s) : 0, 10820818
Agent [candidate] (1.179 s) : 0, 1178681
Total [candidate] (10.85 s) : 0, 10849534
section profiling
Agent [baseline] (1.245 s) : 0, 1245154
Total [baseline] (10.56 s) : 0, 10559996
Agent [candidate] (1.263 s) : 0, 1262595
Total [candidate] (10.696 s) : 0, 10695611

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.063 s	-
Agent	appsec	1.175 s	111.773 ms (10.5%)
Agent	iast	1.184 s	121.408 ms (11.4%)
Agent	profiling	1.245 s	182.385 ms (17.2%)
Total	tracing	10.391 s	-
Total	appsec	10.513 s	122.118 ms (1.2%)
Total	iast	10.821 s	429.824 ms (4.1%)
Total	profiling	10.56 s	169.003 ms (1.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	appsec	1.169 s	120.141 ms (11.5%)
Agent	iast	1.179 s	129.53 ms (12.3%)
Agent	profiling	1.263 s	213.444 ms (20.3%)
Total	tracing	10.326 s	-
Total	appsec	10.454 s	128.359 ms (1.2%)
Total	iast	10.85 s	523.486 ms (5.1%)
Total	profiling	10.696 s	369.563 ms (3.6%)

gantt
title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (678.581 ms) : 0, 678581
BytebuddyAgent [candidate] (669.037 ms) : 0, 669037
GlobalTracer [baseline] (310.454 ms) : 0, 310454
GlobalTracer [candidate] (307.132 ms) : 0, 307132
AppSec [baseline] (51.807 ms) : 0, 51807
AppSec [candidate] (51.283 ms) : 0, 51283
Remote Config [baseline] (699.502 µs) : 0, 700
Remote Config [candidate] (701.782 µs) : 0, 702
Telemetry [baseline] (7.624 ms) : 0, 7624
Telemetry [candidate] (7.517 ms) : 0, 7517
section appsec
BytebuddyAgent [baseline] (682.074 ms) : 0, 682074
BytebuddyAgent [candidate] (678.356 ms) : 0, 678356
GlobalTracer [baseline] (301.305 ms) : 0, 301305
GlobalTracer [candidate] (300.098 ms) : 0, 300098
AppSec [baseline] (157.458 ms) : 0, 157458
AppSec [candidate] (157.904 ms) : 0, 157904
Remote Config [baseline] (612.285 µs) : 0, 612
Remote Config [candidate] (603.19 µs) : 0, 603
Telemetry [baseline] (9.076 ms) : 0, 9076
Telemetry [candidate] (8.422 ms) : 0, 8422
IAST [baseline] (20.943 ms) : 0, 20943
IAST [candidate] (20.811 ms) : 0, 20811
section iast
BytebuddyAgent [baseline] (788.851 ms) : 0, 788851
BytebuddyAgent [candidate] (783.757 ms) : 0, 783757
GlobalTracer [baseline] (297.223 ms) : 0, 297223
GlobalTracer [candidate] (297.091 ms) : 0, 297091
AppSec [baseline] (52.71 ms) : 0, 52710
AppSec [candidate] (51.756 ms) : 0, 51756
Remote Config [baseline] (576.89 µs) : 0, 577
Remote Config [candidate] (584.753 µs) : 0, 585
Telemetry [baseline] (7.951 ms) : 0, 7951
Telemetry [candidate] (7.054 ms) : 0, 7054
IAST [baseline] (23.275 ms) : 0, 23275
IAST [candidate] (24.891 ms) : 0, 24891
section profiling
BytebuddyAgent [baseline] (663.646 ms) : 0, 663646
BytebuddyAgent [candidate] (673.935 ms) : 0, 673935
GlobalTracer [baseline] (388.961 ms) : 0, 388961
GlobalTracer [candidate] (393.93 ms) : 0, 393930
AppSec [baseline] (52.213 ms) : 0, 52213
AppSec [candidate] (52.362 ms) : 0, 52362
Remote Config [baseline] (705.453 µs) : 0, 705
Remote Config [candidate] (706.969 µs) : 0, 707
Telemetry [baseline] (7.474 ms) : 0, 7474
Telemetry [candidate] (7.517 ms) : 0, 7517
ProfilingAgent [baseline] (94.96 ms) : 0, 94960
ProfilingAgent [candidate] (96.309 ms) : 0, 96309
Profiling [baseline] (94.984 ms) : 0, 94984
Profiling [candidate] (96.334 ms) : 0, 96334

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1048357
Total [baseline] (8.492 s) : 0, 8491624
Agent [candidate] (1.047 s) : 0, 1047208
Total [candidate] (8.488 s) : 0, 8487706
section iast
Agent [baseline] (1.18 s) : 0, 1180398
Total [baseline] (8.973 s) : 0, 8973123
Agent [candidate] (1.179 s) : 0, 1178825
Total [candidate] (9.023 s) : 0, 9022958
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.18 s) : 0, 1179623
Total [baseline] (8.978 s) : 0, 8977807
Agent [candidate] (1.188 s) : 0, 1188073
Total [candidate] (8.999 s) : 0, 8999448
section iast_TELEMETRY_OFF
Agent [baseline] (1.172 s) : 0, 1171825
Total [baseline] (8.955 s) : 0, 8954933
Agent [candidate] (1.174 s) : 0, 1173695
Total [candidate] (9.003 s) : 0, 9002694

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	iast	1.18 s	132.041 ms (12.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.18 s	131.266 ms (12.5%)
Agent	iast_TELEMETRY_OFF	1.172 s	123.468 ms (11.8%)
Total	tracing	8.492 s	-
Total	iast	8.973 s	481.5 ms (5.7%)
Total	iast_HARDCODED_SECRET_DISABLED	8.978 s	486.183 ms (5.7%)
Total	iast_TELEMETRY_OFF	8.955 s	463.31 ms (5.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	iast	1.179 s	131.617 ms (12.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.188 s	140.865 ms (13.5%)
Agent	iast_TELEMETRY_OFF	1.174 s	126.487 ms (12.1%)
Total	tracing	8.488 s	-
Total	iast	9.023 s	535.252 ms (6.3%)
Total	iast_HARDCODED_SECRET_DISABLED	8.999 s	511.743 ms (6.0%)
Total	iast_TELEMETRY_OFF	9.003 s	514.989 ms (6.1%)

gantt
title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (668.588 ms) : 0, 668588
BytebuddyAgent [candidate] (668.198 ms) : 0, 668198
GlobalTracer [baseline] (307.105 ms) : 0, 307105
GlobalTracer [candidate] (306.252 ms) : 0, 306252
AppSec [baseline] (50.98 ms) : 0, 50980
AppSec [candidate] (51.047 ms) : 0, 51047
Remote Config [baseline] (688.539 µs) : 0, 689
Remote Config [candidate] (698.977 µs) : 0, 699
Telemetry [baseline] (7.535 ms) : 0, 7535
Telemetry [candidate] (7.495 ms) : 0, 7495
section iast
BytebuddyAgent [baseline] (785.299 ms) : 0, 785299
BytebuddyAgent [candidate] (784.045 ms) : 0, 784045
GlobalTracer [baseline] (297.623 ms) : 0, 297623
GlobalTracer [candidate] (297.012 ms) : 0, 297012
AppSec [baseline] (50.824 ms) : 0, 50824
AppSec [candidate] (51.667 ms) : 0, 51667
IAST [baseline] (25.48 ms) : 0, 25480
IAST [candidate] (24.115 ms) : 0, 24115
Remote Config [baseline] (589.602 µs) : 0, 590
Remote Config [candidate] (568.219 µs) : 0, 568
Telemetry [baseline] (7.078 ms) : 0, 7078
Telemetry [candidate] (7.88 ms) : 0, 7880
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (784.736 ms) : 0, 784736
BytebuddyAgent [candidate] (789.657 ms) : 0, 789657
GlobalTracer [baseline] (298.018 ms) : 0, 298018
GlobalTracer [candidate] (299.389 ms) : 0, 299389
AppSec [baseline] (50.083 ms) : 0, 50083
AppSec [candidate] (50.674 ms) : 0, 50674
IAST [baseline] (23.289 ms) : 0, 23289
IAST [candidate] (22.846 ms) : 0, 22846
Remote Config [baseline] (583.943 µs) : 0, 584
Remote Config [candidate] (603.555 µs) : 0, 604
Telemetry [baseline] (9.387 ms) : 0, 9387
Telemetry [candidate] (11.197 ms) : 0, 11197
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (779.472 ms) : 0, 779472
BytebuddyAgent [candidate] (781.044 ms) : 0, 781044
GlobalTracer [baseline] (296.801 ms) : 0, 296801
GlobalTracer [candidate] (297.273 ms) : 0, 297273
AppSec [baseline] (52.406 ms) : 0, 52406
AppSec [candidate] (52.818 ms) : 0, 52818
IAST [baseline] (22.017 ms) : 0, 22017
IAST [candidate] (20.624 ms) : 0, 20624
Remote Config [baseline] (613.328 µs) : 0, 613
Remote Config [candidate] (611.792 µs) : 0, 612
Telemetry [baseline] (7.019 ms) : 0, 7019
Telemetry [candidate] (7.763 ms) : 0, 7763

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-08-21T09:39:52	2024-08-21T09:46:39
git_branch	master	smola/git-changes-test-verification
git_commit_date	1724232069	1724232595
git_commit_sha	`99cfdba`	`e420db6`
release_version	1.39.0-SNAPSHOT~99cfdba356	1.39.0-SNAPSHOT~e420db6f76
start_time	2024-08-21T09:39:39	2024-08-21T09:46:26

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1724233942	1724233942
ci_job_id	611816335	611816335
ci_pipeline_id	42395068	42395068
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
dateFormat X
axisFormat %s
section baseline
no_agent (1.355 ms) : 1336, 1375
. : milestone, 1355,
appsec (1.727 ms) : 1702, 1752
. : milestone, 1727,
appsec_no_iast (1.713 ms) : 1689, 1737
. : milestone, 1713,
iast (1.461 ms) : 1439, 1483
. : milestone, 1461,
profiling (1.497 ms) : 1473, 1522
. : milestone, 1497,
tracing (1.459 ms) : 1434, 1484
. : milestone, 1459,
section candidate
no_agent (1.349 ms) : 1329, 1368
. : milestone, 1349,
appsec (1.727 ms) : 1704, 1750
. : milestone, 1727,
appsec_no_iast (1.721 ms) : 1697, 1744
. : milestone, 1721,
iast (1.465 ms) : 1442, 1488
. : milestone, 1465,
profiling (1.492 ms) : 1468, 1516
. : milestone, 1492,
tracing (1.461 ms) : 1437, 1486
. : milestone, 1461,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.355 ms [1.336 ms, 1.375 ms]	-
appsec	1.727 ms [1.702 ms, 1.752 ms]	371.633 µs (27.4%)
appsec_no_iast	1.713 ms [1.689 ms, 1.737 ms]	357.728 µs (26.4%)
iast	1.461 ms [1.439 ms, 1.483 ms]	105.824 µs (7.8%)
profiling	1.497 ms [1.473 ms, 1.522 ms]	141.967 µs (10.5%)
tracing	1.459 ms [1.434 ms, 1.484 ms]	103.644 µs (7.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.349 ms [1.329 ms, 1.368 ms]	-
appsec	1.727 ms [1.704 ms, 1.75 ms]	378.548 µs (28.1%)
appsec_no_iast	1.721 ms [1.697 ms, 1.744 ms]	371.963 µs (27.6%)
iast	1.465 ms [1.442 ms, 1.488 ms]	116.052 µs (8.6%)
profiling	1.492 ms [1.468 ms, 1.516 ms]	143.444 µs (10.6%)
tracing	1.461 ms [1.437 ms, 1.486 ms]	112.656 µs (8.4%)

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
dateFormat X
axisFormat %s
section baseline
no_agent (364.429 µs) : 345, 384
. : milestone, 364,
iast (471.273 µs) : 450, 492
. : milestone, 471,
iast_FULL (546.244 µs) : 524, 568
. : milestone, 546,
iast_GLOBAL (496.504 µs) : 475, 518
. : milestone, 497,
iast_HARDCODED_SECRET_DISABLED (476.719 µs) : 456, 498
. : milestone, 477,
iast_INACTIVE (447.687 µs) : 425, 470
. : milestone, 448,
iast_TELEMETRY_OFF (458.909 µs) : 438, 480
. : milestone, 459,
tracing (443.566 µs) : 423, 464
. : milestone, 444,
section candidate
no_agent (362.951 µs) : 344, 382
. : milestone, 363,
iast (477.626 µs) : 456, 499
. : milestone, 478,
iast_FULL (544.967 µs) : 523, 567
. : milestone, 545,
iast_GLOBAL (493.727 µs) : 472, 515
. : milestone, 494,
iast_HARDCODED_SECRET_DISABLED (475.381 µs) : 454, 497
. : milestone, 475,
iast_INACTIVE (448.362 µs) : 426, 470
. : milestone, 448,
iast_TELEMETRY_OFF (459.272 µs) : 438, 480
. : milestone, 459,
tracing (440.115 µs) : 419, 461
. : milestone, 440,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	364.429 µs [345.178 µs, 383.68 µs]	-
iast	471.273 µs [450.377 µs, 492.168 µs]	106.844 µs (29.3%)
iast_FULL	546.244 µs [524.004 µs, 568.485 µs]	181.816 µs (49.9%)
iast_GLOBAL	496.504 µs [474.648 µs, 518.36 µs]	132.075 µs (36.2%)
iast_HARDCODED_SECRET_DISABLED	476.719 µs [455.683 µs, 497.755 µs]	112.29 µs (30.8%)
iast_INACTIVE	447.687 µs [425.052 µs, 470.321 µs]	83.258 µs (22.8%)
iast_TELEMETRY_OFF	458.909 µs [438.077 µs, 479.741 µs]	94.48 µs (25.9%)
tracing	443.566 µs [422.847 µs, 464.285 µs]	79.137 µs (21.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	362.951 µs [343.696 µs, 382.207 µs]	-
iast	477.626 µs [456.398 µs, 498.854 µs]	114.675 µs (31.6%)
iast_FULL	544.967 µs [522.642 µs, 567.293 µs]	182.016 µs (50.1%)
iast_GLOBAL	493.727 µs [472.031 µs, 515.424 µs]	130.776 µs (36.0%)
iast_HARDCODED_SECRET_DISABLED	475.381 µs [454.098 µs, 496.663 µs]	112.429 µs (31.0%)
iast_INACTIVE	448.362 µs [426.226 µs, 470.499 µs]	85.411 µs (23.5%)
iast_TELEMETRY_OFF	459.272 µs [438.433 µs, 480.111 µs]	96.321 µs (26.5%)
tracing	440.115 µs [419.403 µs, 460.827 µs]	77.163 µs (21.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/git-changes-test-verification
git_commit_date	1724232069	1724232595
git_commit_sha	`99cfdba`	`e420db6`
release_version	1.39.0-SNAPSHOT~99cfdba356	1.39.0-SNAPSHOT~e420db6f76

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1724234464	1724234464
ci_job_id	611816336	611816336
ci_pipeline_id	42395068	42395068
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
dateFormat X
axisFormat %s
section baseline
no_agent (1.455 ms) : 1444, 1467
. : milestone, 1455,
appsec (2.218 ms) : 2182, 2254
. : milestone, 2218,
iast (1.976 ms) : 1934, 2018
. : milestone, 1976,
iast_GLOBAL (2.015 ms) : 1972, 2058
. : milestone, 2015,
profiling (1.86 ms) : 1826, 1895
. : milestone, 1860,
tracing (1.823 ms) : 1790, 1855
. : milestone, 1823,
section candidate
no_agent (1.454 ms) : 1443, 1465
. : milestone, 1454,
appsec (2.224 ms) : 2189, 2259
. : milestone, 2224,
iast (1.955 ms) : 1913, 1997
. : milestone, 1955,
iast_GLOBAL (2.002 ms) : 1959, 2044
. : milestone, 2002,
profiling (1.862 ms) : 1828, 1896
. : milestone, 1862,
tracing (1.838 ms) : 1805, 1870
. : milestone, 1838,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.455 ms [1.444 ms, 1.467 ms]	-
appsec	2.218 ms [2.182 ms, 2.254 ms]	762.679 µs (52.4%)
iast	1.976 ms [1.934 ms, 2.018 ms]	520.854 µs (35.8%)
iast_GLOBAL	2.015 ms [1.972 ms, 2.058 ms]	559.728 µs (38.5%)
profiling	1.86 ms [1.826 ms, 1.895 ms]	404.85 µs (27.8%)
tracing	1.823 ms [1.79 ms, 1.855 ms]	367.174 µs (25.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.454 ms [1.443 ms, 1.465 ms]	-
appsec	2.224 ms [2.189 ms, 2.259 ms]	770.251 µs (53.0%)
iast	1.955 ms [1.913 ms, 1.997 ms]	501.346 µs (34.5%)
iast_GLOBAL	2.002 ms [1.959 ms, 2.044 ms]	547.744 µs (37.7%)
profiling	1.862 ms [1.828 ms, 1.896 ms]	408.211 µs (28.1%)
tracing	1.838 ms [1.805 ms, 1.87 ms]	383.728 µs (26.4%)

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
dateFormat X
axisFormat %s
section baseline
no_agent (15.492 s) : 15492000, 15492000
. : milestone, 15492000,
appsec (15.256 s) : 15256000, 15256000
. : milestone, 15256000,
iast (18.785 s) : 18785000, 18785000
. : milestone, 18785000,
iast_GLOBAL (17.926 s) : 17926000, 17926000
. : milestone, 17926000,
profiling (15.169 s) : 15169000, 15169000
. : milestone, 15169000,
tracing (14.965 s) : 14965000, 14965000
. : milestone, 14965000,
section candidate
no_agent (15.41 s) : 15410000, 15410000
. : milestone, 15410000,
appsec (15.182 s) : 15182000, 15182000
. : milestone, 15182000,
iast (18.602 s) : 18602000, 18602000
. : milestone, 18602000,
iast_GLOBAL (17.847 s) : 17847000, 17847000
. : milestone, 17847000,
profiling (15.363 s) : 15363000, 15363000
. : milestone, 15363000,
tracing (15.063 s) : 15063000, 15063000
. : milestone, 15063000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.492 s [15.492 s, 15.492 s]	-
appsec	15.256 s [15.256 s, 15.256 s]	-236.0 ms (-1.5%)
iast	18.785 s [18.785 s, 18.785 s]	3.293 s (21.3%)
iast_GLOBAL	17.926 s [17.926 s, 17.926 s]	2.434 s (15.7%)
profiling	15.169 s [15.169 s, 15.169 s]	-323.0 ms (-2.1%)
tracing	14.965 s [14.965 s, 14.965 s]	-527.0 ms (-3.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.41 s [15.41 s, 15.41 s]	-
appsec	15.182 s [15.182 s, 15.182 s]	-228.0 ms (-1.5%)
iast	18.602 s [18.602 s, 18.602 s]	3.192 s (20.7%)
iast_GLOBAL	17.847 s [17.847 s, 17.847 s]	2.437 s (15.8%)
profiling	15.363 s [15.363 s, 15.363 s]	-47.0 ms (-0.3%)
tracing	15.063 s [15.063 s, 15.063 s]	-347.0 ms (-2.3%)

pr-commenter · 2024-08-19T07:41:33Z

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1724070506	1724070885
end_time	2024-08-19T12:29:40	2024-08-19T12:35:58
git_branch	master	smola/git-changes-test-verification
git_commit_sha	`0c5f7c2`	de8ff6a137ba1be9e5a4f743b122861e4ab94ffc
start_time	2024-08-19T12:28:27	2024-08-19T12:34:46

See matching parameters

	Baseline	Candidate
ci_job_id	608692090	608692090
ci_pipeline_id	42109453	42109453
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1724069861	1724069861

Summary

Found 3 performance improvements and 0 performance regressions! Performance is the same for 5 metrics, 7 unstable metrics.

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:basic	better [-15.615µs; -4.689µs] or [-5.722%; -1.718%]	better [-23.254µs; -6.166µs] or [-7.757%; -2.057%]	better [-24.812µs; -4.496µs] or [-8.046%; -1.458%]	unstable [-47.518µs; +24.800µs] or [-8.126%; +4.241%]	unstable [-21.535op/s; +298.735op/s] or [-0.840%; +11.651%]

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-19.627µs; +21.967µs] or [-7.698%; +8.616%]	unstable [-32017.584ns; +32458.783ns] or [-11.078%; +11.231%]	unstable [-43402.685ns; +44547.579ns] or [-14.380%; +14.759%]	unstable [-112.147µs; +55.555µs] or [-17.615%; +8.726%]	unstable [-102.863op/s; +176.910op/s] or [-3.806%; +6.546%]
scenario:loop	unsure [-39.932µs; -33.645µs] or [-0.386%; -0.325%]	same	same	same	same

Request duration reports for reports

gantt
title reports - request duration [CI 0.99] : candidate=None, baseline=None
dateFormat X
axisFormat %s
section baseline
noprobe (289.008 µs) : 264, 314
. : milestone, 289,
basic (299.79 µs) : 293, 307
. : milestone, 300,
loop (10.531 ms) : 10494, 10568
. : milestone, 10531,
section candidate
noprobe (289.228 µs) : 254, 325
. : milestone, 289,
basic (285.08 µs) : 276, 295
. : milestone, 285,
loop (10.497 ms) : 10456, 10537
. : milestone, 10497,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	289.008 µs [264.122 µs, 313.893 µs]
basic	299.79 µs [292.826 µs, 306.753 µs]
loop	10.531 ms [10.494 ms, 10.568 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	289.228 µs [253.532 µs, 324.925 µs]
basic	285.08 µs [275.5 µs, 294.659 µs]
loop	10.497 ms [10.456 ms, 10.537 ms]

…t.groovy

smola added the tag: do not merge Do not merge changes label Aug 18, 2024

smola force-pushed the smola/git-changes-test-verification branch from 78424c9 to 4af5abdCompare August 18, 2024 18:13

smola force-pushed the smola/git-changes-test branch from eebe257 to 096fc38Compare August 18, 2024 19:45

smola force-pushed the smola/git-changes-test-verification branch from 4af5abd to ce30672Compare August 18, 2024 19:46

smola force-pushed the smola/git-changes-test branch from 096fc38 to cc47be1Compare August 19, 2024 07:18

smola force-pushed the smola/git-changes-test-verification branch from ce30672 to 0dde356Compare August 19, 2024 07:19

smola force-pushed the smola/git-changes-test branch 2 times, most recently from 87b138c to d664538Compare August 19, 2024 11:07

smola force-pushed the smola/git-changes-test-verification branch 2 times, most recently from a182355 to de8ff6aCompare August 19, 2024 12:17

smola force-pushed the smola/git-changes-test branch from f8991a1 to c7f43cbCompare August 19, 2024 13:34

smola force-pushed the smola/git-changes-test-verification branch from de8ff6a to 7ea5ef8Compare August 19, 2024 13:35

smola force-pushed the smola/git-changes-test branch from a7047ea to 31efcdfCompare August 20, 2024 14:32

smola force-pushed the smola/git-changes-test-verification branch from 7ea5ef8 to d70548bCompare August 20, 2024 14:32

smola added 7 commits August 20, 2024 17:58

Run CI only for tasks affected by git changes
ee5dc89

Move code for CI-oriented tasks to gradle/ci_jobs.gradle
d7c8db1

Fix changes path patterns
45e5205

Fix affected projects for multiple changed files
9af9759

Simplify code
e55adae

Simplify code
d8d3066

Clarify prefix matching logic
a89f547

smola force-pushed the smola/git-changes-test branch from 31efcdf to a89f547Compare August 20, 2024 16:18

smola force-pushed the smola/git-changes-test-verification branch from d70548b to a37d9c4Compare August 20, 2024 16:18

[TEST] edit dd-smoke-tests/kafka-2/src/test/groovy/IastKafka2SmokeTes…
e420db6
…t.groovy

smola force-pushed the smola/git-changes-test-verification branch from a37d9c4 to e420db6Compare August 21, 2024 09:30

Base automatically changed from smola/git-changes-test to masterAugust 21, 2024 13:34

smola closed this Aug 21, 2024

smola deleted the smola/git-changes-test-verification branch August 21, 2024 13:35

Navigation Menu

Uh oh!

[TEST] Testing changes for #7464 - #7465

[TEST] Testing changes for #7464#7465
smola wants to merge 8 commits into
masterfrom
smola/git-changes-test-verification

smola commented Aug 18, 2024

Uh oh!

pr-commenterBot commented Aug 18, 2024•
edited
Loading

Uh oh!

pr-commenterBot commented Aug 19, 2024•
edited
Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

smola commented Aug 18, 2024

What Does This Do

Uh oh!

pr-commenterBot commented Aug 18, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

pr-commenterBot commented Aug 19, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Debugger benchmarks

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pr-commenterBot commented Aug 18, 2024•
edited
Loading

pr-commenterBot commented Aug 19, 2024•
edited
Loading