Add `apm-sdks-benchmarks` to CI #10206

sarahchen6 · 2025-12-12T22:48:10Z

What Does This Do

Test https://github.com/DataDog/apm-sdks-benchmarks/pull/44

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-12-13T00:47:30Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/test-merge-base-benchmark
git_commit_date	1765578878	1765583900
git_commit_sha	`276a869`	`e6573d0`
release_version	1.57.0-SNAPSHOT~276a86988b	1.57.0-SNAPSHOT~e6573d03d1

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1765585882	1765585882
ci_job_id	1292690740	1292690740
ci_pipeline_id	86612829	86612829
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-jykowhmr 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-jykowhmr 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 7 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~e6573d03d1, baseline=1.57.0-SNAPSHOT~276a86988b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1080967
Total [baseline] (8.763 s) : 0, 8763416
Agent [candidate] (1.079 s) : 0, 1078959
Total [candidate] (8.718 s) : 0, 8718200
section iast
Agent [baseline] (1.246 s) : 0, 1246289
Total [baseline] (9.442 s) : 0, 9441662
Agent [candidate] (1.219 s) : 0, 1219086
Total [candidate] (9.399 s) : 0, 9399468

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.081 s	-
Agent	iast	1.246 s	165.321 ms (15.3%)
Total	tracing	8.763 s	-
Total	iast	9.442 s	678.245 ms (7.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.079 s	-
Agent	iast	1.219 s	140.127 ms (13.0%)
Total	tracing	8.718 s	-
Total	iast	9.399 s	681.268 ms (7.8%)

gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~e6573d03d1, baseline=1.57.0-SNAPSHOT~276a86988b

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.175 ms) : 0, 1175
BytebuddyAgent [baseline] (649.207 ms) : 0, 649207
BytebuddyAgent [candidate] (648.068 ms) : 0, 648068
GlobalTracer [baseline] (281.603 ms) : 0, 281603
GlobalTracer [candidate] (281.538 ms) : 0, 281538
AppSec [baseline] (32.383 ms) : 0, 32383
AppSec [candidate] (32.307 ms) : 0, 32307
Debugger [baseline] (67.509 ms) : 0, 67509
Debugger [candidate] (67.013 ms) : 0, 67013
Remote Config [baseline] (684.274 µs) : 0, 684
Remote Config [candidate] (639.707 µs) : 0, 640
Telemetry [baseline] (9.096 ms) : 0, 9096
Telemetry [candidate] (8.952 ms) : 0, 8952
Flare Poller [baseline] (3.728 ms) : 0, 3728
Flare Poller [candidate] (3.708 ms) : 0, 3708
section iast
crashtracking [baseline] (1.212 ms) : 0, 1212
crashtracking [candidate] (1.179 ms) : 0, 1179
BytebuddyAgent [baseline] (807.613 ms) : 0, 807613
BytebuddyAgent [candidate] (788.883 ms) : 0, 788883
GlobalTracer [baseline] (260.288 ms) : 0, 260288
GlobalTracer [candidate] (255.245 ms) : 0, 255245
IAST [baseline] (27.67 ms) : 0, 27670
IAST [candidate] (26.835 ms) : 0, 26835
AppSec [baseline] (35.192 ms) : 0, 35192
AppSec [candidate] (33.593 ms) : 0, 33593
Debugger [baseline] (65.598 ms) : 0, 65598
Debugger [candidate] (65.46 ms) : 0, 65460
Remote Config [baseline] (578.134 µs) : 0, 578
Remote Config [candidate] (547.916 µs) : 0, 548
Telemetry [baseline] (8.568 ms) : 0, 8568
Telemetry [candidate] (8.383 ms) : 0, 8383
Flare Poller [baseline] (3.539 ms) : 0, 3539
Flare Poller [candidate] (3.508 ms) : 0, 3508

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~e6573d03d1, baseline=1.57.0-SNAPSHOT~276a86988b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1082815
Total [baseline] (10.992 s) : 0, 10991502
Agent [candidate] (1.094 s) : 0, 1094347
Total [candidate] (10.908 s) : 0, 10907726
section appsec
Agent [baseline] (1.263 s) : 0, 1262651
Total [baseline] (11.048 s) : 0, 11048230
Agent [candidate] (1.264 s) : 0, 1263552
Total [candidate] (10.942 s) : 0, 10941860
section iast
Agent [baseline] (1.222 s) : 0, 1221652
Total [baseline] (11.193 s) : 0, 11192674
Agent [candidate] (1.223 s) : 0, 1222845
Total [candidate] (11.211 s) : 0, 11211288
section profiling
Agent [baseline] (1.204 s) : 0, 1203657
Total [baseline] (10.88 s) : 0, 10880449
Agent [candidate] (1.213 s) : 0, 1212666
Total [candidate] (10.977 s) : 0, 10977053

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.083 s	-
Agent	appsec	1.263 s	179.836 ms (16.6%)
Agent	iast	1.222 s	138.838 ms (12.8%)
Agent	profiling	1.204 s	120.843 ms (11.2%)
Total	tracing	10.992 s	-
Total	appsec	11.048 s	56.728 ms (0.5%)
Total	iast	11.193 s	201.172 ms (1.8%)
Total	profiling	10.88 s	-111.053 ms (-1.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.094 s	-
Agent	appsec	1.264 s	169.205 ms (15.5%)
Agent	iast	1.223 s	128.499 ms (11.7%)
Agent	profiling	1.213 s	118.319 ms (10.8%)
Total	tracing	10.908 s	-
Total	appsec	10.942 s	34.134 ms (0.3%)
Total	iast	11.211 s	303.562 ms (2.8%)
Total	profiling	10.977 s	69.328 ms (0.6%)

gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~e6573d03d1, baseline=1.57.0-SNAPSHOT~276a86988b

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.176 ms) : 0, 1176
crashtracking [candidate] (1.205 ms) : 0, 1205
BytebuddyAgent [baseline] (650.043 ms) : 0, 650043
BytebuddyAgent [candidate] (657.229 ms) : 0, 657229
GlobalTracer [baseline] (282.115 ms) : 0, 282115
GlobalTracer [candidate] (285.032 ms) : 0, 285032
AppSec [baseline] (32.405 ms) : 0, 32405
AppSec [candidate] (32.562 ms) : 0, 32562
Debugger [baseline] (67.976 ms) : 0, 67976
Debugger [candidate] (68.907 ms) : 0, 68907
Remote Config [baseline] (644.309 µs) : 0, 644
Remote Config [candidate] (649.232 µs) : 0, 649
Telemetry [baseline] (9.03 ms) : 0, 9030
Telemetry [candidate] (9.085 ms) : 0, 9085
Flare Poller [baseline] (3.793 ms) : 0, 3793
Flare Poller [candidate] (3.85 ms) : 0, 3850
section appsec
crashtracking [baseline] (1.18 ms) : 0, 1180
crashtracking [candidate] (1.177 ms) : 0, 1177
BytebuddyAgent [baseline] (686.911 ms) : 0, 686911
BytebuddyAgent [candidate] (688.442 ms) : 0, 688442
GlobalTracer [baseline] (258.336 ms) : 0, 258336
GlobalTracer [candidate] (258.647 ms) : 0, 258647
IAST [baseline] (24.433 ms) : 0, 24433
IAST [candidate] (24.551 ms) : 0, 24551
AppSec [baseline] (174.396 ms) : 0, 174396
AppSec [candidate] (174.657 ms) : 0, 174657
Debugger [baseline] (68.15 ms) : 0, 68150
Debugger [candidate] (66.809 ms) : 0, 66809
Remote Config [baseline] (690.437 µs) : 0, 690
Remote Config [candidate] (689.296 µs) : 0, 689
Telemetry [baseline] (9.12 ms) : 0, 9120
Telemetry [candidate] (9.139 ms) : 0, 9139
Flare Poller [baseline] (3.932 ms) : 0, 3932
Flare Poller [candidate] (3.873 ms) : 0, 3873
section iast
crashtracking [baseline] (1.192 ms) : 0, 1192
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (790.44 ms) : 0, 790440
BytebuddyAgent [candidate] (790.343 ms) : 0, 790343
GlobalTracer [baseline] (254.936 ms) : 0, 254936
GlobalTracer [candidate] (255.943 ms) : 0, 255943
IAST [baseline] (26.836 ms) : 0, 26836
IAST [candidate] (26.898 ms) : 0, 26898
AppSec [baseline] (35.293 ms) : 0, 35293
AppSec [candidate] (34.388 ms) : 0, 34388
Debugger [baseline] (65.02 ms) : 0, 65020
Debugger [candidate] (66.107 ms) : 0, 66107
Remote Config [baseline] (547.952 µs) : 0, 548
Remote Config [candidate] (559.447 µs) : 0, 559
Telemetry [baseline] (8.364 ms) : 0, 8364
Telemetry [candidate] (8.437 ms) : 0, 8437
Flare Poller [baseline] (3.453 ms) : 0, 3453
Flare Poller [candidate] (3.533 ms) : 0, 3533
section profiling
crashtracking [baseline] (1.218 ms) : 0, 1218
crashtracking [candidate] (1.229 ms) : 0, 1229
BytebuddyAgent [baseline] (700.974 ms) : 0, 700974
BytebuddyAgent [candidate] (706.693 ms) : 0, 706693
GlobalTracer [baseline] (220.741 ms) : 0, 220741
GlobalTracer [candidate] (222.404 ms) : 0, 222404
AppSec [baseline] (32.013 ms) : 0, 32013
AppSec [candidate] (32.468 ms) : 0, 32468
Debugger [baseline] (68.352 ms) : 0, 68352
Debugger [candidate] (68.757 ms) : 0, 68757
Remote Config [baseline] (637.224 µs) : 0, 637
Remote Config [candidate] (653.649 µs) : 0, 654
Telemetry [baseline] (9.046 ms) : 0, 9046
Telemetry [candidate] (9.134 ms) : 0, 9134
Flare Poller [baseline] (3.743 ms) : 0, 3743
Flare Poller [candidate] (3.791 ms) : 0, 3791
ProfilingAgent [baseline] (96.982 ms) : 0, 96982
ProfilingAgent [candidate] (97.211 ms) : 0, 97211
Profiling [baseline] (97.558 ms) : 0, 97558
Profiling [candidate] (97.783 ms) : 0, 97783

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/test-merge-base-benchmark
git_commit_date	1765578878	1765583900
git_commit_sha	`276a869`	`e6573d0`
release_version	1.57.0-SNAPSHOT~276a86988b	1.57.0-SNAPSHOT~e6573d03d1

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1765586324	1765586324
ci_job_id	1292690741	1292690741
ci_pipeline_id	86612829	86612829
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-svjmrsc0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-svjmrsc0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 0 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load	better [-220.095µs; -113.094µs] or [-7.586%; -3.898%]	unsure [-499.903µs; -148.906µs] or [-6.192%; -1.844%]	unstable [-80.081op/s; +194.456op/s] or [-6.365%; +15.456%]	2.735ms	7.750ms	1315.312op/s	2.901ms	8.074ms	1258.125op/s
scenario:load:petclinic:tracing:high_load	better [-902.823µs; -474.482µs] or [-5.035%; -2.646%]	unsure [-1393.322µs; -437.911µs] or [-4.756%; -1.495%]	unstable [-19.632op/s; +35.132op/s] or [-7.674%; +13.732%]	17.244ms	28.381ms	263.594op/s	17.933ms	29.297ms	255.844op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~e6573d03d1, baseline=1.57.0-SNAPSHOT~276a86988b
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.197 ms) : 1185, 1209
.   : milestone, 1197,
iast (3.275 ms) : 3231, 3319
.   : milestone, 3275,
iast_FULL (5.807 ms) : 5748, 5866
.   : milestone, 5807,
iast_GLOBAL (3.646 ms) : 3591, 3700
.   : milestone, 3646,
profiling (2.372 ms) : 2347, 2396
.   : milestone, 2372,
tracing (1.785 ms) : 1770, 1800
.   : milestone, 1785,
section candidate
no_agent (1.217 ms) : 1205, 1229
.   : milestone, 1217,
iast (3.217 ms) : 3176, 3257
.   : milestone, 3217,
iast_FULL (5.805 ms) : 5746, 5864
.   : milestone, 5805,
iast_GLOBAL (3.484 ms) : 3433, 3535
.   : milestone, 3484,
profiling (2.285 ms) : 2264, 2306
.   : milestone, 2285,
tracing (1.793 ms) : 1779, 1808
.   : milestone, 1793,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.197 ms [1.185 ms, 1.209 ms]	-
iast	3.275 ms [3.231 ms, 3.319 ms]	2.078 ms (173.7%)
iast_FULL	5.807 ms [5.748 ms, 5.866 ms]	4.61 ms (385.2%)
iast_GLOBAL	3.646 ms [3.591 ms, 3.7 ms]	2.449 ms (204.6%)
profiling	2.372 ms [2.347 ms, 2.396 ms]	1.175 ms (98.2%)
tracing	1.785 ms [1.77 ms, 1.8 ms]	588.737 µs (49.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.217 ms [1.205 ms, 1.229 ms]	-
iast	3.217 ms [3.176 ms, 3.257 ms]	2.0 ms (164.3%)
iast_FULL	5.805 ms [5.746 ms, 5.864 ms]	4.588 ms (377.0%)
iast_GLOBAL	3.484 ms [3.433 ms, 3.535 ms]	2.267 ms (186.3%)
profiling	2.285 ms [2.264 ms, 2.306 ms]	1.068 ms (87.8%)
tracing	1.793 ms [1.779 ms, 1.808 ms]	576.469 µs (47.4%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~e6573d03d1, baseline=1.57.0-SNAPSHOT~276a86988b
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.275 ms) : 18086, 18463
.   : milestone, 18275,
appsec (18.863 ms) : 18675, 19050
.   : milestone, 18863,
code_origins (17.79 ms) : 17612, 17969
.   : milestone, 17790,
iast (17.811 ms) : 17632, 17989
.   : milestone, 17811,
profiling (18.61 ms) : 18423, 18797
.   : milestone, 18610,
tracing (18.237 ms) : 18052, 18421
.   : milestone, 18237,
section candidate
no_agent (18.233 ms) : 18046, 18421
.   : milestone, 18233,
appsec (18.577 ms) : 18387, 18767
.   : milestone, 18577,
code_origins (17.659 ms) : 17485, 17833
.   : milestone, 17659,
iast (17.832 ms) : 17656, 18008
.   : milestone, 17832,
profiling (18.54 ms) : 18359, 18722
.   : milestone, 18540,
tracing (17.701 ms) : 17526, 17877
.   : milestone, 17701,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.275 ms [18.086 ms, 18.463 ms]	-
appsec	18.863 ms [18.675 ms, 19.05 ms]	587.938 µs (3.2%)
code_origins	17.79 ms [17.612 ms, 17.969 ms]	-484.356 µs (-2.7%)
iast	17.811 ms [17.632 ms, 17.989 ms]	-464.031 µs (-2.5%)
profiling	18.61 ms [18.423 ms, 18.797 ms]	335.709 µs (1.8%)
tracing	18.237 ms [18.052 ms, 18.421 ms]	-38.021 µs (-0.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.233 ms [18.046 ms, 18.421 ms]	-
appsec	18.577 ms [18.387 ms, 18.767 ms]	343.92 µs (1.9%)
code_origins	17.659 ms [17.485 ms, 17.833 ms]	-574.549 µs (-3.2%)
iast	17.832 ms [17.656 ms, 18.008 ms]	-401.202 µs (-2.2%)
profiling	18.54 ms [18.359 ms, 18.722 ms]	307.044 µs (1.7%)
tracing	17.701 ms [17.526 ms, 17.877 ms]	-532.123 µs (-2.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/test-merge-base-benchmark
git_commit_date	1765578878	1765583900
git_commit_sha	`276a869`	`e6573d0`
release_version	1.57.0-SNAPSHOT~276a86988b	1.57.0-SNAPSHOT~e6573d03d1

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1765586246	1765586246
ci_job_id	1292690742	1292690742
ci_pipeline_id	86612829	86612829
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-h7ujmm8x 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-h7ujmm8x 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~e6573d03d1, baseline=1.57.0-SNAPSHOT~276a86988b
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.065 s) : 15065000, 15065000
.   : milestone, 15065000,
appsec (14.668 s) : 14668000, 14668000
.   : milestone, 14668000,
iast (18.364 s) : 18364000, 18364000
.   : milestone, 18364000,
iast_GLOBAL (17.758 s) : 17758000, 17758000
.   : milestone, 17758000,
profiling (14.715 s) : 14715000, 14715000
.   : milestone, 14715000,
tracing (14.701 s) : 14701000, 14701000
.   : milestone, 14701000,
section candidate
no_agent (15.529 s) : 15529000, 15529000
.   : milestone, 15529000,
appsec (15.117 s) : 15117000, 15117000
.   : milestone, 15117000,
iast (17.875 s) : 17875000, 17875000
.   : milestone, 17875000,
iast_GLOBAL (17.861 s) : 17861000, 17861000
.   : milestone, 17861000,
profiling (15.191 s) : 15191000, 15191000
.   : milestone, 15191000,
tracing (14.745 s) : 14745000, 14745000
.   : milestone, 14745000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.065 s [15.065 s, 15.065 s]	-
appsec	14.668 s [14.668 s, 14.668 s]	-397.0 ms (-2.6%)
iast	18.364 s [18.364 s, 18.364 s]	3.299 s (21.9%)
iast_GLOBAL	17.758 s [17.758 s, 17.758 s]	2.693 s (17.9%)
profiling	14.715 s [14.715 s, 14.715 s]	-350.0 ms (-2.3%)
tracing	14.701 s [14.701 s, 14.701 s]	-364.0 ms (-2.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.529 s [15.529 s, 15.529 s]	-
appsec	15.117 s [15.117 s, 15.117 s]	-412.0 ms (-2.7%)
iast	17.875 s [17.875 s, 17.875 s]	2.346 s (15.1%)
iast_GLOBAL	17.861 s [17.861 s, 17.861 s]	2.332 s (15.0%)
profiling	15.191 s [15.191 s, 15.191 s]	-338.0 ms (-2.2%)
tracing	14.745 s [14.745 s, 14.745 s]	-784.0 ms (-5.0%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~e6573d03d1, baseline=1.57.0-SNAPSHOT~276a86988b
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.481 ms) : 1469, 1492
.   : milestone, 1481,
appsec (3.661 ms) : 3448, 3875
.   : milestone, 3661,
iast (2.211 ms) : 2147, 2275
.   : milestone, 2211,
iast_GLOBAL (2.261 ms) : 2197, 2326
.   : milestone, 2261,
profiling (2.08 ms) : 2028, 2132
.   : milestone, 2080,
tracing (2.05 ms) : 2000, 2101
.   : milestone, 2050,
section candidate
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (3.722 ms) : 3505, 3940
.   : milestone, 3722,
iast (2.217 ms) : 2153, 2281
.   : milestone, 2217,
iast_GLOBAL (2.265 ms) : 2200, 2329
.   : milestone, 2265,
profiling (2.076 ms) : 2023, 2129
.   : milestone, 2076,
tracing (2.059 ms) : 2008, 2110
.   : milestone, 2059,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.469 ms, 1.492 ms]	-
appsec	3.661 ms [3.448 ms, 3.875 ms]	2.181 ms (147.3%)
iast	2.211 ms [2.147 ms, 2.275 ms]	730.647 µs (49.3%)
iast_GLOBAL	2.261 ms [2.197 ms, 2.326 ms]	780.552 µs (52.7%)
profiling	2.08 ms [2.028 ms, 2.132 ms]	599.395 µs (40.5%)
tracing	2.05 ms [2.0 ms, 2.101 ms]	569.852 µs (38.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.464 ms, 1.487 ms]	-
appsec	3.722 ms [3.505 ms, 3.94 ms]	2.247 ms (152.3%)
iast	2.217 ms [2.153 ms, 2.281 ms]	742.024 µs (50.3%)
iast_GLOBAL	2.265 ms [2.2 ms, 2.329 ms]	789.484 µs (53.5%)
profiling	2.076 ms [2.023 ms, 2.129 ms]	600.979 µs (40.7%)
tracing	2.059 ms [2.008 ms, 2.11 ms]	583.928 µs (39.6%)

sarahchen6 added the tag: do not merge Do not merge changes label Dec 12, 2025

sarahchen6 temporarily deployed to ci-build December 12, 2025 22:48 — with GitHub Actions Inactive

sarahchen6 added 2 commits December 12, 2025 16:51

Add apm-sdks-benchmarks to CI

d9d4887

Add thresholds file

732aefd

sarahchen6 force-pushed the sarahchen6/test-merge-base-benchmark branch from c1cba5f to 732aefd Compare December 12, 2025 23:03

sarahchen6 added 2 commits December 12, 2025 17:10

Empty commit to run benchmarks again

0beddad

Empty commit to run benchmarks again

9610907

sarahchen6 had a problem deploying to ci-build December 12, 2025 23:20 — with GitHub Actions Error

sarahchen6 temporarily deployed to ci-build December 12, 2025 23:21 — with GitHub Actions Inactive

Empty commit to run benchmarks again

4fc38a5

sarahchen6 temporarily deployed to ci-build December 12, 2025 23:50 — with GitHub Actions Inactive

Empty commit to run benchmarks again

bb85a00

sarahchen6 temporarily deployed to ci-build December 12, 2025 23:54 — with GitHub Actions Inactive

Empty commit to run benchmarks again

e6573d0

sarahchen6 temporarily deployed to ci-build December 13, 2025 00:00 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add `apm-sdks-benchmarks` to CI #10206

Add `apm-sdks-benchmarks` to CI #10206

sarahchen6 commented Dec 12, 2025

Uh oh!

pr-commenter bot commented Dec 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add apm-sdks-benchmarks to CI #10206

Are you sure you want to change the base?

Add apm-sdks-benchmarks to CI #10206

Conversation

sarahchen6 commented Dec 12, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Dec 13, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add `apm-sdks-benchmarks` to CI #10206

Add `apm-sdks-benchmarks` to CI #10206