Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revert "Avoid performing blocking I/O operation on application thread (#8120)" #8124

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Revert "Avoid performing blocking I/O operation on application thread (#8120)" #8124

wants to merge 1 commit into from

Conversation

smola
Copy link
Member

@smola smola commented Dec 20, 2024 •
edited
Loading

What Does This Do

This reverts commit 8d5f5ac (#8120).

Motivation

On IBM 8 smoke test jobs:

JULInterfaceJBossBackend > check raw file injection FAILED
    org.spockframework.runtime.SpockComparisonFailure at LogInjectionSmokeTest.groovy:173

JULInterfaceJBossBackend128bTid > check raw file injection FAILED
    org.spockframework.runtime.SpockComparisonFailure at LogInjectionSmokeTest.groovy:173

JULInterfaceJBossBackendNoTags > check raw file injection FAILED
    org.spockframework.runtime.SpockComparisonFailure at LogInjectionSmokeTest.groovy:173

JULInterfaceJBossLatestBackend > check raw file injection FAILED
    org.spockframework.runtime.SpockComparisonFailure at LogInjectionSmokeTest.groovy:173

JULInterfaceLog4j2Backend > check raw file injection FAILED
    org.spockframework.runtime.SpockComparisonFailure at LogInjectionSmokeTest.groovy:173

JULInterfaceLog4j2Backend128bTid > check raw file injection FAILED
    org.spockframework.runtime.SpockComparisonFailure at LogInjectionSmokeTest.groovy:173

JULInterfaceLog4j2LatestBackend > check raw file injection FAILED
    org.spockframework.runtime.SpockComparisonFailure at LogInjectionSmokeTest.groovy:173

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@smola smola added the run-tests: all Run all tests label Dec 20, 2024
@smola smola changed the title Revert "Avoid performing blocking I/O operation on application thread… Revert "Avoid performing blocking I/O operation on application thread (#8120)" Dec 20, 2024
@pr-commenter
Copy link

pr-commenter bot commented Dec 20, 2024

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1734712990 1734713397
end_time 2024-12-20T16:44:27 2024-12-20T16:51:13
git_branch master smola/revert-io-pr
git_commit_sha a3e9bda 324f9f2
start_time 2024-12-20T16:43:11 2024-12-20T16:49:57
See matching parameters
Baseline Candidate
ci_job_id 747248512 747248512
ci_pipeline_id 51666044 51666044
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1734712386 1734712386

Summary

Found 5 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 7 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop better
[-1.885ms; -1.426ms] or [-15.874%; -12.004%]		 better
[-1.945ms; -1.471ms] or [-16.085%; -12.170%]		 better
[-1.935ms; -1.447ms] or [-15.905%; -11.891%]		 better
[-2.092ms; -1.455ms] or [-16.665%; -11.588%]		 better
[+9.568op/s; +13.338op/s] or [+11.482%; +16.006%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-38.522µs; +81.271µs] or [-14.235%; +30.032%]		 unstable
[-46.750µs; +90.954µs] or [-15.004%; +29.191%]		 unstable
[-57.367µs; +112.081µs] or [-17.606%; +34.397%]		 unstable
[-415.571µs; +1160.311µs] or [-69.755%; +194.763%]		 same
scenario:basic same same unstable
[-15.634µs; +31.536µs] or [-4.859%; +9.802%]		 unstable
[-213.168µs; +514.977µs] or [-39.654%; +95.798%]		 unstable
[-176.520op/s; +176.520op/s] or [-7.061%; +7.061%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (311.577 µs) : 275, 348
.   : milestone, 312,
basic (310.682 µs) : 303, 319
.   : milestone, 311,
loop (12.09 ms) : 11781, 12398
.   : milestone, 12090,
section candidate
noprobe (333.679 µs) : 248, 419
.   : milestone, 334,
basic (315.006 µs) : 296, 334
.   : milestone, 315,
loop (10.382 ms) : 10341, 10423
.   : milestone, 10382,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 311.577 µs [275.244 µs, 347.911 µs]
basic 310.682 µs [302.818 µs, 318.547 µs]
loop 12.09 ms [11.781 ms, 12.398 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 333.679 µs [248.272 µs, 419.085 µs]
basic 315.006 µs [296.347 µs, 333.665 µs]
loop 10.382 ms [10.341 ms, 10.423 ms]

@smola smola marked this pull request as ready for review December 20, 2024 17:10
@smola smola requested review from a team as code owners December 20, 2024 17:10
@smola smola requested review from cimi and mcculls and removed request for a team December 20, 2024 17:10
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 20, 2024 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@pr-commenter
Copy link

pr-commenter bot commented Dec 20, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/revert-io-pr
git_commit_date 1734694675 1734712386
git_commit_sha a3e9bda 324f9f2
release_version 1.45.0-SNAPSHOT~a3e9bda406 1.45.0-SNAPSHOT~324f9f2de1
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1734714721 1734714721
ci_job_id 747248506 747248506
ci_pipeline_id 51666044 51666044
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 6 performance improvements and 16 performance regressions! Performance is the same for 36 metrics, 5 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent.start worse
[+58.829ms; +66.440ms] or [+4.998%; +5.644%]		 1.240s 1.177s
scenario:startup:insecure-bank:iast:GlobalTracer worse
[+60.930ms; +64.447ms] or [+24.721%; +26.148%]		 309.163ms 246.475ms
scenario:startup:insecure-bank:iast:Remote Config better
[-70.293µs; -28.233µs] or [-10.762%; -4.323%]		 603.879µs 653.142µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start worse
[+43.831ms; +75.688ms] or [+3.694%; +6.379%]		 1.246s 1.187s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:GlobalTracer worse
[+59.362ms; +64.935ms] or [+23.905%; +26.150%]		 310.469ms 248.321ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Remote Config better
[-71.393µs; -34.948µs] or [-10.933%; -5.352%]		 599.802µs 652.973µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start worse
[+42.170ms; +72.985ms] or [+3.573%; +6.184%]		 1.238s 1.180s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:GlobalTracer worse
[+59.518ms; +64.741ms] or [+24.000%; +26.107%]		 310.117ms 247.988ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config better
[-57.926µs; -28.388µs] or [-9.177%; -4.498%]		 588.035µs 631.192µs
scenario:startup:insecure-bank:tracing:Agent.start worse
[+53.177ms; +60.004ms] or [+5.045%; +5.692%]		 1.111s 1.054s
scenario:startup:insecure-bank:tracing:GlobalTracer worse
[+62.405ms; +64.765ms] or [+24.389%; +25.311%]		 319.462ms 255.876ms
scenario:startup:insecure-bank:tracing:AppSec better
[-7.137ms; -2.398ms] or [-11.883%; -3.993%]		 55.292ms 60.059ms
scenario:startup:petclinic:appsec:Agent.start worse
[+57.192ms; +82.651ms] or [+4.829%; +6.979%]		 1.254s 1.184s
scenario:startup:petclinic:appsec:GlobalTracer worse
[+62.683ms; +67.337ms] or [+24.839%; +26.683%]		 317.371ms 252.361ms
scenario:startup:petclinic:iast:Agent.start worse
[+24.043ms; +65.736ms] or [+2.010%; +5.495%]		 1.241s 1.196s
scenario:startup:petclinic:iast:GlobalTracer worse
[+54.956ms; +63.652ms] or [+21.908%; +25.375%]		 310.149ms 250.845ms
scenario:startup:petclinic:iast:Remote Config better
[-108.869µs; -47.834µs] or [-16.213%; -7.124%]		 593.132µs 671.483µs
scenario:startup:petclinic:iast:Telemetry better
[-745.049µs; -226.416µs] or [-8.570%; -2.604%]		 8.208ms 8.694ms
scenario:startup:petclinic:profiling:Agent.start worse
[+63.529ms; +86.705ms] or [+4.992%; +6.813%]		 1.348s 1.273s
scenario:startup:petclinic:profiling:GlobalTracer worse
[+66.878ms; +73.298ms] or [+17.999%; +19.727%]		 441.652ms 371.564ms
scenario:startup:petclinic:tracing:Agent.start worse
[+51.180ms; +66.672ms] or [+4.855%; +6.324%]		 1.113s 1.054s
scenario:startup:petclinic:tracing:GlobalTracer worse
[+60.394ms; +68.938ms] or [+23.542%; +26.872%]		 321.205ms 256.540ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~324f9f2de1, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1054259
Total [baseline] (10.439 s) : 0, 10439267
Agent [candidate] (1.113 s) : 0, 1113185
Total [candidate] (10.481 s) : 0, 10480714
section appsec
Agent [baseline] (1.184 s) : 0, 1184300
Total [baseline] (10.743 s) : 0, 10742762
Agent [candidate] (1.254 s) : 0, 1254222
Total [candidate] (10.747 s) : 0, 10747093
section iast
Agent [baseline] (1.196 s) : 0, 1196332
Total [baseline] (10.992 s) : 0, 10992368
Agent [candidate] (1.241 s) : 0, 1241222
Total [candidate] (11.033 s) : 0, 11033255
section profiling
Agent [baseline] (1.273 s) : 0, 1272551
Total [baseline] (10.82 s) : 0, 10819916
Agent [candidate] (1.348 s) : 0, 1347668
Total [candidate] (10.995 s) : 0, 10994746
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent appsec 1.184 s 130.041 ms (12.3%)
Agent iast 1.196 s 142.073 ms (13.5%)
Agent profiling 1.273 s 218.292 ms (20.7%)
Total tracing 10.439 s -
Total appsec 10.743 s 303.496 ms (2.9%)
Total iast 10.992 s 553.102 ms (5.3%)
Total profiling 10.82 s 380.649 ms (3.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.113 s -
Agent appsec 1.254 s 141.037 ms (12.7%)
Agent iast 1.241 s 128.037 ms (11.5%)
Agent profiling 1.348 s 234.483 ms (21.1%)
Total tracing 10.481 s -
Total appsec 10.747 s 266.379 ms (2.5%)
Total iast 11.033 s 552.541 ms (5.3%)
Total profiling 10.995 s 514.032 ms (4.9%) 
gantt
    title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~324f9f2de1, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (712.82 ms) : 0, 712820
BytebuddyAgent [candidate] (711.783 ms) : 0, 711783
GlobalTracer [baseline] (256.54 ms) : 0, 256540
GlobalTracer [candidate] (321.205 ms) : 0, 321205
AppSec [baseline] (57.238 ms) : 0, 57238
AppSec [candidate] (55.536 ms) : 0, 55536
Remote Config [baseline] (687.546 µs) : 0, 688
Remote Config [candidate] (672.284 µs) : 0, 672
Telemetry [baseline] (11.895 ms) : 0, 11895
Telemetry [candidate] (8.988 ms) : 0, 8988
section appsec
BytebuddyAgent [baseline] (727.835 ms) : 0, 727835
BytebuddyAgent [candidate] (733.004 ms) : 0, 733004
GlobalTracer [baseline] (252.361 ms) : 0, 252361
GlobalTracer [candidate] (317.371 ms) : 0, 317371
AppSec [baseline] (170.629 ms) : 0, 170629
AppSec [candidate] (170.128 ms) : 0, 170128
Remote Config [baseline] (650.494 µs) : 0, 650
Remote Config [candidate] (653.619 µs) : 0, 654
Telemetry [baseline] (8.352 ms) : 0, 8352
Telemetry [candidate] (7.935 ms) : 0, 7935
IAST [baseline] (19.264 ms) : 0, 19264
IAST [candidate] (20.192 ms) : 0, 20192
section iast
BytebuddyAgent [baseline] (840.426 ms) : 0, 840426
BytebuddyAgent [candidate] (828.935 ms) : 0, 828935
GlobalTracer [baseline] (250.845 ms) : 0, 250845
GlobalTracer [candidate] (310.149 ms) : 0, 310149
AppSec [baseline] (58.747 ms) : 0, 58747
AppSec [candidate] (57.524 ms) : 0, 57524
Remote Config [baseline] (671.483 µs) : 0, 671
Remote Config [candidate] (593.132 µs) : 0, 593
Telemetry [baseline] (8.694 ms) : 0, 8694
Telemetry [candidate] (8.208 ms) : 0, 8208
IAST [baseline] (21.79 ms) : 0, 21790
IAST [candidate] (20.877 ms) : 0, 20877
section profiling
BytebuddyAgent [baseline] (700.486 ms) : 0, 700486
BytebuddyAgent [candidate] (706.727 ms) : 0, 706727
GlobalTracer [baseline] (371.564 ms) : 0, 371564
GlobalTracer [candidate] (441.652 ms) : 0, 441652
AppSec [baseline] (54.247 ms) : 0, 54247
AppSec [candidate] (54.198 ms) : 0, 54198
Remote Config [baseline] (651.081 µs) : 0, 651
Remote Config [candidate] (652.167 µs) : 0, 652
Telemetry [baseline] (7.781 ms) : 0, 7781
Telemetry [candidate] (7.845 ms) : 0, 7845
ProfilingAgent [baseline] (96.206 ms) : 0, 96206
ProfilingAgent [candidate] (94.568 ms) : 0, 94568
Profiling [baseline] (96.229 ms) : 0, 96229
Profiling [candidate] (94.595 ms) : 0, 94595
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~324f9f2de1, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1054151
Total [baseline] (8.643 s) : 0, 8642653
Agent [candidate] (1.111 s) : 0, 1110741
Total [candidate] (8.684 s) : 0, 8684081
section iast
Agent [baseline] (1.177 s) : 0, 1177151
Total [baseline] (9.223 s) : 0, 9222964
Agent [candidate] (1.24 s) : 0, 1239786
Total [candidate] (9.265 s) : 0, 9265195
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.187 s) : 0, 1186507
Total [baseline] (9.209 s) : 0, 9209144
Agent [candidate] (1.246 s) : 0, 1246266
Total [candidate] (9.265 s) : 0, 9265106
section iast_TELEMETRY_OFF
Agent [baseline] (1.18 s) : 0, 1180190
Total [baseline] (9.216 s) : 0, 9216065
Agent [candidate] (1.238 s) : 0, 1237768
Total [candidate] (9.284 s) : 0, 9283903
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent iast 1.177 s 123.001 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.187 s 132.356 ms (12.6%)
Agent iast_TELEMETRY_OFF 1.18 s 126.04 ms (12.0%)
Total tracing 8.643 s -
Total iast 9.223 s 580.311 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.209 s 566.491 ms (6.6%)
Total iast_TELEMETRY_OFF 9.216 s 573.412 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.111 s -
Agent iast 1.24 s 129.045 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.246 s 135.525 ms (12.2%)
Agent iast_TELEMETRY_OFF 1.238 s 127.027 ms (11.4%)
Total tracing 8.684 s -
Total iast 9.265 s 581.114 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.265 s 581.026 ms (6.7%)
Total iast_TELEMETRY_OFF 9.284 s 599.822 ms (6.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~324f9f2de1, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (711.504 ms) : 0, 711504
BytebuddyAgent [candidate] (711.564 ms) : 0, 711564
GlobalTracer [baseline] (255.876 ms) : 0, 255876
GlobalTracer [candidate] (319.462 ms) : 0, 319462
AppSec [baseline] (60.059 ms) : 0, 60059
AppSec [candidate] (55.292 ms) : 0, 55292
Remote Config [baseline] (687.118 µs) : 0, 687
Remote Config [candidate] (668.416 µs) : 0, 668
Telemetry [baseline] (11.107 ms) : 0, 11107
Telemetry [candidate] (8.865 ms) : 0, 8865
section iast
BytebuddyAgent [baseline] (827.586 ms) : 0, 827586
BytebuddyAgent [candidate] (828.297 ms) : 0, 828297
GlobalTracer [baseline] (246.475 ms) : 0, 246475
GlobalTracer [candidate] (309.163 ms) : 0, 309163
AppSec [baseline] (57.892 ms) : 0, 57892
AppSec [candidate] (57.456 ms) : 0, 57456
Remote Config [baseline] (653.142 µs) : 0, 653
Remote Config [candidate] (603.879 µs) : 0, 604
Telemetry [baseline] (8.472 ms) : 0, 8472
Telemetry [candidate] (8.247 ms) : 0, 8247
IAST [baseline] (21.181 ms) : 0, 21181
IAST [candidate] (21.106 ms) : 0, 21106
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (834.314 ms) : 0, 834314
BytebuddyAgent [candidate] (832.002 ms) : 0, 832002
GlobalTracer [baseline] (248.321 ms) : 0, 248321
GlobalTracer [candidate] (310.469 ms) : 0, 310469
AppSec [baseline] (58.265 ms) : 0, 58265
AppSec [candidate] (57.839 ms) : 0, 57839
Remote Config [baseline] (652.973 µs) : 0, 653
Remote Config [candidate] (599.802 µs) : 0, 600
Telemetry [baseline] (8.489 ms) : 0, 8489
Telemetry [candidate] (8.302 ms) : 0, 8302
IAST [baseline] (21.472 ms) : 0, 21472
IAST [candidate] (22.027 ms) : 0, 22027
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (830.019 ms) : 0, 830019
BytebuddyAgent [candidate] (825.616 ms) : 0, 825616
GlobalTracer [baseline] (247.988 ms) : 0, 247988
GlobalTracer [candidate] (310.117 ms) : 0, 310117
AppSec [baseline] (57.534 ms) : 0, 57534
AppSec [candidate] (58.004 ms) : 0, 58004
Remote Config [baseline] (631.192 µs) : 0, 631
Remote Config [candidate] (588.035 µs) : 0, 588
Telemetry [baseline] (8.283 ms) : 0, 8283
Telemetry [candidate] (8.041 ms) : 0, 8041
IAST [baseline] (20.784 ms) : 0, 20784
IAST [candidate] (20.485 ms) : 0, 20485
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-12-20T16:44:04 2024-12-20T16:51:04
git_branch master smola/revert-io-pr
git_commit_date 1734694675 1734712386
git_commit_sha a3e9bda 324f9f2
release_version 1.45.0-SNAPSHOT~a3e9bda406 1.45.0-SNAPSHOT~324f9f2de1
start_time 2024-12-20T16:43:50 2024-12-20T16:50:50
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1734713817 1734713817
ci_job_id 747248507 747248507
ci_pipeline_id 51666044 51666044
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~324f9f2de1, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (385.212 µs) : 365, 405
.   : milestone, 385,
iast (500.912 µs) : 479, 523
.   : milestone, 501,
iast_FULL (653.012 µs) : 632, 675
.   : milestone, 653,
iast_GLOBAL (514.841 µs) : 493, 536
.   : milestone, 515,
iast_HARDCODED_SECRET_DISABLED (496.155 µs) : 475, 517
.   : milestone, 496,
iast_INACTIVE (455.331 µs) : 434, 477
.   : milestone, 455,
iast_TELEMETRY_OFF (478.29 µs) : 457, 500
.   : milestone, 478,
tracing (459.008 µs) : 437, 481
.   : milestone, 459,
section candidate
no_agent (371.857 µs) : 352, 392
.   : milestone, 372,
iast (502.494 µs) : 481, 524
.   : milestone, 502,
iast_FULL (652.445 µs) : 631, 674
.   : milestone, 652,
iast_GLOBAL (533.063 µs) : 511, 555
.   : milestone, 533,
iast_HARDCODED_SECRET_DISABLED (493.719 µs) : 472, 516
.   : milestone, 494,
iast_INACTIVE (451.479 µs) : 431, 472
.   : milestone, 451,
iast_TELEMETRY_OFF (488.294 µs) : 466, 511
.   : milestone, 488,
tracing (450.013 µs) : 430, 471
.   : milestone, 450,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.212 µs [365.067 µs, 405.358 µs] -
iast 500.912 µs [479.098 µs, 522.726 µs] 115.699 µs (30.0%)
iast_FULL 653.012 µs [631.501 µs, 674.523 µs] 267.8 µs (69.5%)
iast_GLOBAL 514.841 µs [493.379 µs, 536.303 µs] 129.629 µs (33.7%)
iast_HARDCODED_SECRET_DISABLED 496.155 µs [474.825 µs, 517.485 µs] 110.943 µs (28.8%)
iast_INACTIVE 455.331 µs [434.094 µs, 476.567 µs] 70.118 µs (18.2%)
iast_TELEMETRY_OFF 478.29 µs [457.058 µs, 499.523 µs] 93.078 µs (24.2%)
tracing 459.008 µs [437.384 µs, 480.633 µs] 73.796 µs (19.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 371.857 µs [351.623 µs, 392.091 µs] -
iast 502.494 µs [480.706 µs, 524.281 µs] 130.637 µs (35.1%)
iast_FULL 652.445 µs [630.755 µs, 674.135 µs] 280.588 µs (75.5%)
iast_GLOBAL 533.063 µs [511.183 µs, 554.943 µs] 161.206 µs (43.4%)
iast_HARDCODED_SECRET_DISABLED 493.719 µs [471.791 µs, 515.647 µs] 121.863 µs (32.8%)
iast_INACTIVE 451.479 µs [430.565 µs, 472.394 µs] 79.623 µs (21.4%)
iast_TELEMETRY_OFF 488.294 µs [466.013 µs, 510.576 µs] 116.438 µs (31.3%)
tracing 450.013 µs [429.525 µs, 470.501 µs] 78.157 µs (21.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~324f9f2de1, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.368 ms) : 1349, 1388
.   : milestone, 1368,
appsec (1.752 ms) : 1728, 1775
.   : milestone, 1752,
appsec_no_iast (1.762 ms) : 1736, 1787
.   : milestone, 1762,
iast (1.505 ms) : 1482, 1528
.   : milestone, 1505,
profiling (1.546 ms) : 1521, 1570
.   : milestone, 1546,
tracing (1.507 ms) : 1483, 1531
.   : milestone, 1507,
section candidate
no_agent (1.352 ms) : 1333, 1371
.   : milestone, 1352,
appsec (1.74 ms) : 1716, 1764
.   : milestone, 1740,
appsec_no_iast (1.749 ms) : 1724, 1774
.   : milestone, 1749,
iast (1.5 ms) : 1477, 1523
.   : milestone, 1500,
profiling (1.52 ms) : 1496, 1544
.   : milestone, 1520,
tracing (1.478 ms) : 1453, 1504
.   : milestone, 1478,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.368 ms [1.349 ms, 1.388 ms] -
appsec 1.752 ms [1.728 ms, 1.775 ms] 383.311 µs (28.0%)
appsec_no_iast 1.762 ms [1.736 ms, 1.787 ms] 393.278 µs (28.7%)
iast 1.505 ms [1.482 ms, 1.528 ms] 136.7 µs (10.0%)
profiling 1.546 ms [1.521 ms, 1.57 ms] 177.544 µs (13.0%)
tracing 1.507 ms [1.483 ms, 1.531 ms] 138.648 µs (10.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.333 ms, 1.371 ms] -
appsec 1.74 ms [1.716 ms, 1.764 ms] 388.04 µs (28.7%)
appsec_no_iast 1.749 ms [1.724 ms, 1.774 ms] 396.654 µs (29.3%)
iast 1.5 ms [1.477 ms, 1.523 ms] 147.684 µs (10.9%)
profiling 1.52 ms [1.496 ms, 1.544 ms] 167.738 µs (12.4%)
tracing 1.478 ms [1.453 ms, 1.504 ms] 126.219 µs (9.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/revert-io-pr
git_commit_date 1734694675 1734712386
git_commit_sha a3e9bda 324f9f2
release_version 1.45.0-SNAPSHOT~a3e9bda406 1.45.0-SNAPSHOT~324f9f2de1
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1734714399 1734714399
ci_job_id 747248508 747248508
ci_pipeline_id 51666044 51666044
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~324f9f2de1, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.468 s) : 15468000, 15468000
.   : milestone, 15468000,
appsec (14.892 s) : 14892000, 14892000
.   : milestone, 14892000,
iast (18.88 s) : 18880000, 18880000
.   : milestone, 18880000,
iast_GLOBAL (17.922 s) : 17922000, 17922000
.   : milestone, 17922000,
profiling (14.946 s) : 14946000, 14946000
.   : milestone, 14946000,
tracing (14.847 s) : 14847000, 14847000
.   : milestone, 14847000,
section candidate
no_agent (14.86 s) : 14860000, 14860000
.   : milestone, 14860000,
appsec (14.974 s) : 14974000, 14974000
.   : milestone, 14974000,
iast (18.917 s) : 18917000, 18917000
.   : milestone, 18917000,
iast_GLOBAL (17.703 s) : 17703000, 17703000
.   : milestone, 17703000,
profiling (14.971 s) : 14971000, 14971000
.   : milestone, 14971000,
tracing (15.225 s) : 15225000, 15225000
.   : milestone, 15225000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.468 s [15.468 s, 15.468 s] -
appsec 14.892 s [14.892 s, 14.892 s] -576.0 ms (-3.7%)
iast 18.88 s [18.88 s, 18.88 s] 3.412 s (22.1%)
iast_GLOBAL 17.922 s [17.922 s, 17.922 s] 2.454 s (15.9%)
profiling 14.946 s [14.946 s, 14.946 s] -522.0 ms (-3.4%)
tracing 14.847 s [14.847 s, 14.847 s] -621.0 ms (-4.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.86 s [14.86 s, 14.86 s] -
appsec 14.974 s [14.974 s, 14.974 s] 114.0 ms (0.8%)
iast 18.917 s [18.917 s, 18.917 s] 4.057 s (27.3%)
iast_GLOBAL 17.703 s [17.703 s, 17.703 s] 2.843 s (19.1%)
profiling 14.971 s [14.971 s, 14.971 s] 111.0 ms (0.7%)
tracing 15.225 s [15.225 s, 15.225 s] 365.0 ms (2.5%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~324f9f2de1, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.36 ms) : 2318, 2401
.   : milestone, 2360,
iast (2.092 ms) : 2039, 2145
.   : milestone, 2092,
iast_GLOBAL (2.141 ms) : 2088, 2194
.   : milestone, 2141,
profiling (1.965 ms) : 1922, 2007
.   : milestone, 1965,
tracing (1.95 ms) : 1909, 1990
.   : milestone, 1950,
section candidate
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.357 ms) : 2315, 2399
.   : milestone, 2357,
iast (2.087 ms) : 2034, 2140
.   : milestone, 2087,
iast_GLOBAL (2.133 ms) : 2079, 2186
.   : milestone, 2133,
profiling (2.463 ms) : 2274, 2652
.   : milestone, 2463,
tracing (1.935 ms) : 1895, 1976
.   : milestone, 1935,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.487 ms] -
appsec 2.36 ms [2.318 ms, 2.401 ms] 884.339 µs (59.9%)
iast 2.092 ms [2.039 ms, 2.145 ms] 616.307 µs (41.8%)
iast_GLOBAL 2.141 ms [2.088 ms, 2.194 ms] 665.561 µs (45.1%)
profiling 1.965 ms [1.922 ms, 2.007 ms] 488.986 µs (33.1%)
tracing 1.95 ms [1.909 ms, 1.99 ms] 474.235 µs (32.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.357 ms [2.315 ms, 2.399 ms] 887.77 µs (60.4%)
iast 2.087 ms [2.034 ms, 2.14 ms] 617.727 µs (42.0%)
iast_GLOBAL 2.133 ms [2.079 ms, 2.186 ms] 663.297 µs (45.1%)
profiling 2.463 ms [2.274 ms, 2.652 ms] 994.111 µs (67.7%)
tracing 1.935 ms [1.895 ms, 1.976 ms] 465.87 µs (31.7%)

@mcculls
Copy link
Contributor

mcculls commented Dec 20, 2024

FYI, this is for testing purposes only and not to be merged

The blocking I/O fix has uncovered a separate issue which will be addressed in a separate PR - reverting the change will merely cover up the issue again :)

@PerfectSlayer PerfectSlayer added the tag: do not merge Do not merge changes label Dec 23, 2024
@PerfectSlayer
Copy link
Contributor

Assigning the DNM tag then 😉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jpbempel jpbempel jpbempel approved these changes

@cimi cimi Awaiting requested review from cimi cimi is a code owner automatically assigned from DataDog/debugger-java

@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
run-tests: all Run all tests tag: do not merge Do not merge changes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@smola @mcculls @PerfectSlayer @jpbempel