[Bug]: CLI PHP Processes do not stop and with Apache (mod_php) the connection is closed unexpectedly

[elzekool]

Bug report

On our platform we see two behaviors that seem related to the DataDog PHP tracing agent:

• PHP Processed execute but then don't stop (e.g. keep running)
• Apache kills connections unexpectedly

Both issues only arrise sporadicly and we did not find an exact trigger that makes it hard to provide details on the second issue. We could connect with one stuck process with GDB and extracted the following stacktrace:

root@juju-machine-7:/home/ubuntu# strace -p 1333711
strace: Process 1333711 attached
futex(0x7f9fa43899d0, FUTEX_WAIT, 1333735, NULL

#0 __pthread_clockjoin_ex (threadid=140323631699712, thread_return=thread_return@entry=0x0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, block=block@entry=true) at pthread_join_common.c:145
#1 0x00007f9fb0d1e9f3 in __pthread_join (threadid=, thread_return=thread_return@entry=0x0) at pthread_join.c:24
#2 0x00007f9face1f39a in ddtrace_coms_flush_shutdown_writer_synchronous () at /home/circleci/datadog/tmp/build_extension/ext/coms.c:1237
#3 0x00007f9face1f42f in ddtrace_coms_flush_shutdown_writer_synchronous () at /home/circleci/datadog/tmp/build_extension/ext/coms.c:1197
#4 0x00007f9face157b2 in zm_shutdown_ddtrace (type=, module_number=25) at /home/circleci/datadog/tmp/build_extension/ext/ddtrace.c:897
#5 0x00005566fde21e1f in ?? ()
#6 0x00005566fde1ba50 in ?? ()
#7 0x00005566fde2d639 in zend_hash_graceful_reverse_destroy ()
#8 0x00005566fde1c9e7 in ?? ()
#9 0x00005566fddbc6e2 in php_module_shutdown ()
#10 0x00005566fdc82a09 in ?? ()
#11 0x00007f9fb0d5c083 in __libc_start_main (main=0x5566fdc825e0, argc=3, argv=0x7fff3af15a38, init=, fini=, rtld_fini=, stack_end=0x7fff3af15a28) at ../csu/libc-start.c:308
#12 0x00005566fdc831be in _start ()

I could trace this to the following function call:
https://github.com/DataDog/dd-trace-php/blob/0.96.0/ext/coms.c#L1237

Additional information:

Agent version: 7.50.3
dd-trace: 0.96
PHP version: 7.4.3-4ubuntu2.20
Linux/Ubuntu version: Ubuntu 20.04.6 LTS

PHP version

7.4.3-4ubuntu2.20

Tracer or profiler version

0.96

Installed extensions

[PHP Modules]
amqp
apcu
bcmath
bz2
calendar
Core
ctype
curl
date
dom
enchant
exif
fileinfo
filter
ftp
gd
gearman
geoip
gettext
gmp
hash
iconv
igbinary
imagick
imap
intl
json
ldap
libxml
mbstring
memcache
memcached
mongodb
msgpack
mysqli
mysqlnd
oci8
odbc
openssl
pcntl
pcre
PDO
pdo_mysql
PDO_OCI
PDO_ODBC
pdo_pgsql
pdo_sqlsrv
pgsql
Phar
posix
readline
redis
Reflection
session
shmop
SimpleXML
soap
sockets
sodium
SPL
sqlsrv
ssh2
standard
tidy
tokenizer
xml
xmlreader
xmlrpc
xmlwriter
xsl
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

Output of phpinfo()

ddtrace

Datadog PHP tracer extension
For help, check out the documentation at https://docs.datadoghq.com/tracing/languages/php/
(c) Datadog 2020

Datadog tracing support => enabled
Version => 0.96.0
DATADOG TRACER CONFIGURATION => {
"date": "2024-04-17T07:37:59Z",
"os_name": "Linux juju-machine-7 5.4.0-173-generic #191-Ubuntu SMP Fri Feb 2 13:55:07 UTC 2024 x86_64",
"os_version": "5.4.0-173-generic",
"version": "0.96.0",
"lang": "php",
"lang_version": "7.4.33",
"env": "postnl.1c.eu-central.aws",
"enabled": true,
"service": "staging.shop.postnl.nl",
"enabled_cli": true,
"agent_url": "http://127.0.0.1:1400",
"debug": false,
"analytics_enabled": false,
"sample_rate": -1,
"sampling_rules": [],
"tags": {
"hosting.domain.main": "staging.shop.postnl.nl",
"hosting.domain.alternate": "staging.shop.postnl.nl",
"hosting.context": "acceptance",
"hosting.source": "cli"
},
"service_mapping": [],
"distributed_tracing_enabled": true,
"priority_sampling_enabled": true,
"dd_version": "20240320105200",
"architecture": "x86_64",
"sapi": "cli",
"datadog.trace.request_init_hook": "/opt/datadog/dd-library/0.96.0/dd-trace-sources/bridge/dd_wrap_autoloader.php",
"open_basedir_configured": false,
"uri_fragment_regex": null,
"uri_mapping_incoming": null,
"uri_mapping_outgoing": null,
"auto_flush_enabled": false,
"generate_root_span": true,
"http_client_split_by_domain": false,
"measure_compile_time": true,
"report_hostname_on_root_span": false,
"traced_internal_functions": null,
"auto_prepend_file_configured": true,
"integrations_disabled": "default",
"enabled_from_env": true,
"opcache.file_cache": null
}

                           Diagnostics                               

Diagnostic checks => passed

Directive => Local Value => Master Value
ddtrace.disable => Off => Off
ddtrace.cgroup_file => /proc/self/cgroup => /proc/self/cgroup
datadog.trace.request_init_hook => /opt/datadog/dd-library/0.96.0/dd-trace-sources/bridge/dd_wrap_autoloader.php => /opt/datadog/dd-library/0.96.0/dd-trace-sources/bridge/dd_wrap_autoloader.php
ddtrace.request_init_hook => /opt/datadog/dd-library/0.96.0/dd-trace-sources/bridge/dd_wrap_autoloader.php => /opt/datadog/dd-library/0.96.0/dd-trace-sources/bridge/dd_wrap_autoloader.php
datadog.trace.agent_url => no value => no value
datadog.agent_host => 127.0.0.1 => 127.0.0.1
datadog.dogstatsd_url => no value => no value
datadog.api_key => no value => no value
datadog.distributed_tracing => On => On
datadog.dogstatsd_port => 1401 => 1401
datadog.env => postnl.1c.eu-central.aws => postnl.1c.eu-central.aws
datadog.autofinish_spans => Off => Off
datadog.trace.url_as_resource_names_enabled => On => On
datadog.http_server_route_based_naming => On => On
datadog.integrations_disabled => default => default
datadog.priority_sampling => On => On
datadog.service => staging.shop.postnl.nl => staging.shop.postnl.nl
datadog.service_name => staging.shop.postnl.nl => staging.shop.postnl.nl
datadog.service_mapping => no value => no value
datadog.tags => hosting.domain.main:staging.shop.postnl.nl,hosting.domain.alternate:staging.shop.postnl.nl,hosting.context:acceptance,hosting.source:cli => hosting.domain.main:staging.shop.postnl.nl,hosting.domain.alternate:staging.shop.postnl.nl,hosting.context:acceptance,hosting.source:cli
datadog.trace.global_tags => hosting.domain.main:staging.shop.postnl.nl,hosting.domain.alternate:staging.shop.postnl.nl,hosting.context:acceptance,hosting.source:cli => hosting.domain.main:staging.shop.postnl.nl,hosting.domain.alternate:staging.shop.postnl.nl,hosting.context:acceptance,hosting.source:cli
datadog.trace.agent_port => 1400 => 1400
datadog.trace.analytics_enabled => Off => Off
datadog.trace.append_trace_ids_to_logs => Off => Off
datadog.trace.auto_flush_enabled => Off => Off
datadog.trace.cli_enabled => On => On
datadog.trace.measure_compile_time => On => On
datadog.trace.debug => Off => Off
datadog.trace.enabled => On => On
datadog.instrumentation_telemetry_enabled => On => On
datadog.trace.health_metrics_enabled => Off => Off
datadog.trace.health_metrics_heartbeat_sample_rate => 0.001 => 0.001
datadog.trace.db_client_split_by_instance => Off => Off
datadog.trace.http_client_split_by_domain => Off => Off
datadog.trace.redis_client_split_by_host => Off => Off
datadog.trace.memory_limit => no value => no value
datadog.trace.report_hostname => Off => Off
datadog.trace.flush_collect_cycles => Off => Off
datadog.trace.laravel_queue_distributed_tracing => On => On
datadog.trace.remove_root_span_laravel_queue => On => On
datadog.trace.remove_autoinstrumentation_orphans => Off => Off
datadog.trace.resource_uri_fragment_regex => no value => no value
datadog.trace.resource_uri_mapping_incoming => no value => no value
datadog.trace.resource_uri_mapping_outgoing => no value => no value
datadog.trace.resource_uri_query_param_allowed => no value => no value
datadog.trace.http_url_query_param_allowed => * => *
datadog.trace.http_post_data_param_allowed => no value => no value
datadog.trace.rate_limit => 0 => 0
datadog.trace.sample_rate => -1 => -1
datadog.sampling_rate => -1 => -1
datadog.trace.sampling_rules => [] => []
datadog.trace.sampling_rules_format => regex => regex
datadog.span_sampling_rules => [] => []
datadog.span_sampling_rules_file => no value => no value
datadog.trace.header_tags => no value => no value
datadog.trace.x_datadog_tags_max_length => 512 => 512
datadog.trace.peer_service_mapping => no value => no value
datadog.trace.peer_service_defaults_enabled => Off => Off
datadog.trace.remove_integration_service_names_enabled => Off => Off
datadog.trace.propagate_service => Off => Off
datadog.trace.propagation_style_extract => tracecontext,Datadog,B3,B3 single header => tracecontext,Datadog,B3,B3 single header
datadog.propagation_style_extract => tracecontext,Datadog,B3,B3 single header => tracecontext,Datadog,B3,B3 single header
datadog.trace.propagation_style_inject => tracecontext,Datadog => tracecontext,Datadog
datadog.propagation_style_inject => tracecontext,Datadog => tracecontext,Datadog
datadog.trace.propagation_style => tracecontext,Datadog => tracecontext,Datadog
datadog.trace.traced_internal_functions => no value => no value
datadog.trace.agent_timeout => 500 => 500
datadog.trace.agent_connect_timeout => 100 => 100
datadog.trace.debug_prng_seed => -1 => -1
datadog.log_backtrace => Off => Off
datadog.trace.generate_root_span => On => On
datadog.trace.spans_limit => 1000 => 1000
datadog.trace.128_bit_traceid_generation_enabled => On => On
datadog.trace.128_bit_traceid_logging_enabled => Off => Off
datadog.trace.agent_max_consecutive_failures => 3 => 3
datadog.trace.agent_attempt_retry_time_msec => 5000 => 5000
datadog.trace.bgs_connect_timeout => 2000 => 2000
datadog.trace.bgs_timeout => 5000 => 5000
datadog.trace.agent_flush_interval => 5000 => 5000
datadog.trace.agent_flush_after_n_requests => 10 => 10
datadog.trace.shutdown_timeout => 5000 => 5000
datadog.trace.startup_logs => On => On
datadog.trace.once_logs => On => On
datadog.trace.agent_debug_verbose_curl => Off => Off
datadog.trace.debug_curl_output => Off => Off
datadog.trace.beta_high_memory_pressure_percent => 80 => 80
datadog.trace.sidecar_trace_sender => Off => Off
datadog.trace.agentless => Off => Off
datadog.trace.warn_legacy_dd_trace => On => On
datadog.trace.retain_thread_capabilities => Off => Off
datadog.version => 20240320105200 => 20240320105200
datadog.trace.obfuscation_query_string_regexp => (?i)(?:(?:"|%22)?)(?:(?:old[-]?|new[-]?)?p(?:ass)?w(?:or)?d(?:1|2)?|pass(?:[-]?phrase)?|secret|(?:api[-]?|private[-]?|public[-]?|access[-]?|secret[-]?|app(?:lication)?[-]?)key(?:[-]?id)?|token|consumer[-]?(?:id|key|secret)|sign(?:ed|ature)?|auth(?:entication|orization)?)(?:(?:\s|%20)(?:=|%3D)[^&]+|(?:"|%22)(?:\s|%20)(?::|%3A)(?:\s|%20)*(?:"|%22)(?:%2[^2]|%[^2]|[^"%])+(?:"|%22))|(?:bearer(?:\s|%20)+[a-z0-9.-]+|token(?::|%3A)[a-z0-9]{13}|gh[opsu][0-9a-zA-Z]{36}|eyI-L+.eyI-L+(?:.(?:[\w.+/=-]|%3D|%2F|%2B)+)?|-{5}BEGIN(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY-{5}[^\-]+-{5}END(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY(?:-{5})?(?:\n|%0A)?|(?:ssh-(?:rsa|dss)|ecdsa-[a-z0-9]+-[a-z0-9]+)(?:\s|%20|%09)+(?:[a-z0-9/.+]|%2F|%5C|%2B){100,}(?:=|%3D)*(?:(?:\s|%20|%09)+[a-z0-9.-]+)?) => (?i)(?:(?:"|%22)?)(?:(?:old[-]?|new[-]?)?p(?:ass)?w(?:or)?d(?:1|2)?|pass(?:[-]?phrase)?|secret|(?:api[-]?|private[-]?|public[-]?|access[-]?|secret[-]?|app(?:lication)?[-]?)key(?:[-]?id)?|token|consumer[-]?(?:id|key|secret)|sign(?:ed|ature)?|auth(?:entication|orization)?)(?:(?:\s|%20)(?:=|%3D)[^&]+|(?:"|%22)(?:\s|%20)(?::|%3A)(?:\s|%20)*(?:"|%22)(?:%2[^2]|%[^2]|[^"%])+(?:"|%22))|(?:bearer(?:\s|%20)+[a-z0-9.-]+|token(?::|%3A)[a-z0-9]{13}|gh[opsu][0-9a-zA-Z]{36}|eyI-L+.eyI-L+(?:.(?:[\w.+/=-]|%3D|%2F|%2B)+)?|-{5}BEGIN(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY-{5}[^\-]+-{5}END(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY(?:-{5})?(?:\n|%0A)?|(?:ssh-(?:rsa|dss)|ecdsa-[a-z0-9]+-[a-z0-9]+)(?:\s|%20|%09)+(?:[a-z0-9/.+]|%2F|%5C|%2B){100,}(?:=|%3D)*(?:(?:\s|%20|%09)+[a-z0-9.-]+)?)
datadog.trace.client_ip_enabled => Off => Off
datadog.trace.client_ip_header => no value => no value
datadog.trace.forked_process => On => On
datadog.trace.hook_limit => 1000 => 1000
datadog.trace.agent_max_payload_size => 52428800 => 52428800
datadog.trace.agent_stack_initial_size => 131072 => 131072
datadog.trace.agent_stack_backlog => 12 => 12
datadog.trace.propagate_user_id_default => Off => Off
datadog.dbm_propagation_mode => disabled => disabled
datadog.trace.wordpress_additional_actions => no value => no value
datadog.trace.wordpress_callbacks => Off => Off
datadog.trace.wordpress_enhanced_integration => Off => Off
datadog.trace.otel_enabled => Off => Off
datadog.trace.amqp_enabled => On => On
datadog.trace.amqp_analytics_enabled => Off => Off
datadog.amqp_analytics_enabled => Off => Off
datadog.trace.amqp_analytics_sample_rate => 1 => 1
datadog.amqp_analytics_sample_rate => 1 => 1
datadog.trace.cakephp_enabled => On => On
datadog.trace.cakephp_analytics_enabled => Off => Off
datadog.cakephp_analytics_enabled => Off => Off
datadog.trace.cakephp_analytics_sample_rate => 1 => 1
datadog.cakephp_analytics_sample_rate => 1 => 1
datadog.trace.codeigniter_enabled => On => On
datadog.trace.codeigniter_analytics_enabled => Off => Off
datadog.codeigniter_analytics_enabled => Off => Off
datadog.trace.codeigniter_analytics_sample_rate => 1 => 1
datadog.codeigniter_analytics_sample_rate => 1 => 1
datadog.trace.curl_enabled => On => On
datadog.trace.curl_analytics_enabled => Off => Off
datadog.curl_analytics_enabled => Off => Off
datadog.trace.curl_analytics_sample_rate => 1 => 1
datadog.curl_analytics_sample_rate => 1 => 1
datadog.trace.drupal_enabled => On => On
datadog.trace.drupal_analytics_enabled => Off => Off
datadog.drupal_analytics_enabled => Off => Off
datadog.trace.drupal_analytics_sample_rate => 1 => 1
datadog.drupal_analytics_sample_rate => 1 => 1
datadog.trace.elasticsearch_enabled => On => On
datadog.trace.elasticsearch_analytics_enabled => Off => Off
datadog.elasticsearch_analytics_enabled => Off => Off
datadog.trace.elasticsearch_analytics_sample_rate => 1 => 1
datadog.elasticsearch_analytics_sample_rate => 1 => 1
datadog.trace.eloquent_enabled => On => On
datadog.trace.eloquent_analytics_enabled => Off => Off
datadog.eloquent_analytics_enabled => Off => Off
datadog.trace.eloquent_analytics_sample_rate => 1 => 1
datadog.eloquent_analytics_sample_rate => 1 => 1
datadog.trace.guzzle_enabled => On => On
datadog.trace.guzzle_analytics_enabled => Off => Off
datadog.guzzle_analytics_enabled => Off => Off
datadog.trace.guzzle_analytics_sample_rate => 1 => 1
datadog.guzzle_analytics_sample_rate => 1 => 1
datadog.trace.laminas_enabled => On => On
datadog.trace.laminas_analytics_enabled => Off => Off
datadog.laminas_analytics_enabled => Off => Off
datadog.trace.laminas_analytics_sample_rate => 1 => 1
datadog.laminas_analytics_sample_rate => 1 => 1
datadog.trace.laravel_enabled => On => On
datadog.trace.laravel_analytics_enabled => Off => Off
datadog.laravel_analytics_enabled => Off => Off
datadog.trace.laravel_analytics_sample_rate => 1 => 1
datadog.laravel_analytics_sample_rate => 1 => 1
datadog.trace.laravelqueue_enabled => On => On
datadog.trace.laravelqueue_analytics_enabled => Off => Off
datadog.laravelqueue_analytics_enabled => Off => Off
datadog.trace.laravelqueue_analytics_sample_rate => 1 => 1
datadog.laravelqueue_analytics_sample_rate => 1 => 1
datadog.trace.logs_enabled => On => On
datadog.logs_injection => On => On
datadog.trace.logs_analytics_enabled => Off => Off
datadog.logs_analytics_enabled => Off => Off
datadog.trace.logs_analytics_sample_rate => 1 => 1
datadog.logs_analytics_sample_rate => 1 => 1
datadog.trace.lumen_enabled => On => On
datadog.trace.lumen_analytics_enabled => Off => Off
datadog.lumen_analytics_enabled => Off => Off
datadog.trace.lumen_analytics_sample_rate => 1 => 1
datadog.lumen_analytics_sample_rate => 1 => 1
datadog.trace.magento_enabled => On => On
datadog.trace.magento_analytics_enabled => Off => Off
datadog.magento_analytics_enabled => Off => Off
datadog.trace.magento_analytics_sample_rate => 1 => 1
datadog.magento_analytics_sample_rate => 1 => 1
datadog.trace.memcache_enabled => On => On
datadog.trace.memcache_analytics_enabled => Off => Off
datadog.memcache_analytics_enabled => Off => Off
datadog.trace.memcache_analytics_sample_rate => 1 => 1
datadog.memcache_analytics_sample_rate => 1 => 1
datadog.trace.memcached_enabled => On => On
datadog.trace.memcached_analytics_enabled => Off => Off
datadog.memcached_analytics_enabled => Off => Off
datadog.trace.memcached_analytics_sample_rate => 1 => 1
datadog.memcached_analytics_sample_rate => 1 => 1
datadog.trace.mongo_enabled => On => On
datadog.trace.mongo_analytics_enabled => Off => Off
datadog.mongo_analytics_enabled => Off => Off
datadog.trace.mongo_analytics_sample_rate => 1 => 1
datadog.mongo_analytics_sample_rate => 1 => 1
datadog.trace.mongodb_enabled => On => On
datadog.trace.mongodb_analytics_enabled => Off => Off
datadog.mongodb_analytics_enabled => Off => Off
datadog.trace.mongodb_analytics_sample_rate => 1 => 1
datadog.mongodb_analytics_sample_rate => 1 => 1
datadog.trace.mysqli_enabled => On => On
datadog.trace.mysqli_analytics_enabled => Off => Off
datadog.mysqli_analytics_enabled => Off => Off
datadog.trace.mysqli_analytics_sample_rate => 1 => 1
datadog.mysqli_analytics_sample_rate => 1 => 1
datadog.trace.nette_enabled => On => On
datadog.trace.nette_analytics_enabled => Off => Off
datadog.nette_analytics_enabled => Off => Off
datadog.trace.nette_analytics_sample_rate => 1 => 1
datadog.nette_analytics_sample_rate => 1 => 1
datadog.trace.pcntl_enabled => On => On
datadog.trace.pcntl_analytics_enabled => Off => Off
datadog.pcntl_analytics_enabled => Off => Off
datadog.trace.pcntl_analytics_sample_rate => 1 => 1
datadog.pcntl_analytics_sample_rate => 1 => 1
datadog.trace.pdo_enabled => On => On
datadog.trace.pdo_analytics_enabled => Off => Off
datadog.pdo_analytics_enabled => Off => Off
datadog.trace.pdo_analytics_sample_rate => 1 => 1
datadog.pdo_analytics_sample_rate => 1 => 1
datadog.trace.phpredis_enabled => On => On
datadog.trace.phpredis_analytics_enabled => Off => Off
datadog.phpredis_analytics_enabled => Off => Off
datadog.trace.phpredis_analytics_sample_rate => 1 => 1
datadog.phpredis_analytics_sample_rate => 1 => 1
datadog.trace.predis_enabled => On => On
datadog.trace.predis_analytics_enabled => Off => Off
datadog.predis_analytics_enabled => Off => Off
datadog.trace.predis_analytics_sample_rate => 1 => 1
datadog.predis_analytics_sample_rate => 1 => 1
datadog.trace.psr18_enabled => On => On
datadog.trace.psr18_analytics_enabled => Off => Off
datadog.psr18_analytics_enabled => Off => Off
datadog.trace.psr18_analytics_sample_rate => 1 => 1
datadog.psr18_analytics_sample_rate => 1 => 1
datadog.trace.roadrunner_enabled => On => On
datadog.trace.roadrunner_analytics_enabled => Off => Off
datadog.roadrunner_analytics_enabled => Off => Off
datadog.trace.roadrunner_analytics_sample_rate => 1 => 1
datadog.roadrunner_analytics_sample_rate => 1 => 1
datadog.trace.sqlsrv_enabled => On => On
datadog.trace.sqlsrv_analytics_enabled => Off => Off
datadog.sqlsrv_analytics_enabled => Off => Off
datadog.trace.sqlsrv_analytics_sample_rate => 1 => 1
datadog.sqlsrv_analytics_sample_rate => 1 => 1
datadog.trace.slim_enabled => On => On
datadog.trace.slim_analytics_enabled => Off => Off
datadog.slim_analytics_enabled => Off => Off
datadog.trace.slim_analytics_sample_rate => 1 => 1
datadog.slim_analytics_sample_rate => 1 => 1
datadog.trace.symfony_enabled => On => On
datadog.trace.symfony_analytics_enabled => Off => Off
datadog.symfony_analytics_enabled => Off => Off
datadog.trace.symfony_analytics_sample_rate => 1 => 1
datadog.symfony_analytics_sample_rate => 1 => 1
datadog.trace.web_enabled => On => On
datadog.trace.web_analytics_enabled => Off => Off
datadog.web_analytics_enabled => Off => Off
datadog.trace.web_analytics_sample_rate => 1 => 1
datadog.web_analytics_sample_rate => 1 => 1
datadog.trace.wordpress_enabled => On => On
datadog.trace.wordpress_analytics_enabled => Off => Off
datadog.wordpress_analytics_enabled => Off => Off
datadog.trace.wordpress_analytics_sample_rate => 1 => 1
datadog.wordpress_analytics_sample_rate => 1 => 1
datadog.trace.yii_enabled => On => On
datadog.trace.yii_analytics_enabled => Off => Off
datadog.yii_analytics_enabled => Off => Off
datadog.trace.yii_analytics_sample_rate => 1 => 1
datadog.yii_analytics_sample_rate => 1 => 1
datadog.trace.zendframework_enabled => On => On
datadog.trace.zendframework_analytics_enabled => Off => Off
datadog.zendframework_analytics_enabled => Off => Off
datadog.trace.zendframework_analytics_sample_rate => 1 => 1
datadog.zendframework_analytics_sample_rate => 1 => 1

datadog-profiling

Version => 0.96.0
Profiling Enabled => false
Profiling Experimental Features Enabled => false (profiling disabled)
Experimental CPU Time Profiling Enabled => false (profiling disabled)
Allocation Profiling Enabled => false (profiling disabled)
Experimental Timeline Enabled => false (profiling disabled)
Exception Profiling Enabled => false (profiling disabled)
Endpoint Collection Enabled => false (profiling disabled)
Platform's CPU Time API Works => true
Profiling Log Level => off
Profiling Agent Endpoint => http://127.0.0.1:1400/
Application's Environment (DD_ENV) => postnl.1c.eu-central.aws
Application's Service (DD_SERVICE) => staging.shop.postnl.nl
Application's Version (DD_VERSION) => 20240320105200

Directive => Local Value => Master Value
datadog.profiling.enabled => Off => Off
datadog.profiling.experimental_features_enabled => Off => Off
datadog.profiling.endpoint_collection_enabled => On => On
datadog.profiling.experimental_cpu_time_enabled => On => On
datadog.profiling.experimental_cpu_enabled => On => On
datadog.profiling.allocation_enabled => On => On
datadog.profiling.experimental_allocation_enabled => On => On
datadog.profiling.experimental_timeline_enabled => Off => Off
datadog.profiling.exception_enabled => On => On
datadog.profiling.experimental_exception_enabled => On => On
datadog.profiling.exception_sampling_distance => 100 => 100
datadog.profiling.experimental_exception_sampling_distance => 100 => 100
datadog.profiling.log_level => off => off
datadog.profiling.output_pprof => no value => no value

Upgrading from

No response

[bwoebi]

Hey @elzekool,

That should not happen; is the other thread (which we are waiting for) still running? If yes, can you please dump that threads stacktrace too?

Also, does the issue still occur with 0.98.0 or newer? We've modified the retry logic there, maybe that helps.

Lastly, you could try using the sidecar as a trace sender via datadog.trace.sidecar_trace_sender = 1 - then it's not going to wait, but the sending happens in a separate process.