How do I disable polling traces to an SQS queue?

[murshed-panorama]

Background

I have an application that periodically polls an AWS SQS queue to see if there are jobs available with the Shoryuken gem.

Problem

I'm ultimately exporting these traces to AWS X-Ray, and they aren't all that useful. What is the best way to exclude these traces?

Example Raw Trace from AWS X-Ray

{
    "Id": "1-6c8444c8-baad0ad72511bc0720d5a8c6",
    "Duration": 0.042,
    "LimitExceeded": false,
    "Segments": [
        {
            "Id": "7d15bec774b83caf",
            "Document": {
                "id": "7d15bec774b83caf",
                "name": "SQS",
                "start_time": 1710359055.0314865,
                "trace_id": "1-6c8444c8-baad0ad72511bc0720d5a8c6",
                "end_time": 1710359055.073655,
                "fault": false,
                "error": false,
                "throttle": false,
                "http": {
                    "request": {},
                    "response": {
                        "status": 200,
                        "content_length": 0
                    }
                },
                "aws": {
                    "queue_url": "https://sqs.us-east-1.amazonaws.com/123456789012/my-queue-name",
                    "xray": {
                        "auto_instrumentation": false,
                        "sdk_version": "1.4.0",
                        "sdk": "opentelemetry for ruby"
                    },
                    "region": "us-east-1",
                    "operation": "SendMessageBatch"
                },
                "annotations": {
                    "messaging_system": "aws.sqs",
                    "otel_resource_process_pid": 1,
                    "otel_resource_telemetry_sdk_name": "opentelemetry",
                    "otel_resource_process_runtime_version": "3.1.4",
                    "messaging_url": "https://sqs.us-east-1.amazonaws.com/123456789012/my-queue-name",
                    "otel_resource_telemetry_sdk_language": "ruby",
                    "rpc_system": "aws-api",
                    "otel_resource_process_runtime_description": "ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [x86_64-linux]",
                    "otel_resource_telemetry_sdk_version": "1.4.0",
                    "otel_resource_process_command": "/usr/packages/ruby-3.1.4/bin/shoryuken",
                    "messaging_destination_kind": "queue",
                    "rpc_service": "SQS",
                    "otel_resource_process_runtime_name": "ruby",
                    "messaging_destination": "my-queue-name",
                    "otel_resource_service_name": "my_app"
                }
            }
        }
    ]
}

Potential Solutions

Not Instrument Them

Ideally, I would like my application to just not instrument these calls at all. I've seen documentation from the Instrumentation for the AWS SDK that might do this, but I'm unsure.

I have not found a detailed explanation for what suppress_internal_instrumentation actually does.

Tail Sampling

At my collector, I could use a tail sampling or filter processor to filter these requests out. What would be the appropriate fields to filter by?

[arielvalentin]

The SQS instrumentation is currently unmaintained and I have to admit I don't know much about it.

Generally speaking filtering spans in the collector may work but you'll likely need to match the instrumentation scope name and the span name but without a bit more research I don't know what those would be.

I am not familiar with the AWS SDK configuration options or this instrumentation so I cannot confirm if the option you describe will have any effect on suppressing polling spans.

We have set a precedent
in some instrumentations to suppress polling spans using an instrumentations specific configuration option.

If you're up for it, I'm amenable to accepting a contribution from you that adds this option to the SQS instrumentation.

Would you be interested in helping us out with this?

There's also an open PR for shoryuken that the contributor is not able to maintain longer term so we did not accept it. That instrumentation may provide you with additional insights for non-polling spans if you'd be interested in maintaining it going forward.

[murshed-panorama]

Thanks for the info!

There's also an open PR for shoryuken that the contributor is not able to maintain longer term so we did not accept it. That instrumentation may provide you with additional insights for non-polling spans if you'd be interested in maintaining it going forward.

If you have a link to that PR, I'd love to check it out and see if I can contribute to the project.

[arielvalentin]

Here was the initial submission:

open-telemetry/opentelemetry-ruby-contrib#536