Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems with opentelemetry as INPUT and cloudwatch as OUTPUT #9683

Open
Mistra opened this issue Dec 3, 2024 · 0 comments
Open

Problems with opentelemetry as INPUT and cloudwatch as OUTPUT #9683

Mistra opened this issue Dec 3, 2024 · 0 comments

Comments

@Mistra
Copy link

Mistra commented Dec 3, 2024

Bug Report

Describe the bug
Different outcome from different [OUTPUT] plugins (cloudwatch and stdout) with input opentelemetry

To Reproduce

  • Steps to reproduce the problem:

This is my configuration of fluentbit (latest version)

    [SERVICE]
        Flush        1
        Daemon       Off
        Log_Level debug

    [INPUT]
        Name         opentelemetry
        Listen       0.0.0.0
        Port         4318

    [OUTPUT]
        Name         stdout
        Match        *

    [OUTPUT]
        Name                        cloudwatch_logs
        Match                       *
        region                      eu-west-1
        log_group_name              /aws/containerinsights/test-OTL
        log_stream_prefix           from-fluent-bit-OTL-
        log_retention_days          10
        auto_create_group           true
        role_arn                    xxx
        log_format                  json/emf

I'm feeding to the endpoint "/v1/logs" the following JSON payload

{
	"resourceLogs": [
		{
			"resource": {
				"attributes": [
					{
						"key": "service.name",
						"value": {
							"string_value": "test-service"
						}
					},
					{
						"key": "host.name",
						"value": {
							"string_value": "test-host"
						}
					}
				]
			},
			"scopeLogs": [
				{
					"scope": {
						"name": "test-logger"
					},
					"logRecords": [
						{
							"severityText": "INFO",
							"body": {
								"stringValue": "Test log message from Fluent Bit setup"
							},
							"traceId": "123",
							"spanId": "456",
							"attributes": [
								{
									"key": "environment",
									"value": {
										"string_value": "development"
									}
								},
								{
									"key": "trace.id",
									"value": {
										"string_value": "1a2b3c4d5e6f7890"
									}
								},
								{
									"key": "span.id",
									"value": {
										"string_value": "abc123def456"
									}
								}
							]
						}
					]
				}
			]
		}
	]
}

Expected behavior
In stdout I get this:
[0] v1_logs: [[1733218249.306643377, {"environment"=>"development", "trace.id"=>"1a2b3c4d5e6f7890", "span.id"=>"abc123def456"}], {"log"=>"Test log message from Fluent Bit setup"}]
While in cloudwatch, under the stream "from-fluent-bit-OTL-v1_logs" just this:

{
    "log": "Test log message from Fluent Bit setup"
}

My question is: is it normal not getting the other keys? Is there a way to have on the stream also the attributes keys?

Your Environment

  • Version used: 3.2.2
  • Configuration: Fluentbit is running as a simple test deployment on my k8s cluster
  • Environment name and version: I'm using AWS EKS 1.29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant