Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/journald] Handle MESSAGE field more conveniently #35964

Closed
fuero opened this issue Oct 23, 2024 · 0 comments · Fixed by #36005
Closed

[receiver/journald] Handle MESSAGE field more conveniently #35964

fuero opened this issue Oct 23, 2024 · 0 comments · Fixed by #36005
Labels
bug Something isn't working needs triage New item requiring triage

Comments

@fuero
Copy link
Contributor

fuero commented Oct 23, 2024

Component(s)

No response

What happened?

Description

MESSAGE might be an byte array, if journald thinks there's a non-printable character inside:

journalctl -xe CONTAINER_NAME=logging-test -o json | jq -crs .[].MESSAGE
[72,101,108,108,111,32,87,111,114,108,100,33,13]

The journald receiver forwards this as-is.

Steps to Reproduce

Do printf("Hello World!") from inside a docker container and let its logs be pushed to otel-collector

Expected Result

Forward the log line as unicode string, not byte array. Let the exporter take care of shipping it to the destination correctly.
Otherwise, processing steps become more complex

Actual Result

See above

Collector version

0.111.0

Environment information

Environment

OS: Debian Bookworm
Release: otelcol-contrib debian package

OpenTelemetry Collector configuration

# Relevant portion only
receivers:
  journald:
    directory: /var/log/journal

Log output

{
        "PRIORITY" : "6",
        "_SOURCE_REALTIME_TIMESTAMP" : "1729708971622850",
        "_SYSTEMD_UNIT" : "docker.service",
        "_HOSTNAME" : "xxx",
        "_BOOT_ID" : "78032dd089a94f00b81a810ba32fbd6d",
        "_SYSTEMD_CGROUP" : "/system.slice/docker.service",
        "_CAP_EFFECTIVE" : "1ffffffffff",
        "_CMDLINE" : "/usr/sbin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock",
        "MESSAGE" : [
                72,
                101,
                108,
                108,
                111,
                32,
                87,
                111,
                114,
                108,
                100,
                33,
                13
        ],
        "CONTAINER_ID" : "30ef3e4a2e9a",
        "_GID" : "0",
        "CONTAINER_TAG" : "30ef3e4a2e9a",
        "_COMM" : "dockerd",
        "_SYSTEMD_INVOCATION_ID" : "1450b926b1474106a9ea6af9f5aa3b0a",
        "__MONOTONIC_TIMESTAMP" : "116279775072",
        "CONTAINER_ID_FULL" : "30ef3e4a2e9a541792564c957d8570a70d387134d0d1098807524b1183568c34",
        "_UID" : "0",
        "_PID" : "3548",
        "_SYSTEMD_SLICE" : "system.slice",
        "IMAGE_NAME" : "openjdk:21",
        "_MACHINE_ID" : "5cd5aae0da034744beac106810fdd968",
        "_EXE" : "/usr/sbin/dockerd",
        "CONTAINER_NAME" : "logging-test",
        "SYSLOG_IDENTIFIER" : "30ef3e4a2e9a",
        "__REALTIME_TIMESTAMP" : "1729708971623225",
        "_RUNTIME_SCOPE" : "system",
        "_TRANSPORT" : "journal",
        "_SELINUX_CONTEXT" : "unconfined\n",
        "__CURSOR" : "s=84d41b02e4994e1b82d490a999839343;i=8ea3ed;b=78032dd089a94f00b81a810ba32fbd6d;m=1b12d08f60;t=6252940d18339;x=c2f5ce410bd3c45b"
}

Additional context

No response

@fuero fuero added bug Something isn't working needs triage New item requiring triage labels Oct 23, 2024
@fuero fuero changed the title Journald Receiver should handle MESSAGE field correctly Journald Receiver should handle MESSAGE field more conveniently Oct 23, 2024
@fuero fuero changed the title Journald Receiver should handle MESSAGE field more conveniently [receiver/journald] Handle MESSAGE field more conveniently Oct 24, 2024
djaglowski pushed a commit that referenced this issue Oct 30, 2024
#### Description

Adds message conversion from byte array support to journald receiver.
Introduces a config setting for enabling this behaviour.

#### Link to tracking issue

Fixes #35964 

#### Testing

Adds tests for decoding a message successfully to string

Tested on a real world example as well with the resulting binary

#### Documentation

Adds description of the setting the PR introduces
zzhlogin pushed a commit to zzhlogin/opentelemetry-collector-contrib-aws that referenced this issue Nov 12, 2024
…6005)

#### Description

Adds message conversion from byte array support to journald receiver.
Introduces a config setting for enabling this behaviour.

#### Link to tracking issue

Fixes open-telemetry#35964 

#### Testing

Adds tests for decoding a message successfully to string

Tested on a real world example as well with the resulting binary

#### Documentation

Adds description of the setting the PR introduces
sbylica-splunk pushed a commit to sbylica-splunk/opentelemetry-collector-contrib that referenced this issue Dec 17, 2024
…6005)

#### Description

Adds message conversion from byte array support to journald receiver.
Introduces a config setting for enabling this behaviour.

#### Link to tracking issue

Fixes open-telemetry#35964 

#### Testing

Adds tests for decoding a message successfully to string

Tested on a real world example as well with the resulting binary

#### Documentation

Adds description of the setting the PR introduces
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage New item requiring triage
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant