[receiver/kafka]: Support access to message key

[SoerenHenning]

Component(s)

receiver/kafka

Is your feature request related to a problem? Please describe.

It looks like there is currently no way to access the key of a Kafka message. This feature would be quite helpful as it is a common pattern to carry non-redundant data in the key and this data should also be exported to an OTel backend. Note that exporting message headers is already supported.

Describe the solution you'd like

The easiest solution would probably be to extract the key and add it to the resource attributes, just like it is already possible with headers. I propose adding the following configuration:

• key_extraction (default = false): Allows user to attach the message key to resource attributes in otel piepline

Describe alternatives you've considered

Alternatively, the key could also be added somehow to the body of a log record, but the specific implementation would depend on the encoding. I am therefore much more in favor of adding the key to the resource attributes.

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[SoerenHenning]

I still think this is a relevant feature for productive usage of the Kafka receiver.

[atoulme]

@pavolloffay @MovieStoreGuy please weigh in