Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat]Support fluentd grok parser plugin #861

Merged
merged 3 commits into from
Aug 10, 2023

Conversation

vincent-vinf
Copy link
Contributor

What this PR does / why we need it:

Support fluentd grok parser plugin

Which issue(s) this PR fixes:

Fixes #856

Does this PR introduced a user-facing change?

The following fields were added to the fluentd common.Parse
- 'grokPattern': The pattern of grok.
- 'customPatternPath': Path to the file that includes custom grok patterns.
- 'grokFailureKey': The key has grok failure reason.
- 'multiLineStartRegexp': The regexp to match beginning of multiline. This is only for "multiline_grok".
- 'grokPatternSeries': Specify grok pattern series set.
- 'grok': Grok Sections.

Additional documentation, usage docs, etc.:

- [Usage]: [Grok Parser for Fluentd](https://github.com/fluent/fluent-plugin-grok-parser)

@vincent-vinf
Copy link
Contributor Author

To use grok, you need to install fluent-plugin-grok-parser in the image first.

FROM kubesphere/fluentd:v1.15.3
USER root
RUN fluent-gem install fluent-plugin-grok-parser
USER fluent

Copy link
Member

@wenchajun wenchajun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vincent-vinf
Copy link
Contributor Author

You should add the grok parser plugin here https://github.com/fluent/fluent-operator/blob/master/cmd/fluent-watcher/fluentd/Dockerfile.amd64

Hello, there are multiple related dockerfiles here, which ones should I add grok-related configurations to?
image

@benjaminhuo benjaminhuo requested a review from wanjunlei August 4, 2023 01:59
@benjaminhuo
Copy link
Member

@vincent-vinf Thanks for the PR, we're ready to merge it once the dockerfile is modified

@vincent-vinf
Copy link
Contributor Author

@vincent-vinf Thanks for the PR, we're ready to merge it once the dockerfile is modified

The dockerfile has been modified >_<

@benjaminhuo benjaminhuo merged commit a5b6221 into fluent:master Aug 10, 2023
@vincent-vinf vincent-vinf deleted the flentd-grok branch August 10, 2023 07:00
@vincent-vinf vincent-vinf restored the flentd-grok branch August 30, 2023 08:16
@vincent-vinf vincent-vinf deleted the flentd-grok branch August 30, 2023 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

help request: Using fluentd and grok together
3 participants