Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fluent-Bit] Please enable Offset_Key option for tail input plugin. #1422

Open
mbhegde-orcl opened this issue Nov 27, 2024 · 1 comment · May be fixed by #1437
Open

[Fluent-Bit] Please enable Offset_Key option for tail input plugin. #1422

mbhegde-orcl opened this issue Nov 27, 2024 · 1 comment · May be fixed by #1437
Labels
enhancement New feature or request

Comments

@mbhegde-orcl
Copy link

Is your feature request related to a problem? Please describe.

This is a configuration option available in the upstream fluent-bit (https://docs.fluentbit.io/manual/pipeline/inputs/tail - Cmd+F for "Offset_Key") but not in fluent-bit clusterinput.

If there's a way to bypass the operator CRD and directly specify this, that will also work.

Describe the solution you'd like

Something like

diff --git apis/fluentbit/v1alpha2/plugins/input/tail_types.go apis/fluentbit/v1alpha2/plugins/input/tail_types.go
index 33dd6707..cccc04e0 100644
--- apis/fluentbit/v1alpha2/plugins/input/tail_types.go
+++ apis/fluentbit/v1alpha2/plugins/input/tail_types.go
@@ -101,6 +101,8 @@ type Tail struct {
 	PauseOnChunksOverlimit string `json:"pauseOnChunksOverlimit,omitempty"`
 	//Skips empty lines in the log file from any further processing or output.
 	SkipEmptyLines *bool `json:"skipEmptyLines,omitempty"`
+	//If enabled, Fluent Bit appends the offset of the current monitored file as part of the record.
+	OffsetKey *bool `json:"offsetKey,omitempty"`
 }
 
 func (_ *Tail) Name() string {
@@ -196,5 +198,8 @@ func (t *Tail) Params(_ plugins.SecretLoader) (*params.KVs, error) {
 	if t.SkipEmptyLines != nil {
 		kvs.Insert("Skip_Empty_Lines", fmt.Sprint(*t.SkipEmptyLines))
 	}
+	if t.OffsetKey != nil {
+		kvs.Insert("Offset_Key", fmt.Sprint(*t.OffsetKey))
+	}
 	return kvs, nil
 }

Additional context

Thanks

@mbhegde-orcl mbhegde-orcl changed the title [Fluent-Bit] Please enable Offset_Key option for tail plugin. [Fluent-Bit] Please enable Offset_Key option for tail input plugin. Nov 27, 2024
@jjsiv jjsiv added the enhancement New feature or request label Nov 29, 2024
@jjsiv
Copy link
Collaborator

jjsiv commented Nov 29, 2024

Hey, would you be willing to make a contribution?

@cw-Guo cw-Guo linked a pull request Dec 16, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants