bug: could not watch events when using Kubernetes Events input due to missing verbs in operator RBAC

[thomasgouveia]

Describe the issue

When deploying fluent-bit with the operator with a cluster input configured to collect Kubernetes Events, fluent-bit outputs some warn logs:

[2024/07/30 13:24:04] [ info] [input:kubernetes_events:kubernetes_events.1] Requesting /api/v1/events?watch=1&resourceVersion=297456085
[2024/07/30 13:24:04] [ warn] [input:kubernetes_events:kubernetes_events.1] events watch failure, http_status=403 payload={"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"events is forbidden: User \"system:serviceaccount:fluent-operator:fluent-bit\" cannot watch resource \"events\" in API group \"\" at the cluster scope","reason":"Forbidden","details":{"kind":"events"},"code":403}

A missing verb watch in the RBAC rules of the fluent-bit deployment causes this. I tried to fix it by adding the watch verb to my fluent bit CRD using the rbacRules attribute. As mentioned in the values.yaml file, fluent-bit can only be granted permissions the operator also has. Unfortunately, the operator currently doesn't have the watch verb allowed for events.

To Reproduce

You can use the following manifest once the operator has been deployed in your cluster:

apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterFluentBitConfig
metadata:
  name: fluent-bit-config
  labels:
    app.kubernetes.io/name: fluent-bit
spec:
  service:
    parsersFile: parsers.conf
    httpServer: true
  inputSelector:
    matchLabels:
      fluentbit.fluent.io/enabled: "true"
  filterSelector:
    matchLabels:
      fluentbit.fluent.io/enabled: "true"
  outputSelector:
    matchLabels:
      fluentbit.fluent.io/enabled: "true"
---
apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterInput
metadata:
  name: k8s-events
  labels:
    fluentbit.fluent.io/enabled: "true"
    fluentbit.fluent.io/component: "logging"
spec:
  kubernetesEvents:
    tag: events.*
---
apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterOutput
metadata:
  name: stdout
  labels:
    fluentbit.fluent.io/enabled: "true"
    fluentbit.fluent.io/component: "logging"
spec:
  match: "*"
  stdout:
    format: json
---
apiVersion: fluentbit.fluent.io/v1alpha2
kind: FluentBit
metadata:
  name: fluent-bit
  namespace: fluent-operator
  labels:
    app.kubernetes.io/name: fluent-bit
spec:
  image: ghcr.io/fluent/fluent-operator/fluent-bit:3.1.2
  positionDB:
    hostPath:
      path: /var/lib/fluent-bit/
  resources:
    requests:
      cpu: 10m
      memory: 25Mi
    limits:
      cpu: 500m
      memory: 200Mi
  fluentBitConfigName: fluent-bit-config

  rbacRules:
    - apiGroups:
        - ""
      resources:
        - events
      verbs:
        - list

This manifest will deploy a FluentBit CRD into the fluent-operator namespace, and fluent-bit will be configured to collect Kubernetes events, and send them to stdout.

Expected behavior

Warning logs must disappear and events must be logged instead.

Your Environment

- Fluent Operator version: 3.0.0
- Container Runtime: containerd

How did you install fluent operator?

With the official Helm chart from the repository https://github.com/fluent/helm-charts.

Additional context

As a workaround, it is possible to fix this issue by editing the fluent-operator ClusterRole to add the watch verb for the events:

# charts/fluent-operator/templates/fluent-operator-clusterRole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/name: fluent-operator
  name: fluent-operator
rules:
  # rest of rules .....
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - watch <------ Added here
      - list
  # rest of rules .....

Then, if I add the following rbacRules to my FluentBit CRD, it works properly and events are printed to stdout:

# rest of fluent-bit CRD
  rbacRules:
    - apiGroups:
        - ""
      resources:
        - events
      verbs:
        - list
        - watch

I'll provide a PR to fix this issue, so that users will not need to patch anything independently. IMHO, the operator chart (but also other installation methods) must properly configure this ClusterRole.

You can assign me this issue!