feat: added serviceAccountAnnotations to FluentBit resource

[sharkeyl]

What this PR does / why we need it:

We're deploying the fluent-operator, and corresponding FluentBit resource, on an AWS EKS cluster. As we are using IAM Roles for Service Accounts, we need the ability to add the below annotation to the service account to provide the fluent-bit Pods with access to push to our log sink:

metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/fluent-bit-role

Without this, we see the follow error messages in the fluent-bit Pods:

[2023/08/25 12:41:41] [error] [aws_credentials] STS assume role request failed
[2023/08/25 12:41:41] [ warn] [aws_credentials] No cached credentials are available and a credential refresh is already in progress. The current co-routine will retry.

With the changes in this PR, the service account is correctly annotated to enable the Pods to assume the AWS IAM role.

I wasn't sure whether:

• The chart version should be bumped in the PR or whether this is automated.
• These changes need to be made to the fluent/fluent-operator chart first.

Please let me know if there are any changes required.

To reproduce

You'll need an AWS IAM role with a trust relationship that the fluent-bit Pods can assume via IRSA.

I've installed the Helm chart from thelocal chart directory, i.e. helm-charts/charts/fluent-operator, using the following command:

helm upgrade --install --namespace kube-system fluent-operator . --values local-values.yaml

Where the local-values.yaml file contains:

namespaceOverride: kube-system
containerRuntime: containerd
operator:
  disableComponentControllers: fluentd
fluentbit:
  serviceAccountAnnotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::781632261136:role/dev-aws-for-fluent-bit-access
fluentd:
  crdsEnable: false

Which issue(s) this PR fixes:

Fixes #897

Does this PR introduced a user-facing change?

None

Additional documentation, usage docs, etc.:

[wenchajun]

Shouldn't this be version 2.4.1 here?

[benjaminhuo]

yes, should be 2.4.1
We'll change this to 2.5.0 when v2.5.0 is released

[sharkeyl]

Thank you for the review. I have amended the version to 2.4.1.

Is there anything else I need to change?

[benjaminhuo]

Thank you @sharkeyl