[azure-eventhub input] Switch the run EPH run mode to non-blocking

[zmoog]

What does this PR do?

Update the azure-eventhub input to switch the run mode of the Event Processor Host (EPH) from blocking to non-blocking.

With the changes in the PR, the first call to the input Start() method will set up and start the EPH worker in a non-blocking fashion. The EPH worker will continue until the Filebeat calls the Stop() method to tear it down for live reloading.

Why is it important?

Since 8.3.3, the input Runner changed its behavior during the live reload:

• < 8.3.3: the input Runner stops the old input and starts a new input without waiting for the stop to complete.
• >= 8.3.3: the input Runner stops the old input and waits for the Stop() method to complete before starting a new one.

Unfortunately, if we run Start() with EPH in a blocking mode, the method will not return because it will continue waiting for an os.Signal that will never come.

If the EPH's Start() method does not end, the EPH sync.WaitGroup will continue waiting forever.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

1. Enable live reload

Edit the filebeat.yml file and update the "Filebeat modules" like this one:

# ============================== Filebeat modules ==============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  # reload.enabled: false
  reload.enabled: true

  # Period on which files under path should be checked for changes
  #reload.period: 10s
  reload.period: 10s

# ======================= Elasticsearch template setting =======================

2. Enable the azure module

Copy the azure.yml.disabled file into azure.yml and set up one of the existing inputs.

For example:

  platformlogs:
    enabled: true
    var:
      # adding a new line
      eventhub: "platformlogs"
      consumer_group: "$Default"
      connection_string: "Endpoint=sb://....="
      storage_account: ""
      storage_account_key: ""
      tags:
        - "azure"
        - "whatever4"

3. Testing

Start Filebeat and make sure the platform logs are created in Elasticsearch.

Make some changes to the azure.yml. For example, try to add/remove/change the tags and within 10 seconds Filebeat will trigger a configuration reload.

Related issues

• Closes Azure Logs integration stops collecting logs after a configuration change integrations#4743

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @zmoog? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-12-21T10:25:38.835+0000

• Duration: 79 min 27 sec

Test stats 🧪

Test	Results
Failed	0
Passed	7384
Skipped	338
Total	7722

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[zmoog]

/test

[endorama]

I did not test data forwarding to Elasticsearch but I tested autoreloading configuration (played with platformlogs configs).

At each save the new content was picked up and used to configure the input, with restarts in appropriate cases. I can confirm the stop/start sequence is run:

[...] Starting reload procedure, current runners: 1 [...]
[...] Start list: 1, Stop list: 1 [...]
[...] Stopping runner: azure (platformlogs) [...]
[...] azure-eventhub input worker has started. [...]

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

No Coverage information
0.0% Duplication

[ruflin]

@belimawr Can you please have a look at this as the change in 8.3.3 was made by you: #32309

[belimawr]

LGTM.

That's my first contact with azure-eventhub, but looks good.