You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
2 Broker URLs
1 Topic with 4 Partitions - workflow-executions
1 Consumer Group (raw-data-consumer) with 4 Consumers (Kubernetes PODs)
Currently we are using eachMessage() to process the messages from the Brokers but we are not able to achieve the required throughput.
We are doing a POC of using eachBatch() to process the messages in batches.
While using eachBatch(), we found out that the messages from each batch isn't being processed but the handler function keeps on calling the eachBatch(). Is this the expected behaviour?
The text was updated successfully, but these errors were encountered:
I am facing a similar issue for eachMessage and the only solution I could find till now is setting partitionsConsumedConcurrently to 1 based on this: #1492
Kafka Setup
2 Broker URLs
1 Topic with 4 Partitions - workflow-executions
1 Consumer Group (raw-data-consumer) with 4 Consumers (Kubernetes PODs)
Currently we are using eachMessage() to process the messages from the Brokers but we are not able to achieve the required throughput.
We are doing a POC of using eachBatch() to process the messages in batches.
While using eachBatch(), we found out that the messages from each batch isn't being processed but the handler function keeps on calling the eachBatch(). Is this the expected behaviour?
The text was updated successfully, but these errors were encountered: