Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
The currently used KafkaConsumer.poll(long timeout) method is deprecated in 2.x. The method waits for the metadata response indefinitely until we get it (so in case of an unavailable/stopped kafka cluster until the connection gets recovered). It means the poll(long) acquires and holds a thread exclusively for this period which can lead to starvation of other processors for timer driven threads.
Use poll(Duration timeout) instead which is not simply an overloaded version of the method but has different semantics (https://cwiki.apache.org/confluence/display/KAFKA/KIP-266%3A+Fix+consumer+indefinite+blocking+behavior).
If can return from metadata fetch after the timeout expires and does not occupy a thread continuously and makes the processor able to yield.
Attachments
Issue Links
- links to