KafkaProducer.send does not throw TimeoutException as documented

In the JavaDoc for KafkaProducer#send(ProducerRecord, Callback), it claims that it will throw a TimeoutException if blocking on fetching metadata or allocating memory and surpassing max.block.ms.

Throws:
TimeoutException - If the time taken for fetching metadata or allocating memory for the record has surpassed max.block.ms.

(link)

But this is not the case. As TimeoutException is an ApiException it will hit this catch which will result in a failed future being returned instead of the exception being thrown.

The "allocating memory" part likely changed as part of KAFKA-3720 which changed the base exception for buffer exhaustion exceptions to TimeoutException. Timing out waiting on metadata suffers the same issue, but it is not clear whether this has always been the case.

This is basically a discrepancy between documentation and behavior, so it's a question of which one should be adjusted.

And on that, being able to differentiate between synchronous timeouts (as caused by waiting on metadata or allocating memory) and asynchronous timeouts (eg. timing out waiting for acks) is useful. In the former case we know that the broker has not received the event but in the latter it may be that the broker has received it but the ack could not be delivered, and our actions might vary because of this. The current behavior makes this hard to differentiate since both result in a TimeoutException being delivered via the callback. Currently, I am relying on the exception message string to differentiate these two, but this is basically just relying on implementation detail that may change at any time. Therefore I would suggest to either:

• Revert to the documented behavior of throwing in case of synchronous timeouts
• Correct the javadoc and introduce an exception base class/interface for synchronous timeouts