Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7405

Support for graceful handling of corrupted records in Kafka consumer

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.10.2.2, 0.11.0.3, 1.1.1
    • Fix Version/s: None
    • Component/s: consumer, streams
    • Labels:
      None

      Description

      We have run into issues several times where corrupted records cause the Kafka consumer to throw an error code 2 exception (CRC checksum failure) in the fetch layer. Specifically, when using Kafka streams we run into KAFKA-6977 that throws an IllegalStateException and crashes the service. It would be great if the Kafka consumer could be extended with a setting similar to KIP-161, that would allow one to gracefully ignore corrupted records.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                efeller Eugen Feller
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: