Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10376

Add end timestamp to Time Window key. Update TimeWindow Serializer/Deserializer to set end of window.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.6.0
    • None
    • streams

    Description

      Aggregate stream consumers are not currently able to get the end timestamp for Time Windows from the Deserializer.  This is due to the fact that the end timestamp is not present in the message key.  The end timestamp is not currently added to the end of the key for Time Window aggregates.

      Allowing the Consumer to set the window size in the Deserializer's constructor is not sufficient as an objective consumer would not know what window size was being used.  It also stands to reason that the aggregator could choose to change the window without notifying the consumer: e.g., 1 minute windows during business hours and 1 hour off business hours.  It would also be possible for the same consumer to be consuming several topics at once, each with a different window on each topic.

      Update the code to add the end timestamp long onto the end of the key and update the Deserializer to extract and set the correct end on the Window.

      This issue is peripherally related to KAFKA-4468, KAFKA-9649 and KAFKA-10366.

      Attachments

        Activity

          People

            Unassigned Unassigned
            chromatic BPR
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: