Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16575

Automatically remove KTable aggregation result when group becomes empty

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • streams

    Description

      Using `KTable.groupBy(...).aggregate(...)` can handle updates (inserts, deletes, actual updates) of the input KTable, by calling the provided `Adder` and `Subtractor`. However, when all records from the input table (which map to the same group/row in the result table) get removed, the result entry is not removed automatically.

      For example, if we implement a "count", the count would go to zero for a group by default, instead of removing the row from the result, if all input record for this group got deleted.

      Users can let their `Subtractor` return `null` for this case, to actually delete the row, but it's not well documented and it seems it should be a built-in feature of the table-aggregation to remove "empty groups" from the result, instead of relying on "correct" behavior of user-code.

      (Also the built-in `count()` does not return `null`, but actually zero...)

      An internal counter how many elements are in a group should be sufficient. Of course, there is backward compatibility questions we need to answer.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mjsax Matthias J. Sax
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: