XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.0
    • 2.4.0
    • Structured Streaming
    • None

    Description

      Currently, the EpochCoordinator doesn't enforce a commit order. If a message for epoch n gets lost in the ether, and epoch n + 1 happens to be ready for commit earlier, epoch n + 1 will be committed.

       

      This is either incorrect or needlessly confusing, because it's not safe to start from the end offset of epoch n + 1 until epoch n is committed. EpochCoordinator should enforce this sequencing.

       

      Note that this is not actually a problem right now, because the commit messages go through the same RPC channel from the same place. But we shouldn't implicitly bake this assumption in.

      Attachments

        Activity

          People

            efim.poberezkin Efim Poberezkin
            joseph.torres Jose Torres
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: