Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19911

Add builder interface for Kinesis DStreams

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.1.0
    • 2.2.0
    • DStreams
    • None

    Description

      The ```KinesisUtils.createStream()``` interface for creating Kinesis-based DStreams is quite brittle and requires adding a combinatorial number of overrides whenever another optional configuration parameter is added. This makes incorporating a lot of additional features supported by the Kinesis Client Library such as per-service authorization unfeasible. This interface should be replaced by a builder pattern class (https://en.wikipedia.org/wiki/Builder_pattern) to allow for greater extensibility.

      Attachments

        Issue Links

          Activity

            People

              budde Adam Budde
              budde Adam Budde
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: