Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11276

Write with custom indexName and docId not present in document itself

Details

    • Wish
    • Status: Open
    • P3
    • Resolution: Unresolved
    • 2.25.0
    • None
    • io-java-elasticsearch
    • None
    • Java, Google Dataflow

    Description

      ElasticIO connector write method is taking currently as input a "PCollection<String>" element ; string value will be passed as ElasticSearch document body. With methods withIdFn, and withIndexFn, we can overload Elastic document id and index name at runtime.

      There is some kind of situation where indexName and/or docId are not present in document itself (for example from Google Pubsub metadata or other).

      Is there a way to implement this usecase with current ElasticIO ? Otherwise, if it makes sense for you, what kind of improvment can be performed on connector ?

      Attachments

        Activity

          People

            Unassigned Unassigned
            lionelsetan Lionel Setan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: