Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7048

Fetch GCP-Datastore entity by id inside of a Dataflow transform

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • P3
    • Resolution: Unresolved
    • 2.11.0
    • None
    • io-ideas
    • None

    Description

      I have a dataflow pipeline where first I queried one model, but then for each record that pointed to another model I wanted to be able to fetch that too.

      Browsing the {{apache_beam & google.cloud.proto.datastore.v1 }}source I was able to make this happen:

      https://stackoverflow.com/a/55616668/4458510

      I copied the same mechanism from the other Datastore DoFn's and it seems to work fine, and I think something like this would be a good addition to apache_beam.io.gcp.datastore.v1.datastoreio in the sdk

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            alexindaco Alex Indaco
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: