Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4652

PubsubIO: create subscription on different project than the topic

Details

    • New Feature
    • Status: Resolved
    • P1
    • Resolution: Fixed
    • None
    • 2.6.0
    • io-java-gcp
    • None

    Description

      If you try to read a public pubsub topic in the DirectRunner, it will fail with 403 when trying to create a subscription. This is because it tries to create a subscription on the shared public data set.

      There is an example used in https://github.com/googlecodelabs/cloud-dataflow-nyc-taxi-tycoon and the dataset is projects/pubsub-public-data/topics/taxirides-realtime. I discovered that I could not read this in the DirectRunner even though the codelab works. But that 1.x codelab also does not work in the InProcessPipelineRunner, so it has been broken all along.

      So you cannot read public data or any other read-only data using PubsubIO.

      Attachments

        Issue Links

          Activity

            People

              kenn Kenneth Knowles
              kenn Kenneth Knowles
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 5h 40m
                  5h 40m