Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-7707

[Processor/GCP - PubSub] Avro event re-format when pushed to PubSub

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.11.4
    • 1.12.0
    • Extensions

    Description

      Hi, 

      as discussed on Slack : https://apachenifi.slack.com/archives/C0L9VCD47/p1596463216211700

      When we generate an avro event from record and we publish it using PublishPubSub processor the content-type of the file change from avro/binary to UTF-8 due to the following code part : https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/pubsub/PublishGCPubSub.java#L157

      This make the event unreadable once Consumed :/ 

      The workaround we found is base64 encoding the avro message, publishing it to PubSub, consuming it, and finally decoding it. 

      We expect this part of encoding would be handled by the processors.

       

      Attachments

        Issue Links

          Activity

            People

              pvillard Pierre Villard
              aguitton Guitton Alexandre
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h