Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6422

Handling of field type in BigQuery table schema definition is incorrect

    XMLWordPrintableJSON

Details

    Description

      When setting the field type to "record" in BigQuery schema definition, the following exception is thrown

      }}{{2019-07-03 08:35:24,964 ERROR [Timer-Driven Process Thread-8] o.a.n.p.gcp.bigquery.PutBigQueryBatch PutBigQueryBatch[id=b2b1c6bf-016b-1000-e8c9-b3f9fb5b417e] null: java.lang.NullPointerException
      java.lang.NullPointerException: null
          at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.mapToField(BigQueryUtils.java:42)
          at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.listToFields(BigQueryUtils.java:68)
          at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.schemaFromString(BigQueryUtils.java:80)
          at org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch.onTrigger(PutBigQueryBatch.java:277)
          at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
          at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
          at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
          at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
          at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)

       

      This seems to happen because the type handling is ... not so good

       

      Beside, it seems like the TYPE case is not handled correctly, and RECORD type is absolutly not handled.

      Attachments

        Activity

          People

            Unassigned Unassigned
            riduidel Nicolas Delsaux
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 5h 40m
                5h 40m