Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-3958

Avro decimal logical type fail if database returns Zero precision

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.0
    • 1.3.0
    • Extensions
    • None

    Description

      NIFI-2624 introduced Avro logical type mapping capability. For NUMERIC/DECIMAL types, 'decimal' logical type is used, and positive precision is required to encode a numeric value as 'decimal logical type.

      However, database can return 0 (Zero) precision to represent unlimited precision, if a column is not configured with specific precision (some database engine has default precision, but some don't), or numeric value generated by functions or other means dynamically.

      We need to provide a way to define default precision in case Zero precision is returned, probably by adding another Processor property to specify default precision.

      2017-05-23 16:54:12,414 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.IllegalArgumentException: Invalid decimal precision: 0 (must be positive)
        at org.apache.avro.LogicalTypes$Decimal.validate(LogicalTypes.java:206)
        at org.apache.avro.LogicalType.addToSchema(LogicalType.java:70)
        at org.apache.avro.LogicalTypes$Decimal.addToSchema(LogicalTypes.java:182)
        at org.apache.nifi.processors.standard.util.JdbcCommon.lambda$createSchema$0(JdbcCommon.java:463)
        at org.apache.nifi.processors.standard.util.JdbcCommon.addNullableField(JdbcCommon.java:359)
        at org.apache.nifi.processors.standard.util.JdbcCommon.createSchema(JdbcCommon.java:461)
        at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:192)
        at org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:204)
        at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
        at org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:195)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:144)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. NIFI-7973 Oracle JDBC - ExecuteSqlRecord and other components fail on zero precision

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #1851

          Activity

            People

              ijokarumawak Koji Kawamura
              ijokarumawak Koji Kawamura
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: