Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4824

Null maps / lists and non-provided state support for JSON fields. Numeric types promotion.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.0.0
    • None
    • Storage - JSON
    • None

    Description

      There is incorrect output in case of JSON file with complex nested data.

      JSON:

      example.json
      {
              "Field1" : {
              }
      }
      
      {
              "Field1" : {
                      "InnerField1": {"key1":"value1"},
                      "InnerField2": {"key2":"value2"}
              }
      }
      
      {
              "Field1" : {
                      "InnerField3" : ["value3", "value4"],
                      "InnerField4" : ["value5", "value6"]
              }
      }
      

      Query:

      select Field1 from dfs.`/tmp/example.json`
      

      Incorrect result:

      +---------------------------+
      |          Field1           |
      +---------------------------+
      {"InnerField1":{},"InnerField2":{},"InnerField3":[],"InnerField4":[]}
      {"InnerField1":{"key1":"value1"},"InnerField2" {"key2":"value2"},"InnerField3":[],"InnerField4":[]}
      {"InnerField1":{},"InnerField2":{},"InnerField3":["value3","value4"],"InnerField4":["value5","value6"]}
      +--------------------------+
      

      Theres is no need to output missing fields. In case of deeply nested structure we will get unreadable result for user.

      Correct result:

      +--------------------------+
      |         Field1           |
      +--------------------------+
      |{}                                                                     
      {"InnerField1":{"key1":"value1"},"InnerField2":{"key2":"value2"}}
      {"InnerField3":["value3","value4"],"InnerField4":["value5","value6"]}
      +--------------------------+
      

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              RomanKulyk Roman Kulyk
              Paul Rogers Paul Rogers
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: