Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21240

JSON SerDe Re-Write

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed
    • Patch, Important

    Description

      The JSON SerDe has a few issues, I will link them to this JIRA.

      • Use Jackson Tree parser instead of manually parsing
      • Added support for base-64 encoded data (the expected format when using JSON)
      • Added support to skip blank lines (returns all columns as null values)
      • Current JSON parser accepts, but does not apply, custom timestamp formats in most cases
      • Added some unit tests
      • Added cache for column-name to column-index searches, currently O(n) for each row processed, for each column in the row

      Attachments

        1. HIVE-21240.1.patch
          56 kB
          David Mollitor
        2. HIVE-21240.1.patch
          56 kB
          David Mollitor
        3. HIVE-21240.10.patch
          88 kB
          David Mollitor
        4. HIVE-21240.11.patch
          110 kB
          David Mollitor
        5. HIVE-21240.12.patch
          110 kB
          David Mollitor
        6. HIVE-21240.12.patch
          110 kB
          David Mollitor
        7. HIVE-21240.12.patch
          110 kB
          David Mollitor
        8. HIVE-21240.12.patch
          110 kB
          David Mollitor
        9. HIVE-21240.12.patch
          110 kB
          David Mollitor
        10. HIVE-21240.2.patch
          67 kB
          David Mollitor
        11. HIVE-21240.3.patch
          62 kB
          David Mollitor
        12. HIVE-21240.4.patch
          62 kB
          David Mollitor
        13. HIVE-21240.5.patch
          85 kB
          David Mollitor
        14. HIVE-21240.6.patch
          86 kB
          David Mollitor
        15. HIVE-21240.7.patch
          87 kB
          David Mollitor
        16. HIVE-21240.9.patch
          88 kB
          David Mollitor
        17. HIVE-24240.8.patch
          87 kB
          David Mollitor
        18. kafka_storage_handler.diff
          20 kB
          David Mollitor

        Issue Links

        is depended upon by

        New Feature - A new feature of the product, which has yet to be developed. HIVE-18785 Make JSON SerDe First-Class SerDe

        • Major - Major loss of function.
        • Open
        relates to

        Improvement - An improvement or enhancement to an existing feature or task. HIVE-21356 Upgrade Jackson to 2.9.8

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed
        supercedes

        Bug - A problem which impairs or prevents the functions of the product. HIVE-20375 Json SerDe ignoring the timestamp.formats property

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. HIVE-20900 serde2.JsonSerDe no longer supports timestamp.formats

        • Major - Major loss of function.
        • Patch Available

        Task - A task that needs to be done. HIVE-21317 Unit Test kafka_storage_handler Is Failing Regularly

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        links to

        Web Link GitHub Pull Request #530

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            belugabehr David Mollitor Assign to me
            belugabehr David Mollitor
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 2h 20m
              2h 20m

              Slack

                Issue deployment