Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-11745

Add Hive's ESRI geospatial functions as builtins

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Done
    • None
    • Impala 4.3.0
    • Frontend
    • ghx-label-9

    Description

      Hive has added the Esri Hive UDFs as builtin functions in HIVE-26223.
      The original Hive Esri source: https://github.com/Esri/spatial-framework-for-hadoop

      A simple (but inefficient) way to add postgis style geospatial support in Impala is to import these functions through Hive and add them as builtins. This is possible as Impala supports Hive UDFs, but is much slower than a native implementation could be.

      My plan is to add these functions depending on a flag, e.g. uf geospatial_library=hive_esri. This would allow native implementations to be added later and let users choose between Hive compatibility and speed. It could also make sense override only the more critical functions with native ones, but it may turn out that the binary format used in ESRI is suboptimal for a native implementation.

      Attachments

        Issue Links

        is blocked by

        Bug - A problem which impairs or prevents the functions of the product. IMPALA-11549 Incorrent return type mismatch in Hive generic UDFs

        • Major - Major loss of function.
        • Resolved

        New Feature - A new feature of the product, which has yet to be developed. IMPALA-11746 Support legacy Hive UDFs with variable length argument list

        • Major - Major loss of function.
        • Open

        New Feature - A new feature of the product, which has yet to be developed. IMPALA-11718 Improve support for Hive Generic UDFs

        • Major - Major loss of function.
        • In Progress
        relates to

        Bug - A problem which impairs or prevents the functions of the product. IMPALA-11861 Impala*Writable methods are not overridden to handle the heap-stored values

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. IMPALA-11854 ImpalaStringWritable's underlying array can't be changed in UDFs

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. IMPALA-11918 Fix test_java_udfs_from_impala after IMPALA-11745

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            prozsa Peter Rozsa
            csringhofer Csaba Ringhofer
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment