Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16788

ODBC call SQLForeignKeys leads to NPE if you use PK arguments rather than FK arguments

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • ODBC
    • None

    Description

      This ODBC call is meant to allow you to determine FK relationships either from the PK side or from the FK side.

      Hive only allows you to traverse from the FK side, trying it from the PK side leads to an NPE.

      Example using the table "customer" from TPC-H with FKs defined in Hive:

      === Foreign Keys ===
      Using table as foreign source
      (u'HIVE', u'tpch_bin_flat_orc_2', u'nation', u'n_nationkey', u'HIVE', u'tpch_bin_flat_orc_2', u'customer', u'c_nationkey', 1, 0, 0, u'custome
      r_c2', u'nation_c1', 0)
      Not using table as foreign source
      Got an error from the server for customer!
      

      Compare: Postgres

      === Foreign Keys ===
      Using table as foreign source
      (u'vagrant', u'public', u'nation', u'n_nationkey', u'vagrant', u'public', u'customer', u'c_nationkey', 1, 3, 3, u'customer_c_nationkey_fkey', u'nation_pkey', 7)
      Not using table as foreign source
      (u'vagrant', u'public', u'customer', u'c_custkey', u'vagrant', u'public', u'orders', u'o_custkey', 1, 3, 3, u'orders_o_custkey_fkey', u'customer_pkey', 7)
      

      Note that Postgres allows traversal from either way. The traceback you get in the HS2 logs is this:

      2016-12-04T21:08:55,398 ERROR [8998ca98-9940-49f8-8833-7c6ebd8c96a2 HiveServer2-Handler-Pool: Thread-53] metastore.RetryingHMSHandler: MetaEx
      ception(message:java.lang.NullPointerException)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:5785)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_foreign_keys(HiveMetaStore.java:6474)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)
              at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
              at com.sun.proxy.$Proxy25.get_foreign_keys(Unknown Source)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getForeignKeys(HiveMetaStoreClient.java:1596)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:154)
              at com.sun.proxy.$Proxy26.getForeignKeys(Unknown Source)
              at org.apache.hive.service.cli.operation.GetCrossReferenceOperation.runInternal(GetCrossReferenceOperation.java:128)
              at org.apache.hive.service.cli.operation.Operation.run(Operation.java:324)
              at org.apache.hive.service.cli.session.HiveSessionImpl.getCrossReference(HiveSessionImpl.java:933)
              at org.apache.hive.service.cli.CLIService.getCrossReference(CLIService.java:411)
              at org.apache.hive.service.cli.thrift.ThriftCLIService.GetCrossReference(ThriftCLIService.java:738)
              at org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetCrossReference.getResult(TCLIService.java:1617)
              at org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetCrossReference.getResult(TCLIService.java:1602)
              at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
              at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
              at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
              at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.NullPointerException
              at org.apache.hive.common.util.HiveStringUtils.normalizeIdentifier(HiveStringUtils.java:919)
              at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.<init>(ObjectStore.java:2722)
              at org.apache.hadoop.hive.metastore.ObjectStore$GetListHelper.<init>(ObjectStore.java:2863)
              at org.apache.hadoop.hive.metastore.ObjectStore$11.<init>(ObjectStore.java:8455)
              at org.apache.hadoop.hive.metastore.ObjectStore.getForeignKeysInternal(ObjectStore.java:8455)
              at org.apache.hadoop.hive.metastore.ObjectStore.getForeignKeys(ObjectStore.java:8445)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
              at com.sun.proxy.$Proxy24.getForeignKeys(Unknown Source)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_foreign_keys(HiveMetaStore.java:6465)
              ... 28 more
      

      Attachments

        1. HIVE-16788.patch
          5 kB
          jcamachorodriguez

        Activity

          People

            jcamacho Jesús Camacho Rodríguez
            cartershanklin Carter Shanklin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: