Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22619 Fix unstable tests
  3. HIVE-22626

Fix Replication related tests

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • Test

    Description

      For TestStatsReplicationScenariosACIDNoAutogather:

      this test is running "alone" because but still; it sometimes runs more than 40m which results in a timeout
      a jira search reveals that was pretty common:
      https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestStatsReplicationScenariosACIDNoAutogather%22%20order%20by%20updated%20desc

      from the hive logs:

      • it seems like after a few minutes this test starts there is an exception:
        2019-12-10T22:43:19,594 DEBUG [Finalizer] metastore.HiveMetaStoreClient: Unable to shutdown metastore client. Will try closing transport directly.
        org.apache.thrift.transport.TTransportException: java.net.SocketException: Socket closed
                at org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161) ~[libthrift-0.9.3-1.jar:0.9.3-1]
                at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) ~[libthrift-0.9.3-1.jar:0.9.3-1]
                at org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66) ~[libthrift-0.9.3-1.jar:0.9.3-1]
                at com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436) ~[libfb303-0.9.3.jar:?]
                at com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430) ~[libfb303-0.9.3.jar:?]
                at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:776) [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_102]
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_102]
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_102]
                at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
                at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212) [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
                at com.sun.proxy.$Proxy62.close(Unknown Source) [?:?]
                at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:542) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
                at org.apache.hadoop.hive.ql.metadata.Hive.finalize(Hive.java:514) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
                at java.lang.System$2.invokeFinalize(System.java:1270) [?:1.8.0_102]
                at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98) [?:1.8.0_102]
                at java.lang.ref.Finalizer.access$100(Finalizer.java:34) [?:1.8.0_102]
                at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210) [?:1.8.0_102]
        Caused by: java.net.SocketException: Socket closed
                at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116) ~[?:1.8.0_102]
                at java.net.SocketOutputStream.write(SocketOutputStream.java:153) ~[?:1.8.0_102]
                at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[?:1.8.0_102]
                at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[?:1.8.0_102]
                at org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159) ~[libthrift-0.9.3-1.jar:0.9.3-1]
        
      • after that some NoSuchObjectExceptions follow
      • and then some replications seems to happen

      I don't fully understand this; I'll attach the logs...

      Attachments

        1. qalogs.tgz
          13.84 MB
          Zoltan Haindrich

        Issue Links

          Activity

            People

              ayushtkn Ayush Saxena
              kgyrtkirk Zoltan Haindrich
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1.5h
                  1.5h