Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18091

Failing tests of itests/qtest-spark and itests/hive-unit on branch-1

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Labels:
      None

      Description

      Seen this while looking at ptest results for HIVE-17947 but is probably an older issue.

      Tests under itests/qtest-spark and itests/hive-unit fail when trying to execute the download-spark plugin with:

      [INFO]
      [INFO] ------------------------------------------------------------------------
      [INFO] Building Hive Integration - Unit Tests 1.3.0-SNAPSHOT
      [INFO] ------------------------------------------------------------------------
      [INFO]
      [INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @ hive-it-unit ---
      [INFO]
      [INFO] --- maven-antrun-plugin:1.7:run (download-spark) @ hive-it-unit ---
      [INFO] Executing tasks
      
      main:
           [exec] + /bin/pwd
           [exec] + BASE_DIR=./target
           [exec] + HIVE_ROOT=./target/../../../
           [exec] + DOWNLOAD_DIR=./../thirdparty
           [exec] + mkdir -p ./../thirdparty
           [exec] /home/hiveptest/35.192.99.254-hiveptest-0/apache-github-branch-1-source/itests/hive-unit
           [exec] + download http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.5.0-bin-hadoop2-without-hive.tgz spark
           [exec] + url=http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.5.0-bin-hadoop2-without-hive.tgz
           [exec] + finalName=spark
           [exec] ++ basename http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.5.0-bin-hadoop2-without-hive.tgz
           [exec] + tarName=spark-1.5.0-bin-hadoop2-without-hive.tgz
           [exec] + rm -rf ./target/spark
           [exec] + [[ ! -f ./../thirdparty/spark-1.5.0-bin-hadoop2-without-hive.tgz ]]
           [exec] + tar -zxf ./../thirdparty/spark-1.5.0-bin-hadoop2-without-hive.tgz -C ./target
           [exec] + mv ./target/spark-1.5.0-bin-hadoop2-without-hive ./target/spark
           [exec] + cp -f ./target/../../..//data/conf/spark/log4j.properties ./target/spark/conf/
           [exec] + sed '/package /d' /data/hiveptest/working/apache-github-branch-1-source/itests/../contrib/src/java/org/apache/hadoop/hive/contrib/udf/example/UDFExampleAdd.java
           [exec] sed: can't read /data/hiveptest/working/apache-github-branch-1-source/itests/../contrib/src/java/org/apache/hadoop/hive/contrib/udf/example/UDFExampleAdd.java: No such file or directory
           [exec] + javac -cp /data/hiveptest/working/maven/org/apache/hive/hive-exec/1.3.0-SNAPSHOT/hive-exec-1.3.0-SNAPSHOT.jar /tmp/UDFExampleAdd.java -d /tmp
           [exec] + jar -cf /tmp/udfexampleadd-1.0.jar -C /tmp UDFExampleAdd.class
           [exec] /tmp/UDFExampleAdd.class : no such file or directory
      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD FAILURE
      [INFO] ------------------------------------------------------------------------
      [INFO] Total time: 8.376s
      [INFO] Finished at: Mon Nov 06 22:29:39 UTC 2017
      [INFO] Final Memory: 18M/241M
      [INFO] ------------------------------------------------------------------------
      [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (download-spark) on project hive-it-unit: An Ant BuildException has occured: exec returned: 1
      [ERROR] around Ant part ...<exec dir="/home/hiveptest/35.192.99.254-hiveptest-0/apache-github-branch-1-source/itests/hive-unit" executable="bash" failonerror="true">... @ 4:141 in /home/hiveptest/35.192.99.254-hiveptest-0/apache-github-branch-1-source/itests/hive-unit/target/antrun/build-main.xml
      

      mvn antrun:run@download-spark passes when run locally, so I guess it might be an issue with the way wer're executing ptests.

      Full list of classes with the same error:

      TestAcidOnTez
      TestAdminUser
      TestAuthorizationPreEventListener
      TestAuthzApiEmbedAuthorizerInEmbed
      TestAuthzApiEmbedAuthorizerInRemote
      TestBeeLineWithArgs
      TestCLIAuthzSessionContext
      TestClearDanglingScratchDir
      TestClientSideAuthorizationProvider
      TestCompactor
      TestCreateUdfEntities
      TestCustomAuthentication
      TestDBTokenStore
      TestDDLWithRemoteMetastoreSecondNamenode
      TestDynamicSerDe
      TestEmbeddedHiveMetaStore
      TestEmbeddedThriftBinaryCLIService
      TestFilterHooks
      TestFolderPermissions
      TestHS2AuthzContext
      TestHS2AuthzSessionContext
      TestHS2ClearDanglingScratchDir
      TestHS2ImpersonationWithRemoteMS
      TestHiveAuthorizerCheckInvocation
      TestHiveAuthorizerShowFilters
      TestHiveHistory
      TestHiveMetaStoreTxns
      TestHiveMetaStoreWithEnvironmentContext
      TestHiveMetaTool
      TestHiveServer2
      TestHiveServer2SessionTimeout
      TestHiveSessionImpl
      TestHs2Hooks
      TestJdbcDriver2
      TestJdbcMetadataApiAuth
      TestJdbcWithLocalClusterSpark
      TestJdbcWithMiniHS2
      TestJdbcWithMiniMr
      TestJdbcWithSQLAuthUDFBlacklist
      TestJdbcWithSQLAuthorization
      TestLocationQueries
      TestMTQueries
      TestMarkPartition
      TestMarkPartitionRemote
      TestMetaStoreAuthorization
      TestMetaStoreConnectionUrlHook
      TestMetaStoreEndFunctionListener
      TestMetaStoreEventListener
      TestMetaStoreEventListenerOnlyOnCommit
      TestMetaStoreInitListener
      TestMetaStoreListenersError
      TestMetaStoreMetrics
      TestMetaStoreUtils
      TestMetastoreAuthorizationProvider
      TestMetastoreVersion
      TestMiniHS2
      TestMiniSparkOnYarnCliDriver
      TestMultiAuthorizationPreEventListener
      TestMultiSessionsHS2WithLocalClusterSpark
      TestNoSaslAuth
      TestOperationLoggingAPIWithMr
      TestOperationLoggingAPIWithTez
      TestPartitionNameWhitelistValidation
      TestPasswordWithConfig
      TestRemoteHiveMetaStore
      TestRemoteHiveMetaStoreIpAddress
      TestRemoteUGIHiveMetaStoreIpAddress
      TestRetryingHMSHandler
      TestSSL
      TestSchemaTool
      TestSemanticAnalyzerHookLoading
      TestSerdeWithFieldComments
      TestServerSpecificConfig
      TestServiceDiscovery
      TestSetUGIOnBothClientServer
      TestSetUGIOnOnlyClient
      TestSetUGIOnOnlyServer
      TestSparkCliDriver
      TestSparkNegativeCliDriver
      TestStorageBasedClientSideAuthorizationProvider
      TestStorageBasedMetastoreAuthorizationDrops
      TestStorageBasedMetastoreAuthorizationProvider
      TestStorageBasedMetastoreAuthorizationReads
      TestThriftBinaryCLIService
      TestThriftHttpCLIService
      TestZooKeeperTokenStore
      

      These are all the test classes of the two modules with the exception of TestHiveMetastore.

        Attachments

        1. HIVE-18091.1-branch-1.patch
          0.9 kB
          Daniel Voros
        2. HIVE-18091.2-branch-1.patch
          0.9 kB
          Daniel Voros

          Activity

            People

            • Assignee:
              dvoros Daniel Voros
              Reporter:
              dvoros Daniel Voros
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: