Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-43278

Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.3.0
    • None
    • Java API
    • None

    Description

      Java version: 1.8.0_331, Apache Maven 3.8.4

      I run next steps:

      1. git clone https://github.com/apache/spark.git
      2. git checkout -b v3.3.0 3.3.0
      3.  mvn clean install -DskipTests
      4. copy hive-site.xml to examples/src/main/resources/
      5. execute TPC-H Q6 

       

      public static void main(String[] args) throws InterruptedException {
              SparkConf sparkConf = new SparkConf()
                      .setAppName("demo")
                      .setMaster("local[1]")
                      ;        
               SparkSession sparkSession = SparkSession.builder()
                      .config(sparkConf)
                      .enableHiveSupport()
                      .getOrCreate();
              sparkSession.sql("use local_tpch_sf10_uncompressed_etl");        
              sparkSession.sql(TPCH.SQL6).show();
      } 

       

       

      get the error info:

      Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
          at org.apache.spark.util.io.ChunkedByteBufferOutputStream.toChunkedByteBuffer(ChunkedByteBufferOutputStream.scala:115)
          at org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:325)
          at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:140)
          at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:95)
          at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
          at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:75)
          at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1529)
          at org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat.buildReaderWithPartitionValues(ParquetFileFormat.scala:235)
          at org.apache.spark.sql.execution.FileSourceScanExec.inputRDD$lzycompute(DataSourceScanExec.scala:457)
          at org.apache.spark.sql.execution.FileSourceScanExec.inputRDD(DataSourceScanExec.scala:448)
          at org.apache.spark.sql.execution.FileSourceScanExec.doExecuteColumnar(DataSourceScanExec.scala:547)
          at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeColumnar$1(SparkPlan.scala:221)
          at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:232)
          at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
          at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:229)
          at org.apache.spark.sql.execution.SparkPlan.executeColumnar(SparkPlan.scala:217)

      Attachments

        Activity

          People

            Unassigned Unassigned
            jiangjiguang0719 jiangjiguang0719
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: