Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30165 Eliminate compilation warnings
  3. SPARK-30168

Eliminate warnings in Parquet datasource

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: In Progress
    • Minor
    • Resolution: Unresolved
    • 3.1.0
    • None
    • SQL
    • None

    Description

      1. sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/parquet/ParquetPartitionReaderFactory.scala 
        Warning:Warning:line (120)class ParquetInputSplit in package hadoop is deprecated: see corresponding Javadoc for more information.
          Option[TimeZone]) => RecordReader[Void, T]): RecordReader[Void, T] = {
    Warning:Warning:line (125)class ParquetInputSplit in package hadoop is deprecated: see corresponding Javadoc for more information.
      new org.apache.parquet.hadoop.ParquetInputSplit(
    Warning:Warning:line (134)method readFooter in class ParquetFileReader is deprecated: see corresponding Javadoc for more information.
      ParquetFileReader.readFooter(conf, filePath, SKIP_ROW_GROUPS).getFileMetaData
    Warning:Warning:line (183)class ParquetInputSplit in package hadoop is deprecated: see corresponding Javadoc for more information.
      split: ParquetInputSplit,
    Warning:Warning:line (212)class ParquetInputSplit in package hadoop is deprecated: see corresponding Javadoc for more information.
      split: ParquetInputSplit,
      2. sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java 
        Warning:Warning:line (55)java: org.apache.parquet.hadoop.ParquetInputSplit in org.apache.parquet.hadoop has been deprecated
    Warning:Warning:line (95)java: org.apache.parquet.hadoop.ParquetInputSplit in org.apache.parquet.hadoop has been deprecated
    Warning:Warning:line (95)java: org.apache.parquet.hadoop.ParquetInputSplit in org.apache.parquet.hadoop has been deprecated
    Warning:Warning:line (97)java: getRowGroupOffsets() in org.apache.parquet.hadoop.ParquetInputSplit has been deprecated
    Warning:Warning:line (105)java: readFooter(org.apache.hadoop.conf.Configuration,org.apache.hadoop.fs.Path,org.apache.parquet.format.converter.ParquetMetadataConverter.MetadataFilter) in org.apache.parquet.hadoop.ParquetFileReader has been deprecated
    Warning:Warning:line (108)java: filterRowGroups(org.apache.parquet.filter2.compat.FilterCompat.Filter,java.util.List<org.apache.parquet.hadoop.metadata.BlockMetaData>,org.apache.parquet.schema.MessageType) in org.apache.parquet.filter2.compat.RowGroupFilter has been deprecated
    Warning:Warning:line (111)java: readFooter(org.apache.hadoop.conf.Configuration,org.apache.hadoop.fs.Path,org.apache.parquet.format.converter.ParquetMetadataConverter.MetadataFilter) in org.apache.parquet.hadoop.ParquetFileReader has been deprecated
    Warning:Warning:line (147)java: ParquetFileReader(org.apache.hadoop.conf.Configuration,org.apache.parquet.hadoop.metadata.FileMetaData,org.apache.hadoop.fs.Path,java.util.List<org.apache.parquet.hadoop.metadata.BlockMetaData>,java.util.List<org.apache.parquet.column.ColumnDescriptor>) in org.apache.parquet.hadoop.ParquetFileReader has been deprecated
    Warning:Warning:line (203)java: readFooter(org.apache.hadoop.conf.Configuration,org.apache.hadoop.fs.Path,org.apache.parquet.format.converter.ParquetMetadataConverter.MetadataFilter) in org.apache.parquet.hadoop.ParquetFileReader has been deprecated
    Warning:Warning:line (226)java: ParquetFileReader(org.apache.hadoop.conf.Configuration,org.apache.parquet.hadoop.metadata.FileMetaData,org.apache.hadoop.fs.Path,java.util.List<org.apache.parquet.hadoop.metadata.BlockMetaData>,java.util.List<org.apache.parquet.column.ColumnDescriptor>) in org.apache.parquet.hadoop.ParquetFileReader has been deprecated
      3. sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetCompatibilityTest.scala
      4. sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetInteroperabilitySuite.scala
      5. sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetTest.scala
      6. sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala

      Attachments

        Issue Links

          is duplicated by

          Sub-task - The sub-task of the issue SPARK-30174 Eliminate warnings: part 4

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Pull request #28484 [Github] Pull Request #28484 (PavithraRamachandran)

          Activity

            People

              Unassigned Unassigned
              maxgekk Max Gekk
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: