Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19912

Schema evolution checks prints a log line in INFO mode for each vectorized rowbatch, impacts performance

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.1.0, 3.0.1, 4.0.0-alpha-1
    • Hive
    • None

    Description

      While benchmarking query96, noticed 17K log lines printed for each vector rowbactch

       

      In file ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java

       

      @@ -2554,8 +2554,8 @@ public static TypeDescription getDesiredRowTypeDescr(Configuration conf,
     }

     if (haveSchemaEvolutionProperties) {
-      if (LOG.isInfoEnabled()) {
-        LOG.info("Using schema evolution configuration variables schema.evolution.columns " +
+      if (LOG.isDebugEnabled()) {
+        LOG.debug("Using schema evolution configuration variables schema.evolution.columns " +
             schemaEvolutionColumnNames.toString() +
             " / schema.evolution.columns.types " +
             schemaEvolutionTypeDescrs.toString() +

       

      Attachments

        1. HIVE-19912.1.patch
          0.8 kB
          Nita Dembla
        2. HIVE-19912.2.patch
          1 kB
          Prasanth Jayachandran

        Activity

          People

            ndembla Nita Dembla
            ndembla Nita Dembla
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: