Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14232

Kryo: Perflogger is not transient in MapJoinOperator

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Blocker
    • Resolution: Unresolved
    • 2.2.0
    • None
    • Hive
    • None

    Description

        private final PerfLogger perfLogger = SessionState.getPerfLogger();
      

      This produces a new copy of HiveConf into the plan in serialized form.

      classes (sun.misc.Launcher$AppClassLoader)
      classloader (java.security.ProtectionDomain)
      context (java.security.AccessControlContext)
      acc (org.apache.hadoop.hive.ql.exec.UDFClassLoader)
      classLoader (org.apache.hadoop.hive.conf.HiveConf)
      conf (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics)
      metrics (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope)
      openScopes (org.apache.hadoop.hive.ql.log.PerfLogger)
      perfLogger (org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator)
      childOperators (org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator)
      childOperators (org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator)
      childOperators (org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator)
      childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
      aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
              at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
              at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:366)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:307)
              at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
              at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:606)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:87)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
              at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
              at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
              at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
              at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
      

      Attachments

        Issue Links

          Activity

            People

              ndembla Nita Dembla
              gopalv Gopal Vijayaraghavan
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: