Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4302

NullPointerException in CodecUtils with GzipCodec

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.10.0
    • 0.10.1
    • None
    • None

    Description

      When bufferSizeProp is null, CodecUtils.getDecompressedInputStreamWithBufferSize runs into NullPointerException:

      // code placeholder
      java.io.IOException: java.lang.NullPointerException
          at org.apache.tez.runtime.library.common.shuffle.ShuffleUtils.shuffleToMemory(ShuffleUtils.java:142)
          at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.copyMapOutput(FetcherOrderedGrouped.java:550)
          at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.copyFromHost(FetcherOrderedGrouped.java:283)
          at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.fetchNext(FetcherOrderedGrouped.java:182)
          at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.callInternal(FetcherOrderedGrouped.java:194)
          at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.callInternal(FetcherOrderedGrouped.java:57)
          at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
          at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
          at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
          at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.NullPointerException
          at java.util.Hashtable.get(Hashtable.java:363)
          at java.util.Properties.getProperty(Properties.java:969)
          at org.apache.hadoop.conf.Configuration.get(Configuration.java:1200)
          at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1254)
          at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1479)
          at org.apache.tez.runtime.library.utils.CodecUtils.getDecompressedInputStreamWithBufferSize(CodecUtils.java:81)
          at org.apache.tez.runtime.library.common.sort.impl.IFile$Reader.readToMemory(IFile.java:824)
          at org.apache.tez.runtime.library.common.shuffle.ShuffleUtils.shuffleToMemory(ShuffleUtils.java:121)
          ... 12 more
      

      Configuration.getInt(String name, int defaultValue) assumes theĀ  name is not null.

      Attachments

        Issue Links

          Activity

            People

              jshmchenxi Xi Chen
              jshmchenxi Xi Chen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1.5h
                  1.5h