Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3723

TezIndexRecord#hasData() returns true for empty index record in the Unordered case

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0
    • None
    • None

    Description

      In the case of Unordered writer, use of hasData() returns true for empty partitions since the rawlength is zero (as writer skips writing them out altogether) and the check in hasData will fail as rawlength == header.length +2 fails.

      This currently works for the ordered case(and the method is used here unlike in unordered writer) but after TEZ-3605, it will fail for both scenarios. Even if we don't use the method, the logic should be fixed.

      Attachments

        1. TEZ-3723.001.patch
          0.7 kB
          Kuhu Shukla
        2. TEZ-3723.002.patch
          2 kB
          Kuhu Shukla
        3. TEZ-3723.003.patch
          3 kB
          Kuhu Shukla
        4. TEZ-3723.004.patch
          3 kB
          Kuhu Shukla

        Activity

          People

            kshukla Kuhu Shukla
            kshukla Kuhu Shukla
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: