Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-481

Why does writer not use HDFS hflush when close file?

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 1.2.1
    • None
    • Java
    • None
    • HDP 2.6.5

    Description

      We use ORC for writing to HDFS. Today we found some corrupted files without PostScript.

      Code that writes files is looked as:

      writer.close()
      moveFile()

      So it is guaranted file was closed without error. The only idea I have is that footer had not been flushed before file was moved. Is it possible because writer does not call HDFS hflush()?

      Attachments

        Activity

          People

            Unassigned Unassigned
            d.zhaldakov Dmitry Zhaldakov
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: