Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2079

Don't fail when write to scratch dir results in error.

    XMLWordPrintableJSON

Details

    Description

      Currently, a query fails when it tries to spill data to a disk and that write results in an error. This can happen for multiple reasons: a) no write permission, b) bad disk.

      The fix should do two things:

      • Check for write permissions on startup
      • Don't fail queries that hit an error at runtime (e.g. disk full) if there is another functional scratch disk that can be used instead.

      Attachments

        Issue Links

          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-3200 Replace BufferedBlockMgr with new buffer pool

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-2298 Skip full disks when writing tmp files

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-6078 IMPALA query fails if a scratch directory goes bad with a disk failure

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-3866 Improve error reporting for scratch write errors

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-4683 Intelligently blacklist scratch disks with physical I/O errors

          • Major - Major loss of function.
          • Open

          Activity

            People

              tarmstrong Tim Armstrong
              alan@cloudera.com Alan Choi
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: