Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5598

ExecQueryFInstances RPC recv side timeouts (observed in stress test in insecure+release build)

    XMLWordPrintableJSON

Details

    Description

      A run of the stress test in a 2.9-based release build has found a high number of queries that fail with one of these types of messages:

      Query failed: Sender timed out waiting for receiver fragment instance: db4a82d9ad43fa4b:b8ced000000014

      Query failed: ExecQueryFInstances rpc query_id=414ae4b1fade911a:fddb792300000000 failed: RPC recv timed out: Client hostname:22000 timed-out during recv call.

      Roughly 400 in 10,000 which feels very high. I don't recall seeing this in 2.8-based builds. I can check if asked. Still gathering logs.

      Attachments

        1. queries per second.png
          160 kB
          Michael Brown

        Issue Links

          is broken by

          Sub-task - The sub-task of the issue IMPALA-2550 Switch to per-query exec rpc

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-5275 Avoid printing Status stack trace on hot paths

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              joemcdonnell Joe McDonnell
              mikeb Michael Brown
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: