Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9395

RuntimeFilter causes impalad crash

    XMLWordPrintableJSON

Details

    • ghx-label-10

    Description

      impalad crashed while running TestRuntimeFilters::test_basic_filters:

      16:55:53  TestRuntimeFilters.test_basic_filters[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: seq/none] 
      16:55:53 query_test/test_runtime_filters.py:55: in test_basic_filters
      16:55:53     test_file_vars={'$RUNTIME_FILTER_WAIT_TIME_MS' : str(WAIT_TIME_MS)})
      16:55:53 /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/common/impala_test_suite.py:659: in run_test_case
      16:55:53     result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
      16:55:53 /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/common/impala_test_suite.py:594: in __exec_in_impala
      16:55:53     result = self.__execute_query(target_impalad_client, query, user=user)
      16:55:53 /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/common/impala_test_suite.py:933: in __execute_query
      16:55:53     return impalad_client.execute(query, user=user)
      16:55:53 /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/common/impala_connection.py:205: in execute
      16:55:53     return self.__beeswax_client.execute(sql_stmt, user=user)
      16:55:53 /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/beeswax/impala_beeswax.py:187: in execute
      16:55:53     handle = self.__execute_query(query_string.strip(), user=user)
      16:55:53 /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/beeswax/impala_beeswax.py:365: in __execute_query
      16:55:53     self.wait_for_finished(handle)
      16:55:53 /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/beeswax/impala_beeswax.py:386: in wait_for_finished
      16:55:53     raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
      16:55:53 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      16:55:53 E    Query aborted:ExecQueryFInstances rpc query_id=1b4b52ada7e0b713:a6ca09e200000000 failed: Exec() rpc failed: Network error: Client connection negotiation failed: client connection to 127.0.0.1:27002: connect: Connection refused (error 111)
      

      minidump:

      Operating system: Linux
                        0.0.0 Linux 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64
      CPU: amd64
           family 6 model 85 stepping 4
           1 CPU
      
      GPU: UNKNOWN
      
      Crash reason:  SIGABRT
      Crash address: 0x7d100006057
      Process uptime: not available
      
      Thread 122 (crashed)
       0  libc-2.17.so + 0x351f7
          rax = 0x0000000000000000   rdx = 0x0000000000000006
          rcx = 0xffffffffffffffff   rbx = 0x0000000007208c80
          rsi = 0x0000000000006366   rdi = 0x0000000000006057
          rbp = 0x00007fb39ecf7da0   rsp = 0x00007fb39ecf7a38
           r8 = 0x0000000000000000    r9 = 0x00007fb39ecf78b0
          r10 = 0x0000000000000008   r11 = 0x0000000000000202
          r12 = 0x0000000007208d00   r13 = 0x000000000000004b
          r14 = 0x0000000007210644   r15 = 0x0000000007208c80
          rip = 0x00007fb40b9501f7
          Found by: given as instruction pointer in context
       1  libc-2.17.so + 0x368e8
          rbp = 0x00007fb39ecf7da0   rsp = 0x00007fb39ecf7a40
          rip = 0x00007fb40b9518e8
          Found by: stack scanning
       2  impalad!google_breakpad::ExceptionHandler::HandleSignal(int, siginfo_t*, void*) + 0x1e0
          rbp = 0x00007fb39ecf7da0   rsp = 0x00007fb39ecf7ac8
          rip = 0x0000000004e866f0
          Found by: stack scanning
       3  impalad!boost::detail::lexical_converter_impl<std::string, impala::extdatasource::TColumnDesc>::try_convert(impala::extdatasource::TColumnDesc const&, std::string&) [converter_lexical.hpp : 508 + 0x1]
          rbp = 0x00007fb39ecf7da0   rsp = 0x00007fb39ecf7ae0
          rip = 0x0000000001f90130
          Found by: stack scanning
       4  impalad!google::LogMessage::Flush() + 0x157
          rbx = 0x0000000000000000   rbp = 0x00007fb40c5006a0
          rsp = 0x00007fb39ecf7db0   rip = 0x0000000004e4e837
          Found by: call frame info
       5  impalad!google::LogMessageFatal::~LogMessageFatal() + 0xe
          rbx = 0x00007fb39ecf7e70   rbp = 0x00007fb39ecf7f80
          rsp = 0x00007fb39ecf7e30   r12 = 0x0000000000000001
          r13 = 0x0000000007217b78   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x0000000004e51dfe
          Found by: call frame info
       6  impalad!impala::RuntimeFilter::SetFilter(impala::BloomFilter*, impala::MinMaxFilter*) [runtime-filter.cc : 29 + 0x10]
          rbx = 0x0000000000000001   rbp = 0x00007fb39ecf7f80
          rsp = 0x00007fb39ecf7e50   r12 = 0x0000000000000001
          r13 = 0x0000000007217b78   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x0000000002207e37
          Found by: call frame info
       7  impalad!impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) [runtime-filter-bank.cc : 359 + 0x21]
          rbx = 0x0000000002532b2a   rbp = 0x00007fb39ecf82a0
          rsp = 0x00007fb39ecf7f90   r12 = 0x0000000000000000
          r13 = 0x000000001737d5a0   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x00000000021c953c
          Found by: call frame info
       8  impalad!impala::QueryState::PublishFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) [query-state.cc : 722 + 0x28]
          rbx = 0x0000000000000000   rbp = 0x00007fb39ecf82f0
          rsp = 0x00007fb39ecf82b0   r12 = 0x0000000000000000
          r13 = 0x000000001737d5a0   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x00000000021b8127
          Found by: call frame info
       9  impalad!impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*) [data-stream-service.cc : 132 + 0x2b]
          rbx = 0x0000000000000000   rbp = 0x00007fb39ecf8490
          rsp = 0x00007fb39ecf8300   r12 = 0x0000000000000000
          r13 = 0x000000001737d5a0   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x00000000022e4c26
          Found by: call frame info
      10  impalad!impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::{lambda(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)#8}::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const + 0x3e
          rbx = 0x00000000129d96c0   rbp = 0x00007fb39ecf84c0
          rsp = 0x00007fb39ecf84a0   r12 = 0x00000000176cf1a0
          r13 = 0x000000001737d5a0   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x00000000023a59ec
          Found by: call frame info
      11  impalad!std::_Function_handler<void(const google::protobuf::Message*, google::protobuf::Message*, kudu::rpc::RpcContext*), impala::DataStreamServiceIf::DataStreamServiceIf(const scoped_refptr<kudu::MetricEntity>&, const scoped_refptr<kudu::rpc::ResultTracker>&)::<lambda(const google::protobuf::Message*, google::protobuf::Message*, kudu::rpc::RpcContext*)> >::_M_invoke [functional : 2039 + 0x20]
          rbx = 0x00000000129d96c0   rbp = 0x00007fb39ecf8510
          rsp = 0x00007fb39ecf84d0   r12 = 0x00000000176cf1a0
          r13 = 0x000000001737d5a0   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x00000000023a7975
          Found by: call frame info
      12  impalad!std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const [functional : 2439 + 0x44]
          rbx = 0x00000000023a790e   rbp = 0x00007fb39ecf8560
          rsp = 0x00007fb39ecf8520   r12 = 0x00000000176cf1a0
          r13 = 0x000000001737d5a0   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x0000000002a95c36
          Found by: call frame info
      13  impalad!kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) [service_if.cc : 139 + 0x2a]
          rbx = 0x000000001737d5a0   rbp = 0x00007fb39ecf85e0
          rsp = 0x00007fb39ecf8570   r12 = 0x00000000129d96c0
          r13 = 0x00007fffe2bb36a0   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x0000000002a95617
          Found by: call frame info
      14  impalad!impala::ImpalaServicePool::RunThread() [impala-service-pool.cc : 271 + 0x3a]
          rbx = 0x0000000002a95380   rbp = 0x00007fb39ecf86f0
          rsp = 0x00007fb39ecf85f0   r12 = 0x0000000000000000
          r13 = 0x00007fffe2bb36a0   r14 = 0x00000000145d27c8
          r15 = 0x0000000000000000   rip = 0x00000000020a9b89
          Found by: call frame info
      

      Attachments

        Issue Links

          Activity

            People

              tarmstrong Tim Armstrong
              attilaj Attila Jeges
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: