Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
Impala 2.5.0
Description
http://sandbox.jenkins.cloudera.com/job/impala-cdh5-trunk-exhaustive-rhel7/312
There are a few different failures/crashes happening in this build, and I can't tell if they're related or not. henryr I've assigned to you because of the runtime filters DCHECK, please file other JIRAs if you think some of these problems are unrelated.
Failure 1:
metadata.test_compute_stats.TestComputeStats.test_compute_stats
metadata.test_compute_stats.TestCorruptTableStats.test_compute_stats (I think py.test may have screwed up and mislabeled this one, it looks like it's another test_compute_stats vector)
Stacktrace
metadata/test_compute_stats.py:41: in test_compute_stats
self.run_test_case('QueryTest/compute-stats', vector, unique_database)
common/impala_test_suite.py:311: in run_test_case
pytest.config.option.update_results)
common/test_result_verifier.py:357: in verify_raw_results
VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:203: in verify_query_result_is_equal
assert expected_results == actual_results
E assert Comparing QueryTestResults (expected vs actual):
E 100,3,'7.73KB','NOT CACHED','NOT CACHED','TEXT','false',regex:.* != 100,2,'7.73KB','NOT CACHED','NOT CACHED','TEXT','false','hdfs://localhost:20500/test-warehouse/test_compute_stats_650013a0.db/alltypesnopart'
Why are there only two files instead of one? Maybe because an impalad crashed? (see below failures)
Failure 2:
impalad with process ID 20640 hit DCHECK:
F0506 15:17:08.873009 3054 coordinator.cc:1988] Check failed: filter_routing_table_complete_ Filter received before routing table complete
Failure 3:
impalad with process ID 20921 crashed with following stack:
... #5 <signal handler called> #6 0x000000000125306a in std::_Rb_tree<std::string, std::pair<std::string const, impala::RuntimeProfile::Counter*>, std::_Select1st<std::pair<std::string const, impala::RuntimeProfile::Counter*> >, std::less<std::string>, std::allocator<std::pair<std::string const, impala::RuntimeProfile::Counter*> > >::_M_begin (this=0x306b6e7568635f9b) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/stl_tree.h:518 #7 0x0000000001251a66 in std::_Rb_tree<std::string, std::pair<std::string const, impala::RuntimeProfile::Counter*>, std::_Select1st<std::pair<std::string const, impala::RuntimeProfile::Counter*> >, std::less<std::string>, std::allocator<std::pair<std::string const, impala::RuntimeProfile::Counter*> > >::lower_bound (this=0x306b6e7568635f9b, __k=...) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/stl_tree.h:927 #8 0x0000000001250473 in std::map<std::string, impala::RuntimeProfile::Counter*, std::less<std::string>, std::allocator<std::pair<std::string const, impala::RuntimeProfile::Counter*> > >::lower_bound (this=0x306b6e7568635f9b, __x=...) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/stl_map.h:902 #9 0x000000000124f204 in std::map<std::string, impala::RuntimeProfile::Counter*, std::less<std::string>, std::allocator<std::pair<std::string const, impala::RuntimeProfile::Counter*> > >::operator[] (this=0x306b6e7568635f9b, __k=...) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/stl_map.h:496 #10 0x000000000124d884 in impala::RuntimeProfile::total_time_counter (this=0x306b6e7568635f73) at /data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/util/runtime-profile.h:523 #11 0x00000000018083f5 in impala::InstanceComparator::operator() (this=0x7fc498105f90, a=..., b=...) at /data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/runtime/coordinator.cc:1641 #12 0x000000000181b76c in __gnu_cxx::__ops::_Val_comp_iter<impala::InstanceComparator>::operator()<std::pair<impala::RuntimeProfile*, bool>, __gnu_cxx::__normal_iterator<std::pair<impala::RuntimeProfile*, bool>*, std::vector<std::pair<impala::RuntimeProfile*, bool>, std::allocator<std::pair<impala::RuntimeProfile*, bool> > > > > (this=0x7fc498105f90, __val=..., __it=...) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/predefined_ops.h:166 #13 0x000000000181a28c in std::__unguarded_linear_insert<__gnu_cxx::__normal_iterator<std::pair<impala::RuntimeProfile*, bool>*, std::vector<std::pair<impala::RuntimeProfile*, bool>, std::allocator<std::pair<impala::RuntimeProfile*, bool> > > >, __gnu_cxx::__ops::_Val_comp_iter<impala::InstanceComparator> > (__last=..., __comp=...) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/stl_algo.h:1827 #14 0x0000000001817bd9 in std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<impala::RuntimeProfile*, bool>*, std::vector<std::pair<impala::RuntimeProfile*, bool>, std::allocator<std::pair<impala::RuntimeProfile*, bool> > > >, __gnu_cxx::__ops::_Iter_comp_iter<impala::InstanceComparator> > (__first=..., __last=..., __comp=...) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/stl_algo.h:1854 #15 0x00000000018149e7 in std::__final_insertion_sort<__gnu_cxx::__normal_iterator<std::pair<impala::RuntimeProfile*, bool>*, std::vector<std::pair<impala::RuntimeProfile*, bool>, std::allocator<std::pair<impala::RuntimeProfile*, bool> > > >, __gnu_cxx::__ops::_Iter_comp_iter<impala::InstanceComparator> > (__first=..., __last=..., __comp=...) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/stl_algo.h:1889 #16 0x000000000181111f in std::__sort<__gnu_cxx::__normal_iterator<std::pair<impala::RuntimeProfile*, bool>*, std::vector<std::pair<impala::RuntimeProfile*, bool>, std::allocator<std::pair<impala::RuntimeProfile*, bool> > > >, __gnu_cxx::__ops::_Iter_comp_iter<impala::InstanceComparator> > (__first=..., __last=..., __comp=...) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/stl_algo.h:1970 #17 0x000000000180ddf9 in std::sort<__gnu_cxx::__normal_iterator<std::pair<impala::RuntimeProfile*, bool>*, std::vector<std::pair<impala::RuntimeProfile*, bool>, std::allocator<std::pair<impala::RuntimeProfile*, bool> > > >, impala::InstanceComparator> (__first=..., __last=..., __comp=...) at /tmp/impala-deps/gcc-4.9.2/include/c++/4.9.2/bits/stl_algo.h:4716 #18 0x000000000180ab85 in impala::RuntimeProfile::SortChildren<impala::InstanceComparator> (this=0x9b75b00, cmp=...) at /data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/util/runtime-profile.h:445 #19 0x00000000018043a7 in impala::Coordinator::ReportQuerySummary (this=0xc015500) at /data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/runtime/coordinator.cc:1740 #20 0x00000000017fd6c3 in impala::Coordinator::Wait (this=0xc015500) at /data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/runtime/coordinator.cc:1050 #21 0x00000000013ca7a1 in impala::ImpalaServer::QueryExecState::WaitInternal (this=0xe0c6000) at /data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/service/query-exec-state.cc:618 #22 0x00000000013ca474 in impala::ImpalaServer::QueryExecState::Wait (this=0xe0c6000) at /data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/service/query-exec-state.cc:594 ...