Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12556

test_two_statestored_with_force_active fails or flaky

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • Impala 4.4.0
    • Impala 4.4.0
    • Distributed Exec
    • None
    • ghx-label-11

    Description

      custom_cluster.test_statestored_ha.TestStatestoredHA.test_two_statestored_with_force_active failed in a precommit run.
      Symptom:

      common/custom_cluster_test_suite.py:208: in setup_method
          self._start_impala_cluster(cluster_args, **kwargs)
      common/custom_cluster_test_suite.py:330: in _start_impala_cluster
          check_call(cmd + options, close_fds=True)
      ../toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/subprocess.py:190: in check_call
          raise CalledProcessError(retcode, cmd)
      E   CalledProcessError: Command '['/home/ubuntu/Impala/bin/start-impala-cluster.py', '--state_store_args=--statestore_update_frequency_ms=50     --statestore_priority_update_frequency_ms=50     --statestore_heartbeat_frequency_ms=50', '--cluster_size=3', '--num_coordinators=3', '--log_dir=/home/ubuntu/Impala/logs/custom_cluster_tests', '--log_level=1', '--state_store_args=--statestore_force_active=true ', '--enable_statestored_ha', '--impalad_args=--default_query_options=']' returned non-zero exit status 1
      

      The test dies with a FATAL log entry in catalogd's log:

      DCHECK found in log file: /home/ubuntu/Impala/logs/custom_cluster_tests/catalogd.FATAL
      
      Log file created at: 2023/11/11 23:36:24
      Running on machine: ip-172-31-52-128
      Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
      F1111 23:36:24.798915 2270244 statestore-subscriber.cc:336] Check failed: !statestore_is_active || !statestore2_is_active 
      

      Offending precommit run: https://jenkins.impala.io/job/ubuntu-20.04-from-scratch/874/ (preserved).

      Attachments

        Issue Links

          Activity

            People

              wzhou Wenzhe Zhou
              laszlog Laszlo Gaal
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: