Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-19539

Test Failure: test_bootstrap_with_reset_bootstrap_state

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Normal
    • Resolution: Unresolved
    • 5.x
    • CI, Test/dtest/python
    • None
    • Correctness - Test Failure
    • Normal
    • Normal
    • User Report
    • All
    • None

    Description

      Failing on trunk:

      ccmlib.node.TimeoutError: 03 Apr 2024 19:41:13 [node3] after 180.22/180 seconds Missing: ['Starting listening for CQL clients'] not found in system.log:
       Head: INFO  [main] 2024-04-03 19:37:59,845 YamlConfigura
       Tail: ...19 - Got error from /127.0.0.1:7000: TIMEOUT when sending TCM_COMMIT_REQ, retrying on CandidateIterator{candidates=[/127.0.0.1:7000], checkLive=true}
      self = <bootstrap_test.TestBootstrap object at 0x7f682b09ae20>
      
          @since('2.2')
          def test_bootstrap_with_reset_bootstrap_state(self):
              """Test bootstrap with resetting bootstrap progress"""
              cluster = self.cluster
              cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', 'True')
              cluster.set_configuration_options(values={'stream_throughput_outbound_megabits_per_sec': 1})
              cluster.populate(2).start()
          
              node1 = cluster.nodes['node1']
              node1.stress(['write', 'n=100K', '-schema', 'replication(factor=2)'])
              node1.flush()
          
              # kill node1 in the middle of streaming to let it fail
              t = InterruptBootstrap(node1)
              t.start()
          
              # start bootstrapping node3 and wait for streaming
              node3 = new_node(cluster)
              try:
                  node3.start()
              except NodeError:
                  pass  # node doesn't start as expected
              t.join()
              node1.start()
          
              # restart node3 bootstrap with resetting bootstrap progress
              node3.stop(signal_event=signal.SIGKILL)
              mark = node3.mark_log()
              node3.start(jvm_args=["-Dcassandra.reset_bootstrap_progress=true"])
              # check if we reset bootstrap state
              node3.watch_log_for("Resetting bootstrap progress to start fresh", from_mark=mark)
              # wait for node3 ready to query, 180s as the node needs to bootstrap
      >       node3.wait_for_binary_interface(from_mark=mark, timeout=180)
      
      bootstrap_test.py:513: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:709: in wait_for_binary_interface
          self.watch_log_for("Starting listening for CQL clients", **kwargs)
      ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:608: in watch_log_for
          TimeoutError.raise_if_passed(start=start, timeout=timeout, node=self.name,
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      start = 1712173092.936025, timeout = 180
      msg = "Missing: ['Starting listening for CQL clients'] not found in system.log:\n Head: INFO  [main] 2024-04-03 19:37:59,845...00: TIMEOUT when sending TCM_COMMIT_REQ, retrying on CandidateIterator{candidates=[/127.0.0.1:7000], checkLive=true}\n"
      node = 'node3'
      
          @staticmethod
          def raise_if_passed(start, timeout, msg, node=None):
              if start + timeout < time.time():
      >           raise TimeoutError.create(start, timeout, msg, node)
      E           ccmlib.node.TimeoutError: 03 Apr 2024 19:41:13 [node3] after 180.22/180 seconds Missing: ['Starting listening for CQL clients'] not found in system.log:
      E            Head: INFO  [main] 2024-04-03 19:37:59,845 YamlConfigura
      E            Tail: ...19 - Got error from /127.0.0.1:7000: TIMEOUT when sending TCM_COMMIT_REQ, retrying on CandidateIterator{candidates=[/127.0.0.1:7000], checkLive=true}
      
      ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:56: TimeoutError
      

      https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2680/workflows/8b1c0d0a-7458-4b43-9bba-ac96b9bfe64f/jobs/58929/tests#failed-test-0
      https://ci-cassandra.apache.org/job/Cassandra-trunk/1859/#showFailuresLink

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              e.dimitrova Ekaterina Dimitrova
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: