Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-17872

Dtests failing intermittently on Jolokia agent

    XMLWordPrintableJSON

Details

    Description

      Some apparently unrealeted Python dtests fail with an output of the form:

      Error Message
      subprocess.CalledProcessError: Command '('/usr/lib/jvm/java-8-openjdk-amd64/bin/java', '-cp', '/usr/lib/jvm/java-8-openjdk-amd64/lib/tools.jar:/home/cassandra/cassandra/cassandra-dtest/tools/../lib/jolokia-jvm-1.7.1-agent.jar', 'org.jolokia.jvmagent.client.AgentLauncher', '--host', '127.0.0.1', 'start', '706')' returned non-zero exit status 1.
      Stacktrace
      self = <auth_test.TestAuthRoles object at 0x7fc6cb4313a0>
      
      (...)
      
          
              mbean = make_mbean('auth', type='RolesCache')
      >       with JolokiaAgent(self.cluster.nodelist()[0]) as jmx:
      
      auth_test.py:1888: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      tools/jmxutils.py:309: in __enter__
          self.start()
      tools/jmxutils.py:187: in start
          subprocess.check_output(args, stderr=subprocess.STDOUT)
      /usr/lib/python3.8/subprocess.py:415: in check_output
          return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      input = None, capture_output = False, timeout = None, check = True
      popenargs = (('/usr/lib/jvm/java-8-openjdk-amd64/bin/java', '-cp', '/usr/lib/jvm/java-8-openjdk-amd64/lib/tools.jar:/home/cassandr...t/tools/../lib/jolokia-jvm-1.7.1-agent.jar', 'org.jolokia.jvmagent.client.AgentLauncher', '--host', '127.0.0.1', ...),)
      kwargs = {'stderr': -2, 'stdout': -1}
      process = <subprocess.Popen object at 0x7fc6c9afb910>
      stdout = b"Couldn't start agent for PID 706\nPossible reason could be that port '8778' is already occupied.\nPlease check the standard output of the target process for a detailed error message.\n"
      stderr = None, retcode = 1
      
      (...)
      
                  if check and retcode:
      >               raise CalledProcessError(retcode, process.args,
                                               output=stdout, stderr=stderr)
      E               subprocess.CalledProcessError: Command '('/usr/lib/jvm/java-8-openjdk-amd64/bin/java', '-cp', '/usr/lib/jvm/java-8-openjdk-amd64/lib/tools.jar:/home/cassandra/cassandra/cassandra-dtest/tools/../lib/jolokia-jvm-1.7.1-agent.jar', 'org.jolokia.jvmagent.client.AgentLauncher', '--host', '127.0.0.1', 'start', '706')' returned non-zero exit status 1.
      
      /usr/lib/python3.8/subprocess.py:516: CalledProcessError
      

      Here is a of bunch hits in different tests across multiple branches:

      Note the common with JolokiaAgent(self.cluster.nodelist()[0]) and "Possible reason could be that port '8778' is already occupied." parts.

      So far, the issue doesn't seem to reproduce on 3.0.

      Attachments

        Issue Links

          Activity

            People

              brandon.williams Brandon Williams
              adelapena Andres de la Peña
              Brandon Williams
              Andres de la Peña
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: