Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8849

ChaosMonkey should cuase chaos in a more reproducible manner

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 6.0
    • None
    • None

    Description

      Looking into the ChaosMonkey code a bit, and it seems like this class – particularly the way monkeyThread is defined – uses randomness in a way that makes it extremely unlikely that it will ever create reproducible failures.

      Obviously in any test where there are multiple concurrent threads, timing issues might prevent test reproducibility – but in this case, even the sequence of "chaos" actions the monkeyThread takes won't be reproducible if anyother concurrent test thread accesses LuceneTestCase.random() ...

            public void run() {
              while (!stop) {
                try {
          
                  Random random = LuceneTestCase.random();
                  // ... lots of stuff using random, or calling methods that use LuceneTestCase.random() directly
      

      It seems like it would be a lot better if ChaosMonkey's constructor created it's own private Random chaosRand using LuceneTestCase.random() as a seed, and then used chaosRand to make all random choices in it's methods.

      That way at least the sequence of chaotic operations made by ChaosMonkey would be consistent for a given test seed, even if the exact timing/interleaving of those operations relative to other operations by other threads couldn't be garunteed.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            hossman Chris M. Hostetter
            hossman Chris M. Hostetter
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment