Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12421

Migrate elasticsearchio tests to test containers

    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • P2
    • Resolution: Done
    • None
    • 2.31.0
    • io-java-elasticsearch
    • None

    Description

      There are more and more flakiness on this IO due to load. I think it is time to migrate from in-jvm ES test framework to test containers.

      See discussion: https://lists.apache.org/thread.html/reb68f37c435995a64ded19100e09dfc31c5cf6227feae16494226100%40%3Cdev.beam.apache.org%3E

      egalpin I took the liberty to assign to you, feel free to unassign if it is a problem.

      lboutros I know you worked on that already, maybe you have a testContainers code for ES to give (I remember you had a starting impl in your github) can you send it to serve as a base for PR ?

      To do this migration I think ES2 should be removed. A poll was done among users last year and it concluded to removal of ES 2 (no more supported by elastic since some years)

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. BEAM-5172 org.apache.beam.sdk.io.elasticsearch/ElasticsearchIOTest is flaky

          • P1 - Critical bug: data loss, total loss of function, or loss of testing signal due to test failures or flakiness
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. BEAM-10485 Failure / flake: ElasticsearchIOTest > testWriteWithIndexFn

          • P1 - Critical bug: data loss, total loss of function, or loss of testing signal due to test failures or flakiness
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. BEAM-10504 Failure / flake in ElasticSearchIOTest > testWriteFullAddressing and testWriteWithIndexFn

          • P1 - Critical bug: data loss, total loss of function, or loss of testing signal due to test failures or flakiness
          • Resolved
          links to

          Web Link GitHub Pull Request #14933

          Activity

            People

              egalpin Evan Galpin
              echauchot Etienne Chauchot
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 40m
                  3h 40m