Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-1251 Python 3 Support
  3. BEAM-7365

apache_beam.io.avroio_test.TestAvro.test_dynamic_work_rebalancing_exhaustive is very slow

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Not applicable
    • Component/s: io-py-avro
    • Labels:
      None

      Description

      $ python setup.py test -s apache_beam.io.avroio_test.TestFastAvro.test_dynamic_work_rebalancing_exhaustive
      test_dynamic_work_rebalancing_exhaustive (apache_beam.io.avroio_test.TestFastAvro) ... WARNING:root:After 101 concurrent splitting trials at item #2, observed only failure, giving up on this item
      WARNING:root:After 101 concurrent splitting trials at item #21, observed only failure, giving up on this item
      WARNING:root:After 101 concurrent splitting trials at item #22, observed only failure, giving up on this item
      WARNING:root:After 1014 total concurrent splitting trials, considered only 25 items, giving up.
      ok
      ----------------------------------------------------------------------
      Ran 1 test in 172.223s
       
      

      Compare this with

      $ python setup.py test -s apache_beam.io.avroio_test.TestAvro.test_dynamic_work_rebalancing_exhaustive
      test_dynamic_work_rebalancing_exhaustive (apache_beam.io.avroio_test.TestAvro) ... ok
      
      ----------------------------------------------------------------------
      Ran 1 test in 0.623s
      
      OK
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tvalentyn Valentyn Tymofieiev
                Reporter:
                robertwb Robert Bradshaw
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 40m
                  1h 40m