Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12391

WriteToAvro fails if fastavro loads its python implementation of writer

Details

    • Bug
    • Status: Resolved
    • P3
    • Resolution: Fixed
    • 2.25.0, 2.26.0, 2.27.0, 2.28.0, 2.29.0
    • Missing
    • io-py-avro

    Description

      It's possible for fastavro to fail to correctly load its cython implementation of the Writer class in which case it will silently fall back to a pure python implementation. If this happens there's no outward indication, but line 621 in io/avroio.py will fail because writer.fo is only present on the cython implementation.

      To reproduce you can modify fastavro's write.py to only use its fallback:

      
      #from . import _write
      
      from . import _write_py as _write
      
      

      And then run a workflow that sinks to WriteToAvro(use_fastavro=True).

      Attachments

        Issue Links

          Activity

            People

              johnjcasey John Casey
              cchandler@squareup.com Chris Chandler
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 40m
                  3h 40m