Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12329

Avoid AWS S3 Filesystem warnings about not all bytes have been read

Details

    • Improvement
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.31.0
    • io-java-aws
    • None

    Description

      While running a pipeline on AWS Glue I noticed a recurrent warning:


      WARN S3AbortableInputStream: Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use.

      This seems to be related to not fully consuming the input stream when reading files from AWS S3.

      Attachments

        Issue Links

          Activity

            People

              iemejia Ismaël Mejía
              iemejia Ismaël Mejía
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m