Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-1465

Error with multipart upload to S3 for large files

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.1.1
    • None
    • jclouds-blobstore
    • None

    Description

      Hi,

      We have a problem with multipart upload to S3 for files above or even to 20GB. The exception is here:

      2018-10-31 18:04:32,804 ERROR [EventQueueWorker-ThreadPool-16] s3.jclouds.ssl.CustomJavaUrlHttpCommandExecutorService - error after writing 24576/16800342 bytes to "some s3 container" java.io.IOException: Error writing request body to server at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(HttpURLConnection.java:3536) at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3519) at com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:53) at com.google.common.io.ByteStreams.copy(ByteStreams.java:179) at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService. java:298)

       

      We are using the last jclouds version (2.1.1). As we make transfers to S3 on a daily basis, the expected behavior is: files to be successfully uploaded. Now they fail while uploading the first part of the multipart file. I have examined similar issue: http://mail-archives.apache.org/mod_mbox/jclouds-dev/201504.mbox/%3CCAHtp1K2vRaZxQCBAR8ZCjXoXxbnz58hNZCtFmmrNjYKSJ7dHgw@mail.gmail.com%3E and it seems that it is not resolved. Have you identify it and start working on it? If not can you suggest any workaround?

      Best Regards,

      Blago

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            blagolaj Blagoi Anastasov
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: