Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-1469

Error with multipart upload to S3 custom endpoint when using AWS V4 signature

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.1.1
    • None
    • jclouds-blobstore
    • None

    Description

      Hello, when uploading in multipart mode file(no matter how big it is) to a custom S3 endpoint, using V2 signature it's ok. But when I change in S3HttpApiModule.java bindRequest Signer

      protected void bindRequestSigner() {| |bind(RequestAuthorizeSignature.class).to(RequestAuthorizeSignatureV4.class).in(Scopes.SINGLETON);| *}

      *
      to point to RequestAuthorizeSignatureV4 it fails with:
       
      Caused by: com.google.common.util.concurrent.UncheckedExecutionException: org.jclouds.http.HttpResponseException: unable to reset unrepeatable payload stream after calculating AWS4 signature. connecting to PUT https://xxx.xxx.xxx.xxx:9002/xxx.xxx.xxx.xxx/upload/5lv.jpg?partNumber=1&uploadId=a992b147-8831-41c3-a4c9-eebb0574ff79 HTTP/1.1
      at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525)
      at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511)
      at org.jclouds.blobstore.internal.BaseBlobStore.putMultipartBlob(BaseBlobStore.java:378)
      at org.jclouds.blobstore.internal.BaseBlobStore.putMultipartBlob(BaseBlobStore.java:349)
      at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:262)
      at xxx.xxx.xxx.xxx.xxx.xxx.xxx.S3ConnectorImpl.singleUpload(S3ConnectorImpl.java:363)
      at xxx.xxx.xxx.xxx.xxx.xxx.xxx.S3ConnectorImpl.upload(S3ConnectorImpl.java:285)
      ... 82 more
      Caused by: org.jclouds.http.HttpResponseException: unable to reset unrepeatable payload stream after calculating AWS4 signature. connecting to PUT https://xxx.xxx.xxx.xxx:9002/xxx.xxx.xxx.xxx/upload/5lv.jpg?partNumber=1&uploadId=a992b147-8831-41c3-a4c9-eebb0574ff79 HTTP/1.1
      at xxx.xxx.xxx.xxx.xxx.xxx.xxx.xxx.xxx.CustomJavaUrlHttpCommandExecutorService.invoke(CustomJavaUrlHttpCommandExecutorService.java:107)
      at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
      at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
      at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
      at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
      at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
      at com.sun.proxy.$Proxy175.uploadPart(Unknown Source)
      at org.jclouds.s3.blobstore.S3BlobStore.uploadMultipartPart(S3BlobStore.java:393)
      at org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:398)
      at org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:385)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
      at java.lang.Thread.getStackTrace(Thread.java:1559)
      at org.jclouds.blobstore.internal.BaseBlobStore.putMultipartBlob(BaseBlobStore.java:376)
      ... 86 more
      Caused by: org.jclouds.http.HttpException: unable to reset unrepeatable payload stream after calculating AWS4 signature.
      at org.jclouds.s3.filters.Aws4SignerForAuthorizationHeader.closeOrResetPayloadStream(Aws4SignerForAuthorizationHeader.java:214)
      at org.jclouds.s3.filters.Aws4SignerForAuthorizationHeader.calculatePayloadContentHash(Aws4SignerForAuthorizationHeader.java:199)
      at org.jclouds.s3.filters.Aws4SignerForAuthorizationHeader.getPayloadHash(Aws4SignerForAuthorizationHeader.java:175)
      at org.jclouds.s3.filters.Aws4SignerForAuthorizationHeader.sign(Aws4SignerForAuthorizationHeader.java:144)
      at org.jclouds.s3.filters.RequestAuthorizeSignatureV4.signForAuthorizationHeader(RequestAuthorizeSignatureV4.java:91)
      at org.jclouds.s3.filters.RequestAuthorizeSignatureV4.filter(RequestAuthorizeSignatureV4.java:55)
      at xxx.xxx.xxx.xxx.xxx.xxx.xxx.xxx.xxx.CustomJavaUrlHttpCommandExecutorService.invoke(CustomJavaUrlHttpCommandExecutorService.java:80)
      at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
      at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
      at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
      at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
      at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
      at com.sun.proxy.$Proxy175.uploadPart(Unknown Source)
      at org.jclouds.s3.blobstore.S3BlobStore.uploadMultipartPart(S3BlobStore.java:393)
      at org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:398)
      at org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:385)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      ... 3 more
      Caused by: java.io.IOException: Mark not supported
      at com.google.common.io.ByteStreams$LimitedInputStream.reset(ByteStreams.java:592)
      at org.jclouds.s3.filters.Aws4SignerForAuthorizationHeader.closeOrResetPayloadStream(Aws4SignerForAuthorizationHeader.java:211)
      ... 19 more
       
      Have you examine this before? It's strange because when pointing to AWS region, using aws-s3 provider instead of s3 api it works with AWS V4 signature...

      Attachments

        Activity

          People

            Unassigned Unassigned
            blagolaj Blagoi Anastasov
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: