Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.0.1
-
None
-
Devstack on Ubuntu 14.04
Description
If a large object is PUT to a key that contains multibyte Unicode characters, the manifest upload fails because the Content-Length header sent is incorrect.
A small project that reproduces the issue against the latest Devstack Swift can be found here: https://github.com/UniversityofWarwick/jclouds-highbyte-slo-bug
In short:
>> "[{"path":"slo-test/Ni[0xc5][0x9f]an.rar/slo/1491214092.299000/38547913/0/00000001","etag":"58f06dd588d8ffb3beb46ada6309436b","size_bytes":33554432},{"path":"slo-test/Ni[0xc5][0x9f]an.rar/slo/1491214092.299000/38547913/0/00000002","etag":"2b4b81733d0a2e4abe89516639627408","size_bytes":4993481}]"
>> PUT http://137.205.194.8:8080/v1/AUTH_c7a3e66567ec442080a360d6d23f2dbe/slo-test/Ni%C5%9Fan.rar?multipart-manifest=put HTTP/1.1
>> Content-Length: 272
14657 ERROR org.jclouds.http.internal.JavaUrlHttpCommandExecutorService error after writing 0/272 bytes to http://137.205.194.8:8080/v1/AUTH_c7a3e66567ec442080a360d6d23f2dbe/slo-test/Ni%C5%9Fan.rar?multipart-manifest=put
java.io.IOException: too many bytes written
at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3505)
at com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:53)
at com.google.common.io.ByteStreams.copy(ByteStreams.java:179)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:298)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:171)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:65)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:99)
at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
at com.sun.proxy.$Proxy77.replaceManifest(Unknown Source)
at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.completeMultipartUpload(RegionScopedSwiftBlobStore.java:522)
at uk.ac.warwick.slo.AbstractJCloudsSLOTest.putSLO(AbstractJCloudsSLOTest.java:70)
at uk.ac.warwick.slo.AbstractJCloudsSLOTest.assertCanPutAndGetSLO(AbstractJCloudsSLOTest.java:74)
at uk.ac.warwick.slo.AbstractJCloudsSLOTest.sloWithHighByteChars(AbstractJCloudsSLOTest.java:91)
at uk.ac.warwick.slo.SwiftJCloudsSLOTest.sloWithHighByteChars(SwiftJCloudsSLOTest.java:7)
The cause of this is org.jclouds.openstack.swift.v1.binders.BindManifestToJsonPayload:61, where the ContentLength is set to the length of the JSON String in Java, which is not the same as the String byte length. This is actually unnecessary, as request.setPayload(String) creates a StringPayload, and the constructor here sets the content length to the correct byte length - so I think this code is left-over from a previous version where this isn't the case.