Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
Patch
Description
I noticed that sometimes upload fails with SignatureDoesNotMatch error. I tried to debug this, and I got the following notice in the error message from S3:
<StringToSign>PUT\nE8F2ok2KBiJTfNy91PLz+A==\n\n1452529414\n/mybucket/11.png?partNumber=1&uploadId=DQGexmin_N3usw4giAtLWoRbCWWRWQij2a20xPO_BgSRENZRpgGqhgMm9goSPoHkmkheIMBrPcI_Z2xWZtqMmZcmekuVhpTBre4cuS.nfyP6DQuTPEuAYop5abOHbm2t</StringToSign>
and in the code libcloud/storage/drivers/s3.py:147 string_to_sign equals the following
string_to_sign = "PUT\nE8F2ok2KBiJTfNy91PLz+A==\n\n1452529414\n/mybucket/11.png?uploadId=DQGexmin_N3usw4giAtLWoRbCWWRWQij2a20xPO_BgSRENZRpgGqhgMm9goSPoHkmkheIMBrPcI_Z2xWZtqMmZcmekuVhpTBre4cuS.nfyP6DQuTPEuAYop5abOHbm2t&partNumber=1"
Obviously the order of GET parameters is not preserved. For me the following patch do the trick, it keeps parameters of multipart upload in order:
diff --git a/libcloud/storage/drivers/s3.py b/libcloud/storage/drivers/s3.py
index 77a86de..1b5a5c0 100644
--- a/libcloud/storage/drivers/s3.py
+++ b/libcloud/storage/drivers/s3.py
@@ -20,6 +20,8 @@ import sys
from hashlib import sha1
+from collections import OrderedDict
+
try:
from lxml.etree import Element, SubElement
except ImportError:
@@ -520,7 +522,9 @@ class BaseS3StorageDriver(StorageDriver):
bytes_transferred = 0
count = 1
chunks = []
- params = {'uploadId': upload_id}
+ params = OrderedDict()
+ params['partNumber'] = 0
+ params['uploadId'] = upload_id
# Read the input data in chunk sizes suitable for AWS
for data in read_in_chunks(iterator, chunk_size=CHUNK_SIZE,