Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-823

UnicodeDecodeError when uploading via Azure Blobs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • Storage
    • None
    • Ubuntu 14.04.3
      Python 2.7.6
      apache-libcloud 0.20.1

    Description

      I take regular mongo backups which are compressed as tgz files and then uploaded to the storage platform of the cloud provider hosting the DB servers, either S3 and Azure Blobs. The following code works fine for S3:

      from libcloud.storage.types import Provider
      from libcloud.storage.providers import get_driver
      
      cls = get_driver(Provider.S3_EU_WEST)
      driver = cls(cfg['storage']['user'],
                   cfg['storage']['pass'])
      
      try:
              container = driver.create_container(cfg['storage']['container'])
      except:
              container = driver.get_container(cfg['storage']['container'])
      
      driver.upload_object(
              '/mnt/resource/backup/full_20160526_080013.tgz',
              container,
              'mongodb/full_20160526_080013.tgz',
      )
      

      However, running this against Azure Blobs, returns the following error:

      Traceback (most recent call last):
        File "/usr/local/scripts/mongodb_backup.py", line 645, in main
          upload_local(backup_root, backup_dir, ml, mc, sp, container)
        File "/usr/local/scripts/mongodb_backup.py", line 156, in upload_local
          verify_hash=True):
        File "/usr/local/lib/python2.7/dist-packages/libcloud/storage/drivers/azure_blobs.py", line 795, in upload_object
          use_lease=ex_use_lease)
        File "/usr/local/lib/python2.7/dist-packages/libcloud/storage/drivers/azure_blobs.py", line 932, in _put_object
          iterator=iterator)
        File "/usr/local/lib/python2.7/dist-packages/libcloud/storage/base.py", line 654, in _upload_object
          **upload_func_kwargs)
        File "/usr/local/lib/python2.7/dist-packages/libcloud/storage/drivers/azure_blobs.py", line 664, in _upload_in_chunks
          params=params)
        File "/usr/local/lib/python2.7/dist-packages/libcloud/common/base.py", line 782, in request
          headers=headers)
        File "/usr/lib/python2.7/httplib.py", line 979, in request
          self._send_request(method, url, body, headers)
        File "/usr/lib/python2.7/httplib.py", line 1013, in _send_request
          self.endheaders(body)
        File "/usr/lib/python2.7/httplib.py", line 975, in endheaders
          self._send_output(message_body)
        File "/usr/lib/python2.7/httplib.py", line 833, in _send_output
          msg += message_body
      UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128)
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            jamiecressey89 Jamie Cressey
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: