Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
When attempting to use the `deprecated` param on `GCENodeImage.deprecate()` [1], the call was failing with:
2018-08-20 11:52:08 - DEBUG: Delete at = 2018-08-23T10:52:08.061384+00:00 Traceback (most recent call last): File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py", line 6371, in ex_deprecate_image timestamp_to_datetime(value) File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py", line 57, in timestamp_to_datetime ts = datetime.datetime.strptime(timestamp[:-10], '%Y-%m-%dT%H:%M:%S') File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 565, in _strptime_datetime tt, fraction = _strptime(data_string, format) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 365, in _strptime data_string[found.end():]) ValueError: unconverted data remains: .06 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scripts/packer-image-manager.py", line 135, in <module> main() File "scripts/packer-image-manager.py", line 126, in main deprecate_images() File "scripts/packer-image-manager.py", line 56, in deprecate_images dry_run=args.dry_run) File "/Users/fatmcgav/Work/infra/infra/packer2.0/lib/utils.py", line 82, in deprecate_image if image.deprecate(replacement_image, 'DEPRECATED', deleted=delete_at): File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py", line 699, in deprecate deprecated, obsolete, deleted) File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py", line 6375, in ex_deprecate_image attribute) ValueError: deleted must be an RFC3339 timestamp
It would appear that the date parsing logic in `timestamp_to_datetime` [2] is flawed, assuming that the length of `microseconds + tz offset` will always be 10 characters, whereas the above example has 12 characters.
Patch applied as follows:
diff --git a/libcloud/compute/drivers/gce.py b/libcloud/compute/drivers/gce.py index 3dfc87d3..efcdcb87 100644 --- a/libcloud/compute/drivers/gce.py +++ b/libcloud/compute/drivers/gce.py @@ -52,9 +52,9 @@ def timestamp_to_datetime(timestamp): :return: Datetime object corresponding to timestamp :rtype: :class:`datetime.datetime` """ - # We remove timezone offset and microseconds (Python 2.5 strptime doesn't - # support %f) - ts = datetime.datetime.strptime(timestamp[:-10], '%Y-%m-%dT%H:%M:%S') + # We remove timezone offset + ts = datetime.datetime.strptime(timestamp[:timestamp.index('+')], + '%Y-%m-%dT%H:%M:%S.%f') tz_hours = int(timestamp[-5:-3]) tz_mins = int(timestamp[-2:]) * int(timestamp[-6:-5] + '1') tz_delta = datetime.timedelta(hours=tz_hours, minutes=tz_mins)
However once that issue was resolved, the Google Compute API call failed with:
2018-08-20 12:04:59 - DEBUG: Delete at = 2018-08-23T11:04:59.525758+00:00
Traceback (most recent call last):
File "scripts/packer-image-manager.py", line 135, in <module>
main()
File "scripts/packer-image-manager.py", line 126, in main
deprecate_images()
File "scripts/packer-image-manager.py", line 56, in deprecate_images
dry_run=args.dry_run)
File "/Users/fatmcgav/Work/infra/infra/packer2.0/lib/utils.py", line 82, in deprecate_image
if image.deprecate(replacement_image, 'DEPRECATED', deleted=delete_at):
File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py", line 699, in deprecate
deprecated, obsolete, deleted)
File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py", line 6381, in ex_deprecate_image
self.connection.request(request, method='POST', data=image_data).object
File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py", line 124, in request
response = super(GCEConnection, self).request(*args, **kwargs)
File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/common/google.py", line 808, in request
*args, **kwargs)
File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/common/base.py", line 637, in request
response = responseCls(**kwargs)
File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/common/base.py", line 155, in __init__
self.object = self.parse_body()
File "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/common/google.py", line 288, in parse_body
raise InvalidRequestError(message, self.status, code)
libcloud.common.google.InvalidRequestError: {'domain': 'global', 'reason': 'invalid', 'message': "Invalid value for field 'deprecationStatus.deleted': '2018-08-23T11:04:59.525758'. Must be a string representation of a full-date or date-time in valid RFC3339 format with either 0 or 3 digits for fractional seconds"}
Tracked this down to the `ex_deprecate_image` [3] call, which appeared to be 'validating' the format of the timestamp, but not actually storing the converted result.
Another patch applied:
diff --git a/libcloud/compute/drivers/gce.py b/libcloud/compute/drivers/gce.py index 3dfc87d3..a3329133 100644 --- a/libcloud/compute/drivers/gce.py +++ b/libcloud/compute/drivers/gce.py @@ -6523,11 +6523,11 @@ class GCENodeDriver(NodeDriver): continue try: - timestamp_to_datetime(value) + value = timestamp_to_datetime(value) except: raise ValueError('%s must be an RFC3339 timestamp' % attribute) - image_data[attribute] = value + image_data[attribute] = value.isoformat(timespec='milliseconds') request = '/global/images/%s/deprecate' % (image.name)
Pull request incoming with both patches.
[1] https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/gce.py#L6485-L6486
[2] https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/gce.py#L44-L61
[3] https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/gce.py#L6525-L6530
Attachments
Issue Links
- links to
