pyodata 1.8.0 - Error if Edm.DateTimeOffset is not set

[metaodi]

This issue is based on an issue I got on a library that uses pyodata (metaodi/swissparlpy#17)

Some calls fail when pyodata tries to parse dates:

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/model.py in parse_datetime_literal(value)
 388 try:
--> 389 return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M:%S.%f')
 390 except ValueError:

/opt/tljh/user/lib/python3.7/_strptime.py in _strptime_datetime(cls, data_string, format)
 576 format string."""
--> 577 tt, fraction, gmtoff_fraction = _strptime(data_string, format)
 578 tzname, gmtoff = tt[-2:]

/opt/tljh/user/lib/python3.7/_strptime.py in _strptime(data_string, format)
 358 raise ValueError("time data %r does not match format %r" %
--> 359 (data_string, format))
 360 if len(data_string) != found.end():

ValueError: time data '0000-00-00T00:00:00' does not match format '%Y-%m-%dT%H:%M:%S.%f'

And the trace is the following:

/opt/tljh/user/lib/python3.7/site-packages/swissparlpy/client.py in __init__(self, entity_request, variables)
 59 class SwissParlResponse(object):
 60 def __init__(self, entity_request, variables):
---> 61 self.entities = entity_request.execute()
 62 self.count = self.entities.total_count
 63 self.variables = variables

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/service.py in execute(self)
 326 self._logger.debug(' body: <cannot be decoded>')
 327 
--> 328 return self._handler(response)
 329 
 330 def custom(self, name, value):

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/service.py in get_entities_handler(response)
 1407 result = ListWithTotalCount(total_count)
 1408 for props in entities:
-> 1409 entity = EntityProxy(self._service, self._entity_set, self._entity_set.entity_type, props)
 1410 result.append(entity)
 1411 

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/service.py in __init__(self, service, entity_set, entity_type, proprties, entity_key, etag)
 766 else:
 767 # null value is in literal form for now, convert it to python representation
--> 768 self._cache[type_proprty.name] = type_proprty.from_literal(type_proprty.typ.null_value)
 769 
 770 # then, assign all navigation properties

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/model.py in from_literal(self, value)
 837 return None
 838 
--> 839 return self.typ.traits.from_literal(value)
 840 
 841 def to_literal(self, value):

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/model.py in from_literal(self, value)
 554 tz_sign = -1 if match.group('sign') == '-' else 1
 555 tz_info = datetime.timezone(tz_sign * tz_offset)
--> 556 return parse_datetime_literal(datetime_part).replace(tzinfo=tz_info)
 557 except (ValueError, AttributeError):
 558 raise PyODataModelError(f'Cannot decode datetimeoffset from value {value}.')

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/model.py in parse_datetime_literal(value)
 395 return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M')
 396 except ValueError:
--> 397 raise PyODataModelError(f'Cannot decode datetime from value {value}.')
 398 
 399 

PyODataModelError: Cannot decode datetime from value 0000-00-00T00:00:00.

I'm not 100% I understand everything that happens, but to me it seems, that in pyodata/v2/service.py (Line 768) the defined null_value is passed to from_literal and the null_value of Edm.DateTimeOffset is set to 'datetimeoffset\'0000-00-00T00:00:00Z\''

The new code introduced in #184 doesn't seem to check for this case.

My current workaround is this to monkey patch this:

def patched_parse_datetime_literal(value):
 print(value)
 if value == '0000-00-00T00:00:00':
 return datetime.datetime(1970, 1, 1)
 try:
 return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M:%S.%f')
 except ValueError:
 try:
 return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M:%S')
 except ValueError:
 try:
 return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M')
 except ValueError:
 raise PyODataModelError(f'Cannot decode datetime from value {value}.')
pyodata.v2.model.parse_datetime_literal = patched_parse_datetime_literal

But maybe this is not the best place to fix this issue.

cc @rettichschnidi

[metaodi]

Here is the server response that leads to this error: Transcript.txt

[rettichschnidi]

0. The default value is poorly chosen (invalid month, invalid day). Will create a PR.
1. Python does not like the year being 0:

$ ipython3 
Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.20.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import datetime; datetime.datetime(0,1,1)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-4df0df8009df> in <module>
----> 1 import datetime; datetime.datetime(0,1,1)

ValueError: year 0 is out of rang

I do not know how to work around this and whether it would be needed in any case. Need to look up if OData allows specifying dates in year 0 or before.

For now, I will fix this by defaulting to a suitable default year (1970?).

[rettichschnidi]

Need to look up if OData allows specifying dates in year 0 or before.

Did not dig into this, but it seems like we have never been able to deal with values before 01 Jan 0001 anyway. Went on to create PR #197.

[phanak-sap]

This should be fixed in #195 and part of release 1.9.0. Please reopen if still reproducible.