silence warning for decode_cf_datetime?

[mathause]

Code Sample, a copy-pastable example if possible

import xarray as xr
import numpy as np

x = np.arange(100) * 365
x = xr.coding.times.decode_cf_datetime(x, 'days since 2400-01-01', 
        calendar='proleptic_gregorian')

Problem description

xarray still throws an error when decoding out-of-bounds dates for proleptic_gregorian - should this be silenced?

Output of xr.show_versions()

Details

# Paste the output here xr.show_versions() here NSTALLED VERSIONS ------------------ commit: None python: 3.7.1 | packaged by conda-forge | (default, Nov 13 2018, 18:33:04) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.4.172-86-default machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.2

xarray: 0.11.3
pandas: 0.24.1
numpy: 1.16.1
scipy: 1.2.0
netCDF4: 1.4.2
pydap: None
h5netcdf: 0.6.2
h5py: 2.9.0
Nio: None
zarr: None
cftime: 1.0.3.4
PseudonetCDF: None
rasterio: 1.0.17
cfgrib: None
iris: None
bottleneck: 1.2.1
cyordereddict: None
dask: 1.1.1
distributed: 1.25.3
matplotlib: 3.0.2
cartopy: 0.17.0
seaborn: 0.9.0
setuptools: 40.7.3
pip: 19.0.1
conda: None
pytest: 4.2.0
IPython: 7.2.0
sphinx: 1.8.4

[spencerkclark]

Thanks @mathause -- I'm assuming you are referring to this warning, rather than an error?

//anaconda/envs/xarray-dev-37/bin/ipython:2: SerializationWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using dummy cftime.datetime objects instead, reason: dates out of range

The rationale for keeping this warning is that for standard calendars (like 'proleptic_gregorian') we use pandas/NumPy dates by default; therefore falling back to using cftime in this circumstance is a deviation from the default behavior, which might come as a surprise to some users.

I wonder if it might make sense to offer an option in xarray to always decode times to cftime dates, even for standard calendar dates between years 1678 and 2262 (which, if set, could also silence this warning); this would be another possible solution for #1263.

[mathause]

Yes, I meant this warning.

Thanks, I understand better now - it makes sense to throw a warning when you unexpectedly get a cftime instead of pandas dates.

An option might be a good way to get consistent datetimes. Especially as the capabilities with cftime are progressing. What is still missing once 0.12 is released (plotting and resampling)? I don't have the full overview there.

Btw: I am really thrilled to be able to have datetime irrespective of the calendar.

[mathause]

I just ran into the problem trying to xr.concat two files - one that runs from 2000 to 2150, the second from 2151 to 2300. They have a mix of Timestamp and cftime.DatetimeGregorian in the time vector which, obviously can not be saved to disk. (I.e. concat worked but not to_netcdf).

[shoyer]

+1 for adding an option to always use CF datetimes. I would probably start with putting it on open_dataset() rather than making it global -- local options make it much easier to follow control flow.

[shoyer]

From a design perspective, I would consider adding an optional use_cftime argument:

• The default value use_cftime=None would match current behavior
  use_cftime=True` would mean always use cftime (no warnings)
• use_cftime=False would mean never use cftime (error instead)

[spencerkclark]

Thanks for your thoughts @shoyer; I like your suggested approach.

[spencerkclark]

What is still missing once 0.12 is released (plotting and resampling)? I don't have the full overview there.

@mathause indeed we've made a lot of progress. A few remaining missing things that come to mind (there could be others) are:

• Support for missing values in arrays of cftime objects (I should probably create an issue for this)
• Possible PeriodIndex-like functionality with cftime objects (Implement CFPeriodIndex #2481)
• coarsen (coming in the next version of xarray) still needs to be updated to work with cftime coordinates (waiting on fix datetime_to_numeric and Variable._to_numeric #2668 before starting on that)

Out of those three, coarsen is probably the highest priority on my list at the moment (the other two would require some more careful thought).